No API on earth returns a flat list of numbers. It returns users, each with a profile, each with a list of orders, each with line items. Real data is STRUCTURES INSIDE STRUCTURES — and once you can read a nested shape at a glance and walk it without fear, every JSON payload in the world becomes yours.
What you'll learn
What
Nested data structures are containers holding containers: a list of dicts (records/table), a dict of lists (columns/groups), dicts of dicts (lookup of records), and arbitrary JSON-shaped mixtures. Access chains one step per level: data['users'][0]['email'].
Why
This is the shape of everything: API responses, config files, scraped data, MongoDB documents, and — crucially — a pandas DataFrame is conceptually a dict of columns. The navigation and reshaping skills here (extract, flatten, invert) are pre-pandas data wrangling itself.
Where it's used
Consuming any API, reading JSON/YAML config, log processing, building report structures, and every 'transform this payload into that table' ticket.
Where this runs in production
A repo's pull-request list is a list of dicts, each holding a 'user' dict, a 'labels' list of dicts, and nested 'head'/'base' branch objects — the canonical practice payload.
Document databases store exactly what you're learning to navigate: dicts with nested dicts and arrays, queried by path — 'customer.address.city'.
pd.DataFrame({'name': [...], 'score': [...]}) — the constructor you'll use in weeks takes a dict of lists. Records-vs-columns conversion is this lesson's core reshape.
order = {
'id': 'A-101',
'customer': {'name': 'Ada', 'tier': 'pro'},
'items': [
{'sku': 'KB-201', 'qty': 2},
{'sku': 'MS-330', 'qty': 1},
],
}
print(order['customer']['name']) # dict → dict → value
print(order['items'][1]['sku']) # dict → list → dict → valueRead chains left to right, asking 'what type am I holding now?' after each step: order is a dict, ['customer'] gives a dict, ['name'] gives a string. Mixing ['key'] and [index] is normal — the brackets adapt to whatever container you're currently in.
Nested data is an office filing system: a CABINET (dict) whose labeled drawers hold FOLDERS (lists) of DOCUMENTS (dicts) with named fields. 'Get the second invoice's total from the Clients drawer, Acme folder' — cabinet['clients']['acme'][1]['total']. Nobody finds a document in one motion; you open one container at a time, and each container tells you how it opens: drawers by label, folder contents by position.
A table nests two ways. ROW-oriented — a list of dicts: [{'name': 'Ada', 'score': 95}, ...] — one dict per record; natural for APIs, JSON lines, and iteration. COLUMN-oriented — a dict of lists: {'name': ['Ada', ...], 'score': [95, ...]} — one list per field; natural for computation (sum a column directly) and it's literally the DataFrame constructor's shape. Converting between them is a rite of passage — and two loops.
rows = [{'name': 'Ada', 'score': 95}, {'name': 'Kai', 'score': 88}]
# rows → columns
cols = {'name': [r['name'] for r in rows], 'score': [r['score'] for r in rows]}
print(cols)
print(sum(cols['score'])) # columns make math easyOne comprehension per column: walk the records, pluck one field. The reverse (columns → rows) zips the lists back together. pandas does both with .to_dict('records') and pd.DataFrame(...) — you're learning what those buttons do.
Copying nested structures shallowly shares the innards: config2 = dict(config) makes a NEW outer dict whose values are the SAME inner objects — editing config2['limits']['max'] changes config too. This is the aliasing lesson from Lists vs NumPy, one level down, and it's the #1 'my data changed by itself' bug. When you truly need an independent copy of a nested structure, copy.deepcopy(config) exists — but first ask whether you should be building a new structure instead of copying and mutating.
resp['data']['orders'][0]['total'] — click each hop to see what type you're holding.
Pull one fact out of a two-level structure.
dict → list → element: ['members'] opens the drawer, [1] picks the folder. len() works at whatever level you point it at — here, the member list.
kai 3
Your task
A music service payload holds playlists, each with a list of track dicts. Produce: each playlist's name with its track count and total minutes, and the single longest track title anywhere (flatten + max with a key). Guard the possibly-missing 'tracks' key.
Focus: 2 tracks, 10.5 min Empty Vibes: 0 tracks, 0 min Run: 1 tracks, 3.5 min longest: Deep Flow
Write your solution in the editor on the right, then hit Run.
data = {'items': [{'id': 7}, {'id': 9}]} — how do you read the second id?
Which structure is 'row-oriented' table data?
user.get('address', {}).get('city', 'n/a') — why the {} in the first .get()?
What happened?
Convert columns to rows: cols = {'city': ['oslo', 'lima'], 'temp': [-3.5, 19.5]} → print a list of dicts [{'city': 'oslo', 'temp': -3.5}, {'city': 'lima', 'temp': 19.5}]. (Hint: loop over range(len(...)).)
orgs = [{'team': 'data', 'members': ['ada', 'mia']}, {'team': 'infra', 'members': ['kai']}, {'team': 'data', 'members': ['zoe']}] — build headcount, a dict of team → TOTAL member count (merging duplicate teams), and print it. Expected: {'data': 3, 'infra': 1}
flat = [x for row in grid for x in row] with grid = [[1, 2], [3], [4, 5]] — what is flat?
Compare row-oriented (list of dicts) and column-oriented (dict of lists) table layouts. When is each better?
Row-oriented — [{'name': 'Ada', 'score': 95}, ...] — puts one self-describing dict per record. It's the natural shape for iteration (process each record), for streaming (records arrive one at a time), for ragged data (records can omit fields), and it's what JSON APIs return. Column-oriented — {'name': [...], 'score': [...]} — puts one list per field, which makes whole-column computation trivial (sum(cols['score']) with no extraction loop) and is memory-friendlier since field names are stored once, not per record. It's also literally pandas' constructor shape, because DataFrames are column stores internally — that's what makes vectorized column math fast. The practical skill is converting: records-to-columns is one comprehension per field; columns-to-records zips parallel lists by index. Knowing which shape you're holding — and which shape the next tool wants — is half of data wrangling.
How do you safely extract a deeply nested optional field from a JSON payload?
First decide policy per branch, exactly as with flat dicts: required paths get loud bracket access so malformed payloads fail at the source; optional branches get defensive access. The core defensive idiom is chained .get() with type-preserving defaults — user.get('address', {}).get('city', 'unknown') — where the intermediate {} matters because a bare .get() returns None and None.get crashes with AttributeError; each level's default must be the container type the NEXT step expects ({} before a key access, [] before iteration). For lists, .get('orders', []) lets len/sum/loops work unchanged on absent branches. Beyond a couple of levels, extract a helper — def dig(d, *keys, default=None) looping .get — or lean on a schema validator (pydantic) at the boundary so the interior code can use honest brackets. The anti-pattern to name: wrapping everything in try/except KeyError, which silences genuinely malformed data along with expected gaps.
What's the difference between a shallow and a deep copy of a nested structure, and when does it bite?
A shallow copy — dict(d), list(l), .copy(), slicing — creates a new OUTER container whose slots reference the SAME inner objects. A deep copy — copy.deepcopy — recursively copies every level, producing a fully independent structure. Shallow copying bites whenever you copy-then-mutate below the first level: copy a config template, edit copy['limits']['max'], and every other 'copy' plus the template change together, because 'limits' was one shared dict all along. The same mechanism produces the multiplied-row bug: [[0] * 3] * 2 is two references to ONE row list. Diagnosis tip: `a['x'] is b['x']` returning True reveals the sharing. Mitigations in preference order: build new structures instead of copy-and-mutate (comprehensions naturally produce fresh containers), deepcopy when you genuinely need an independent clone, and treat shared nested data as read-only by convention.
Common Mistakes to Avoid
1) Wrong bracket for the level — ['key'] on a list or [0] on a dict; ask 'what type am I holding?' at each hop. 2) .get() chains without container defaults — None.get crashes; use .get('k', {}) / .get('k', []). 3) Shallow-copying then mutating inner structures — shared innards change everywhere. 4) Overwriting when duplicate keys should accumulate — .get(k, 0) + value, not plain assignment. 5) Processing only [0] when the payload is a list — loop it. 6) Building three-level structures when flat records + a key field would do — flatten early, nest only for output.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'Show me a gnarly JSON payload and quiz me on access paths.' • 'Drill me on records↔columns conversion both directions.' • 'Walk through the shallow-copy config bug with is-checks.' • 'Help me flatten this three-level structure step by step.' • 'Interview mode: ask me row vs column orientation and grade my answer.'
Glossary
Nested structure — a container holding containers. Access chain — successive brackets, one per level (d['a'][0]['b']). Row-oriented — list of dicts; one record per dict. Column-oriented — dict of lists; one list per field. Flattening — converting nested collections to one flat list (double comprehension). Leaf — a non-container value at the bottom of a structure. Shallow copy — new outer container, shared inner objects. Deep copy — fully recursive copy (copy.deepcopy). Ragged data — records with differing/missing fields. Chained .get() — defensive navigation with container defaults.
Recommended Resources
• Docs: the json module page — loads/dumps turn this lesson's shapes into text and back. • Read: any public API's example response (GitHub's REST docs are ideal) and sketch its shape as nested brackets before touching code. • Practice: take the Industry Example payload, add a third nesting level (order → items), and extend the flattener to produce one row per ITEM. • Next in DSM: you now know all four structures and their compositions — Choosing the Right Structure is the capstone: a decision framework, performance intuition, and a mini-project wiring everything together.
Recap
✓ Read shapes by brackets: [ = list (positions), { = dict (names); chain access one level per hop. ✓ Row shape (list of dicts) iterates; column shape (dict of lists) computes — convert deliberately. ✓ Optional branches: chained .get() with {} / [] defaults; required ones: loud brackets. ✓ Flatten nested lists with double comprehensions; aggregate duplicates with .get(k, 0) +. ✓ Shallow copies share innards — deepcopy or build fresh when independence matters. ✓ Nested-in, flat-out: most ingestion code reduces payloads to flat records. Next up: Choosing the Right Structure. Lists, tuples, dicts, sets, and their nestings are all in hand — the finale is judgment: picking the structure the PROBLEM wants, with performance intuition to back it.
Run your code to see the output here.