How many UNIQUE customers ordered this month? Which users appear in both the trial list and the churned list? Which required columns are missing from this CSV? Three questions, one structure: the set — a bag of values where duplicates simply cannot exist.
What you'll learn
What
A set is an unordered, mutable collection of unique, immutable elements: {3, 1, 4}. Adding an existing value does nothing; membership tests are hash-fast; and sets support union, intersection, and difference — the Venn-diagram operations.
Why
Data questions are constantly about uniqueness and overlap: distinct values, deduplication, who's-in-both, what's-missing. Sets answer all of them in one line each, at hash speed — where the list equivalents are slow loops you'd have to write and debug.
Where it's used
Deduplication, distinct counts, membership testing against allowlists/blocklists, comparing expected vs actual columns, and cohort overlap analysis.
Where this runs in production
Which listeners are in BOTH the 'podcast fans' and 'daily active' segments? Segment membership is set intersection — run across hundreds of millions of IDs.
Every request checks its IP against threat sets. `ip in blocked` must be instant at any size — hash-based membership, the set's core power.
Expected columns minus actual columns = what's missing; actual minus expected = what's unexpected. Data pipeline tests are set differences.
votes = ['py', 'sql', 'py', 'r', 'py']
distinct = set(votes)
print(distinct) # {'py', 'sql', 'r'} — order not guaranteed
print(len(distinct)) # 3 distinct values
print('sql' in distinct) # True — hash-fastset() is the one-line dedup. Note the output order may differ run to run — sets are UNORDERED; if you need order, sort: sorted(distinct). And `in` is the same constant-time lookup dicts have — sets ARE essentially dicts with only keys.
A set is a party guest list. Names, not seat numbers — nobody 'comes first' (unordered). Writing a name twice doesn't invite them twice (unique). And the bouncer's question — 'are you on the list?' — is answered by looking up the name directly, not reading the list top to bottom (hash membership). Comparing two parties' lists — who's invited to both, who only to yours — is the set algebra.
Union a | b: everything in either. Intersection a & b: only what's in both. Difference a - b: in a but not b (order matters!). Symmetric difference a ^ b: in exactly one. These four answer every overlap question — 'who churned AND was on trial' (&), 'required columns we don't have' (required - actual), 'total unique users across both apps' (|). Named methods exist too (.union(), .intersection(), ...) and accept any iterable.
trial = {'ada', 'kai', 'mia', 'zoe'}
paid = {'kai', 'zoe', 'raj'}
print(trial & paid) # converted: in both
print(trial - paid) # trialed but never paid
print(paid - trial) # paid without a trial
print(trial | paid) # everyone we've seenFour business questions, four operators, zero loops. Note a - b ≠ b - a: difference is directional — 'mine minus yours' and 'yours minus mine' are different questions.
Sets have no indexing (s[0] is a TypeError), no slicing, and no stable display order — code that depends on the order of set iteration is a latent bug that surfaces on another machine or Python version. Need uniqueness AND order? Sort the set when you present it, or dedup with dict.fromkeys(items) which preserves first-seen order. And set comprehension exists: {t.lower() for t in tags} — braces, no colon.
trial = {ada, kai, mia, zoe} · paid = {kai, zoe, raj}. Click each operation for the business question it answers.
Select a type to see its full definition, operations, and data science usage.
How many different products were sold today?
set() collapses duplicates; len() counts distincts; sorted() imposes order for display (returning a list). Three built-ins, complete answer.
3 ['cap', 'mug', 'tee']
Your task
A job posting requires certain skills; a candidate lists theirs (with duplicates). Compute: the candidate's distinct skill count, the required skills they're missing, the extra skills they bring, and whether they fully qualify (no missing skills).
distinct skills: 4 missing: ['git', 'statistics'] extra: ['excel', 'tableau'] qualified: False
Write your solution in the editor on the right, then hit Run.
What does len({1, 2, 2, 3, 3, 3}) return?
How do you create an EMPTY set?
a = {1, 2, 3, 4}; b = {3, 4, 5} — what is a - b?
What's the fix, and why does it work?
emails = ['a@x.io', 'b@x.io', 'a@x.io', 'c@x.io', 'b@x.io'] — print the number of duplicate entries (total minus distinct), then the distinct emails sorted. Expected: 2 ['a@x.io', 'b@x.io', 'c@x.io']
Which element can NOT go in a set?
Site A visitors: ['ada', 'kai', 'mia', 'kai']; site B: ['mia', 'raj', 'ada', 'raj']. Print: how many unique people visited EITHER site, the sorted list of who visited BOTH, and the sorted list of who visited exactly ONE site. Expected: 4 ['ada', 'mia'] ['kai', 'raj']
When do you reach for a set over a list, and what does it cost you?
Three triggers: uniqueness (deduplication and distinct counts — set(items) is the idiom), repeated membership testing (each `x in s` is constant-time hash lookup versus a list's linear scan — decisive inside loops or against large collections), and overlap questions (union, intersection, difference replace hand-written double loops). The costs: sets are unordered — no indexing, no slicing, no stable iteration order, so presentation needs sorted(); elements must be hashable, so no lists-of-lists; and duplicates are destroyed, so if multiplicity matters — counting how MANY times each value appears — you want the dict counting pattern or collections.Counter instead. My one-line heuristic: 'is this value present?' → set; 'how many / in what order?' → dict or list.
Explain the four set operations with a concrete analytics example of each.
Take june and july as sets of active user IDs. Union (june | july) is everyone active in either month — the deduplicated reach number across sources. Intersection (june & july) is users active in both — retention's numerator, and the general 'in both cohorts' question. Difference is directional: june - july is churned users (active then, gone now), while july - june is new users — reversing the operands reverses the business question, which is worth saying out loud because it's a real bug source. Symmetric difference (june ^ july) is users active in exactly one month — the churned plus the new, the 'unstable' cohort. Every one runs in roughly linear time over the sets, replacing nested loops that would be quadratic — and each accepts any iterable via the method forms (.union(list_of_ids)).
Your colleague deduplicates with `unique = []; for x in items: if x not in unique: unique.append(x)`. Critique it.
It's correct and even preserves first-seen order — but it's quadratic: each `x not in unique` scans the growing list, so 100k items means on the order of 5 billion comparisons. The fixes depend on whether order matters. If not: unique = set(items) — one pass, done. If order matters: either keep the loop shape but test against a parallel SET (seen = set(); if x not in seen: seen.add(x); result.append(x)) which keeps order at hash speed, or use the standard idiom list(dict.fromkeys(items)) — dicts preserve insertion order and keys are unique, so it dedups order-preservingly in one line. The deeper interview point: the pattern 'membership test inside a loop against a growing/large collection' should ALWAYS trigger the set reflex, whatever the surrounding code looks like.
Common Mistakes to Avoid
1) {} for an empty set — that's an empty dict; use set(). 2) Relying on set order — unordered by contract; sorted() for display. 3) Indexing a set (s[0]) — TypeError; convert or iterate. 4) Putting lists in sets — unhashable; use tuples. 5) Getting difference backwards — a - b is 'in a, not b'; state the question before writing the operands. 6) Using a set when duplicates carry meaning — counts need the dict pattern, not dedup. 7) Repeated `in some_list` checks at scale — the set conversion pays for itself immediately.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'Quiz me: which operation answers each of these eight cohort questions?' • 'Show the quadratic-dedup fix step by step with timings explained.' • 'Drill me on a-b vs b-a with business phrasing.' • 'When do I need Counter instead of a set?' • 'Interview mode: ask me set vs list vs dict trade-offs and grade my answer.'
Glossary
Set — unordered, mutable collection of unique, hashable elements. set() — the empty set (and the dedup conversion). Membership — `x in s`, hash-fast. Union (|) — in either set. Intersection (&) — in both. Difference (-) — in the left, not the right; directional. Symmetric difference (^) — in exactly one. .add()/.discard()/.remove() — insert; remove-quietly; remove-loudly. Hashable — stable hash required of elements (same rule as dict keys). Set comprehension — {expr for x in items}. dict.fromkeys(items) — order-preserving dedup idiom. frozenset — an immutable set, usable inside other sets.
Recommended Resources
• Docs: 'Sets' in the official Python tutorial and the set-types reference (all operators and methods). • Read: collections.Counter — when uniqueness isn't enough and multiplicity matters. • Practice: take two months of any repeated-entries data you can invent and produce retained/lost/new plus a retention rate — the cohort example, from scratch. • Next in DSM: you now hold lists, tuples, dicts, and sets — Nested Data Structures combines them into the real-world shapes (lists of dicts, dicts of lists) that JSON and APIs actually deliver.
Recap
✓ Sets hold unique, hashable elements — duplicates collapse on entry; set() for empty. ✓ set(items) is one-line dedup; len(set(items)) is the distinct count. ✓ Membership (`in`) is hash-fast — repeated lookups against big collections demand a set. ✓ The algebra: | either, & both, - directional difference, ^ exactly-one. ✓ Unordered by contract: sort for display, never depend on iteration order. ✓ Duplicates meaningful? That's a counting dict, not a set. Next up: Nested Data Structures. Lists, tuples, dicts, and sets are your bricks — now build with them: lists of dicts, dicts of lists, and the navigation skills real JSON demands.
Run your code to see the output here.