Ninety percent of data analysis is asking 'show me just these rows and these columns.' Do it clumsily and you get warnings, wrong answers, and a corrupted DataFrame you don't notice until it's in production. Master three tools — .loc, .iloc, and boolean masks — and selection becomes surgical.
What you'll learn
What
Data selection is the act of pulling a subset of a DataFrame: specific rows, specific columns, or the cells where a condition holds. pandas gives you label-based selection (.loc), position-based selection (.iloc), and condition-based selection (boolean masks).
Why
Every filter, every feature you engineer, every subset you feed a model starts with selection. Choosing the right tool keeps your code correct and readable; choosing the wrong one — especially chained indexing — introduces silent bugs.
Where it's used
Filtering customers by segment, isolating a date range, grabbing feature columns for a model, or extracting the rows that failed a quality check — all of it is selection.
Where this runs in production
Analysts select the rows where experiment_group == 'treatment' and region == 'US' with a single boolean mask, then compare watch-time against control — the selection step defines the whole experiment.
A fraud team masks the transactions DataFrame for status == 'failed' and amount > 500, using .loc to pull just the columns they need for review, leaving the multi-million-row table untouched.
To study surge pricing, data scientists select trips between 18:00 and 21:00 with a mask on the hour column, then read out fare and distance columns with .loc for modelling.
.loc[rows, cols] selects by label — the index labels and column names. .iloc[rows, cols] selects by integer position — counting from zero, ignoring labels.
import pandas as pd
df = pd.DataFrame(
{'name': ['Amara','Jamal','Priya','Wei'],
'age': [28, 34, 29, 42],
'city': ['Lagos','Cairo','Delhi','Xian']},
index=['a','b','c','d']
)
print(df.loc['b', 'name']) # label row 'b', col 'name' -> Jamal
print(df.iloc[1, 0]) # position row 1, col 0 -> JamalBoth land on Jamal, but by different routes: .loc uses the label 'b' and the name 'name'; .iloc uses positions 1 and 0. When the index isn't the default 0,1,2, only one of them will do what you mean.
mask = df['age'] > 30 # boolean Series: b and d are True older = df[mask] # keep rows where mask is True older_names = df.loc[df['age'] > 30, 'name'] # rows + one column
df[mask] filters rows. df.loc[mask, 'name'] filters rows and selects a column in one call — the preferred, unambiguous pattern.
A boolean mask is a stencil laid over your DataFrame: everywhere the stencil says True, the data shows through; everywhere it says False, it's blocked. The stencil has the same length as the column, one hole per row.
Python's and/or expect single True/False values, but a mask is a whole Series — so pandas raises 'The truth value of a Series is ambiguous.' Use the bitwise operators & (and), | (or), ~ (not), and wrap every condition in parentheses: (df['age'] > 30) & (df['city'] == 'Cairo'). The parentheses are mandatory because & binds tighter than >.
Writing df[df['age'] > 30]['name'] = 'X' chains two selections: pandas may operate on a temporary copy, so your assignment silently vanishes and you get SettingWithCopyWarning. Always assign through a single .loc call: df.loc[df['age'] > 30, 'name'] = 'X'. One set of brackets, one operation, guaranteed to write to the original.
Click each selector to see what it takes, what it returns, and when to reach for it.
Select a type to see its full definition, operations, and data science usage.
From an employee table, you just need the list of names.
A two-column DataFrame with a default 0,1,2 index.
Your task
A logistics company tracks deliveries. Complete the selection tasks to surface the shipments that need attention, using safe, single-call patterns.
Urgent: ['S4'] Weights: [5, 0, 8, 0, 0]
Write your solution in the editor on the right, then hit Run.
Which expression returns a one-column DataFrame (not a Series)?
What is the key difference between .loc and .iloc?
Why does `df[(df['a'] > 1) and (df['b'] < 5)]` raise an error?
What went wrong and how should it be written?
Given the DataFrame below, print the total revenue (price * qty summed) for orders from the 'EU' region only. Print a single number. df = pd.DataFrame({'region':['EU','US','EU','APAC','EU'], 'price':[10,20,15,30,25], 'qty':[2,1,4,1,2]})
For `df` with a default index, which selects the first 3 rows and the columns 'a' and 'b'?
When would you use .loc versus .iloc?
I use .loc when I'm selecting by meaning — a specific index label, a column name, a date range, or a boolean mask. I use .iloc when I'm selecting by position — the first N rows, the last row, or a column by its integer order regardless of its name. The distinction matters most when the index isn't the default range: after filtering or sorting, .loc[0] looks for the label 0, which may no longer exist, while .iloc[0] always gives me the first row. As a rule I default to .loc for readability and only drop to .iloc when position is genuinely what I mean.
What is the chained-indexing problem and how do you avoid it?
Chained indexing is when you select twice in a row, like df[df['x'] > 0]['y'] = 1. The first selection can return either a view or a copy of the data, and pandas can't reliably tell which, so the assignment might land on a throwaway copy and silently do nothing — you get a SettingWithCopyWarning but no error. I avoid it by collapsing selection and assignment into one .loc call: df.loc[df['x'] > 0, 'y'] = 1. That's a single indexed operation, so pandas always targets the original frame. The same rule applies to reads when I care about getting a stable, independent object — I take an explicit .copy() rather than relying on a chained slice.
How do boolean masks work under the hood, and what are the performance implications on large DataFrames?
A boolean mask is a Series of True/False the same length as the DataFrame, produced by a vectorized comparison that runs in NumPy's C layer. When I pass it back with df[mask] or df.loc[mask], pandas uses it to build a new frame from the True positions. Because the comparison and the selection are both vectorized, masking millions of rows is fast — far faster than any Python-level loop. The costs to watch are memory and combination overhead: each mask allocates a full-length boolean array, so building many intermediate masks on a huge frame adds up, and combining them with & and | creates more temporaries. For very large data I lean on query() for readability, categorical dtypes to shrink comparisons, or chunked processing, but for the vast majority of work a straightforward boolean mask is both the clearest and the fastest option.
Common Mistakes to Avoid
1) Using `and`/`or` to combine masks instead of `&`/`|` — pandas raises 'truth value is ambiguous'. 2) Forgetting parentheses around each condition: `df['a'] > 1 & df['b'] < 5` misbinds because & has higher precedence than the comparisons. 3) Chained indexing for assignment — df[mask]['col'] = v may write to a copy; use df.loc[mask, 'col'] = v. 4) Reaching for .loc[0] after filtering and expecting the first row — the label 0 may be gone; use .iloc[0] for positional access. 5) Assuming .loc label slices are end-exclusive — unlike .iloc and Python lists, df.loc[0:2] includes label 2.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'ELI5: what's the difference between .loc and .iloc?' • 'Show me the chained-indexing bug and walk me through the fix.' • 'Quiz me on combining boolean conditions with & and |.' • 'Explain why df.loc[0:2] includes label 2 but df.iloc[0:2] stops at 1.' • 'Interview mode: ask me when I'd choose a boolean mask over .loc.'
Glossary
.loc[] — label-based selection of rows and columns; slices are inclusive. .iloc[] — integer-position selection; slices are end-exclusive. Boolean mask — a True/False Series from a comparison, used to filter rows. & | ~ — bitwise and/or/not for combining masks element-wise. Chained indexing — two back-to-back selections that may hit a copy, breaking assignment. SettingWithCopyWarning — pandas' warning that an assignment may not reach the original DataFrame. View — a window onto the original data. Copy — an independent duplicate. reset_index(drop=True) — replace the current index with a fresh 0-based one.
Recommended Resources
• Docs: 'Indexing and selecting data' in the pandas User Guide — the definitive reference for .loc, .iloc, and masks, including the returning-a-view-versus-copy rules. • Read: the pandas 'Returning a view versus a copy' section for the full chained-indexing story. • Practice: take any DataFrame, write the same selection three ways (bracket, .loc, mask) and confirm they match; then deliberately trigger and fix a SettingWithCopyWarning. • Next in DSM: you can pull out any subset — next, in Adding & Modifying Columns, you'll write new data back, building derived columns with fast vectorized operations.
Recap
✓ df['col'] returns a Series; df[['col']] (a list) returns a DataFrame. ✓ .loc selects by label; .iloc selects by integer position — they diverge once the index isn't 0,1,2. ✓ Boolean masks filter rows; pair them with .loc to also choose columns. ✓ Combine conditions with & and |, wrapping each in parentheses. ✓ Avoid chained indexing for assignment — use one df.loc[mask, 'col'] = value call. ✓ .loc label slices are inclusive; .iloc position slices are end-exclusive. Next up: Adding & Modifying Columns. You can select any slice of a DataFrame — next you'll create and update columns, turning raw fields into the derived features every analysis and model depends on.
Run your code to see the output here.