df['price'].sum() returns '19.995.0012.50' — pandas happily concatenated your prices, because a CSV of numbers with one stray '£' in it loads as strings. Nothing crashed. Nothing warned you. The column just isn't what you think it is, and every operation on it is quietly wrong.
What you'll learn
What
Type coercion is converting columns to their intended dtypes: astype() when the data is clean, pd.to_numeric() and pd.to_datetime() with errors='coerce' when it isn't, and astype('category') for low-cardinality labels. It's the step that makes arithmetic, comparison, sorting, and date math mean what they should.
Why
dtypes decide behaviour. Strings sort alphabetically ('10' < '9'), sum by concatenation, and refuse date arithmetic. A wrong dtype doesn't error — it produces plausible nonsense, which is far more dangerous. Coercion converts silent wrongness into either correct values or visible NaNs you can count.
Where it's used
Immediately after every read_csv in every pipeline. Loading, cleaning, feature engineering, and database writes all begin with getting dtypes right — and memory-heavy workflows lean on category dtype to fit data in RAM.
Where this runs in production
Market data arrives from dozens of exchanges as text with mixed formats — '1,234.50', '1.234,50' — and is coerced to numeric with strict validation, because a misparsed price becomes a wrong trade.
Hospital records mix date formats across systems; ingestion pipelines run to_datetime with explicit formats and audit the coercion failures, since a swapped day/month changes a patient's timeline.
Basket analysis over hundreds of millions of rows stores product department and aisle labels as category dtype, cutting memory severalfold so the analysis fits on ordinary machines.
print(df.dtypes) # price object <- suspicious: should be float # quantity int64 # order_date object <- suspicious: should be datetime
info() and dtypes are the detection tools. Any object column with a numeric or date-like name deserves a look at its unique values to find the contaminating entries.
df['col'].astype(int), astype(float), astype(str), astype('category') — astype converts every value or raises a ValueError on the first one it can't handle. That strictness is a feature for clean data: it guarantees the whole column converted. For dirty data it's the wrong tool, because one '£4.99' aborts the entire cast.
pd.to_numeric(df['price'], errors='coerce') # bad values -> NaN pd.to_datetime(df['date'], errors='coerce') # bad dates -> NaT converted = pd.to_numeric(df['price'], errors='coerce') print(converted.isna().sum() - df['price'].isna().sum()) # NEW NaNs = failures
The forgiving converters: errors='coerce' turns unconvertible values into NaN (NaT for dates) instead of raising. The crucial habit is the last line — count how many NEW nulls coercion created. That number is your failure count; investigate it before moving on.
df['price'] = pd.to_numeric(
df['price'].str.replace(',', '', regex=False)
.str.replace('₹', '', regex=False)
.str.strip(),
errors='coerce',
)Strip separators and symbols first so values like '₹1,299.00' convert to 1299.0 instead of becoming NaN. Coercing without cleaning throws away recoverable data.
astype('category') stores each distinct value once and replaces the column with small integer codes. A million-row 'city' column with 40 cities shrinks dramatically and groups faster. Use it for low-cardinality labels (status, department, city); skip it for high-cardinality or free-text columns, where the codes table saves nothing.
NumPy's int64 has no missing-value representation, so a column with nulls silently becomes float64 (1 → 1.0) and astype(int) on it raises. Options: fill the nulls first, keep floats, or use pandas' nullable 'Int64' (capital I) dtype, which stores integers alongside pd.NA. That's why ID columns often arrive as floats — a single null demoted them.
A warehouse where one shelf's weights are recorded in kilograms and another's in the word 'heavy' can't answer 'total weight?'. Coercion is relabelling everything into proper units: afterwards the questions become answerable, and anything that couldn't be relabelled is flagged (NaN) rather than quietly mixed in.
Click each node to follow a column from object dtype to its true type.
See what a wrong dtype actually does before fixing it.
Quoted digits: the column loads as object (strings), not numbers.
Your task
A sales export loaded every column as text. Convert amount (strip the $ and commas first), parse the date column, and count how many values coercion failed to convert.
[1200.0, 350.0, nan, 780.0] datetime64[ns] 2
Write your solution in the editor on the right, then hit Run.
A CSV column of a million numbers contains one entry 'N/A'. What dtype does read_csv give the column?
What's the key behavioural difference between astype(float) and pd.to_numeric(..., errors='coerce')?
Why does an integer ID column often show up as float64 after loading?
What should actually happen next?
Convert the 'ordered' column to datetime (coercing failures) and print how many rows were ordered in March 2026. df = pd.DataFrame({'ordered': ['2026-03-02', '2026-02-27', '2026-03-15', 'corrupted', '2026-04-01']})
For which column does astype('category') save the MOST memory?
A numeric column loaded as object dtype. Walk me through your fix.
First I find out why: df['col'].unique() or a value_counts on a sample usually reveals the contaminant — currency symbols, thousands separators, 'N/A' strings, stray whitespace. If the junk is systematic and recoverable, I clean first: .str.replace for symbols and separators, .str.strip for whitespace. Then I coerce with pd.to_numeric(col, errors='coerce') rather than astype, because dirty data would abort a strict cast. The step people skip is the audit: I compare null counts before and after coercion — every new NaN is a value the conversion destroyed. A handful of genuine junk entries is fine; thousands of failures mean there's another format I should be cleaning rather than discarding. Only when a column is verifiably clean do I use astype directly, precisely because its all-or-nothing failure is then a guarantee, not a nuisance.
When would you choose the category dtype, and how does it work under the hood?
category stores the distinct values once, in a categories array, and replaces the column data with small integer codes pointing into it. So a million-row column with 40 city names holds 40 strings plus a million small ints, instead of a million string objects — often an order-of-magnitude memory saving, and groupbys get faster because they operate on the codes. I reach for it when cardinality is low relative to length: status flags, departments, cities, plan names. It's counterproductive for near-unique columns like IDs or free text, where the categories table is as big as the original data. Two caveats from practice: string methods require going through .str which may cast back, and comparisons or merges between different category sets can behave surprisingly — so I apply it as a final storage optimisation, not in the middle of heavy string cleaning.
How do you make dtype handling robust in a production pipeline rather than a one-off notebook?
The notebook habit is letting read_csv infer dtypes and patching afterwards; the production habit is pinning them. I pass explicit dtype= mappings and parse_dates= to read_csv so every load either produces the expected schema or fails loudly — a format drift in Tuesday's file becomes a visible parse error instead of a silent object column. Where source data is known-dirty, the pipeline cleans then coerces with errors='coerce', but always records the failure count as a metric; a sudden spike in coercion failures is an upstream schema change announcing itself. For dates I specify the format explicitly rather than letting inference guess, since day/month ambiguity ('03/04/2026') is a correctness bug, not a style choice. Finally I validate post-load — assert dtypes match the expected schema — so the type contract is enforced at the boundary, and everything downstream can trust it.
Common Mistakes to Avoid
1) Trusting that a numeric-looking column is numeric — check dtypes after every load. 2) Using astype on dirty data — one bad value aborts the cast; coerce instead. 3) Coercing without counting the new NaNs — that count is destroyed data. 4) Discarding recoverable values ('₹1,299') that a .str.replace would have saved. 5) Letting to_datetime guess ambiguous formats — '03/04/2026' needs an explicit format. 6) astype(int) on a column with nulls — it raises; use fillna first or nullable 'Int64'. 7) Applying category to high-cardinality columns and wondering where the memory saving went.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'ELI5: why did my price column sum to a giant string?' • 'Show me clean-then-coerce on messy currency data step by step.' • 'Quiz me: astype vs to_numeric — which tool for which column?' • 'Explain why one null turns an int column into floats.' • 'Interview mode: grill me on making dtypes robust in a production pipeline.'
Glossary
dtype — a column's data type (int64, float64, object, datetime64, category). object — the catch-all dtype, usually strings; the red flag. astype() — strict conversion; raises on any unconvertible value. pd.to_numeric / pd.to_datetime — converters with an errors parameter. errors='coerce' — turn unconvertible values into NaN/NaT instead of raising. NaT — Not a Time, the datetime missing value. Nullable Int64 — pandas integer dtype that can hold pd.NA. category — dtype storing distinct values once plus integer codes per row. Coercion audit — counting NEW nulls created by a conversion.
Recommended Resources
• Docs: pandas user guide sections 'Basics — dtypes' and 'Time series' (for to_datetime formats), plus 'Categorical data'. • Read: the read_csv API docs on dtype=, parse_dates=, and converters= — schema pinning is a production superpower. • Practice: export a spreadsheet with currency formatting to CSV, load it, and recover every value with clean-then-coerce and a zero-failure audit. • Next in DSM: types are right, but the text INSIDE string columns is still messy — String Cleaning tackles casing, whitespace, and extraction.
Recap
✓ object dtype on a numeric- or date-shaped column is the red flag; one bad value demotes a whole CSV column. ✓ astype is strict (raises on failure) — right for clean columns; to_numeric/to_datetime with errors='coerce' handle dirty ones. ✓ Clean before coercing: strip symbols and separators so recoverable values don't become NaN. ✓ Audit every coercion by counting new NaNs — that's destroyed data, not a solved problem. ✓ Nulls force ints to float64; use nullable 'Int64' when you need integers with gaps. ✓ astype('category') slashes memory for low-cardinality labels and speeds up groupbys. Next up: String Cleaning. The dtypes are correct, but 'Delhi', ' delhi ', and 'DELHI' are still three different strings — the .str accessor fixes that.
Run your code to see the output here.