Everything in this course was training for this: a raw dataset, a stakeholder question, and 90 minutes to produce findings someone will act on. No lesson scaffolding, no pre-cleaned columns — the full loop, the way it happens on the job. By the end you'll have run a complete professional EDA and written the report that proves it.
What you'll learn
What
A guided project: you receive a realistic multi-table e-commerce dataset (orders, customers) with authentic quality problems, a stakeholder brief ('what should we know about our business?'), and you execute the four-phase workflow end to end — frame, profile/clean, explore (univariate → bivariate → multivariate), communicate.
Why
Skills consolidate only under integration pressure. Separately you can groupby, merge, and spot skew; the job is doing them in the right order, catching the trap the data set for you, and producing findings with numbers and caveats. This project is the difference between knowing pandas and doing analysis.
Where it's used
This IS the job: the first-week analysis at a new company, the due-diligence pass on a new data source, the take-home assignment in most data-analyst and data-scientist interviews.
Where this runs in production
Analyst interviews at marketplaces hand candidates order data and ask 'what do you see?' — evaluating exactly this project's loop: audit first, honest statistics, findings with caveats.
Internal analysts run this workflow on merchant data weekly: join orders to accounts, clean, segment, and report — with mix effects and small-cell discipline deciding what ships to leadership.
Quarterly category deep-dives are this project at scale: revenue concentration, segment behaviour, quality caveats on returns data — the same phases, bigger tables.
import pandas as pd
import numpy as np
orders = pd.DataFrame({
'order_id': [1001,1002,1003,1004,1005,1006,1007,1008,1008,1010,1011,1012],
'cust_id': ['C1','C2','C1','C3','C2','C9','C4','C3','C3','C1','C4','C2'],
'amount': ['1,200','450','2,100','800','-50','950','1,500','700','700','9000','650','1,100'],
'channel': ['web','app','web','APP','app','web',' web','app','app','web','app','Web'],
'order_date':['2026-05-02','2026-05-03','2026-05-10','2026-05-11','2026-05-12',
'2026-05-15','2026-05-18','2026-05-20','2026-05-20','2026-05-21',
'2026-06-01','2026-06-02'],
})
customers = pd.DataFrame({
'cust_id': ['C1','C2','C3','C4','C5'],
'segment': ['premium','regular','regular','premium','regular'],
'signup': ['2025-01-10','2025-06-05','2026-04-28','2025-11-11','2026-05-30'],
})Your data. Twelve orders, five customers — small enough to verify by hand, seeded with the course's real traps: amounts as strings with separators and a negative, channel labels with casing/whitespace variants, a duplicated order row, an orphan customer key, an extreme amount, and one customer with no orders.
Turn the vague brief into 4–6 answerable questions. A solid set: Q1 How much revenue, and how concentrated across customers? Q2 Does order value differ by channel? Q3 Does behaviour differ by segment? Q4 Any trend across the period? Q5 What data quality risks limit the answers? Write hypotheses too — 'premium customers order larger' is now testable.
Run the audit: shape, dtypes, isna, duplicated, value_counts on channel, describe on amount (after coercion). Then fix in order: types first (amount is strings with commas — clean-then-coerce), strings next (channel needs strip+lower), duplicates (order 1008 appears twice — exact copy, safe drop), then judge the invalid (-50: a refund miscoded? exclude with disclosure) and the extreme (9000: possible bulk order — investigate, don't delete). COUNT every fix for the report.
Univariate: amount's shape (mean vs median — that 9000 will split them), channel and segment balance, date coverage. Bivariate: amount by channel (medians!), amount by segment (requires the merge — validate it: many_to_one, count rows before and after, count the orphan C9), orders over time. Multivariate: does the channel difference hold WITHIN each segment, or is it a mix effect? That's the conditioning check that separates this course's graduates from groupby operators.
This dataset will punish shortcuts: 1) Sum amount without coercion → string concatenation. 2) groupby channel without string cleaning → 'web', ' web', 'Web', 'APP' as separate groups. 3) Skip dedup → order 1008 double-counts 700. 4) Keep -50 silently → revenue understated with no disclosure. 5) Report mean order value with the 9000 in → inflated by ~60%. 6) Inner-join to customers → C9's order silently vanishes. 7) Compare channels without the segment split → possible mix story. Each trap is a lesson from this course; finding all seven is the real mastery test.
The report format: one line per finding — claim, number, caveat. Example: 'Revenue is highly concentrated: the top customer (C1) drives 58% of clean revenue [caveat: 11-order sample, one 9000 bulk order pending verification].' Plus a quality appendix: what you fixed, what you excluded, what you couldn't verify. End with next questions ('why does C9 exist in orders but not customers?').
Every previous lesson taught one manoeuvre in isolation — takeoffs, stalls, crosswind landings. A check-ride strings them together over unfamiliar terrain while the examiner watches whether you run your checklists under pressure. This dataset is your terrain; the seven traps are the examiner's engine-failure card. Pass here and the license means something.
Click each phase — the full course compressed into one working session.
Before any analysis: run the ritual and list what's wrong.
amount is object — strings with commas and (peek at the values) a negative. And one fully duplicated row lurks (order 1008).
Your task
Run the core of the project pipeline yourself: clean the amounts and channels, drop duplicates, exclude negatives, and produce the median order value per (cleaned) channel.
4
{'app': 450.0, 'web': 1200.0}Write your solution in the editor on the right, then hit Run.
In the project pipeline, why must channel strings be cleaned BEFORE dropping duplicates?
The -50 amount was excluded 'with disclosure'. What does disclosure add over just filtering it out?
Why did the project use a LEFT join (not inner) from orders to customers?
What's the correct explanation?
Complete the merge step with full discipline: left-join orders to customers on cust_id with validate and indicator, then print the number of unmatched orders and total revenue including them. orders = pd.DataFrame({'cust_id':['C1','C2','C9'], 'amount':[100.0, 200.0, 300.0]}) customers = pd.DataFrame({'cust_id':['C1','C2'], 'segment':['premium','regular']})
The pivot showed web's channel advantage shrinks within the regular segment. What is the honest way to report the channel finding?
You're handed a take-home: order data, 'tell us something interesting', 48 hours. How do you structure your work and your submission?
I run the four-phase loop with the clock in mind. Hour one: framing — 4–6 questions the business would pay to answer (revenue concentration, segment behaviour, trend, quality risks), written in the notebook's first cell, because reviewers grade structure before statistics. Then the audit and cleaning with a counted trail: dtype fixes, string normalisation, dedup, invalid-value judgements — each fix one line in a quality log, since take-homes routinely plant traps and finding them IS the test. Exploration goes univariate (shapes decide mean-vs-median), bivariate (the segment and channel cuts with counts), then at least one conditioning check on my headline finding, because a finding that survives Simpson-checking is the differentiator. The submission leads with the report: 5–8 findings, each one sentence with a number and caveat, the quality appendix, and next questions — code after, not instead. What I deliberately don't do: forty charts, causal verbs, or means on distributions I never looked at. Reviewers can't distinguish those from not knowing better.
Across this whole course, what's the single discipline that most improves analysis quality, and why?
Auditing before analysing — in the broad sense: never computing on data whose shape, types, and quirks I haven't checked. It compounds across every module. The five-command audit catches the string-typed amounts before they concatenate, the channel variants before they split groups, the duplicate before it double-counts, the orphan key before a join swallows it. Univariate shape-checking is the same discipline pointed at distributions — it catches the outlier before the mean ships. The conditioning check is the discipline pointed at findings — it catches the mix effect before the headline. In each case the pattern is identical: a cheap, systematic look BEFORE the expensive conclusion, converting silent errors into visible, countable ones. Its absence is also the common thread in every analysis disaster I've described in interviews: nobody looked before computing. The habit costs minutes; skipping it costs credibility — usually in a meeting, three weeks later, when someone asks why revenue doesn't reconcile.
How would you extend this project's analysis if you had the company's full data and a month?
Three directions, in order of value. First, depth on the raised questions: verify the bulk order and refund handling with finance, resolve the orphan-customer pipeline gap with engineering, and rebuild the numbers on the corrected base — quality work is analysis work at real companies. Second, time: with more than one month of orders I'd add the window-function layer — rolling revenue trends, cohort retention by signup month, week-over-week growth with shift(7) seasonality discipline — because trends answer 'is this getting better?', which snapshots can't. Third, graduation from slicing to modelling: my segment cuts condition on one or two variables before cells thin out; with the full customer base I'd fit simple models (regression for order value, a churn classifier) that condition on many variables simultaneously, using the EDA findings as feature candidates — and the leakage discipline from window functions when any feature involves time. Throughout, the deliverable cadence stays the same: findings with numbers and caveats shipped weekly, not a month-end reveal — because analysis compounds when stakeholders can steer it.
Common Mistakes to Avoid
1) Diving into groupbys before the audit — this dataset punishes it seven ways. 2) Cleaning without counting: 'I fixed some stuff' is not an audit trail. 3) Dropping the -50 and the 9000 with the same reflex — one is invalid (exclude, disclose), the other is possible (verify, keep pending). 4) Inner-joining away the orphan order and never noticing revenue shrank. 5) Reporting means on a distribution with an 80% mean-median gap. 6) Shipping the channel finding without the segment conditioning check. 7) Submitting the notebook as the deliverable — the report leads, code follows. 8) Ending without next questions: an EDA that raises none has looked at too little.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'Grade my project report draft against the findings+caveats format.' • 'I found 5 of the 7 planted traps — quiz me toward the rest.' • 'Generate a fresh messy dataset with different traps and let me run the loop again.' • 'Roleplay the VP: interrogate my findings and caveats.' • 'Help me plan the Kaggle version of this project for my portfolio.'
Glossary
Capstone — an integrative project exercising a course's full skill set. Stakeholder brief — the (often vague) request an analysis must convert into questions. Audit trail — the counted record of every fix and exclusion. Orphan key — a foreign key with no match in the lookup table. Quality appendix — the report section disclosing fixes, exclusions, and unverifiable items. Findings report — claim + number + caveat, 5–8 lines, decision-ready. Take-home — the interview format this project mirrors. Conditioning check — re-testing a finding within third-variable levels before shipping it.
Recommended Resources
• Datasets for round two: Kaggle's Olist e-commerce set, NYC TLC taxi trips, or any open-data portal — pick one with obvious business questions. • Read: a well-regarded public EDA notebook (Kaggle grandmaster kernels) and study its STRUCTURE, not its charts — framing, audit, levels, report. • Practice: re-run this project's loop on your chosen dataset this week; publish with the findings report as the first screen. • Next in DSM: the Data Analysis domain is complete — the SQL domain teaches the same querying instincts where most company data actually lives.
Recap
✓ The full loop ran end to end: vague brief → framed questions → counted cleaning → validated join → three-level exploration → findings with caveats. ✓ Seven traps, seven course lessons: string amounts (coercion), channel variants (string cleaning), the dupe (dedup), the -50 (invalid values), the 9000 (outlier judgement), the orphan (join discipline), the mix effect (conditioning). ✓ Every fix counted, every exclusion disclosed, every headline statistic chosen by the distribution's shape. ✓ The deliverable is the report — findings, numbers, caveats, quality appendix, next questions — not the notebook. ✓ Repeat on a real public dataset this week; three such projects are a portfolio. 🎓 Data Analysis domain complete. You can load, clean, transform, and interrogate tabular data like a professional. Next up: the SQL domain — the same analytical instincts, applied where most of the world's data actually lives.
Run your code to see the output here.