Ice cream sales correlate beautifully with drowning deaths. Nobody drowns from sundaes — summer causes both. Bivariate analysis is where EDA gets powerful ('orders differ by city!') and dangerous ('X causes Y!') at the same time. The techniques take an afternoon to learn; knowing what they can't tell you is the actual skill.
What you'll learn
What
Bivariate analysis examines two variables together, and the type pair picks the tool: numeric×numeric → correlation and scatter thinking; numeric×categorical → group comparisons (the groupby you know, now with EDA judgement); categorical×categorical → cross-tabs with pd.crosstab. Plus the two great caveats: correlation ≠ causation, and Pearson only sees straight lines.
Why
Almost every actionable finding is bivariate in shape: 'churn differs by plan', 'spend rises with tenure', 'conversion depends on channel'. It's also where analyses most often go wrong — spurious correlations, means compared across skew, percentages computed against the wrong denominator. Master the tools AND the traps.
Where it's used
Feature-target exploration before modelling, A/B readouts, segment comparisons in every business review, correlation screens across metrics, and the 'what moves with what' phase of any investigation.
Where this runs in production
Analysts correlate dozens of engagement metrics with retention to shortlist candidates for causal testing — the correlation screen finds candidates; A/B tests earn the causal claims.
Epidemiology starts with cross-tabs — condition by exposure group, normalized by row — before any causal modelling; the humble crosstab is medicine's first analytical tool.
People who see more ads buy more — but heavy users see more ads AND buy more anyway. Meta runs holdout experiments precisely because the raw ad-exposure/purchase correlation overstates causation.
Numeric × numeric → correlation (df['a'].corr(df['b'])), scatter-plot thinking. Numeric × categorical → group comparison (groupby(cat)[num].describe()) — 'does the distribution of X differ by group?'. Categorical × categorical → contingency table (pd.crosstab) with normalized shares. Three pairings, three tools; misapplying one to another's job is the first class of bivariate error.
df['tenure'].corr(df['spend']) # Pearson r: -1 .. +1 df[['a','b','c']].corr().round(2) # correlation matrix (screening) df['a'].corr(df['b'], method='spearman') # rank-based: monotonic, robust
Pearson r measures LINEAR co-movement: +1 perfect line up, -1 down, 0 no linear trend. Spearman correlates the RANKS instead — it catches any monotonic relationship (curved but always-increasing) and shrugs off outliers. When Pearson and Spearman disagree sharply, the data is telling you 'nonlinear or outlier-driven — look closer'.
1) Nonlinearity: a perfect U-shaped relationship (spend vs age) can score r ≈ 0 — 'no correlation' while the relationship is total. 2) Outliers: one extreme point can manufacture r = 0.9 from noise, or bury a real relationship. 3) Anscombe's lesson: four datasets with identical r can look wildly different. Consequence: never interpret a correlation you haven't visualised (or at least Spearman-checked and outlier-checked).
# Numeric across groups: distributions, not just means
df.groupby('plan')['spend'].agg(['median', 'mean', 'count'])
# medians for skew-honesty, counts because a median of 3 rows is a rumourGroup comparison is univariate analysis per group: the same centre/spread/shape discipline, now side by side. Report medians when the variable is skewed, and ALWAYS carry group sizes — differences between tiny groups are noise wearing a costume.
pd.crosstab(df['plan'], df['churned']) # counts pd.crosstab(df['plan'], df['churned'], normalize='index') # row %: churn rate per plan pd.crosstab(df['plan'], df['churned'], normalize='columns') # col %: plan mix among churners
The normalize direction IS the question. normalize='index' answers 'what share of EACH PLAN churned?' (compare rates across plans); normalize='columns' answers 'what do churners consist of?'. Choosing the wrong denominator produces a true number that answers a different question — the sneakiest bivariate error.
A correlation between X and Y admits five explanations: X→Y, Y→X (reverse), Z→both (confounding: summer → ice cream AND drownings), selection effects (who ends up in the data), and chance (especially after screening many pairs). EDA's job is to FIND the relationship and list plausible explanations; experiments and causal methods (later domains) get to pick one. In writing: 'is associated with', never 'drives', until earned.
Correlation says two dancers move together; it doesn't say who leads, whether both follow the music (a third factor), or whether you happened to watch the one song where they synced by chance. Watching more carefully (visualisation), checking different songs (segments, later data), and finally asking them to dance without music (an experiment) — that's the escalation from association to causation.
Click through: the type pair picks the tool; the caveats apply to all three.
Do longer-tenured customers spend more?
Six customers, tenure in months against monthly spend.
Your task
An app dataset: correlate sessions with purchases, compare purchases across device types with the honest statistic, and build the conversion crosstab with the right denominator.
0.99
{'android': 1.0, 'ios': 5.0}
{'android': 0.33, 'ios': 0.8}Write your solution in the editor on the right, then hit Run.
Which tool fits a numeric × categorical pair, like spend across plan types?
A Pearson r of 0.02 between age and spend means:
Pearson gives 0.95 but Spearman gives 0.20 for the same pair. Most likely explanation?
What's the error?
Print two correlations between x and y, rounded to 2 decimals: Pearson and Spearman. df = pd.DataFrame({'x':[1, 2, 3, 4, 5, 50], 'y':[2, 1, 4, 3, 5, 60]})
Users who enable feature X retain 2× better. Why can't EDA alone conclude 'X improves retention'?
How do you analyse the relationship between two variables? Walk through the cases.
The type pair routes the analysis. Two numerics: correlation — Pearson for linear strength, Spearman as the robustness check — but never interpreted blind, because Pearson misses curves and gets manufactured by outliers; a big Pearson-Spearman gap sends me to a scatter plot. Numeric across categories: a group comparison — groupby with median, mean, AND count, applying the univariate discipline per group: skewed variable means medians, and small groups mean the difference is a rumour until sized. Two categoricals: pd.crosstab, where the entire skill is choosing normalize to match the question — normalize='index' compares rates across groups, normalize='columns' describes composition, and quoting one as the other is the classic percentage error. In all three cases the finding is stated as association ('churn differs by plan: 50% vs 25%'), with the causal question explicitly left open — EDA nominates relationships; experiments elect them.
Give me concrete reasons a strong correlation might NOT mean causation, with examples.
Five distinct mechanisms. Reverse causation: sales and ad spend correlate — but many companies set ad budgets as a percentage of sales, so sales drive spend. Confounding: ice cream sales and drownings — summer drives both; in product data, user engagement confounds almost everything (engaged users adopt features AND retain AND spend). Selection effects: hospitals with the best surgeons show worse mortality because they take the hardest cases — who lands in the data isn't random. Chance under multiple comparisons: screen 190 metric pairs and several will correlate impressively by luck; spurious-correlations galleries (Nicolas Cage films vs pool drownings) exist because time series with trends correlate by default. And measurement artifacts: two metrics computed from the same underlying field correlate mechanically. The discipline: for any striking correlation, write down which of the five could apply before presenting — and phrase as 'associated with' until a design (randomization, natural experiment) rules the alternatives out.
You're comparing a metric across customer segments. What could make the comparison misleading, and how do you guard it?
Four standard hazards. Skew and outliers: one whale in a segment inflates its mean — I compare medians (or trimmed means) and look at spread per segment, not just centres. Sample size asymmetry: a 15-customer segment 'outperforming' by 20% is likely noise — I always show n per group and treat small-group differences as hypotheses; formally, that's a significance question for the stats domain. Denominator confusion: 'segment A is 60% of complainers' versus 'segment A complains at 12%' answer different questions — I fix the question first, then pick the normalization. And composition/mix effects, the gateway to next lesson: segment A can beat B overall while losing within every subgroup, if the segments' internal mixes differ (Simpson's paradox) — so for any decision-bearing comparison I check whether the result survives splitting by the obvious third variables (region, tier, tenure). Guarding all four turns 'A is better than B' from a groupby output into a defensible sentence.
Common Mistakes to Avoid
1) Interpreting a correlation you never visualised — outliers and curves hide in bare r values. 2) Reading r ≈ 0 as 'no relationship' when it only means 'no LINEAR relationship'. 3) Comparing group means on skewed data — one whale rewrites the ranking; use medians. 4) Presenting differences between tiny groups without their sizes. 5) The denominator swindle: quoting composition ('80% of churners are monthly') as risk ('monthly users churn more'). 6) Mining a 20-column correlation matrix and reporting the winners as findings — screening inflates false positives. 7) Causal verbs ('drives', 'boosts') on observational associations. 8) Forgetting corr() pairwise-drops NaNs — cells may describe different subsets.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'ELI5: dance partners vs puppet strings.' • 'Build me a dataset where Pearson and Spearman disagree, and explain why.' • 'Quiz me: which tool for each variable pair you invent?' • 'Give me five correlations; I'll name the likeliest non-causal explanation for each.' • 'Interview mode: I present a segment comparison, you attack its denominators and sample sizes.'
Glossary
Bivariate — two variables analysed together. Pearson r — linear correlation, -1 to +1, via .corr(). Spearman — rank-based correlation; robust, catches monotonic patterns. Correlation matrix — all pairwise correlations (df.corr()); a screening tool. Group comparison — a numeric variable's distribution across category groups. Contingency table / crosstab — counts for category combinations (pd.crosstab). normalize='index'/'columns' — row-wise rates vs column-wise composition. Confounder — a third variable driving both members of a pair. Selection effect — non-random inclusion distorting a comparison. Association — co-occurrence; the honest word until causation is earned.
Recommended Resources
• Docs: pandas corr and crosstab references. • Look up: Anscombe's quartet (identical r, wildly different data) and the 'Spurious Correlations' gallery — ten minutes that permanently immunise. • Read: a primer on confounding and Simpson's paradox before the next lesson. • Practice: on any dataset, find your three strongest correlations, then try to BREAK each one: outlier check, Spearman check, segment split, and the five-explanations list. • Next in DSM: Multivariate Analysis — what happens to your bivariate findings when a third variable enters the room.
Recap
✓ The type pair picks the tool: correlation (num×num), group comparison (num×cat), crosstab (cat×cat). ✓ Pearson sees only straight lines and bends to outliers; Spearman is the rank-based cross-check, and a gap between them is an alarm. ✓ Group comparisons inherit univariate discipline: medians under skew, group sizes always attached. ✓ In crosstabs the normalize direction IS the question — rates (index) versus composition (columns). ✓ Correlation matrices generate shortlists, not findings — screening breeds chance 'discoveries'. ✓ Every association has five candidate explanations; EDA lists them, experiments choose. Write 'associated with'. Next up: Multivariate Analysis. Your bivariate finding looks solid — until a third variable flips it. Segmentation, interaction effects, and Simpson's paradox await.
Run your code to see the output here.