data_dir + '\\' + fname + '.csv' works on your Windows laptop and dies on the Linux server. A hardcoded '/Users/ada/data/' dies on everyone else's machine. Path bugs are the most preventable failures in data work — and Python solved them in one import: pathlib.
What you'll learn
What
pathlib.Path represents filesystem paths as OBJECTS: join with /, inspect with .name/.stem/.suffix, test with .exists(), list directories with .iterdir(), find files with .glob('*.csv'), and create output folders with .mkdir(). One API, every operating system.
Why
Every dataset lives at a path, every output needs a home, and every batch job walks directories. Path-as-string code accumulates os.path.join calls, separator bugs, and extension surgery via slicing; Path objects make the same operations readable, portable, and composable — and glob turns 'process every CSV in this folder' into one line.
Where it's used
Locating input data, building output paths, batch-processing folders of files, project-relative configs, and every 'works on my machine' path incident you'll now prevent.
Where this runs in production
Competition kernels begin with Path('/kaggle/input') and glob for the CSVs — the first cell of thousands of winning notebooks is literally this lesson.
Data-version-control tools track datasets as content-addressed paths; pipeline stages declare input/output paths. Reproducible ML is disciplined path handling.
Billions of uploads land in sharded directory trees (ab/cd/abcd1234.jpg) built by path composition — the mkdir(parents=True) pattern at planetary scale.
from pathlib import Path
p = Path('data') / 'raw' / 'sales_2026.csv'
print(p.name) # sales_2026.csv (final component)
print(p.stem) # sales_2026 (name minus extension)
print(p.suffix) # .csv (the extension)
print(p.parent) # data/raw (containing directory)The four accessors that replace all filename slicing: no more fname[:-4] to drop an extension or split('/')[-1] to get a name. p.with_suffix('.parquet') swaps extensions; p.parent / 'clean' / p.name relocates a file to a sibling folder — path algebra instead of string surgery.
A string path is handwritten directions in local dialect: 'C:\\data\\raw' means something on Windows and gibberish on Linux. A Path object is a structured address card — country, city, street in labeled fields. The postal service (each OS) renders it in its own format, you query fields directly (.name is the addressee, .parent the street), and composing a new address is filling fields, not editing a sentence with scissors. Same information, but one is DATA and the other is prose you keep re-parsing.
p.exists() (anything there?), p.is_file(), p.is_dir() — cheap questions that make absence a DECISION instead of a crash. Policy mirrors the exceptions lesson: for a REQUIRED input, skip the check and let open() raise FileNotFoundError loudly (or check and raise your own, richer error). For OPTIONAL files, check-and-default reads clearly. Remember the race caveat from EAFP: exists() then open() can still lose to a deleting process, so the open stays ready to fail.
from pathlib import Path
raw = Path('data') / 'raw'
for p in sorted(raw.glob('*.csv')):
print(p.name)
# subfolders too: raw.glob('**/*.csv') or raw.rglob('*.csv')glob is the batch-work engine: '*.csv' matches every CSV in the folder, '**/' recurses. It returns Path objects (not strings!) ready to open. One habit worth gold: sorted(...) — glob's order is OS-dependent, and unsorted iteration is the classic 'results differ between runs' mystery.
out_dir.mkdir(parents=True, exist_ok=True) — the incantation to memorize: parents=True builds intermediate folders ('reports/2026/07' in one call), exist_ok=True makes reruns idempotent instead of raising FileExistsError. Every batch job's first act: compose the output dir, mkdir it, then write into it. Without this, jobs die on fresh machines where the folder tree doesn't exist yet.
1) Never hardcode absolute personal paths ('/Users/ada/...') — compose from a base or accept a parameter. 2) Output dir ≠ input dir: write to a separate folder so a bug can't clobber sources (the mode-'w' lesson, at directory scale). 3) Filter what glob returns — '*.csv' can catch '.csv.bak' style strays if you glob loosely ('*'); check .suffix when it matters. 4) Case sensitivity differs (Linux: Sales.CSV ≠ sales.csv) — normalize names on write. 5) Paths in configs and logs: str(p) converts explicitly when a string is truly required.
Path('data')/'raw'/'sales_2026.csv' — every part has an accessor. Click each piece.
Build a path portably and read its parts.
The / operator joins; the accessors dissect. No separator characters appear anywhere in the source — this exact code runs identically on Windows, Mac, and Linux.
notes.txt notes .txt projects/dsm
Your task
Organize a mixed drop folder: create shoot/ with three files (two .jpg, one .txt) via write_text; glob the .jpg files (sorted); for each, compose a destination in organized/ named '<stem>_web<suffix>' and write the source's content there (simulating a copy); create organized/ safely first. Print each 'name -> dest' mapping and finish with the sorted names in organized/.
beach.jpg -> organized/beach_web.jpg sunset.jpg -> organized/sunset_web.jpg ['beach_web.jpg', 'sunset_web.jpg']
Write your solution in the editor on the right, then hit Run.
What is Path('data') / 'raw' / 'x.csv'?
For p = Path('logs/app_2026.log'), what is p.stem?
Why call out_dir.mkdir(parents=True, exist_ok=True) before writing outputs?
What discipline was violated?
Given paths = [Path('a/report.csv'), Path('b/data.parquet'), Path('c/notes.csv')], print the .name of each path whose .suffix is '.csv'. Expected: report.csv notes.csv
A nightly job processes sorted(inbox.glob('*.csv')). Why the sorted()?
Create staging/ containing x.csv ('1\n2\n') and y.csv ('3\n') via write_text. Glob it (sorted), sum all numbers across files (per-line int() with try/except counting skips), then write 'total=<n>' to results/summary.txt — creating results/ safely — and print that file's content. Expected: total=6
Why prefer pathlib over string paths and os.path? What actually goes wrong with strings?
Strings make paths PROSE the program keeps re-parsing; pathlib makes them structured data. The concrete failures strings invite: hardcoded separators ('data\\raw' breaks on Linux, 'data/raw' half-works on Windows until some API disagrees); extension surgery via slicing (fname[:-4] silently corrupts '.jsonl' or 'archive.tar.gz'); filename extraction via split('/') that misses backslashes; and doubled or missing separators from naive concatenation. os.path fixes portability but yields awkward nested calls — os.path.join(os.path.dirname(p), name) — where pathlib's object algebra reads as intent: p.parent / name, p.with_suffix('.parquet'), p.stem. Beyond ergonomics, Path unifies the whole file workflow — the same object composes (/), inspects (.name/.suffix), questions the filesystem (.exists/.is_file), discovers (.glob), prepares (.mkdir), and opens (open(p) or p.read_text) — so path handling becomes one coherent API instead of string tricks plus three modules. It's also a nice dunder story: / working on paths is just __truediv__, the protocol lesson applied by the standard library.
Design the file layout and path handling for a daily batch job. What habits prevent incidents?
Layout: a base root with separate trees per role — inbox/ (or raw/) for inputs, treated as strictly read-only; processed or reports organized by run date (reports/2026/07/16/) so history accumulates and any run can be audited or replayed; optionally an archive/ where consumed inputs MOVE (never in-place edits). Habits: compose every path from a configurable base (never hardcode personal absolute paths — accept a parameter or anchor on Path(__file__).parent); start each run with out_dir.mkdir(parents=True, exist_ok=True) so fresh machines and reruns behave identically; discover inputs with sorted(glob('*.csv')) — the sort buys determinism across filesystems; derive output names from input names via .stem/.suffix algebra rather than string slicing; and keep the input tree untouchable so the job is idempotent — a rerun after a mid-job crash must be safe, which it is exactly when inputs are read-only and outputs land in a dated, separate tree. Failure policy ties in: required inputs missing → raise with the full path in the message; optional ones → explicit defaults. Most 'the batch job destroyed the data' stories violate exactly one of these: outputs written into the input tree.
When do exists() checks make sense versus letting open() fail — and what's the race condition caveat?
The split follows the EAFP/LBYL reasoning from exception handling. exists()/is_file() reads well when absence is a NORMAL, expected state with a defined alternative: optional config → default values; a marker file controlling behavior; choosing between candidate locations. Letting open() raise (or checking and raising your own richer error) is right when the file is REQUIRED: a missing mandatory input is a deployment or upstream failure that should stop the job immediately, ideally with the absolute path in the message so the fix is obvious. The caveat: check-then-open is a TOCTOU (time-of-check-to-time-of-use) race — the file can vanish, appear, or change between exists() and open(), because the filesystem is shared mutable state; so an exists() check never removes the need for open() to survive failure, it only makes the COMMON absence case read cleanly. For the drop-folder pattern specifically (another process still writing files as you glob), robust jobs handle partially-written files too — by convention (writers use temp names then rename atomically) rather than by checks, since no amount of pre-checking closes the race. The mental model to state: filesystem checks are optimizations for readability and expected cases, never guarantees.
Common Mistakes to Avoid
1) Separator characters in source ('data\\raw', 'a/b' concatenation) — compose with /. 2) Hardcoded absolute personal paths — anchor on a base or parameter. 3) Writing outputs into the input tree — one bug from unrecoverable; separate trees. 4) Unsorted glob — OS-dependent order, irreproducible runs. 5) mkdir without parents=True/exist_ok=True — dies on fresh machines or reruns. 6) Extension surgery by slicing — .stem/.suffix/.with_suffix exist. 7) read_text on huge files — the streaming with-loop still owns big data. 8) exists() as a guarantee — the open must still handle failure.
Ask the AI Tutor
Try these prompts in the AI Tutor panel: • 'Quiz me: .name/.stem/.suffix/.parent for these eight paths.' • 'Refactor this os.path + string-slicing script to pathlib with me.' • 'Design a dated output layout for my scraper and critique my draft.' • 'Show the TOCTOU race between exists() and open().' • 'Interview mode: ask me the batch-job path-discipline question and grade my answer.'
Glossary
pathlib.Path — the OS-neutral path object. / (on paths) — the join operator (__truediv__). .name/.stem/.suffix/.parent — final component / name-sans-extension / extension / directory. .with_suffix() — swap extensions safely. .exists()/.is_file()/.is_dir() — filesystem questions. .glob(pattern)/.rglob — discover matching files ('**/' recurses). .iterdir() — list a directory. .mkdir(parents=True, exist_ok=True) — idempotent directory creation. .read_text()/.write_text() — small-file one-liners. Path.cwd() — the (variable!) working directory. Idempotent — safe to rerun. TOCTOU — the check-then-use race window.
Recommended Resources
• Docs: the pathlib module page — skim the 'Basic use' section and the table mapping os.path functions to Path methods. • Read: any Kaggle kernel's first cell — real Path/glob usage in the wild. • Practice: point a glob loop at your own Downloads folder and report file counts by .suffix (read-only — no writes!). • Next in DSM: Errors & File I/O complete — the Python for Data Science module begins with Package Management: pip, virtual environments, and requirements.txt — how the ecosystem's tools reach your machine.
Recap
✓ Path objects compose with / and render per-OS — separators never appear in source. ✓ .name/.stem/.suffix/.parent replace all filename string surgery. ✓ Required files fail loudly with the path in the message; optional ones default explicitly. ✓ sorted(dir.glob('*.csv')) = deterministic batch discovery; '**/' recurses. ✓ out.mkdir(parents=True, exist_ok=True) before writing; outputs NEVER share the input tree. ✓ read_text/write_text for small files; the streaming with-loop for everything big. Next up: Package Management. The Error Handling module is complete — now the bridge to the data stack: pip, virtual environments, requirements.txt, and how NumPy and pandas actually arrive on your machine.
Run your code to see the output here.