DSM
40 XP
20 minsBeginner40 XP

Package Management (pip, venv, conda)

ModuleNotFoundError: No module named 'pandas'. Every data scientist has hit this wall — and half the fixes people try (reinstalling Python, sudo pip, copying files around) make it worse. The actual fix is a mental model: where packages live, who installs them, and why each project deserves its own sealed box.

What you'll learn

  • Explain what pip installs and where packages actually go
  • Create and activate a venv, and know when it's active
  • Freeze dependencies into requirements.txt and rebuild from it
  • Diagnose ModuleNotFoundError as an environment question, not a code bug
  • Know when conda is the better tool and when it isn't

What

pip is Python's installer — it downloads packages from PyPI (the Python Package Index) into an environment. A virtual environment (venv) is an isolated folder holding one project's Python + packages. requirements.txt is the manifest that lets anyone rebuild that environment. conda is an alternative installer + environment manager popular in data science.

Why

The data stack is third-party code: NumPy, pandas, matplotlib, scikit-learn — none ship with Python. Without isolation, projects fight over versions (project A needs pandas 1.x, project B needs 2.x) and 'works on my machine' becomes your job title. Environments + manifests turn setup from folklore into one command.

Where it's used

Starting any project, onboarding teammates, deploying jobs to servers, reproducing a colleague's analysis, and every Dockerfile and CI pipeline you'll ever read.

Where this runs in production

NetflixThousands of isolated notebook environments

Netflix's internal notebook platform provisions a dedicated dependency environment per project so a data scientist upgrading pandas can't break a colleague's recommendation model.

Anacondaconda built for the scientific stack

conda exists because early NumPy/SciPy builds needed compiled C and Fortran libraries pip couldn't ship; conda packages the binaries too, which is why it dominates in heavy scientific computing.

GitHubrequirements.txt as onboarding

Open any popular data science repo: setup is 'create a venv, pip install -r requirements.txt'. The manifest file IS the onboarding doc — new contributors are running in minutes.

Theory

The core ideas, in plain language.

Python's superpower is its ecosystem: PyPI hosts over half a million packages, and pip is the client that fetches them. pip install pandas downloads pandas (and everything pandas itself needs — NumPy, dateutil, pytz) into your environment's site-packages folder. From then on, import pandas finds it. The key question behind every install problem: WHICH environment did it land in?
# Terminal, not Python — these are shell commands
python -m venv .venv          # create an isolated environment in .venv/
source .venv/bin/activate     # activate it (Mac/Linux)
# .venv\Scripts\activate      # activate it (Windows)

pip install pandas numpy      # installs INTO .venv only
pip list                      # see what's installed here

python -m venv .venv creates a self-contained folder with its own Python and its own site-packages. Activating it rewires your shell so python and pip point inside that folder. The (.venv) prefix in your prompt is the tell that it's active. Deactivate with the deactivate command.

Analogy: A toolbox per job site

Installing everything globally is one communal toolbox for every job site: the plumber swaps in a metric wrench set and the electrician's imperial project breaks. A venv is a per-site toolbox: each project gets exactly the tools (and tool VERSIONS) it needs, sealed in its own box. requirements.txt is the packing list taped to the lid — hand it to anyone and they can assemble an identical box. Nobody's job breaks because of someone else's tools.

Key Concept
requirements.txt: the reproducibility contract

pip freeze > requirements.txt writes every installed package with its exact version (pandas==2.2.1). pip install -r requirements.txt rebuilds that set anywhere — teammate's laptop, CI server, production. This one file is the difference between 'clone and run' and a week of version archaeology. Commit it to git; regenerate it when you add or upgrade packages.

# requirements.txt — exact pins from pip freeze
numpy==1.26.4
pandas==2.2.1
python-dateutil==2.9.0
pytz==2024.1

# rebuild anywhere:
# pip install -r requirements.txt

Note that pandas' own dependencies appear too — freeze captures the whole tree. The == pins mean everyone gets identical versions. Looser specs exist (pandas>=2.0) and trade reproducibility for flexibility; for analyses you want to reproduce, exact pins win.

Key Concept
Diagnosing ModuleNotFoundError

It is almost never 'Python is broken.' The import failed because the RUNNING interpreter's environment lacks the package. Check three things: 1) is your venv active (prompt prefix)? 2) does pip list show the package HERE? 3) is the script running with the same interpreter you installed into (an IDE pointed at the wrong environment is the classic cause)? python -m pip install ... sidesteps a whole class of mismatch by guaranteeing pip belongs to the python you're invoking.

Watch out
Habits that prevent environment pain

1) One venv per project, created in the project folder as .venv (and .gitignore it — commit requirements.txt, never the environment itself). 2) Never sudo pip install — installing into the system Python can break OS tools. 3) Never install into the global Python 'just this once' — that's how version conflicts start. 4) After adding a package, refresh requirements.txt in the same commit as the code that needs it. 5) When an environment gets confused, deleting .venv and rebuilding from requirements.txt is cheap and safe — that's the point of the manifest.

Visual Learning

See the concept, then explore it.

From PyPI to import pandas

The supply chain of a package: index → installer → environment → your script. Click each stage.

Worked Examples

Watch it built up, one line at a time.

Very EasyWhich Python am I running?

The first diagnostic for any environment confusion — ask Python where it lives and what it can import.

Step 1 of 1

sys.executable is the exact interpreter path — if it points inside .venv, your venv is live; if it points at a system path, your install and your run are probably different environments. This one print solves most 'but I installed it!' mysteries.

Code
01import sys
02 
03print(sys.executable)
04print(sys.version.split()[0])
Output
/home/ada/project/.venv/bin/python
3.11.9
Practice Coding

Your turn — write the code.

Your task

Build a tiny environment doctor: given required = {'numpy': '1.26.4', 'pandas': '2.2.1', 'matplotlib': '3.8.4'} and installed = {'numpy': '1.26.4', 'pandas': '2.0.0'}, report each requirement as OK (versions match), WRONG VERSION (present but different), or MISSING. Finish by printing the pip command to fix the gaps: pip install followed by each non-OK 'name==version' pin, space-separated, in the dict's order.

Expected output
numpy: OK
pandas: WRONG VERSION (have 2.0.0, need 2.2.1)
matplotlib: MISSING
fix: pip install pandas==2.2.1 matplotlib==3.8.4

Write your solution in the editor on the right, then hit Run.

Exercises

Prove it. Reach 80% to complete the lesson.

Mastery Gate0% / 80% required
Easy0/2 solved

What does pip install pandas actually do?

Why create a virtual environment per project?

Medium0/3 solved

You ran pip install pandas successfully, but your script still raises ModuleNotFoundError. Most likely cause?

ScenarioA teammate clones your analysis repo, creates a venv, and asks 'now what do I install?' Your repo contains only .py files — you've been installing packages as needed and never wrote them down.

What was missing from your workflow?

Given manifest = 'numpy==1.26.4\npandas==2.2.1\npytz==2024.1', parse each line and print '<name> pinned to <version>'. Expected: numpy pinned to 1.26.4 pandas pinned to 2.2.1 pytz pinned to 2024.1

Line iteration:Splits the manifest on newlines and processes each line
Pin parsing:Splits each line on '==' into name and version
Hard0/2 solved

When is conda the better choice over venv + pip?

Simulate an environment check: required = ['numpy', 'pandas', 'requests'], installed = {'numpy', 'pandas'}. Print '<name>: ok' or '<name>: MISSING' for each requirement in order, then 'environment ready' if nothing is missing or 'missing <n> package(s)' otherwise. Expected: numpy: ok pandas: ok requests: MISSING missing 1 package(s)

Set membership:Uses 'in' against the installed set per requirement
Summary logic:Counts misses and branches the final line on the count
Complete 80% more exercises to unlock.
Interview Prep

How this shows up in real interviews.

Walk me through how you set up the Python environment for a new data project, and why each step exists.

Show model answer

Create the project folder, then python -m venv .venv inside it — an isolated interpreter + site-packages so this project's versions can't collide with any other's. Activate it (source .venv/bin/activate, or Scripts\activate on Windows) and confirm via the prompt prefix or sys.executable. Install with python -m pip install pandas numpy — the -m form guarantees pip belongs to the interpreter I'm using. Immediately pip freeze > requirements.txt and commit it alongside the code; add .venv/ to .gitignore because the environment is a rebuildable artifact, not source. From then on the loop is: add a package → refresh the manifest in the same commit. The payoff shows at handoff: any teammate, CI runner, or server rebuilds my exact environment with python -m venv .venv && pip install -r requirements.txt. If the environment ever gets into a weird state, I delete .venv and rebuild from the manifest — two minutes, zero risk, which is precisely the property the manifest exists to provide.

A script that worked last month now crashes with an AttributeError inside pandas. Nothing in the script changed. What's your diagnosis path?

Show model answer

Unchanged code + new failure points at a changed ENVIRONMENT. First: was this running in a pinned venv, or against a global/shared Python someone may have upgraded? Check pandas.__version__ (and sys.executable to confirm which environment is even running) against what the code was written for — an AttributeError inside a library is the classic symptom of an API that moved between major versions, e.g. pandas 1.x methods removed in 2.x. If there's a requirements.txt, diff pip freeze against it: any drift means the environment no longer matches the contract, and pip install -r requirements.txt (or rebuilding the venv from scratch) restores it. If there ISN'T a manifest, that's the root cause — the fix is to pin the working versions once found, so this class of failure becomes impossible. The general principle I'd state: 'nothing changed' almost never includes the environment, so version-drift is hypothesis #1 whenever untouched code breaks — and pinned, per-project environments are the prevention, not better debugging.

What problem do virtual environments actually solve, and what happens on teams that skip them?

Show model answer

They solve shared mutable state at the dependency level. One global site-packages means every project reads and writes the same package set: project B upgrading pandas to 2.x silently breaks project A written against 1.x, installing a new tool pulls transitive dependencies that conflict with existing ones, and on some systems sudo pip installs can break OS tooling that depends on the system Python. Teams that skip isolation accumulate the symptoms: 'works on my machine' disputes (each laptop has a unique, undocumented package soup), onboarding measured in days of dependency archaeology, un-reproducible analyses (nobody knows which versions produced last quarter's numbers), and fear-driven ops — nobody dares upgrade anything because the blast radius is unknown. Environments make dependencies per-project and DISPOSABLE (delete and rebuild from the manifest), and the manifest makes them reproducible across machines and time. The same idea scales up the stack: Docker images and lockfile tools (Poetry, uv) are the industrial-strength versions of venv + requirements.txt — same principle, stronger guarantees.

Common Mistakes to Avoid

1) Installing into whatever Python is ambient — always check the venv is active first. 2) sudo pip install — can break system tools; never needed with venvs. 3) Committing .venv/ to git — commit requirements.txt instead. 4) Installing packages and never freezing — the manifest drifts from reality. 5) Mixing conda install and pip install in one environment carelessly — pick a primary installer per project. 6) 'Fixing' ModuleNotFoundError by reinstalling Python — it's an environment mismatch, not a broken interpreter. 7) One giant shared venv for all projects — that's global installs with extra steps.

Ask the AI Tutor

Try these prompts in the AI Tutor panel: • 'Quiz me on the venv setup commands until I can type them cold.' • 'My import works in the terminal but not in my IDE — walk me through the diagnosis.' • 'Explain what pip freeze captures that reading my imports wouldn't.' • 'When would you pick conda over venv? Give me three concrete cases.' • 'Interview mode: ask me the unchanged-code-now-crashes question and grade my answer.'

Glossary

pip — Python's package installer. PyPI — the public package index pip downloads from. site-packages — the folder inside an environment where installed packages live. venv — a per-project isolated environment (python -m venv .venv). activate — rewire the shell to use a venv's python/pip. pip freeze — list installed packages with exact versions. requirements.txt — the committed manifest that rebuilds an environment. pin (==) — an exact version requirement. conda — alternative installer/environment manager that also ships non-Python binaries. wheel — pip's prebuilt package format. sys.executable — the running interpreter's path, the #1 diagnostic.

Recommended Resources

• Docs: the Python Packaging User Guide's 'Installing packages' tutorial — the official walkthrough of venv + pip. • Read: pip's 'Requirements files' documentation page — what -r actually supports. • Practice: create a throwaway venv, install two packages, freeze, delete the venv, and rebuild it from the manifest — the full lifecycle in ten minutes. • Next in DSM: with the stack installable, NumPy Operations goes deep on the library that powers all of it — ufuncs, axes, reshaping, and broadcasting.

Recap

✓ pip installs from PyPI into ONE environment — whichever is active. ✓ One venv per project: python -m venv .venv, activate, confirm via prompt or sys.executable. ✓ pip freeze > requirements.txt after every install; commit the manifest, .gitignore the venv. ✓ pip install -r requirements.txt rebuilds the environment anywhere — environments are disposable, manifests are precious. ✓ ModuleNotFoundError = environment mismatch: check active venv, pip list, sys.executable. ✓ conda earns its place when compiled non-Python dependencies get heavy; venv + pip is the default. Next up: NumPy Operations. The stack is installed — now master its foundation: ufuncs, aggregations along axes, reshape, and the broadcasting rules behind every vectorized pipeline.

scratchpad — preview this lesson's challenge anytime
env_doctor.pyPython
Ready
Output

Run your code to see the output here.