Managing Python Dependencies with Poetry

requirements.txt has served us well, but it shows its age. Poetry offers a modern alternative: dependency resolution, virtual environments, and packaging in one tool.

Why Poetry?

The Problems with requirements.txt

# requirements.txt
django>=3.0
requests
celery

# Which versions actually installed?
# How do we know they're compatible?
# Where's the lockfile?

What Poetry Provides

# pyproject.toml
[tool.poetry.dependencies]
python = "^3.9"
django = "^3.2"
requests = "^2.25"
celery = "^5.0"

# poetry.lock
# Exact versions locked
# Hashes for verification
# Dependency tree resolved

Getting Started

Installation

# Recommended: pipx
pipx install poetry

# Or curl
curl -sSL https://install.python-poetry.org | python3 -

New Project

poetry new myproject

Creates:

myproject/
├── pyproject.toml
├── README.md
├── myproject/
│   └── __init__.py
└── tests/
    └── __init__.py

Existing Project

cd existing_project
poetry init

Interactive prompts guide you through setup.

Core Commands

Adding Dependencies

# Add a dependency
poetry add django

# Add with version constraint
poetry add django@^3.2

# Add development dependency
poetry add --group dev pytest black flake8

# Add from git
poetry add git+https://github.com/user/repo.git

# Add optional dependency
poetry add --optional slow-library

Removing Dependencies

poetry remove requests
poetry remove --group dev pytest

Installing Dependencies

# Install all dependencies
poetry install

# Install without dev dependencies
poetry install --without dev

# Install specific groups only
poetry install --only main

Updating

# Update all packages
poetry update

# Update specific package
poetry update django

# Show outdated packages
poetry show --outdated

pyproject.toml

[tool.poetry]
name = "myproject"
version = "0.1.0"
description = "A sample project"
authors = ["Your Name <you@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9"
django = "^3.2"
djangorestframework = "^3.12"
celery = {version = "^5.0", optional = true}

[tool.poetry.group.dev.dependencies]
pytest = "^7.0"
pytest-django = "^4.5"
black = "^22.0"
flake8 = "^4.0"

[tool.poetry.extras]
async = ["celery"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Version Constraints

# Caret: Compatible releases
django = "^3.2"      # >=3.2.0 <4.0.0
requests = "^2.25.1" # >=2.25.1 <3.0.0

# Tilde: Patch-level changes only
django = "~3.2.0"    # >=3.2.0 <3.3.0

# Exact version
django = "3.2.0"     # Only 3.2.0

# Range
django = ">=3.0,<4.0"

# Wildcard
django = "3.2.*"     # Any 3.2.x

Virtual Environments

Poetry manages virtualenvs automatically:

# Run command in virtualenv
poetry run python manage.py runserver

# Activate virtualenv shell
poetry shell

# Show virtualenv info
poetry env info

# List virtualenvs
poetry env list

# Remove virtualenv
poetry env remove python3.9

Configuration

# Create virtualenv in project directory
poetry config virtualenvs.in-project true
# Creates .venv/ in project root

Lock File

poetry.lock pins exact versions:

# Generate/update lock file
poetry lock

# Install exactly what's in lock file
poetry install --no-update

Commit poetry.lock to version control for reproducibility.

Scripts

Define runnable scripts:

[tool.poetry.scripts]
myapp = "myproject.cli:main"
serve = "myproject.server:run"

poetry run myapp
poetry run serve

Integration with Django

Project Structure

myproject/
├── pyproject.toml
├── poetry.lock
├── manage.py
├── myproject/
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── apps/
    └── core/

Running Django

poetry run python manage.py runserver
poetry run python manage.py migrate

# Or activate shell
poetry shell
python manage.py runserver

Docker Integration

FROM python:3.9-slim

# Install poetry
RUN pip install poetry

# Copy dependency files
COPY pyproject.toml poetry.lock ./

# Install dependencies
RUN poetry config virtualenvs.create false \
    && poetry install --no-dev --no-interaction

# Copy application
COPY . .

CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

CI/CD

GitHub Actions

- name: Install Poetry
  uses: snok/install-poetry@v1
  with:
    version: 1.4.0
    virtualenvs-create: true
    virtualenvs-in-project: true

- name: Load cached venv
  uses: actions/cache@v2
  with:
    path: .venv
    key: venv-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
  run: poetry install --no-interaction

- name: Run tests
  run: poetry run pytest

Migration from requirements.txt

# Import existing requirements
cat requirements.txt | xargs poetry add

Or manually transfer:

# Old
pip install -r requirements.txt

# New
poetry install

Building and Publishing

# Build package
poetry build
# Creates dist/myproject-0.1.0.tar.gz and .whl

# Publish to PyPI
poetry publish

# Publish to private registry
poetry config repositories.private https://private.pypi.example.com
poetry publish -r private

Tips

Speed Up Resolution

# Skip hash checking for faster installs (dev only)
poetry install --no-cache

Export to requirements.txt

# For compatibility with tools that need requirements.txt
poetry export -f requirements.txt --output requirements.txt
poetry export --without-hashes -f requirements.txt > requirements.txt

Monorepo Support

# Reference local packages
[tool.poetry.dependencies]
shared-lib = {path = "../shared-lib", develop = true}

Final Thoughts

Poetry brings modern package management to Python. It solves real problems: dependency resolution, environment management, and reproducible builds.

Switch to Poetry for new projects. The learning curve is gentle, and the benefits are immediate.

Dependencies managed, sanity maintained.