The Center of Your Python Project: Understanding pyproject.toml

If you’ve started a new Python project recently or looked at the structure of modern libraries, you’ve undoubtedly encountered a file named pyproject.toml sitting in the project root. What is this file, and why has it become so important? Gone are the days of juggling setup.py, setup.cfg, requirements.txt, MANIFEST.in, and separate configuration files for every single tool (.isort.cfg, .flake8, .coveragerc, etc.). pyproject.toml aims to bring order to this configuration chaos.

From setup.py Scripts to Declarative Config

Historically, Python packaging relied heavily on setup.py, a Python script containing calls to functions from setuptools (or its predecessor distutils). While flexible (it’s arbitrary Python code, after all), this approach had downsides:

• Build Dependencies: Running setup.py required its dependencies (like setuptools itself, or maybe numpy for C extensions) to be installed before you could even figure out what those dependencies were. This created complex bootstrapping problems.
• Lack of Standardization: Different tools evolved different configuration files, scattering project settings across the repository.
• Security Risks: Running arbitrary code from setup.py during package installation posed potential security risks.
• Imperative vs. Declarative: setup.py described how to build the package, rather than declaratively stating the package’s metadata and build requirements.

Enter pyproject.toml: Standardization Strikes Back

Several Python Enhancement Proposals (PEPs) paved the way for pyproject.toml:

• PEP 518: Introduced pyproject.toml primarily to specify build system requirements. It allows a project to declare what packages are needed to build itself (like setuptools or wheel) before the build process even starts. pip can read this, install those build dependencies in an isolated environment, and then proceed.
• PEP 517: Defined a standard way to invoke build backends. Instead of relying on setup.py’s specific commands, PEP 517 allows projects to specify which backend (e.g., setuptools, flit-core, hatchling, poetry-core) should build the package. The build frontend (like pip) communicates with the backend via a defined interface.
• PEP 621: Standardized the project metadata itself (name, version, dependencies, etc.) within pyproject.toml, moving away from backend-specific files like setup.cfg.
• PEP 660: Standardized how to perform editable installs for PEP 517 builds.

Together, these PEPs allow pyproject.toml to serve as the central, declarative configuration file for building Python packages and configuring development tools.

Anatomy of pyproject.toml

This file uses the TOML (Tom’s Obvious, Minimal Language) format, which is designed to be easy to read. It’s structured using tables (sections in [square_brackets]):

1. [build-system] (PEP 518): Declaring the Build Process

This table is crucial for pip and other build tools. It tells them what they need to build your project.

[build-system]
# Minimum build tools needed to build the package
requires = ["setuptools>=61.0", "wheel"]
# The build backend entry point
build-backend = "setuptools.build_meta"
# Specify backend-specific behavior (optional)
backend-path = ["."]

• requires: A list of packages needed to run the build backend (e.g., setuptools and wheel are common for Setuptools-based builds). pip installs these into an isolated environment before building.
• build-backend: The Python object path (dotted notation) to the backend implementation that follows the PEP 517 interface. Common backends include:
  • setuptools.build_meta: For standard Setuptools builds.
  • hatchling.build: For the Hatch build backend.
  • flit_core.buildapi: For the Flit build backend.
  • poetry.core.masonry.api: For the Poetry build backend.

2. [project] (PEP 621): Core Project Metadata

This table holds the standardized metadata about your library.

[project]
name = "my-cool-library"
version = "1.2.3"
description = "A short description of my library."
readme = "README.md"
requires-python = ">=3.8"
license = { file = "LICENSE" }
authors = [
    { name = "Your Name", email = "you@example.com" },
]
classifiers = [
    "Development Status :: 4 - Beta",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
]

# Core dependencies
dependencies = [
    "httpx ~= 0.23",
    'importlib_metadata; python_version < "3.10"', # Conditional dep
]

# Optional dependencies (extras)
[project.optional-dependencies]
cli = ["click >= 8.0"]
dev = [
    "my-cool-library[cli]",
    "pytest",
    "ruff",
]

# URLs (Homepage, Repository, etc.)
[project.urls]
Homepage = "https://example.com/my-cool-library"
Repository = "https://github.com/you/my-cool-library"

# Console scripts
[project.scripts]
my-cli-tool = "my_cool_library.cli:main"

# GUI scripts (less common)
# [project.gui-scripts]
# my-gui-tool = "my_cool_library.gui:main"

# Entry points (for plugins etc.)
# [project.entry-points."my_library.plugins"]
# plugin_one = "my_cool_library.plugins:PluginOne"

Key fields include name, version, description, dependencies (core and optional), required Python version, license, author details, classifiers for PyPI, and entry points for scripts.

3. [tool.*] : Configuring Everything Else

This is where the magic of unification happens. Instead of countless dotfiles (.isort.cfg, .coveragerc, etc.), many modern tools now look for their configuration under the [tool] table:

# Example: Ruff configuration
[tool.ruff]
line-length = 88
select = ["E", "F", "W", "I", "U", "C4"]

# Example: Pytest configuration
[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
    "tests",
]

# Example: Coverage.py configuration
[tool.coverage.run]
source = ["my_cool_library"]
omit = ["*/__main__.py"]

# Example: MyPy configuration
[tool.mypy]
ignore_missing_imports = true

# Example: Build backend specific options (Hatch)
[tool.hatch.version]
path = "my_cool_library/__init__.py"

[tool.hatch.build.targets.wheel]
packages = ["my_cool_library"]

Tools like ruff, black, isort, pytest, coverage, mypy, tox, and build backends like hatch or poetry can all place their settings here, keeping your project root cleaner and configuration centralized.

Benefits Recap

• Unified Configuration: One file to rule (most of) them all.
• Build System Isolation: Reliable builds without polluting the global environment.
• Standardized Metadata: Clear, declarative project information (PEP 621).
• Tool Interoperability: Build backends and frontends work together seamlessly.
• Improved Security: Avoids running arbitrary setup.py code during installation.

pyproject.toml represents a major step forward for the Python packaging ecosystem. By embracing its declarative nature and consolidating configuration, library authors can create projects that are easier to build, maintain, and understand for both themselves and their users.