Python Packaging — Trivia & Interesting Facts¶
Surprising, historical, and little-known facts about Python packaging.
Python packaging has been called the language's biggest weakness¶
The Python packaging ecosystem has been described by core developers themselves as confusing and fragmented. Between distutils, setuptools, easy_install, pip, pipenv, poetry, flit, hatch, pdm, and conda, Python has had more packaging tools than arguably any other mainstream language. PEP 517/518 (2017) was specifically designed to standardize the chaos.
PyPI has served over 800 billion package downloads¶
The Python Package Index (PyPI) hosts over 500,000 projects and serves billions of downloads per month. The infrastructure is maintained by a small team at the Python Software Foundation. PyPI's predecessor, the Cheese Shop (named after a Monty Python sketch), launched in 2003. The modern PyPI (Warehouse) replaced it in 2018 after years of development.
pip was created because easy_install could not uninstall packages¶
Ian Bicking created pip in 2008 because easy_install (part of setuptools) had no uninstall command. The name "pip" stands for "Pip Installs Packages" (a recursive acronym). pip became the recommended installer in 2013 and was bundled with Python starting in 3.4 (2014) via the ensurepip module.
The setup.py file is being replaced after 20 years¶
distutils' setup.py was the standard way to build Python packages since 2000. PEP 517 (2017) introduced pyproject.toml as a declarative replacement, and PEP 621 (2021) standardized the metadata format within it. setuptools still supports setup.py for backward compatibility, but new projects are strongly encouraged to use pyproject.toml exclusively.
Virtual environments solve a problem that did not exist in Python 1.x¶
The concept of isolated Python environments emerged because Python installs packages globally by default, and different projects often need conflicting versions. Ian Bicking's virtualenv (2007) was the first solution. Python 3.3 (2013) added the venv module to the standard library, making virtual environments an official feature. Before these tools, dependency conflicts were known as "dependency hell."
Wheels replaced eggs and made installation 10x faster¶
The egg format (introduced by setuptools in 2004) required running arbitrary setup.py code during installation. The wheel format (PEP 427, 2013) is a pre-built binary distribution that requires no code execution — just unpacking a ZIP file. This made pip installs up to 10x faster and eliminated the need for a compiler to install packages with C extensions on common platforms.
The dependency resolver in pip was broken for its first 12 years¶
pip did not have a proper dependency resolver until version 20.3 (November 2020). Before that, pip installed packages in the order it encountered them, potentially installing incompatible versions. The new resolver (funded by a $407,000 grant from Mozilla and the Chan Zuckerberg Initiative) uses backtracking to find a consistent set of versions.
conda is not a Python tool — it is a general-purpose package manager¶
Conda, created by Anaconda Inc. in 2012, manages packages for any language (Python, R, C++, Rust). It handles binary dependencies that pip cannot, like CUDA libraries, MKL, and system-level shared objects. Conda and pip can conflict because they maintain separate package databases and do not communicate with each other about what is installed.
The left-pad equivalent incident hit Python too¶
In 2022, several popular PyPI packages were found to contain malware uploaded via typosquatting (names like "colourama" instead of "colorama"). The Python ecosystem has since introduced Trusted Publishers (OIDC-based publishing from CI), mandatory 2FA for critical projects, and PEP 708 (provenance attestations). Supply chain security is now a top priority for PyPI.
Python packages can contain arbitrary code that runs at install time¶
When pip installs a source distribution (sdist) that uses setup.py, it executes arbitrary Python code as your user. This is a fundamental security concern — pip install evil-package can run any code on your machine during installation. This is one of the strongest arguments for the wheel format, which is installed without executing any code.