setuptools vs distutils why is distutils still a thing

The Python ecosystem thrives on its vibrant community and the ease with which developers can share their code. At the heart of this sharing mechanism lies Python’s packaging tools. For years, the discussion around Python packaging has often revolved around the classic setuptools vs. distutils debate. While setuptools has long been the de facto standard for building and distributing Python packages, a lingering question often arises: why is distutils still a thing, especially when it seems so limited compared to its more powerful successor? This article delves into the historical context, technical intricacies, and ongoing relevance of both tools, shedding light on why distutils remains embedded in the Python landscape and what the future holds for Python package management.

Understanding Python’s Packaging Landscape

To truly grasp the relationship between setuptools and distutils, it’s essential to understand their respective origins and roles within Python’s packaging evolution. The way we create, distribute, and install Python packages has seen significant changes over the years, driven by the community’s need for more robust and flexible solutions. From simple scripts to complex applications with multiple dependencies, the packaging tools have had to adapt.

The Origins of distutils

distutils was the original standard library module introduced in Python 1.6 (released in 2000) for building and distributing Python modules. Its primary purpose was to provide a basic framework for creating source distributions (sdist) and binary distributions (bdist), allowing developers to easily install Python packages. It brought much-needed standardization to a previously fragmented process, enabling commands like python setup.py install to become commonplace. However, distutils came with significant limitations; it lacked crucial features like dependency management, support for modern distribution formats, and the ability to easily extend its functionality.

The Rise of setuptools

Recognizing the shortcomings of distutils, the community developed setuptools around 2004. setuptools was designed as an extension to distutils, building upon its foundation while introducing a wealth of new features critical for modern Python development. It provided robust dependency resolution, support for Python Eggs (an early distribution format), and more powerful mechanisms for package metadata and entry points. setuptools quickly became indispensable for anyone publishing packages to the Python Package Index (PyPI), largely replacing direct distutils usage for complex projects. Its introduction marked a significant leap forward in Python’s packaging capabilities, making it easier to manage intricate project dependencies and deployment pipelines.

The Core Differences: setuptools vs. distutils

The fundamental distinction between setuptools and distutils lies in their feature sets and design philosophies. While distutils offers a minimal, foundational layer for simple package creation, setuptools extends this functionality dramatically to meet the demands of contemporary software development. Understanding these differences is crucial for any Python developer managing project dependencies or preparing a package for distribution.

For instance, distutils primarily handles basic tasks like compiling C extensions, installing files into the correct Python site-packages directory, and generating a simple source distribution. It operates under the assumption that all dependencies are already met, offering no built-in mechanism for automatically downloading or installing required packages. This made it impractical for projects with complex dependency trees, leading to manual and often error-prone setup processes for users.

Conversely, setuptools revolutionized Python packaging by introducing essential features that distutils lacked. It brought sophisticated dependency management, allowing developers to declare project requirements in setup.py and rely on tools like pip (which leverages setuptools) to resolve and install them automatically. Furthermore, setuptools supports modern distribution formats like wheels, which are pre-compiled distributions that significantly speed up installation times by avoiding source compilation on the user’s machine. This is a key reason why setuptools is the go-to tool for virtually all Python projects distributed on PyPI today.

The primary difference between setuptools and distutils is that setuptools provides extended capabilities like dependency management, automatic script generation, and support for modern distribution formats (like wheels), built upon the foundational, albeit limited, functionalities of distutils for basic package installation and metadata handling. This allows setuptools to manage complex project dependencies and distribution processes seamlessly, a critical feature missing from distutils.

• Dependency Resolution: setuptools excels at automatically identifying and installing project dependencies, a feature completely absent in distutils.
• Modern Formats: setuptools supports the creation of Wheel distributions (.whl), which offer faster and more reliable installations compared to source distributions alone.
• Entry Points: It allows packages to define console scripts and GUI entry points, making executables easily available after installation.
• Extensibility: setuptools is designed to be extensible, allowing for custom commands and build processes.

Why distutils Persists: Legacy and Integration

Despite the overwhelming advantages of setuptools, distutils remains a core component of the Python standard library. This persistence is primarily due to a combination of historical inertia, backward compatibility requirements, and the fundamental role distutils plays as the underlying mechanism for certain Python operations. It’s not so much that distutils is actively chosen over setuptools for new projects, but rather that its foundational elements are deeply integrated into the Python ecosystem.

One key reason for its continued existence is its inclusion in Python’s standard library. Removing distutils entirely would break a vast amount of legacy code and internal Python mechanisms that still rely on its basic functionalities. Many older packages and tools might still call distutils directly, and a sudden deprecation without a clear, gradual transition path could cause significant disruption. The Python core developers prioritize stability and backward compatibility, meaning that changes to core modules are carefully considered and often involve long deprecation periods.

Furthermore, setuptools itself was initially built directly on top of distutils. While setuptools has increasingly replaced distutils’s internal components with its own more robust implementations over time, it still relies on certain distutils primitives for fundamental operations. Think of distutils as a foundational layer, and setuptools as a sophisticated building constructed upon it, with many newer rooms and features. Even as setuptools becomes more independent, some of the original “foundation” pieces from distutils are still implicitly there. For a deeper dive into the relationship and history of these tools, the Python Packaging User Guide provides Question & Answer :

Python has a confusing history of tools that can be used to package and describe projects: these include distutils in the Standard Library, distribute, distutils2, and setuptools (and maybe more). It appears that distribute and distutils2 were discontinued in favor of setuptools, which leaves two competing standards.

To my understanding setuptools offers far more options (e.g. declaring dependencies, tests, etc.) than distutils, however it is not included in the Python standard library (yet?).

The Python Packaging User Guide[1] recommends now:

Use setuptools to define projects and create Source Distributions.

And explains:

Although you can use pure distutils for many projects, it does not support defining dependencies on other projects and is missing several convenience utilities for automatically populating package metadata correctly that are provided by setuptools. Being outside the standard library, setuptools also offers a more consistent feature set across different versions of Python, and (unlike distutils), setuptools will be updated to produce the upcoming “Metadata 2.0” standard formats on all supported versions.

Even for projects that do choose to use distutils, when pip installs such projects directly from source (rather than installing from a prebuilt wheel file), it will actually build your project using setuptools instead.

However, looking into various project’s setup.py files reveals that this does not seem to be an actual standard. Many packages still use distutils and those that support setuptools often mix setuptools with distutils e.g. by doing a fallback import:

try: from setuptools import setup except ImportError: from distutils.core import setup 

Followed by an attempt to find a way to write a setup that can be installed by both setuptools and distutils. This often includes various ways of error-prone dependency checking, since distutils does not support dependencies in the setup function.

Why are people still making the extra effort to support distutils - is the fact that setuptools is not in the standard library the only reason? What are the advantages of distutils and are there any drawbacks of writing setup.py files that only support setuptools.

Have a look at this SO question. It explains all the packaging methods very well, and might help answer your question to some extent: Differences between distribute, distutils, setuptools and distutils2?

Distutils is still the standard tool for packaging in Python. It is included in the standard library (Python 2 and Python 3.0 to 3.3). It is useful for simple Python distributions, but lacks features. It introduces the distutils Python package that can be imported in your setup.py script.

Setuptools was developed to overcome Distutils’ limitations, and is not included in the standard library. It introduced a command-line utility called easy_install. It also introduced the setuptools Python package that can be imported in your setup.py script, and the pkg_resources Python package that can be imported in your code to locate data files installed with a distribution. One of its gotchas is that it monkey-patches the distutils Python package. It should work well with pip. The latest version was released in July 2013.

So, as you can see setuptools should be preferred to distutils, and I see where your question comes from, however I don’t see distutils losing support anytime soon, as, simply put, it is used in many cases with some popular legacy programs. And as you probably know changing these sorts of things in legacy programs can be quite a pain and come with quite a few problems, for example incompatibilities, which would then lead to the developer having to rewrite the source code. So there is that, and also the fact that distutils is a part of the standard python library whereas setuptools is not. So, if you are creating a python program, in this day and age, use setuptools, however keep in mind that without distutils, setuptools would have never existed.