Why Poetry
Context and problem statement
Managing the packages we depend on (e.g. Django) in Python isn’t “simply” installing it on your computer. The issue with installing Python packages on your computer is dependency conflicts that occur, where the “dependency trees” of one package require a different dependency tree than another one. So installing a package that requires a specific version of one package, while another package you already have installed requires another version can cause system problems.
The solution to this problem is by using “virtual environments” in Python. These are environments that are separate from the rest of the system and can install any packages without impacting other environments.
There are multiple ways of creating these virtual environments and managing package dependencies, and the optional solution depends on the project.
Decision drivers
- Package dependency management is a critical task, so we need a tool for it.
- Ensuring we have some way of building our Python projects in a way that is less likely to be a “it works on my computer” issue.
Considered options
There are (unfortunately) multiple tools to manage package dependencies in Python, for example, listed at Python Packaging User Guide and at Awesome Python. There are also comparison sites like Ritza Articles, warp, and this, and this at dev.to that go into more detail about many of these tools.
We’ll only cover these more popular tools:
Pipenv
Pipenv was designed to combine the existing functionality of pip and virtualenv, rather than being designed and built specifically for the purpose of package management.
Benefits
- Is very popular and widely used
- Has been around for a while
- Only handles package dependencies (no other features)
Drawbacks
- Design is a bit older and not as intuitive/clear to use
- Documentation is a bit too verbose/technical
- Package caching doesn’t seem to be well designed/robust, so packages could get unnecessarily re-installed
- Only handles package dependency management, which means we need another tool to develop software (like a Python package)
Poetry
Poetry was built and designed to address some of the short comings of pipenv and is specifically designed with package management and project development in mind.
Benefits
- Popular and widely used
- Very well designed website and documentation
- Handles package dependencies with lock files (detailed list of packages and versioning)
- Can set up and help manage a Python project (e.g. Python package)
- Designed from the ground up to consider the needs of package development and dependency resolution
- Allows for external plug-ins for further customization
- Similar dependency management to other languages
- Installable with
pipx - Integrates well with existing IDE/editors (PyCharm and VSCode)
Hatch
Hatch, like Poetry, was designed to manage package dependencies and Python projects.
Benefits
- Very similar to Poetry
- Is fully PEP-compliant
- Is relatively new
- Is very opinionated
- Good documentation
- Now maintained under the official PyPA
- Because of being PEP-compliant and hosted by PyPA, may be the better option in the future
Drawbacks
- Is very opinionated
- Is relatively new, so might still be working through things
- Not as widely used compared to the others
PDM
PDM, like both Poetry and Hatch, is designed for managing packages and Python projects.
Benefits
- Fully PEP-compliant
- Decent documentation
Drawbacks
- Relatively new
- Not as widely used or as popular
- Doesn’t have distribution specific installation builds (need to use
curlto install, which isn’t the most secure way of installing) - Can’t install through
pipx
Decision outcome
We decided on Poetry because it has amazing documentation, is well-designed, and is very popular and widely used.
Consequences
- Because Poetry isn’t fully PEP compliant, we might encounter some issues
- Doesn’t have the “official” support (hosted on the GitHub PyPA organization) that Hatch has, so we might reconsider the decision to use Poetry at a future date