Python Package Management (4 Part Series)
This is part of a series explaining the different options for managing Python dependencies. Make sure you've read the
previous posts, this one will build on that information.
Poetry addresses many of my complaints about the pipenv and vanilla stacks by providing a much nicer development experience. On the downside, it's a much newer tool which is still developing some important features, and it's not backed by the Python Package Authority (PyPA) which has lead to some conflicts.
- poetry: Used to manage dependencies, virtual environments, building, and publishing your project.
- pyproject.toml: Contains all your project metadata
- poetry.lock: A file used to synchronize exact versions of dependencies between environments
- pip: Used for some distribution cases
Poetry provides by far the best developer experience of the three stacks discussed.
- 👍 Dev Requirements: Just like Pipenv you can easily declare developer dependencies in the same file as all your other dependencies.
- 😍 Reproducibility: The most sane management of Python packages. Poetry will install from your lock file, it only updates your dependencies when you tell it to.
- ✅ Adding new packages: This is done exactly the way it should be done. If you do
poetry add somepackage, it will get the latest version of somepackage and set the constraint to be compatible per Semantic Versioning. So if the latest version is 1.4.8, your constraint is 1.4, meaning it can update to 1.4.9 or 1.5, but not 2.0.
- 😊 Alternative sources: Even easier than Pipenv. Add your source to pyproject.toml and setup your credentials with
poetry config. Much easier than trying to get environment variables to work cross platform.
- 😌 Setup: A breeze.
poetry installcreates a virtual environment if one doesn't exist and is so much faster than Pipenv. One slight tradeoff is Poetry supports multiple versions of Python for one project, which means you have to specify which version you want to use for any given point in time. The recommendation is to use pyenv on -nix systems. I've found the built in py launcher for Windows works fine too.
- 🙂 Usage: Pretty much the same story as Poetry, though there are some more bells and whistles.
- Installing requirements:
poetry run python [your script]
- Activating the environment:
pipenv shell(also spawns a new shell)
- Installing requirements:
Poetry makes almost everything better than Pipenv. If you're doing source or prebuilt distribution, you'll be a little worse off.
- 🥰 Definitions: Your pyproject.toml which defines your dependencies also defines all the other metadata for your project! No more setup.py.
- 😁 Building:
poetry build. Downside? Only pure Python wheels are supported, no C code yet.
- ⚡ PyPI:
- 👌 Source Distribution: You have to install Poetry on your server, but it has much less overhead than Pipenv so I think ti's worth it.
- 😥 Prebuilt Dependencies: The released version of Poetry has no way to do this. My solution today is to use the Alpha version of Poetry (which I've had no issues with) and export a requirements.txt to use with pip
Poetry is awesome! It's still quite young (still on its way to 1.0) but already better than the existing tools. Given the direction it's heading, the community supporting it, and the friendly and supportive maintainer, I think it should be everyone's tool of choice for Python dependency management.
That's probably the end of this series. There is, of course, always more to this topic, so if I missed anything or you have any questions please leave a comment below!
Here are tools that I'm aware of but don't have enough experience with to give a full review on. If you think one of these is worth spending time with and writing another part of this series on, please let me know.
Hatch is a productivity tool designed to make your workflow easier and more efficient, while also reducing the number of other tools you need to know It aims to make the 90% use cases as pleasant as possible.
For me personally, Hatch has entirely replaced the manual (or entire!) use of these:
- Cookiecutter PyPackage
python setup.py ...
/^\/^\ _|__| O| \/ /~ \_/ \ \____|__________/ \ \_______ \ `\ \ \ | | \ / / \ / / \ / / \ \ / / \ \ / / _----_ \ \ / / _-~ ~-_ | | ( ( _-~ _--_ ~-_ _/ | \ ~-____-~ _-~ ~-_ ~-_-~ / ~-_ _-~ ~-_ _-~ ~--______-~ ~-___-~
📦 🔥 Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.
DepHell -- project management for Python.
Why it is better than all other tools:
- Format agnostic. You can use DepHell with your favorite format: setup.py, requirements.txt, Pipfile, poetry. DepHell supports them all and much more.
- Use your favorite tool on any project. Want to install a poetry based project, but don't like poetry? Just tell DepHell to convert the project's meta information into a setup.py and install it with pip. Or work directly work with the project from DepHell, because DepHell can do everything that you usually want to do with packages.
- DepHell doesn't try to replace your favorite tools. If you use poetry, you have to use poetry's file formats and commands. However, DepHell can be combined with any other tool or can even combine all these tools together by converting formats. You can use DepHell, poetry, and pip all at the same time.
- Easily extendable…
As software gets more and more integrated into our lives, the industrialization of its crafting process becomes inevitable. But the over-generalization of software engineering can be crushing the creative side of programming.