This is part of a series explaining the different options for managing Python dependencies. Make sure you've read the
previous posts, this one will build on that information.
TL;DR
Poetry addresses many of my complaints about the pipenv and vanilla stacks by providing a much nicer development experience. On the downside, it's a much newer tool which is still developing some important features, and it's not backed by the Python Package Authority (PyPA) which has lead to some conflicts.
Components
- poetry: Used to manage dependencies, virtual environments, building, and publishing your project.
- pyproject.toml: Contains all your project metadata
- poetry.lock: A file used to synchronize exact versions of dependencies between environments
- pip: Used for some distribution cases
Development
TL;DR
Poetry provides by far the best developer experience of the three stacks discussed.
Defining Requirements
- 👍 Dev Requirements: Just like Pipenv you can easily declare developer dependencies in the same file as all your other dependencies.
- 😍 Reproducibility: The most sane management of Python packages. Poetry will install from your lock file, it only updates your dependencies when you tell it to.
- ✅ Adding new packages: This is done exactly the way it should be done. If you do
poetry add somepackage, it will get the latest version of somepackage and set the constraint to be compatible per Semantic Versioning. So if the latest version is 1.4.8, your constraint is ^1.4, meaning it can update to 1.4.9 or 1.5, but not 2.0. - 😊 Alternative sources: Even easier than Pipenv. Add your source to pyproject.toml and setup your credentials with
poetry config. Much easier than trying to get environment variables to work cross platform.
Virtual Environments
- 😌 Setup: A breeze.
poetry installcreates a virtual environment if one doesn't exist and is so much faster than Pipenv. One slight tradeoff is Poetry supports multiple versions of Python for one project, which means you have to specify which version you want to use for any given point in time. The recommendation is to use pyenv on -nix systems. I've found the built in py launcher for Windows works fine too. - 🙂 Usage: Pretty much the same story as Poetry, though there are some more bells and whistles.
- Installing requirements:
poetry install - Running:
poetry run python [your script] - Activating the environment:
poetry shell(also spawns a new shell) - Deactivating:
exit
- Installing requirements:
Distribution
TL;DR
Poetry makes almost everything better than Pipenv. If you're doing source or prebuilt distribution, you'll be a little worse off.
Build
- 🥰 Definitions: Your pyproject.toml which defines your dependencies also defines all the other metadata for your project! No more setup.py.
- 😁 Building:
poetry build. Downside? Only pure Python wheels are supported, no C code yet.
Deploy
- ⚡ PyPI:
poetry publish - 👌 Source Distribution: You have to install Poetry on your server, but it has much less overhead than Pipenv so I think ti's worth it.
- 😥 Prebuilt Dependencies: The released version of Poetry has no way to do this. My solution today is to use the Alpha version of Poetry (which I've had no issues with) and export a requirements.txt to use with pip
Conclusion
Poetry is awesome! It's still quite young (still on its way to 1.0) but already better than the existing tools. Given the direction it's heading, the community supporting it, and the friendly and supportive maintainer, I think it should be everyone's tool of choice for Python dependency management.
That's probably the end of this series. There is, of course, always more to this topic, so if I missed anything or you have any questions please leave a comment below!
Notable Mentions
Here are tools that I'm aware of but don't have enough experience with to give a full review on. If you think one of these is worth spending time with and writing another part of this series on, please let me know.
-
Hatch
CI/CD 
Docs 
Package Meta
Hatch is a modern, extensible Python project manager.
Features
- Standardized build system with reproducible builds by default
- Robust environment management with support for custom scripts
- Easy publishing to PyPI or other indexes
- Version management
- Configurable project generation with sane defaults
- Responsive CLI, ~2-3x faster than equivalent tools
Documentation
The documentation is made with Material for MkDocs and is hosted by GitHub Pages.
License
Hatch is distributed under the terms of the MIT license.
-
dephell
/
dephell
📦 🔥 Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.
THE PROJECT IS ARCHIVED
Forks: https://github.com/orsinium/forks
DepHell -- project management for Python.
Why it is better than all other tools:
- Format agnostic. You can use DepHell with your favorite format: setup.py, requirements.txt, Pipfile, poetry. DepHell supports them all and much more.
- Use your favorite tool on any project. Want to install a poetry based project, but don't like poetry? Just tell DepHell to convert the project's meta information into a setup.py and install it with pip. Or directly work with the project from DepHell, because DepHell can do everything that you usually want to do with packages.
- DepHell doesn't try to replace your favorite tools. If you use poetry, you have to use poetry's file formats and commands. However, DepHell can be combined with any other tool or can even combine all these tools together by converting formats. You can use DepHell, poetry, and pip all at…
Top comments (11)
Thanks Dylan, these series has been really informative! I'll definitely take a look at Poetry.
The fact that it can export to
requirements.txtshouldn't make deploy a big deal, most PaaS services supportrequirements.txt. I guess that if it becomes a bigger project they'll add support for its own lock files, the same way it happened for Pipenv.Didn't know about
hatchanddephellseems a tool created by someone that was so fed up with the current state of package management that they said to themselves "f*ck it, I'm going nuclear, I'll support everything"I'm testing it already with a project, the UI is much better and the tool seems faster.
The
poetry.lockis also much more readable for a human (it's common that one wants to know exactly which version of a transitive dependency is installed).poetry initwas a great experience with its search and everything.poetry updateis insanely faster thanPipenv's. Search and publish are game changers. Love theversioncommand as well.The export functionality definitely needs some love but I guess I'm a convert now, thanks again Dylan! :D
I’m glad you tried it and liked it! Pretty sure there are already improvements to export underway, it’s just a matter of time.
Another great thing I didn’t mention is that the code quality is very good. While they don’t have the Python API documented yet, it didn’t take me long at all to figure out how to make a custom build script which adds some extra export functionality just by reading the code.
wow that's great!
Nice article! BTW, I'm the author of DepHell :) The coolest thing about DepHell is you can use it along with poetry. DepHell can audit dependencies, show licenses list, make requirements.txt, bump project version etc.
If you find lack of some features -- please, open an issue. I am happy to make DepHell better :)
Great to hear from you! We actually have a smattering of projects using Pipenv that I’ll someday have to convert to poetry. I found DepHell when looking for a tool to convert between the two (since Poetry can’t do it natively yet).
Using it as a better poetry export tool sounds like a good idea too, I can more easily convince people to use the non-alpha version of Poetry.
Okay, I guess I’ll take a closer look at it and give it a go, especially now that you’re on DEV :).
I've registered here for you
<3I'm looking at all mentions of DepHell to get feedback and make it better.I’ve just been reading through the docs, they are very thorough. Being able to really understand what a tool does just by reading the docs is rare, well done!
The one thing I didn’t see any mention of is private PyPI repositories. Seems like dephell uses its own resolver and isn’t just a wrapper around Poetry, so it probably doesn’t use the configured credentials. I think this means it wouldn’t work for converting between formats if they use private repos and wouldn’t work for installing/adding/etc. if some packages were private.
Please correct me if I’m wrong, but if I’m not, private repo support is the only thing holding me back from using this.
Private warehouse on the way:
github.com/dephell/dephell/issues/53
You can subscribe on this issue and get a notification when it is done :)
I also have some projects with a local warehouse (artifactory), so I interested in this feature too. For a while, I use DepHell for environments management in these projects.
Ey, nice article!
I think you have a mistake. In the "Virtual Environments" paragraph, you wrote:
pipenvshell (also spawns a new shell)I think you wanted to write: poetry shell, isn't it?
Yes, you are correct. Thank you and fixed!