loading...

Productive Python with Poetry

dbanty profile image Dylan Anthony Updated on ・3 min read

This is part of a series explaining the different options for managing Python dependencies. Make sure you've read the
previous posts, this one will build on that information.

TL;DR

Poetry addresses many of my complaints about the pipenv and vanilla stacks by providing a much nicer development experience. On the downside, it's a much newer tool which is still developing some important features, and it's not backed by the Python Package Authority (PyPA) which has lead to some conflicts.

Components

  • poetry: Used to manage dependencies, virtual environments, building, and publishing your project.
  • pyproject.toml: Contains all your project metadata
  • poetry.lock: A file used to synchronize exact versions of dependencies between environments
  • pip: Used for some distribution cases

Development

TL;DR

Poetry provides by far the best developer experience of the three stacks discussed.

Defining Requirements

  • 👍 Dev Requirements: Just like Pipenv you can easily declare developer dependencies in the same file as all your other dependencies.
  • 😍 Reproducibility: The most sane management of Python packages. Poetry will install from your lock file, it only updates your dependencies when you tell it to.
  • ✅ Adding new packages: This is done exactly the way it should be done. If you do poetry add somepackage, it will get the latest version of somepackage and set the constraint to be compatible per Semantic Versioning. So if the latest version is 1.4.8, your constraint is ^1.4, meaning it can update to 1.4.9 or 1.5, but not 2.0.
  • 😊 Alternative sources: Even easier than Pipenv. Add your source to pyproject.toml and setup your credentials with poetry config. Much easier than trying to get environment variables to work cross platform.

Virtual Environments

  • 😌 Setup: A breeze. poetry install creates a virtual environment if one doesn't exist and is so much faster than Pipenv. One slight tradeoff is Poetry supports multiple versions of Python for one project, which means you have to specify which version you want to use for any given point in time. The recommendation is to use pyenv on -nix systems. I've found the built in py launcher for Windows works fine too.
  • 🙂 Usage: Pretty much the same story as Poetry, though there are some more bells and whistles.
    • Installing requirements: poetry install
    • Running: poetry run python [your script]
    • Activating the environment: poetry shell (also spawns a new shell)
    • Deactivating: exit

Distribution

TL;DR

Poetry makes almost everything better than Pipenv. If you're doing source or prebuilt distribution, you'll be a little worse off.

Build

  • 🥰 Definitions: Your pyproject.toml which defines your dependencies also defines all the other metadata for your project! No more setup.py.
  • 😁 Building: poetry build. Downside? Only pure Python wheels are supported, no C code yet.

Deploy

  • ⚡ PyPI: poetry publish
  • 👌‍ Source Distribution: You have to install Poetry on your server, but it has much less overhead than Pipenv so I think ti's worth it.
  • 😥 Prebuilt Dependencies: The released version of Poetry has no way to do this. My solution today is to use the Alpha version of Poetry (which I've had no issues with) and export a requirements.txt to use with pip

Conclusion

Poetry is awesome! It's still quite young (still on its way to 1.0) but already better than the existing tools. Given the direction it's heading, the community supporting it, and the friendly and supportive maintainer, I think it should be everyone's tool of choice for Python dependency management.

That's probably the end of this series. There is, of course, always more to this topic, so if I missed anything or you have any questions please leave a comment below!

Notable Mentions

Here are tools that I'm aware of but don't have enough experience with to give a full review on. If you think one of these is worth spending time with and writing another part of this series on, please let me know.

  • GitHub logo ofek / hatch

    A modern project, package, and virtual env manager for Python

    Hatch

    Latest PyPI version Travis CI AppVeyor CI Codecov Supported Python versions License

    Hatch is a productivity tool designed to make your workflow easier and more efficient, while also reducing the number of other tools you need to know It aims to make the 90% use cases as pleasant as possible.

    For me personally, Hatch has entirely replaced the manual (or entire!) use of these:

               /^\/^\
             _|__|  O|
    \/     /~     \_/ \
     \____|__________/  \
            \_______      \
                    `\     \                 \
                      |     |                  \
                     /      /                    \
                    /     /                       \
                  /      /                         \ \
                 /     /                            \  \
               /     /             _----_            \   \
              /     /           _-~      ~-_         |   |
             (      (        _-~    _--_    ~-_     _/   |
              \      ~-____-~    _-~    ~-_    ~-_-~    /
                ~-_           _-~          ~-_       _-~
                   ~--______-~                ~-___-~
    



  • GitHub logo

    dephell
    /
    dephell



    📦 🔥 Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.




    DepHell

    pypi MIT License Travis CI Powered by DepHell

    DepHell -- project management for Python.

    Why it is better than all other tools:

    1. Format agnostic. You can use DepHell with your favorite format: setup.py, requirements.txt, Pipfile, poetry. DepHell supports them all and much more.
    2. Use your favorite tool on any project. Want to install a poetry based project, but don't like poetry? Just tell DepHell to convert the project's meta information into a setup.py and install it with pip. Or directly work with the project from DepHell, because DepHell can do everything that you usually want to do with packages.
    3. DepHell doesn't try to replace your favorite tools. If you use poetry, you have to use poetry's file formats and commands. However, DepHell can be combined with any other tool or can even combine all these tools together by converting formats. You can use DepHell, poetry, and pip all at the same time.
    4. Easily extendable

Posted on by:

dbanty profile

Dylan Anthony

@dbanty

Backend Dev obsessed with security

Discussion

pic
Editor guide
 

Thanks Dylan, these series has been really informative! I'll definitely take a look at Poetry.

The fact that it can export to requirements.txt shouldn't make deploy a big deal, most PaaS services support requirements.txt. I guess that if it becomes a bigger project they'll add support for its own lock files, the same way it happened for Pipenv.

Didn't know about hatch and dephell seems a tool created by someone that was so fed up with the current state of package management that they said to themselves "f*ck it, I'm going nuclear, I'll support everything"

 

I'm testing it already with a project, the UI is much better and the tool seems faster.

The poetry.lock is also much more readable for a human (it's common that one wants to know exactly which version of a transitive dependency is installed).

poetry init was a great experience with its search and everything. poetry update is insanely faster than Pipenv's. Search and publish are game changers. Love the version command as well.

The export functionality definitely needs some love but I guess I'm a convert now, thanks again Dylan! :D

 

I’m glad you tried it and liked it! Pretty sure there are already improvements to export underway, it’s just a matter of time.

Another great thing I didn’t mention is that the code quality is very good. While they don’t have the Python API documented yet, it didn’t take me long at all to figure out how to make a custom build script which adds some extra export functionality just by reading the code.

it didn’t take me long at all to figure out how to make a custom build script which adds some extra export functionality just by reading the code

wow that's great!

 

Nice article! BTW, I'm the author of DepHell :) The coolest thing about DepHell is you can use it along with poetry. DepHell can audit dependencies, show licenses list, make requirements.txt, bump project version etc.

If you find lack of some features -- please, open an issue. I am happy to make DepHell better :)

 

Great to hear from you! We actually have a smattering of projects using Pipenv that I’ll someday have to convert to poetry. I found DepHell when looking for a tool to convert between the two (since Poetry can’t do it natively yet).

Using it as a better poetry export tool sounds like a good idea too, I can more easily convince people to use the non-alpha version of Poetry.

Okay, I guess I’ll take a closer look at it and give it a go, especially now that you’re on DEV :).

 

especially now that you’re on DEV

I've registered here for you <3 I'm looking at all mentions of DepHell to get feedback and make it better.

 

I’ve just been reading through the docs, they are very thorough. Being able to really understand what a tool does just by reading the docs is rare, well done!

The one thing I didn’t see any mention of is private PyPI repositories. Seems like dephell uses its own resolver and isn’t just a wrapper around Poetry, so it probably doesn’t use the configured credentials. I think this means it wouldn’t work for converting between formats if they use private repos and wouldn’t work for installing/adding/etc. if some packages were private.

Please correct me if I’m wrong, but if I’m not, private repo support is the only thing holding me back from using this.

 

Private warehouse on the way:
github.com/dephell/dephell/issues/53
You can subscribe on this issue and get a notification when it is done :)

I also have some projects with a local warehouse (artifactory), so I interested in this feature too. For a while, I use DepHell for environments management in these projects.

 

Ey, nice article!

I think you have a mistake. In the "Virtual Environments" paragraph, you wrote:

  • Activating the environment: pipenv shell (also spawns a new shell)

I think you wanted to write: poetry shell, isn't it?

 

Yes, you are correct. Thank you and fixed!