My team has struggled a lot with Pipenv. We use Python in a ton of different environments and have come across several weaknesses that should be considered before jumping in.
It is slow. Talking 20+ minutes to sync (install from lock, no resolving necessary) a large web app.
When you add a new dependency, it also always updates your other dependencies, which means adding takes a long time and can break things you weren’t intending to touch.
They’re recommended way of doing things is to declare dependencies with * version, meaning any time you update it will grab whatever the latest is, even when not compatible. This becomes quite frustrating, especially with how long it takes to fix it. If you’re going to use Pipenv I strongly recommend declaring dependencies like flask = "~=1" which means use a version compatible with 1 (semantic versioning rules)
Private repositories are weakly supported. You can do it by declaring an additional source in your Pipfile but if you have credentials protecting your repo (which you should), you’ll need to store them in environment variables and access the like ${USERNAME} in your Pipfile (which doesn’t always work).
Lately I’ve been working with Poetry which solves all of those problems, though had a few of its own. The good news is they’re actively working on building features to cover all of the flaws I’ve found so far.
You should! Great post by the way, you should totally do posts on some more advanced usages as you try it. Information tends to be scattered on the topic and dependency management is important!
Which issues you found with Poetry? There are so many dependency managers out there. I settled with Pipenv for now because it's better than nothing but the issues you listed are not news to me.
Getting dependency management right in Python seems quite an ordeal!
Maybe I should do a whole post on my experience with different dependency systems for Python 😅.
The TL;DR is this:
For Python-pure libraries it works great, best solution I've found, but If you have some C-code or something in your lib, it won't work yet (hopefully eventually)
For deploying an app it's a lot trickier, Poetry basically only gives you the option of building a wheel and then installing that in whatever environment you're deploying to. If that works for what you're doing, great! Otherwise you end up having to export a requirements.txt file (Alpha version of Poetry only) to deploy the old fashioned way.
There are future features planned for integrating some deployment options, but they don't exist yet. Poetry is still very young which means it comes with the usual drawbacks of young projects.
I have limited experience with anaconda. It ended up causing me more trouble than it was worth. I guess all the bundled packages they include are useful for data scientists but they were unnecessary for me.
Personally I use pyenv on Unix-like systems for installing/managing more than one version of Python. If you try out pyenv make sure to have this page open, I’ve needed to run a command from it on every install (particularly on macOS).
On Windows the normal Python installer comes with the handy “py” launcher which has always been good enough for me.
I can’t say I’ve ever missed any package that conda has and I don’t. But again, I’m not a data scientist.
Pyenv works well, I have 4 different versions of Python installed on my Mac, plus the system python plus I think one from brew, never had a conflict. It's Pipenv that's slow sometimes
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
My team has struggled a lot with Pipenv. We use Python in a ton of different environments and have come across several weaknesses that should be considered before jumping in.
It is slow. Talking 20+ minutes to sync (install from lock, no resolving necessary) a large web app.
When you add a new dependency, it also always updates your other dependencies, which means adding takes a long time and can break things you weren’t intending to touch.
They’re recommended way of doing things is to declare dependencies with * version, meaning any time you update it will grab whatever the latest is, even when not compatible. This becomes quite frustrating, especially with how long it takes to fix it. If you’re going to use Pipenv I strongly recommend declaring dependencies like
flask = "~=1"which means use a version compatible with 1 (semantic versioning rules)Private repositories are weakly supported. You can do it by declaring an additional source in your Pipfile but if you have credentials protecting your repo (which you should), you’ll need to store them in environment variables and access the like
${USERNAME}in your Pipfile (which doesn’t always work).Lately I’ve been working with Poetry which solves all of those problems, though had a few of its own. The good news is they’re actively working on building features to cover all of the flaws I’ve found so far.
Thanks for your comment, Dylan.
I've just found pipenv recently and tried it just today.
I'm really glad to know some potential issues regarding pipenv from you.
I'll try Poerty and write a post about it :)
You should! Great post by the way, you should totally do posts on some more advanced usages as you try it. Information tends to be scattered on the topic and dependency management is important!
Which issues you found with Poetry? There are so many dependency managers out there. I settled with Pipenv for now because it's better than nothing but the issues you listed are not news to me.
Getting dependency management right in Python seems quite an ordeal!
Maybe I should do a whole post on my experience with different dependency systems for Python 😅.
The TL;DR is this:
For Python-pure libraries it works great, best solution I've found, but If you have some C-code or something in your lib, it won't work yet (hopefully eventually)
For deploying an app it's a lot trickier, Poetry basically only gives you the option of building a wheel and then installing that in whatever environment you're deploying to. If that works for what you're doing, great! Otherwise you end up having to export a requirements.txt file (Alpha version of Poetry only) to deploy the old fashioned way.
There are future features planned for integrating some deployment options, but they don't exist yet. Poetry is still very young which means it comes with the usual drawbacks of young projects.
Got it! I'l keep an eye on it for when it matures :D Please write the post!
Thanks, your discussion helps me a lot to think of what package manager I should use.
By the way, any thought on anaconda? I've been struggling with using both anaconda and pip, and they messed me up :(
I have limited experience with anaconda. It ended up causing me more trouble than it was worth. I guess all the bundled packages they include are useful for data scientists but they were unnecessary for me.
Personally I use pyenv on Unix-like systems for installing/managing more than one version of Python. If you try out pyenv make sure to have this page open, I’ve needed to run a command from it on every install (particularly on macOS).
On Windows the normal Python installer comes with the handy “py” launcher which has always been good enough for me.
I can’t say I’ve ever missed any package that conda has and I don’t. But again, I’m not a data scientist.
Pyenv works well, I have 4 different versions of Python installed on my Mac, plus the system python plus I think one from brew, never had a conflict. It's Pipenv that's slow sometimes