Like the articles? Buy the book! Dead Simple Python by Jason C. McDonald is available from No Starch Press.
Virtual environments. If you've eve...
For further actions, you may consider blocking this person and/or reporting abuse
Check out pipenv, it so very nicely married pip and venv/virtualenv. I've quite enjoyed that it tracks the requirements for me as I go.
If you're gonna start with Python 3.6+, venv module (built in) is my recommendation, of course only if you will work with 3.6+ version and you're not looking for previous versions.
Good start for this good serie !
Nice one! Could add
pip freezestep forrequirements.txtcreation, it results in better dependency management, as it handles transitive cases too.What are your thoughts on using Anaconda? I've done a couple online courses and so far they've all started with installing python via anaconda. It has its own package management and virtual environment stuff (or at least its own commands for them).
Honestly, I've never used it. I know Anaconda is a Python distribution platform geared specifically towards data science. If you like it, by all means, enjoy it. :3 But you should know how regular Python works, and then find out what makes Anaconda different. I don't know much about it at all.
Makes sense. Everywhere I've seen commands they look pretty similar. I wouldn't be surprised if conda is using a lot of the same stuff behind the scenes, but I don't actually know. (it is definitely recommended to NOT use pip to install anything if you're using conda, though...or if you absolutely must, do it last.)
I'm not sure how I like it, comparatively, having not done anything with Python without it. Sometimes it seems like more overhead than necessary, though.
Hello Jason,
I found this article very informative and point-to-point coverage on topics.
Though I still have doubts on she-bang. As you said above if we write !#/usr/bin/env as first line in python script we don't need to call it with 'python' command on terminal. So I tried this by writing 'test.py' on Linux but this script is not getting executed without typing 'python' onto the beginning of the terminal command and it gives error : 'test.py' is not a command..
So I didn't understand the use of she-bang properly.
Are there any exceptions to these?
That error has nothing to do with Python at all. Running just...
...is going to search your system environment paths for an executable file called
test.py, and I can practically guarantee your project folder isn't part of that environment path.Thus, you need to run...
The
.means "current working directory", and then you build the relative path from there.Hi, nice article, but honestly I've abandoned virtualenv in favor of Docker. What are your thoughts about this tool?
Docker has a lot of uses, but merely as a replacement for virtual environments? It's overkill. If a virtualenv is a sandbox, Docker is a clean room. You can use it, naturally, but if the only thing you were needing was a controlled set of Python packages, it's a lot of wasted time and disk space.
Now, if you actually need to control your environment beyond what virtualenv allows you to control, Docker is fine. (It makes sense for web development, for example.)
Just not as a straight virtualenv replacement.
I do not share the point of docker being "overkill" because if one is fluent with it, there would be no "time wasted". And disk space is cheap - as long as you aren't on a Mac (SCNR - But admittedly one of my box is a Mac and I am too guilty of suffering from the lack of a bunch of GBs).
I put it more mildly: It has no advantages over virtual envs if you do not use docker for deployment.
Well, and see, that's why I said that if it was merely a replacement for virtualenvs, it was overkill. A virtual environment only replicates the bare minimum necessary, in terms of the Python interpreter and libraries you're using.
If you're actually deploying with Docker, that's another story altogether.
First, not everyone has Docker configured on their machine. Second, resources like disk space are not always "cheap" on any system. (Macs aren't the only machines to suffer from low disk space.) Internet may be limited, slow, or inexpensive, making a Docker build prohibitive. Time may be of the essence, RAM might be limited, or CPU might be in demand. There are so many scenarios in which Docker is just not going to be work.
Virtual environments are the one canonical "works everywhere" solution for sandboxing Python. Anyone and everyone who can run Python 3 (and most versions of 2) can create a virtual environment. They require very little overhead, generally minimal network time, and nothing in the way of extra processing power. Ironically, they're one of the fastest and most reliable ways to deploy a Python project in a Docker!
To this aim, your project should be configured properly so that it can be run in a virtual environment. (It doesn't take that much work; certainly less work than a Dockerfile does.)
The third problem is, if the Python project in question has anything to do with GUI or the like, you're going to be beyond the "just start a Docker container" situation. You'll have to configure and work with VNC, which for testing a local application is generally impractical. The same could be said of working with local filesystems (you have to plan what to access and mount it), system integration features, and the list goes on.
Docker certainly has a purpose in some Python development situations, but it should be considered a separate tool altogether, and never as a "replacement" for virtual environments.
Hey Jason, thanks for the great write-up! It certainly solves a lot of questions for me. But one is still there – what to do when I have databases and I don't want to have pq, mysql and mongo installed on my system in various versions?
Assuming SQLite is not an option, this is exactly the sort of scenario where Docker is a good solution: you aren't just needing the sandbox the Python environment itself (virtual environment), but the actual system environment.
Virtual environments really become extra helpful here, too, because it simplifies your Dockerfiles. Assuming your Python project has a
requirements.txtfile, you'd only need the following:You'll note, I had no need to "activate" the virtual environment. You can just use the venv's binaries directly; they know to use the virtual environment.
Thanks for your reply, Jason.
I enjoy using Docker, mostly because I always have a database or something similar standing next to the actual application, plus deploying Docker is super easy, but perhaps I should try using venv more. Thank you :)
Both docker and virtualenvs try to solve the same problem: separating the system side of things from the user side of things. In case of virtual envs you are getting basically a bit "shell magic" where paths are tweaked in such a way that things work as you were interfering with the system's python but you aren't. That makes it impossible to have interfering installations of python packages and versions.
Docker does the same on a more fundamental level:
Docker divides the kernel-land from the user-land. So it behaves more or less like a virtual environment; but instead of relying on "shell magic" it relies on "kernel magic".
From that point of view there is no real difference or advantage of using one over the other. It totally depends on your deployment model. If your deployment involves docker, it would make more sense to use docker from the first line of code written - and it would be a minimal smoketest for what you are doing anyways. If your deployment model doesn't involve docker, I see no advantage of using docker over a virtual environment.
Awesome! Quick question tho on
sudo pip. I've had encounters before with it.Why is it wrong,
and what happens if you do do it,
and how do you "undo" such an action?
It's not exactly wrong, it might not do what you are expecting.
People might run pip install requests and fail because of permissions. They might try sudo pip instead of pip install --user.
Sudo pip will install things in the system's python, rather than the user's python.
Good explanation, @jcsvveiga .
The reason I say you should "never" use
sudo pipis, on any system maintained by a dedicated package manager, it can easily conflict with the system-maintained packages, which can make a real mess, especially where dependencies are concerned.Ah! That makes sense. So it would be much better then to do
pip install --userthan sudo?Yes, or if you want a ditectory/project dependencies, use a virtual environment
Great article Jason, i love how easy is to understand virtualenvs using this.
Recently i came across a problem trying to teach a colleague how to set up his own virtualenv to simulate what i had done with mine since we were trying to mimic the entire environment of mine we faced a struggle and ended up failing :(
A day later we came across pipenv wich reproduced the exact ambient for us and problem solved... now i have to ask whats your stance in this virtualenv vs pipenv that seems to be happening around the python world?
and also are you planning to add it to your dead simple series? i will love to use it to teach my colleagues at college, i love the style of your writing (fan girl screaming on the background).
greetings and lots of tacos for you.
Honestly, I never planned on adding
pipenvto this, but I may have to cave and add it anyway! Everyone keeps asking. :PSo glad you enjoyed it!
Just started with Python a week ago and the first advice I came across: pipenv! Combine venv, pip and whatnot in one simple command. It seems to be "the new standard" and until now seems to work flawlessly.
This is excellent! I did something wrong awhile back and ended up in a situation that makes that second xkcd look very familiar. 😬
(Speaking of xkcd, I don't see any attribution for the comics. While I'm not exactly sure what legal attribution compliance with the license looks like, a link to the source material seems like an appropriate minimum.)
This has me excited to try python again, this time following your clear and concise instructions, and taking advantage of venv! Thank you!
So glad you liked the article!
I went ahead and added an attribution at the bottom, thanks. XKCD is so widespread at this point, they get embedded rather commonly anymore (so much so, that Randall just hands out embed links, which I used). That's why I overlooked attribution. It's polite, anyhow. ;)
This has some quality content. Thanks for explaining this. Starts to put things together a little more for me
Thank you for this introductory post to begin the series. I look forward to going through the rest as they are released.
This article is a lifesaver for Python enthusiasts struggling with virtual environments. The author's candid admission of past struggles adds a relatable touch. The analogy of virtual environments as sandboxes brilliantly simplifies their purpose. Explaining the intricacies of package installations with humor (cue the XKCD comics) makes it an enjoyable read.
The step-by-step guide on creating, activating, and leaving virtual environments is like having a tech-savvy friend guiding you. The inclusion of commands for various operating systems ensures inclusivity. Kudos to the author for demystifying pip and making it sound "deliciously simple." Thanks a ton for this gem, Jason C. McDonald! Your efforts in breaking down complexities are truly appreciated.
• For similar topics, read this article:
Thanks for this clear post! What is the difference between:
virtualenv -p python3 venvand
python3 -m venv myvirtualenvThere basically isn't one. The latter is preferred now, though, because it feels a little more natural feeling to create a virtual environment using a specific version of Python:
After countless searching of blogs/websites
Finally found the best and easiest tutorial of virtualenv 😄
I’m very excited for this series! Amazing so far! 😁
Great recommendations!
Very helpful tutorial--thanks!
I've coded in Python for a while, but didn't know about virtual-environments.
So I already have python2 and python3 on my Mac. How to handle that? Should I remove python2?
You can absolutely leave them both installed. That's part of the beauty of a virtual environment! Just specify which one you want to use when you create the virtualenv. For example...
On what directory should I run the "venv\Scripts\activate.bat" command?
Ah i got it. I should run it in the same directory as venv/ and if i want to run the python i should use the one inside the 'venv\script\python' instead of only typing 'python'.