Virtual Environments in Python

I’ve been struggling a lot with understanding how to package python scripts & libraries. Some of the tool I use heavily rely on older version of the language. Some even rely on specific version of the older libraries. Working daily at a Java shop, this isn’t all that surprising. The bit that really caught me off-guard was how to make it all work when there’s different libraries to work with. e.g. some libraries require Python 2.7 while one of my other projects requires Python 3.4 or above. It seems the solution is to use VirtualEnv.

VirtualEnv allows you to isolate different packages & even python versions. You need to manually activate / deactivate a virtualenv but once you are in one of these, each virtualenv can have its own copy of the Python and dependencies in the site-packages directory. This allows you to create multiple environments within the same devlopment box / laptop. Typically, you create one virtual env for each workspace / project but you can just as create multiple envs for the same project to see how your library works in different situations. It’s a pretty sweet deal.

Installation

To set it all up, followthe installation guide

sudo pip install virtualenv

To create an new virtualenv you basically go to your project & activate

cd my_project_root_folder
virtualenv -p /usr/bin/python2.7 my_project
# now you activate
source my_project/bin/activate
# now do your dirty work like...
pip install -r requirements.txt
# when yo done, deactivate
deactivate

More Resources

Some links that really help me understand how packaging works in python

• Application Dependencies - overview of python dependencies
• Alice in Python projectland - takes you through how packaging works at a very simplified level from a simple script to a fully fleged application / library
• setup.py for Humans - sample project explain the different values within setup.py & the moar relevant resources
• setup.py vs requirements.txt - differentiates between setup.py & requirements.txt and explains why both are required.
• Hitchiker’s Guide To VirtualEnv

Misc links I found on the way

• A template to make good README.md

Note

You have to always manually activate / deactivate your virutalenv. This makes you prone to human errors if you are not paying attention. There’s tool to automate this step as well but I’ve found them to not work as well (so far…)

Another issue I’ve faced so far is inability to isolate dependencies into their own shell. Something along the lines of JarJar. I’m not convinced this is a bad thing but it’s still a minor annoyance I need to work around with (by creating a project / microserver).

Comments

[Kamal Mustafa]

You have to always manually activate / deactivate your virutalenv.

This I think one of the anti-pattern of virtualenv. In our team, we make it compulsory to use full path to the python instead of using activate. For example:-

/home/kamal/projects/juita/.env/bin/python manage.py runserver

This also make communication easier. There's no need for back and forth communication like "Are you sure you're using the correct python ? Have you run activate ?".

[Rishi Baldawa]

I just python for hobby projects so this hasn't happened to be as much but if there's a way to change the terminal prompt & reflect that you're in the wrong virualenv, the Q/As might drop drastically.

I definitely dig using the full path. I'll give that a shot next time I'm working on a project.