virtualenv in Python is a very well known tools. Almost all tutorial will recommend you to use it. In the past, I'd also wrote about why you should not use the system python.
Being essential it is, starting with python 3.4 it was bundled within python itself as
There's nothing magic about
virtualenv actually. It just a copy (or symlink) to the python interpreter that you already have in your system, plus a couple of other files.
As we have learned when we kids, the best way to learn and understand stuff is by breaking it, or try to build it from scratch. So let's try to build venv/virtualenv without using the built-in module.
As mentioned before, virtualenv is just a collection of directories and files.
mkdir myenv mkdir myenv/bin
Then we need to find out where is the system python interpreter.
which python3 /usr/local/bin/python3
Let's copy it into our "virtualenv":-
cp /usr/local/bin/python3 myenv/bin/
Now let's try to invoke our "new" python interpreter:-
We should see the usual prompt such as:-
Python 3.6.0 (default, Jan 24 2017, 16:44:16) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
Now our "virtualenv" got it own interpreter, we also must make sure that it has it own
site-packages directory, where all the packages we're going to install will live. This is the main reason we use virtualenv, so that packages we're installing do not mess up with the system python or other project virtualenv. Run
myenv/bin/python3 and run this code:-
>>> sys.path ['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/kamal/Library/Python/3.6/lib/python/site-packages', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages'] >>> sys.prefix '/Library/Frameworks/Python.framework/Versions/3.6'
Notice that our
myenv directory is none in the output. The docs on
sys.prefix says this:-
If a virtual environment is in effect, this value will be changed in site.py to point to the virtual environment. The value for the Python installation will still be available, via base_prefix.
And following the link on virtual environment it says this:-
A virtual environment is a directory tree which contains Python executable files and other files which indicate that it is a virtual environment.
But what indicate a virtual environment? I cheated a bit here and created a new virtualenv using
python -mvenv tmp_env and in the newly created virtualenv, I noticed a file called
pyenv.cfg which contain this:-
home = /usr/local/bin include-system-site-packages = false version = 3.6.0
So let's try adding this file in our virtualenv as
myenv/pyenv.cfg. After adding this file, our interpreter will give the following output:-
>>> import sys >>> sys.path ['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload'] >>> sys.prefix '/Users/kamal/myenv'
sys.prefix value. Now we're onto something. It correctly pointed to our virtualenv directory. However our virtualenv still not in
sys.path. I forgot something! We still haven't created the
lib directory. So let's do it now:-
mkdir -p myenv/lib/python3.6/site-packages
And check our
>>> sys.path ['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/kamal/myenv/lib/python3.6/site-packages']
Now our virtualenv is added to
sys.path! Time to install some packages:-
myenv/bin/python3 -mpip install requests
But we will get the following error:-
/Users/kamal/myenv/bin/python3: No module named pip
site-packages directory still empty, so we don't have pip yet. Fortunately getting pip is not that hard.
wget https://bootstrap.pypa.io/get-pip.py myenv/bin/python3 get-pip.py Collecting pip Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl Collecting setuptools Using cached https://files.pythonhosted.org/packages/37/06/754589caf971b0d2d48f151c2586f62902d93dc908e2fd9b9b9f6aa3c9dd/setuptools-40.6.3-py2.py3-none-any.whl Collecting wheel Using cached https://files.pythonhosted.org/packages/ff/47/1dfa4795e24fd6f93d5d58602dd716c3f101cfd5a77cd9acbe519b44a0a9/wheel-0.32.3-py2.py3-none-any.whl Installing collected packages: pip, setuptools, wheel Successfully installed pip-18.1 setuptools-40.6.3 wheel-0.32.3
Now we have
pip installed, let's try again installing
myenv/bin/python3 -mpip install requests
requests will be installed without problem. Let's check it get installed into the correct location:-
myenv/bin/python3 >>> import requests >>> requests <module 'requests' from '/Users/kamal/myenv/lib/python3.6/site-packages/requests/__init__.py'>
It's correct, we have a fully functioning virtualenv now!
Astute reader might noticed that all this far, we have been invoking our python as
myenv/bin/python3. What if we want to invoke it simply as
python3? In virtualenv, there's a concept of
activate that you do after you created the new env. This basically just adding the newly created env directory to the
PATH environment variable, which is a list of search path used by the OS shell to find where the particular program exists. The
activate script basically look like this:-
deactivate, we replace
PATH back to
Alternatively, we can also start a subshell with PATH containing the new PATH:-
And to deactivate, we simply
exit from the subshell. Personally I don't recommend this "activate" thing. I prefer to invoke the virtualenv interpreter directly, using it full path.