P ython EX ecutables are awesome , they bring everything you need to run your code with them. Just like virtualenv used to do but without the need of having to initialize a environment or doing that pip path import hack I recently started using more often (as I didn't want to have to distribute an entire environment)
In a nutshell, you install all your requirements with
pip install -t .pip -r requirements.txt
All modules will be installed into .pip which you can then use when you add the path to your PYTHONPATH. You can do that inside your app/script by doing the following
#!/usr/bin/env python
import sys
import os
sys.path.append(sys.path.append(os.path.abspath('.pip')))
import your.modules.from.requirements.txt
So, above isn’t too bad, and you can relatively easy distribute the .pip directory without needing an entire environment and you also won't need to install any requirements onto the destination machine or container (or environment).
Doing the above isn’t bad, but we can do better! Here is where PEX comes in, a python executable that one builds either by using pants (python ant) or by using the pex module. In this example, we'll be using the pex module.
Install pex
$ (sudo) pip install pex
Yep, that’s all you have to do to install pex ;)
Building a .pex file
So, unlike virtualenv and the .pip hack, you cannot easily just add a python script and execute it as part of the .pex file as you (or I) may have expected.
To build a .pex file that functions just like virtualenv or .pip with all our requirements installed, you can do the following short steps, ensure you have installed pex (see above) first.
$ pex module1 module2 module3 -o myenv.pex
To use this myenv.pex (which now has all your required modules available), simply run
$ ./myenv.pex
By default, this will just spin up python itself, then you can import the modules you’ve added to the .pex file or, to run a script, do a simple
$ ./myenv.pex script.py
Using a requirements.txt file may be easier to create your environment though
$ pex -r requirements.txt -o myenv.pex
Running your script inside the .pex, not as a script
So, it turns out you can run your script inside the .pex file, it’s just simply a tiny bit more work than one would like. Isn’t that great!? Think of a go static binary, but it's python.
So, let’s assume you’re wanting to run a script that prints Hello! when executed, obviously that wouldn't need its own environment, etc. but it's just an exercise.
__init__.py, hello.py & setup.py
First, we need to create the necessary directory structure and files to make this a installable python module/package. For this, we will need the following
./hello/__init__.py ./hello/setup.py ./hello/hello.py
__init__.py can just be empty - yay!
setup.py does need some two lines of boilerplate
from distutils.core import setup
setup( name='hello', version='0.0.1', py_modules=['hello'] )
hello.py will just do what we ask it to - print hello
def hello():
print "hello!"
Yep, that’s all we need — great isn’t it?
Build the .pex file
From outside the “hello” module/package/path, run pex to create the pex
$ pex hello -e hello:hello -o hello.pex
This will install the “hello” module that we created into the .pex file, the -e hello:hello declares an entrypoint which will load the module "hello" and execute the function "hello" when the .pex is being run.
Final result
Output
vagrant@local:/vagrant$ pex hello -e hello:hello -o hello.pex
vagrant@local:/vagrant$ ls
hello
hello.pex
vagrant@local:/vagrant$ ./hello.pex
hello!
File / Dir size comparison
- .pex file with ansible installed : 3 MB
- .pip dir with ansible installed : 17 MB
- virtualenv with ansible installed : 28 MB
Top comments (0)