pex — python executables

P ython EX ecutables are awesome , they bring everything you need to run your code with them. Just like virtualenv used to do but without the need of having to initialize a environment or doing that pip path import hack I recently started using more often (as I didn't want to have to distribute an entire environment)

In a nutshell, you install all your requirements with

pip install -t .pip -r requirements.txt

All modules will be installed into .pip which you can then use when you add the path to your PYTHONPATH. You can do that inside your app/script by doing the following

#!/usr/bin/env python

import sys
import os

sys.path.append(sys.path.append(os.path.abspath('.pip')))

import your.modules.from.requirements.txt

So, above isn’t too bad, and you can relatively easy distribute the .pip directory without needing an entire environment and you also won't need to install any requirements onto the destination machine or container (or environment).

Doing the above isn’t bad, but we can do better! Here is where PEX comes in, a python executable that one builds either by using pants (python ant) or by using the pex module. In this example, we'll be using the pex module.

Install pex

$ (sudo) pip install pex

Yep, that’s all you have to do to install pex ;)

Building a .pex file

So, unlike virtualenv and the .pip hack, you cannot easily just add a python script and execute it as part of the .pex file as you (or I) may have expected.

To build a .pex file that functions just like virtualenv or .pip with all our requirements installed, you can do the following short steps, ensure you have installed pex (see above) first.

$ pex module1 module2 module3 -o myenv.pex

To use this myenv.pex (which now has all your required modules available), simply run

$ ./myenv.pex

By default, this will just spin up python itself, then you can import the modules you’ve added to the .pex file or, to run a script, do a simple

$ ./myenv.pex script.py

Using a requirements.txt file may be easier to create your environment though

$ pex -r requirements.txt -o myenv.pex

Running your script inside the .pex, not as a script

So, it turns out you can run your script inside the .pex file, it’s just simply a tiny bit more work than one would like. Isn’t that great!? Think of a go static binary, but it's python.

So, let’s assume you’re wanting to run a script that prints Hello! when executed, obviously that wouldn't need its own environment, etc. but it's just an exercise.

__init__.py, hello.py & setup.py

First, we need to create the necessary directory structure and files to make this a installable python module/package. For this, we will need the following

./hello/__init__.py ./hello/setup.py ./hello/hello.py

__init__.py can just be empty - yay!

setup.py does need some two lines of boilerplate

from distutils.core import setup
setup( name='hello', version='0.0.1', py_modules=['hello'] )

hello.py will just do what we ask it to - print hello

def hello(): 
 print "hello!"

Yep, that’s all we need — great isn’t it?

Build the .pex file

From outside the “hello” module/package/path, run pex to create the pex

$ pex hello -e hello:hello -o hello.pex

This will install the “hello” module that we created into the .pex file, the -e hello:hello declares an entrypoint which will load the module "hello" and execute the function "hello" when the .pex is being run.

Final result

Output

vagrant@local:/vagrant$ pex hello -e hello:hello -o hello.pex

vagrant@local:/vagrant$ ls
 hello 
 hello.pex

vagrant@local:/vagrant$ ./hello.pex
hello!

File / Dir size comparison

• .pex file with ansible installed : 3 MB
• .pip dir with ansible installed : 17 MB
• virtualenv with ansible installed : 28 MB

---