DEV Community

Cover image for Deploying Python projects to air-gapped systems
borisuu
borisuu

Posted on • Edited on

Deploying Python projects to air-gapped systems

#python #programming #tutorial #learning

Motivation

A lot of times in the real world we have to deploy Python projects behind a company firewall, restricted systems or even an air-gapped system. This is most certainly a big hassle for most people.
I've found a pretty good setup which allows us to develop using modern tooling like poetry, but also deploy to any system without the need to pull any packages from the internet.

Project setup

The project is setup as follows

airgap/
├── airgap/
│   ├── __init__.py
│   └── main.py
├── poetry.lock
├── pyproject.toml
├── README.md
Enter fullscreen mode Exit fullscreen mode

and we're pulling some dependencies into our project (I've used arrow which is an awesome date and time manipulation library):

[tool.poetry]
name = "airgap"
version = "0.1.0"
description = "A project running on an air-gapped system."
authors = ["..."]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
arrow = "^1.3.0"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Enter fullscreen mode Exit fullscreen mode

We can have as many dependencies as we want, but for demonstration purposes I'll keep it short.

Preparation before distribution

We'll need to generate a requirements.txt file for our next steps. Let's use poetry to do that:

EDIT 2025-03-06: poetry doesn't ship with the export command by default any more. You'll need to install a plugin for that:

poetry self add poetry-plugin-export
Enter fullscreen mode Exit fullscreen mode

There are also other options to do that, so make sure to check out the docs on the export command

poetry export -f requirements.txt -o requirements.txt
Enter fullscreen mode Exit fullscreen mode

Command breakdown:

  • export the command used by poetry to convert the pyproject.toml file
  • -f: the format to use, only contstraints.txt and requirements.txt supported
  • -o: the name of the output file

After running you'll get a requirements.txt similar to this one (except the hashes which I've omitted for brevity):

arrow==1.3.0 ; python_version >= "3.10" and python_version < "4.0"
python-dateutil==2.8.2 ; python_version >= "3.10" and python_version < "4.0"
six==1.16.0 ; python_version >= "3.10" and python_version < "4.0"
types-python-dateutil==2.8.19.14 ; python_version >= "3.10" and python_version < "4.0"
Enter fullscreen mode Exit fullscreen mode

Now comes the funky part. We'll create wheels from these dependencies and store them locally. We'll use pip to get the wheels:

pip wheel --no-deps --wheel-dir ./wheels -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Command breakdown:

  • wheel: the pip command used to generate wheels docs
  • --no-deps: do not install dependencies (we've already got them all in the requirements.txt)
  • --wheel-dir: the directory to store the wheels (doesn't have to exist, it will be created for you)
  • -r: which requirements file to use

Our requirements.txt includes hashes, which means that pip wheel will imply the --require-hashes option. In turn this will verify the packages when installing.

Now our directory wheels will hold four packages (the ones from the requirements.txt). The final part is our own package source. We'll use poetry to build the wheel and place it in the wheels directory.

poetry build && mv dist/*.whl ./wheels
Enter fullscreen mode Exit fullscreen mode

Command breakdown:

  • build: the poetry command which builds our project
  • mv: move files
  • dist/*.whl: source all files ending in .whl in the dist/ directory
  • ./wheels: the destination for the sourced files

At this point we're done with the preparation. Just zip the ./wheels directory and distribute it to your air-gapped server. We'll just create a zip of the files:

zip -r airgap-dist-0.1.0.zip ./wheels/*
Enter fullscreen mode Exit fullscreen mode

Command breakdown:

  • -r: compress the archive
  • iargap-dist-0.1.0.zip: name of the compressed archive
  • ./wheels/*: the sources to put in the archive

Deployment

Get your zip to your air-gapped system, unpack it and install. The best-practice I stick to is to use the /opt directory to deploy custom software. Let's put it there:

mkdir /opt/airgap
unzip airgap-dist-0.1.0.zip -d /opt/airgap
Enter fullscreen mode Exit fullscreen mode

Note: you might have to run the mkdir command using sudo. It's very dependant on your setup, user, OS version etc.

Just as an example you might need to do this:

sudo mkdir /opt/airgap
sudo chown youruser:yourgroup /opt/airgap
sudo chmod 755 /opt/airgap
Enter fullscreen mode Exit fullscreen mode

Now in general I wouldn't advocate to install any project in the global Python site, but since we're striving for simplicity we'll do that. The next step is very easy, install our wheels:

pip install --no-cache /opt/airgap/wheels/*
Enter fullscreen mode Exit fullscreen mode

Bonus

Just in case you want to use venv you can do the following:

python -m venv /opt/airgap/venv
Enter fullscreen mode Exit fullscreen mode

Command breakdown:

  • -m venv: tell Python to use the module venv (should be installed by default)
  • /opt/airgap/venv: the location where to create the virtual environment, note that the directory name venv is just a convention, but you can use any name you like.

Activate the newly created virtual environment:

source /opt/airgap/venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

And then do the install:

# verify you're using the venv pip
which pip
$ /opt/airgap/venv/bin/pip

# install your packages
pip install --no-cache /opt/airgap/wheels/*
Enter fullscreen mode Exit fullscreen mode

Note: to exit the virtual environment you can run deactivate in your terminal. It's a command automatically available as soon as you activate a virtual environment

Conclusion

We've seen how to extract the dependencies from our new pyproject.toml file, build wheels and zip them up for distribution. Consequently how we can deploy our project to our air-gapped system. Happy coding!

Top comments (2)

Collapse
 
allen_kennedy_a30bd6dfa08 profile image
Allen Kennedy •

I feel like you got 90% of the way then fell on your face.
How do I run the program after all this?

Collapse
 
borisuu profile image
borisuu •

It depends what kind of application you're deploying.

A simple example would be to activate the venv and run your main in the background

source /opt/airgap/venv/bin/activate
# run once
python -m airgap.main

# run in the background
python -m airgap.main &
Enter fullscreen mode Exit fullscreen mode

That's assuming you've got a if __name__ == '__main__' block in your main.py file.

You'll notice that I didn't discuss the internals of the application at all, as the variations are endless. If you have a more specific use-case, I'd be glad to take a look.