DevOps 101: Packaging Software

#devops #programming #productivity #discuss

Packaging code is a key step in the software development life cycle. It is the process of taking source code, installing any dependencies that are required and converting it to code that can be executed.

Anyone who develops and maintains software will need to package a new version of their code any time the source code or its dependencies change. Doing this on a frequent basis will need to automate the process, so how do we do it?

Packaging with Shell Scripts

Firstly, we need to start with packaging our code. The most common way to package software is using shell commands (BASH or ZSH) on Linux. Most server environments run Linux and we can run them on Mac directly or on Windows with Windows Subsystem for Linux (WSL). Using these commands we need to copy our code into an empty directory, install our dependencies and run our unit tests. An example package script for a Python application could look like this:

#!/usr/bin/env bash

set -e

# Make sure our distribution directory exists
mkdir -p ./dist/

# Cleanup any old files from previous packaging
rm -rf ./dist/*

# Install any dependencies listed in requirements
python3 -m pip install -r ./requirements.txt -t ./dist/

# Copy our source code into our distribution
cp -rf ./src/* ./dist/

( cd ./dist/; python3 -m unit_test)

By automating our packaging process with a simple readable script like this we can constantly package and test our code. We can also add and modify steps within the process very easily to use any language specific tooling or testing frameworks.

Where to Run Our Script

Packaging software can be done locally or on remote build agents. It is a good idea to have the flexibility to do either so that developers can locally work on their code whilst still being able to deliver changes with central tooling for transparency and traceability.

To package our code locally we can simply execute the script from a compatible terminal. To run the script in a central tool like GitLab we can add a CICD configuration file that runs the script whenever we commit changes.

Testing Our Code

Unit testing should be done as part of our packaging process in order to maintain confidence that our software works or to get fast feedback if it doesn't. These tests will validate that our code executes and provides the expected responses based on our inputs. They don't require configuration to integrate with other systems and should be able to run without connecting to other services.

We can run our unit tests before installing dependencies or after packaging our files, depending on how much our code integrates with other libraries. An example set of unit tests that runs within a Python module might look like this:

#!/usr/bin/env python3

from . import add

print("Unit Testing Mathematics")
assert add(1, 1) == 2, "Failed addition test A001"

Storing Our Package

Once we have tested our package we can upload it to a central storage location such as AWS S3, Google Cloud Storage, Azure Blob Storage or an FTP server. This will give us access to the working software that is ready for configuration and deployment without needing to repackage it. This is highly specific to the location of the upload but the services will usually provide an appropriate cli tool that allows us to handle this upload.

Summary

Packaging code is the first step in this process of delviering software and should be automated to that we can quickly and effectively deliver working software to our customers on a frequent basis. By choosing flexible tools and including testing as part of the process we can speed up software development whilst keeping the process as simple as clicking a button.

After packaging our code we need to configure it for deployment and release. In an upcoming article we will discuss the process of automating configuration and how it can be done without relying on complex tools or repetitive manual work.

Follow me here for more content or contact me: