Dynamically Managing Dependencies In Your Python Projects

In today's software development landscape, ensuring successful project deployment can be challenging due to the variety of environments programmers use to build their projects. Each environment presents unique challenges to project deployment from Windows to ArchLinux, Mac to Ubuntu.

One major challenge developers face is conflicting environment dependencies, which can cause project failure when deployed. Although Docker solves this problem, its complex setup requirements make it inaccessible to many developers.

This tutorial offers a solution by demonstrating how to dynamically install Python dependencies based on the environment in which the project will run.

Prerequisites

With only a basic understanding of Python and Git, any developer can follow the methods outlined in this tutorial to improve their project's deployment success.
The only requirement to have installed for this tutorial is Python. To install Python, visit their official downloads page.

What is the problem?

When using Postgres database with Django, psycopg2 database adapter is a required dependency for this to work. But there is a problem, psycopg2 will install successfully on a Windows environment but won't on a Linux environment, you will need to install psycopg2-binary instead.

Well, if your development and deployment environment is similar or you use Docker, you won't have a problem with this situation. The problem occurs when you have differing environments or multiple collaborators working on the project with you. You don't want them editing the requirements.txt file so as not to cause git conflicts.

How can this problem be solved?

Two major methods will be shared today to help you dynamically install Python dependencies based on your environment or Python version.

Conditional pip requirements syntax

Suppose that the dependencies listed in your requirements.txt file are as follows:

requests==2.26.0
six==1.16.0
urllib3==1.26.7
psycopg2==2.9.6

That can be modified to ensure that when you install the requirements on a Linux environment, the required database adapter for Linux will be installed.

requests==2.26.0
six==1.16.0
urllib3==1.26.7
psycopg2-binary==2.9.3; sys_platform == "linux"
psycopg2==2.9.6; sys_platform == "win32"

You can also program the dependencies to be installed based on the Python version installed on the environment.

requests==2.26.0; python_version > '3.0'
requests==2.26.0; python_version == '2.6'
six==1.16.0
urllib3==1.26.7
psycopg2-binary==2.9.3; sys_platform == "linux"
psycopg2==2.9.6; sys_platform == "win32"

They are defined in PEP 508 and PEP 0345 (Environment Markers), but the syntax appears to follow the draft PEP 0496.

To know all the possible values for sys_platform, visit this StackOverflow answer.

Now when you run pip install -r requirements.txt, depending on your environment, the appropriate dependency and version will be installed.

This is the easiest and recommended method for dynamically installing dependencies based on the environment or Python version.

Python install script

You could create an install.py script to install dependencies based on the environment or Python version.

• Create a file install.py in your project's root directory
  

  import pip
  
  _all_ = (
      "requests==2.26.0",
      "six==1.16.0",
      "urllib3==1.26.7",
  )  # All platforms
  
  windows = ("psycopg2==2.9.6",)  # Windows
  
  linux = ("psycopg2-binary==2.9.3",)  # Linux
  
  darwin = []  # MacOS
  
  def install(packages):
      print("Installing packages: {}".format(packages))
      for package in packages:
          pip.main(['install', package])
  
  if __name__ == '__main__':
  
      from sys import platform
  
      print("Platform: {}".format(platform))
  
      install(_all_)
  
      if platform == 'win32':
          install(windows)
      if platform.startswith('linux'):
          install(linux)
      if platform == 'darwin':
          install(darwin)
  
      print("Completed installation of packages.")
  

This script works by checking what platform the program is running on using the Python built-in sys package. So it installs the defined dependencies for that platform by calling pip by script. This method is more complex and time-consuming, but it gives you more control and flexibility on the program control flow.

• Run the program to install the packages
  

  python install.py
  

Conclusion

In conclusion, conflicting environment dependencies can be a huge headache for software developers when deploying their projects. Fortunately, several solutions are available, including Docker and the dynamic installation of dependencies based on the environment. In this article, we have provided two different methods for dynamically installing Python dependencies based on the environment, which can help avoid conflicts and ensure the project runs smoothly across different platforms. These methods allow developers to save time, reduce errors, and focus on building great software. I hope that this article has been helpful in providing useful tips and tricks for managing environment dependencies in your Python projects.

Thanks for reading, I'd also love to connect with you at Twitter | LinkedIn | GitHub | Portfolio. 🤓