Are you managing multiple requirements files for your Python project? Are you copy-pasting rows from one file to the other when changing versions? Maybe you have one requirement file for development and one for production. Or maybe a separate file for that docker image. Fear no more! Here's how to do it.
Starting point
Let's jump straight into an example. In this application, I have two requirements files, one for production install, and one for development.
requirements.txt:
requests>=2.25.0
pandas>=1.1.0
requirements-dev.txt:
requests>=2.25.0
pandas>=1.1.0
pytest>=6.1.2
responses==0.12.1
As you can see we have duplicated the requests and pandas rows. This can lead to trouble when updating dependencies. Forgetting to update requirements.txt could, in the worst-case scenario, lead to downtime in production.
There must be a better way? -- Raymond Hettinger
Yes, there are at least two better ways!
Multiple -r Flags when Installing
You can actually use multiple files when running the pip install command. First, let's edit the files.
requirements.txt:
requests>=2.25.0
pandas>=1.1.0
requirements-dev.txt:
pytest>=6.1.2
responses==0.12.1
Now each dependency is just defined once! To install the full development environment we would run:
$ pip install -r requirements.txt -r requirements-dev.txt
While this works and is better than before, it still doesn't really feel right. This is will probably trip people up when joining the project. Let's look at the final and best way to solve this!
requirements.txt in requirements-dev.txt
When installing with pip we use the -r flag to refer to a requirements file. We can do this inside another requirements file as well.
requirements.txt:
requests>=2.25.0
pandas>=1.1.0
requirements-dev.txt:
-r requirements.txt
pytest>=6.1.2
responses==0.12.1
Now you can install all development dependencies using only: pip install -r requirements-dev.txt. To install the production environment you still use pip install -r requirements.txt.
Top comments (2)
Really useful, you probably saved lives (and jobs ...)
Glad you found it useful 🙂