Python Environment Variables (Env Vars): A Primer

Max — Thu, 22 Aug 2024 13:58:21 +0000

Python developers often deal with data they don't want anyone else to see, like API keys, API secrets, database names, etc.

One way programmers store these secrets is in environment variables. In this article, you will learn everything you need to know about using environment variables in Python. We’ll show you how to set Python environment variables, how to get them, and the different ways you can keep all of your secrets safe. We’ll finish with a real-life example of how we use them here at Vonage.

Use the links below to skip ahead in the article:

What are Environment Variables?

Environment variables (sometimes called "env vars") are variables you store outside your program that can affect how it runs. For example, you can set environment variables that contain the key and secret for an API. Your program might then use those variables when it connects to the API.

Storing your secrets in your environment instead of your source code has a few advantages:

Security: Environment variables help you safeguard sensitive data like API keys and passwords. For example, you may not want whoever downloads your source code access to an API key you are using.
Configuration Management: With environment variables, it’s easy to store and manage configuration settings.
Portability: You can adapt applications to different environments without code changes. For example, if your code behaves differently on different operating systems, you can use an environment variable to manage this.
Testing and Debugging: Developers can simplify testing and debugging by adjusting environment variables rather than changing the code.

In essence, environment variables allow you to change your program’s behavior without changing the program itself.

You can store environment variables in your operating system, but there are other ways to use them that we will learn about shortly.

How to Use Environment Variables in Python

To follow along with this tutorial, you’ll need to have Python installed on your machine. You can install Python by downloading it from the official Python website.

You get and set environment variables in Python using the built-in os module.

How to Get Python Environment Variables

You can view all of the environment variables in your program by saving the following code in a Python file and then running this Python program:

Your Python interpreter should print out all your operating system's environment variables when running this code. You can access the different environment variables in os.environ like a Python dictionary. Here are two ways to get environment variables with Python:

The last two lines in your Python code above do the same thing: both get the USER environment variable from your operating system. However, when you use the first way, Python throws an exception if it does not find the variable.

A good practice is to use os.environ['MY_ENVIRONMENT_VARIABLE'] if the environment variable is required for your Python application to run and os.environ.get('MY_ENVIRONMENT_VARIABLE') if it is optional.

How to Set Environment Variables

To set environment variables in Python, you can add them to the os.environ object like you would with a dictionary. However, only strings are permitted, as these are passed directly to the shell your interpreter is running in.

To update an environment variable, simply override it in the exact same way:

How to Store Python Environment Variables (5 Methods)

There are different reasons that you might want to store environment variables, so there are also different ways to store them. Sometimes, you just need something to be set on your local machine, whereas in other cases, you might need to run your application in production - these use cases require different approaches!

Below, we’ll show you five ways to store environment variables for Python:

Operating System Storage
File Storage
Cloud Storage
Universal Secret Manager Storage
CI/CD System Storage

Store Environment Variables on Your Operating System

Sometimes, you don’t want to create an environment variable through Python if you just want to set something quickly. Luckily, this can be done through the command line. The examples I’ll give here work for Unix-like systems (Mac, Linux, etc.), but if you’re using Windows, you can learn how to get and set environment variables in this tutorial.

In the previous example, USER was an environment variable set by your operating system, representing who is using your computer. Although your operating system creates this variable automatically, you can also create your own environment variables.

You can see all the environment variables from your command line by opening up a command line and typing the following (on a Unix-like system):

export

This will give you a list of every environment variable that your command-line shell has access to.

Here’s how to create an environment variable on your command line:

export VONAGE_API=your_api

This creates a variable called VONAGE_API and sets its value to your_api. You can print any variable’s value like this:

echo $VONAGE_API

When you run the code above, you’ll see your_api as the output.

Store Environment Variables in Files

When you create a new environment variable using your terminal/command line, it only exists for that session. When you close your terminal, the environment variable no longer exists. Often when you’re programming, you want your environment variables to persist so they can be used every time you run your code. One way to accomplish this is to store them in a file: for example, a .env file.

Let’s create an example project to demonstrate how to use a .env file to store environment variables.

First, use your terminal to create a new folder for this tutorial, move into it and create a .env file inside:

mkdir env_variables_tutorial cd env_variables_tutorial touch .env

Add this line to your .env file:

VONAGE_API=your_api

python3 -m venv venv . ./venv/bin/activate

Now we’re inside a Python Virtual Environment, we can install the package we need.

pip install python-dotenv

Now, we can use the dotenv module from this package to load environment variables from the .env file to the environment Python can access with the os module. Create a Python file with the following content:

The load_dotenv function brings environment variables from the .env file into os.environ, which can then be used like any other environment variables set by your operating system.

How to Store Environment Variables in the Cloud

When you create software for production, you probably won't run it from your computer. Instead, you most likely will run your code on a server.

That means you need to know how to set and get environment variables from wherever you run your code in production.

Here is a list of cloud providers and where you can get more information about dealing with environment variables using them:

Store Environment Variables in CI/CD systems

If you use cloud-based CI/CD systems such as GitHub Actions, CircleCI, Travis, or Jenkins, you can also store environment variables in their systems.

If you’re using GitHub for your project, you can store environment variables in the settings for your repo by navigating to the "Settings" tab and setting API keys etc., in the "Secrets and variables" setting under "Security".

Here, you can create a new repository secret by clicking the option and setting values. This will now be available in your GitHub Actions runs.

There’s lots of information about setting up environment variables with GitHub Actions, as well as CircleCI, Travis and Jenkins, as well as information for other providers.

Store Environment Variables with a Universal Secrets Manager

Storing your secrets in a .env file persists your environment variables but does have some problems.

For example, say you are on a team with ten people. Everyone is tracking their secrets in .env files, and one of the secrets changes (say you get a new API key). In that case, ten people all have to update their .env file, which is not very efficient.

Or, what if you decide to switch from Heroku to AWS? In that case, you will have to learn how to deal with secrets on a new platform, which requires extra work.

To solve these problems, some programmers use a universal secrets manager like Doppler. A universal secrets manager allows you to store your secrets in one place, so everyone on your team can access them.

With a universal secrets manager, your secrets are independent of your local machine or a cloud provider, so you can bring them with you no matter where you run your code.

Bonus: Using Environment Variables with External APIs

Sometimes, if you’re using an external API’s sample project or sample code snippets, tooling is provided by the owner/maintainer to make it easier to use. Often, you’ll need to use environment variables to get the tooling set up optimally.

Here at Vonage, we make use of environment variables in our Python code samples for sending SMS, making phone calls, sending verification codes, checking fraud scores, and much more.

Let’s say you want to use Vonage’s Number Insight API to get information about some phone numbers. In this case, you might want to use a code sample like this one to get things working quickly in Python.

To use this sample code, first create a Vonage account (don’t worry, it’s free!) to get an API key and secret to authenticate your API calls.

Next, clone the repo at https://github.com/Vonage/vonage-python-code-snippets/. Once this is done, create a new Python virtual environment and install the required dependencies with

You should see information about the phone number you entered, meaning you used Vonage’s Number Insight API to look up the phone number from your environment variables, without needing to edit the Python file itself at all! This is a major advantage of using environment variables.

deactivate # if the virtual environment from earlier is still active python3 -m venv venv . ./venv/bin/activate pip install -r requirements.txt

Finally, rename the .env.dist file to .env and add your API key and secret to the file, as well as changing the value for INSIGHT_NUMBER to be the number you want to look up. Now you’re ready to run the code with

python number-insight/ni-basic.py

Final Thoughts

Getting and setting environment variables is an essential part of creating production software. If you’ve completed this tutorial, you’re now familiar with how to get and set environment variables using Python.

You also now understand your different options for storing your secrets: temporarily setting them using your OS, storing them in a .env file, keeping them in the cloud, and using a universal secrets manager. The method you use will depend on the circumstances of the project you are working on. If you want to read more, this post on using private keys in environment variables has more information.

If you want to start using Vonage APIs, you can sign up for a free developer account (with free credits!) If you have any questions about this tutorial, feel free to reach out to us on our Vonage Community Slack and ask us over there or by messaging us on X, previously known as Twitter.

Now, go ahead and try using environment variables in your own Python projects!

DEV Community: Max