Credentials substitution at runtime in Python

Introduction

Often we hear that credentials should not be hardcoded in the code or Version Control (like git). You can leverage the dynamic substitution of such credentials using Environment variables tied to the code as a variable.

Pre-requisite

• Python 3.8 or up
• PyYaml - To read YAML configuration file

Steps

• Create a yaml file as the following

# credentials.yml

aws:
  access_key_id: "${AWS_ACCESS_KEY_ID}"
  secret_access_key: "${AWS_SECRET_ACCESS_KEY}"

• We will read this yaml file and get the python dictionary object

# main.py
from typing import Dict
import yaml

def read_credentials_file() -> Dict:
  try:
    with open("credentials.yml", mode="r", encoding="utf-8") as cred_file:
      return yaml.safe_load(cred_file)
  except yaml.YAMLError as yaml_err:
    print(f"Unable to read the file. Error: {yaml_err}")
    raise

• We will use the string.Template from the standard library to substitute the value from the environment variables

# main.py
import string
import os

def substitute_creds(creds: Dict) -> None:
  creds["aws"]["access_key_id"] = string.Template(
    creds["aws"]["access_key_id"]
  ).substitute(
    {"AWS_ACCESS_KEY_ID": os.getenv("AWS_ACCESS_KEY_ID")}
  )

  creds["aws"]["secret_access_key"] = string.Template(
    creds["aws"]["secret_access_key"]
  ).substitute(
    {"AWS_SECRET_ACCESS_KEY": os.getenv("AWS_SECRET_ACCESS_KEY")}
  )

• Add the following two variables as part of the environment. They can be added in the .env file as well

$ export AWS_ACCESS_KEY_ID="abcd1234"
$ export AWS_SECRET_ACCESS_KEY="secret123"

Complete Program

# main.py
from typing import Dict
import string
import yaml
import os

def read_credentials_file() -> Dict:
  try:
    with open("credentials.yml", mode="r", encoding="utf-8") as cred_file:
      return yaml.safe_load(cred_file)
  except yaml.YAMLError as yaml_err:
    print(f"Unable to read the file. Error: {yaml_err}")
    raise

def substitute_creds(creds: Dict) -> None:
  creds["aws"]["access_key_id"] = string.Template(
    creds["aws"]["access_key_id"]
  ).substitute(
    {"AWS_ACCESS_KEY_ID": os.getenv("AWS_ACCESS_KEY_ID")}
  )

  creds["aws"]["secret_access_key"] = string.Template(
    creds["aws"]["secret_access_key"]
  ).substitute(
    {"AWS_SECRET_ACCESS_KEY": os.getenv("AWS_SECRET_ACCESS_KEY")}
  )

if __name__ == "__main__":
  credentials = read_credentials_file()
  substitute_creds(credentials)

  # WARNING: Never print or log credentials or any sensitive information
  print(f"Credentials: {credentials}")

• The above code when run should produce the following output

$ python3 main.py
Credentials: {"aws": {"access_key_id": "abcd1234", "secret_access_key": "secret123"}}