Upgrade Python Version on AWS Elastic Beanstalk using Pyenv

Ahiwe Onyebuchi Valentine — Mon, 06 Mar 2023 20:09:14 +0000

Hello there,

Another exciting article has hit.

I present to you this valuable solution where I describe some of my custom work with Elastic Beanstalk. Recently, I got a case where the pre-built SSL library readily available with the Beanstalk Python environment was too outdated for my client's needs. In this article, I share my experience and what I did to solve the problem.

Imagine that your app needs a more current Python runtime version than the one which is readily managed within your AWS Elastic Beanstalk environment.

As niche as this might sound, it is quite common for your app to require a more up-to-date Python runtime version than what is available in AWS Elastic Beanstalk. This becomes a blocker and is particularly important to you if your app's functionality depends on a library that is pre-built with the Python installation but is outdated in the Beanstalk environment.

Before you throw your hands in the air, and say, oh no!! the python version cadence is too slow for my app's needs on Elastic Beanstalk!! Well, this project provides a solution. It shows you how to deploy a Flask app on AWS Elastic Beanstalk and upgrade the server's Python version with Pyenv.

The code for the following project is documented in the attached repo: repo, and I invite you to please feel free to help yourself to the buffet. You are also most welcome to keep reading for some explanation on how to use the repo.

The repo includes a Flask application example for deployment on Elastic Beanstalk. To use it, follow these steps:

Create an Elastic Beanstalk environment using the AWS Management Console or AWS CLI.
Set environment variables by editing configuration options on the Elastic Beanstalk environment dashboard. Navigate to "Configuration" ==> "Software", then "Edit". The relevant environment variables that need to be set include:


 shell
PYTHON_VERSION: The python version that you need to be installed on the server.
PYTHONPATH: The path to the application\'s virtual environment.

The PYTHONPATH is set by default by Elastic Beanstalk to /var/app/venv/staging-LQM1lest/bin.

Deploy the new application version to the Elastic Beanstalk environment using either the AWS Management Console or eb CLI with:



eb deploy

The images below demonstrate the BEFORE and AFTER versions of the Python installation on the AWS Elastic Beanstalk server.

Python Version: 3.8.5. Before, the Python Version was upgraded.

Python Version: 3.10.0. After the Python Version has been upgraded.

The pyenv upgrade is neatly handled through the code block shown below. This block can also be found in the repo as the "00_upgrade_python.config" file in the ".ebextensions" directory. It upgrades the server's python version using Pyenv to the specified version set with the environment variables.



#!/bin/bash -e

# This should be set in the environment variables of the Elastic Beanstalk environment. e.g. 3.8.5
# You can find the available versions of python on pyenv by running the following command:
# - pyenv install --list
PYTHON_VERSION=$(/opt/elasticbeanstalk/bin/get-config environment -k PYTHON_VERSION)

# This should be set in the environment variables of the Elastic Beanstalk environment by default
# Default value: /var/app/venv/staging-LQM1lest/bin
PYTHONPATH=$(/opt/elasticbeanstalk/bin/get-config environment -k PYTHONPATH)

# Check if the PYTHON_VERSION environment variable is set
if [ -z "$PYTHON_VERSION" ]; then
    echo "PYTHON_VERSION is not set. Please set the PYTHON_VERSION environment variable."
    exit 1
fi

# Check if the PYTHONPATH environment variable is set
if [ -z "$PYTHONPATH" ]; then
  echo "PYTHONPATH is not set. Please set the PYTHONPATH environment variable."
  exit 1
fi

# Remove the /bin suffix from the PYTHONPATH environment variable using sed
PYTHONPATH=$(echo $PYTHONPATH | sed 's/\/bin$//')

# Check if the desired version of python is already installed
if [ -x /usr/bin/.pyenv/versions/$PYTHON_VERSION/bin/python3 ]; then
  echo "Python version $PYTHON_VERSION is already installed in pyenv"

  # Check if the virtual environment exists
  if [ ! -d $PYTHONPATH ]; then
    echo "Virtual environment does not exist. Creating a new virtual environment with the correct version of python"
    /usr/bin/.pyenv/versions/$PYTHON_VERSION/bin/python3 -m venv $PYTHONPATH
    exit 0
  fi

  # Get the version of python in the virtual environment
  PYTHON_VERSION_VENV=$($PYTHONPATH/bin/python3 --version)

  # Check if the version of python in the virtual environment is the same as the desired version
  if [ "$PYTHON_VERSION_VENV" == "Python $PYTHON_VERSION" ]; then
    echo "Python version $PYTHON_VERSION is already installed in the virtual environment"
    exit 0
  fi

  # Remove the old virtual environment
  rm -rf $PYTHONPATH

  # Create a new python virtual environment with the correct version of python
  /usr/bin/.pyenv/versions/$PYTHON_VERSION/bin/python3 -m venv $PYTHONPATH
  exit 0
fi

# Install the following dependencies, which are necessary to build the versions of python via pyenv:
sudo yum groupinstall -y "Development Tools"
sudo yum install -y openssl11 openssl11-libs readline-devel \
  zlib-devel bzip2 bzip2-devel sqlite sqlite-devel \
  libffi-devel xz-devel openssl11-devel

# If you run into a '**command not found**' error for --allowerasing,
# as an alternative you can use the 'yum swap' command to swap
# the installed openssl-devel package for the openssl11-devel package:
sudo yum swap openssl-devel openssl11-devel

# Install pyenv and then add the pyenv directory and binary to your shell environment:

# Set the pyenv root to a readable location
export PYENV_ROOT=/usr/bin/.pyenv

# Download the pyenv installer
curl -sL https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer -O

# Check if pyenv is not already installed
if [ ! -d $PYENV_ROOT ]; then
  bash pyenv-installer
fi

# Setup the pyenv environment variables
grep -q 'export PYENV_ROOT="/usr/bin/.pyenv"' ~/.profile || echo 'export PYENV_ROOT="/usr/bin/.pyenv"' >> ~/.profile
grep -q 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' ~/.profile || echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.profile
grep -q 'eval "$(pyenv init -)"' ~/.profile || echo 'eval "$(pyenv init -)"' >> ~/.profile
grep -q 'eval "$(pyenv virtualenv-init -)"' ~/.profile || echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.profile

# Install the desired version of python, for example, 3.8.5:
export PYENV_ROOT="/usr/bin/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
pyenv install $PYTHON_VERSION

# Enable the new version as the global:
pyenv global $PYTHON_VERSION

# Verify the version of python is indeed the version we specified in the previous command:
/usr/bin/.pyenv/versions/$PYTHON_VERSION/bin/python3 --version

# Remove the old virtual environment
rm -rf $PYTHONPATH

# Create a new python virtual environment with the correct version of python
/usr/bin/.pyenv/versions/$PYTHON_VERSION/bin/python3 -m venv $PYTHONPATH

That's it. I hope you found this read helpful, and you can use this ready-to-use solution to upgrade your python runtime versions on elastic beanstalk with no hassles.

See you in the next article!

DEV Community: Ahiwe Onyebuchi Valentine

Upgrade Python Version on AWS Elastic Beanstalk using Pyenv