DEV Community

Python-T Point
Python-T Point

Posted on • Originally published at pythontpoint.in

⚙️ Terraform create AWS EC2 instance with Python environment

#kubernetes #cloud #devops #tutorial

Terraform can provision an AWS EC2 instance and set up a Python virtual environment in a single, reproducible run — the whole workflow is declarative and version‑controlled.

📑 Table of Contents

  • 💻 Terraform — How to Provision an EC2 Instance
  • 🔧 AWS Provider — Configuring Credentials
  • 🐍 Python Environment — Setting up a Virtualenv on the Instance
  • 📦 Installing Python and venv
  • 📦 Activating and Using the Environment
  • 📦 User Data — Automating Installation with Terraform
  • 🟩 Final Thoughts
  • ❓ Frequently Asked Questions
  • How do I store the Terraform state securely?
  • Can I use a different Linux distribution for the EC2 instance?
  • Is it possible to attach an Elastic IP to the instance?
  • 📚 References & Further Reading

💻 Terraform — How to Provision an EC2 Instance

A Terraform configuration file describes the desired state of AWS resources; applying it makes the real cloud match that state.

First, install Terraform (version 1.5.0 or newer). The binary is a single executable, so the operating system loads it directly into memory and the process performs HTTP requests to AWS endpoints.

$ terraform version
Terraform v1.5.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v5.12.0
Enter fullscreen mode Exit fullscreen mode

Next, create a main.tf that declares an aws_instance resource. The provider block authenticates with AWS using either environment variables or a shared credentials file.

# main.tf
terraform { required_version = ">= 1.5.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.12" } }
} provider "aws" { region = "us-east-1"
} resource "aws_instance" "app_server" { ami = "ami-0c02fb55956c7d316" # Amazon Linux 2 instance_type = "t3.micro" # User data will be defined later user_data = data.template_file.init.rendered tags = { Name = "terraform-ec2-python" }
}
Enter fullscreen mode Exit fullscreen mode

Running terraform init contacts the provider registry, downloads the provider plugin, and stores it under .terraform. The generated .terraform.lock.hcl file records exact plugin checksums, guaranteeing that subsequent runs use the same binary.

$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.12"...
- Installing hashicorp/aws v5.12.0...
- Installed hashicorp/aws v5.12.0 (signed by HashiCorp)
Terraform has been successfully initialized!
Enter fullscreen mode Exit fullscreen mode

When terraform apply is invoked, Terraform first builds a directed acyclic graph of resource dependencies. The graph ensures, for example, that a VPC is created before the EC2 instance that depends on it. After the plan is approved, Terraform issues a CreateInstance API call; AWS returns an instance ID that Terraform records in terraform.tfstate.

$ terraform apply -auto-approve
aws_instance.app_server: Creating...
aws_instance.app_server: Still creating... [10s elapsed]
aws_instance.app_server: Creation complete after 13s [id=i-0abcd1234ef567890] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Enter fullscreen mode Exit fullscreen mode

Key point: Terraform’s declarative model guarantees that the EC2 instance shown in the console matches exactly what is described in HCL, and the state file provides a reliable history of changes.

🔧 AWS Provider — Configuring Credentials

The AWS provider authenticates using the standard AWS SDK credential chain, which includes environment variables, shared credentials files, and EC2 instance profiles.

Set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables before running Terraform. The SDK reads these values directly from the process environment, avoiding file I/O on each request.

$ export AWS_ACCESS_KEY_ID=AKIAEXAMPLE
$ export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENGbPxRfiCYEXAMPLEKEY
$ env | grep AWS_
AWS_ACCESS_KEY_ID=AKIAEXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENGbPxRfiCYEXAMPLEKEY
Enter fullscreen mode Exit fullscreen mode

Alternatively, store the keys in ~/.aws/credentials. Terraform reads the same file the AWS CLI uses, ensuring consistency across tools.

$ cat ~/.aws/credentials
[default]
aws_access_key_id = AKIAEXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENGbPxRfiCYEXAMPLEKEY
Enter fullscreen mode Exit fullscreen mode

According to the official Terraform AWS Provider documentation, the provider automatically picks up these credentials and validates them before any resource operations, emitting an error if the keys are missing or malformed. (Also read: 🚀 Deploy Flask App AWS Free Tier — Easy EC2 & Nginx Setup)

$ terraform plan
Error: Failed to configure provider "aws": No valid credential sources found for AWS Provider.
Enter fullscreen mode Exit fullscreen mode

Using an IAM role attached to the instance (via an instance profile) eliminates static keys, which improves security by rotating credentials automatically.

🐍 Python Environment — Setting up a Virtualenv on the Instance

A Python virtual environment isolates the interpreter and site‑packages directory, preventing dependency conflicts between projects.

📦 Installing Python and venv

Amazon Linux 2 ships with Python 3.7, but the python3-venv package is not installed by default. The user‑data script runs yum, which resolves package dependencies via the RPM database before invoking rpm to unpack the files. (More onPythonTPoint tutorials)

$ sudo yum install -y python3 python3-venv
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Resolving Dependencies -> Running transaction check --> Package python3.x86_64 0:3.7.9-1.amzn2 will be installed --> Package python3-venv.x86_64 0:3.7.9-1.amzn2 will be installed
...
Installed: python3-3.7.9-1.amzn2.x86_64 python3-venv-3.7.9-1.amzn2.x86_64 Complete!
Enter fullscreen mode Exit fullscreen mode

After installing, create a directory for the virtual environment and invoke the venv module. The command spawns a new process that copies the Python binary and creates a lib directory containing symlinks to the standard library.

$ python3 -m venv /home/ec2-user/pyenv
Created virtual environment in /home/ec2-user/pyenv
Enter fullscreen mode Exit fullscreen mode

📦 Activating and Using the Environment

Activating the environment prepends its bin directory to $PATH, so subsequent python and pip invocations resolve to the copies inside the virtualenv.

$ source /home/ec2-user/pyenv/bin/activate
(pyenv) $ python -version
Python 3.7.9
(pyenv) $ pip install requests
Collecting requests Downloading https://files.pythonhosted.org/packages/6b/47/.../requests-2.28.1-py3-none-any.whl (62 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2022.12.7 charset-normalizer-3.1.0 idna-3.4 requests-2.28.1 urllib3-1.26.15
Enter fullscreen mode Exit fullscreen mode

Key point: The virtual environment guarantees that any Python packages installed later, including those required by your application, will not affect the system Python or other users on the same instance.

📦 User Data — Automating Installation with Terraform

EC2 user‑data runs a shell script exactly once, during the first boot; the script runs as root and can perform any provisioning steps.

Define a template_file data source that renders a Bash script. Terraform substitutes variables at plan time, so the final script contains the exact instance ID and region.

data "template_file" "init" { template = file("${path.module}/user_data.sh") vars = { instance_name = aws_instance.app_server.tags["Name"] }
}
Enter fullscreen mode Exit fullscreen mode

The user_data.sh script installs Python, creates a virtual environment, and writes a small app.py that prints a greeting. Because the script runs before any user logs in, the environment is ready for immediate SSH access.

#!/bin/bash
set -e # Update packages
yum update -y # Install Python and venv
yum install -y python3 python3-venv # Create a virtualenv for the app
python3 -m venv /home/ec2-user/appenv
source /home/ec2-user/appenv/bin/activate # Install a sample package
pip install flask # Write a simple Flask app
cat >> /home/ec2-user/app.py <<EOF
from flask import Flask
app = Flask(__name__) @app.route('/')
def hello(): return "Hello from Terraform-provisioned EC2!" if __name__ == '__main__': app.run(host='0.0.0.0')
EOF # Start the app in the background
nohup python /home/ec2-user/app.py > /home/ec2-user/app.log 2>&1 & # Signal completion
echo "User data completed"
Enter fullscreen mode Exit fullscreen mode

When the instance boots, the EC2 hypervisor injects the script into /var/lib/cloud/instances/…/user-data.txt. The cloud‑init daemon reads the file, executes it, and records the exit status. A non‑zero exit code marks the instance as failed in the console.

$ sudo cloud-init status -long
status: done
detail: Cloud-init v. 22.2-1.el8 running 'modules:final' ... finished.
Enter fullscreen mode Exit fullscreen mode

After the instance is running, verify that the Flask server listens on port 80.

$ curl http://$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4)/
Hello from Terraform-provisioned EC2!
Enter fullscreen mode Exit fullscreen mode

Key point: Embedding the entire Python environment setup in user‑data makes the instance immutable — any replacement instance will execute the same script, guaranteeing identical runtime behavior.

🟩 Final Thoughts

By combining Terraform’s declarative resource model with EC2 user‑data, you can automate the full lifecycle of a Python application host, from instance creation to package installation, without manual SSH steps. The resulting workflow is reproducible, version‑controlled, and easily extensible: add more aws_security_group rules, swap the AMI, or integrate a load balancer, and Terraform will orchestrate the changes safely.

For developers, the practical benefit is a single source of truth that describes both infrastructure and runtime dependencies. When a new environment is needed for testing or production scaling, a single terraform apply command produces a ready‑to‑run Python virtual environment on the new EC2 instance.

❓ Frequently Asked Questions

How do I store the Terraform state securely?

Use an S3 bucket with server‑side encryption and enable DynamoDB locking. The backend block in terraform {} configures this automatically, preventing concurrent writes and ensuring durability.

Can I use a different Linux distribution for the EC2 instance?

Yes. Replace the AMI ID in the aws_instance resource with an Ubuntu or Amazon Linux image. Ensure the user‑data script matches the package manager (apt for Ubuntu, yum for Amazon Linux).

Is it possible to attach an Elastic IP to the instance?

Define an aws_eip resource and associate it with the instance ID. Terraform will allocate the address, attach it, and update the state so subsequent runs keep the same Elastic IP.

📚 References & Further Reading

  • Official Terraform AWS Provider docs — detailed description of all supported arguments: developer.hashicorp.com
  • AWS EC2 User Data guide — explains how cloud‑init processes the script: docs.aws.amazon.com
  • Python virtual environments documentation — official guide to venv and isolation: docs.python.org

Top comments (0)