Preparation
We need to prepare some tools, such as:
1.HashiCorp Packer. You can download it in here. In this article, I use Packer v1.9.4
2.AWS CLI. You can install it in here. In this article, I use aws-cli v2.8.3
3.Your Favorite IDE (Integrated Development Environment).
4.HashiCorp Terraform (Optional, if you'd like to follow along on the provisioning Jenkins EC2 Using Terraform). You can download it in here. In this article, I use Terraform v1.3.2
HashiCorp Packer Introduction
HashiCorp Packer is an open-source tool that automates the creation of any machine image for multiple platforms. Packer is no replacement for configuration management tools like Ansible, Puppet, or Chef. Packer uses these tools to install and configure software and dependencies while creating images (AMI for EC2).
Packer uses a configuration file to create a machine image. Then, it uses builders to spin up an instance on the target platform and runs provisioners to configure applications or services. Once setup is done, it shuts down the temporary instance and saves the new images with any needed post-processing.
Here are the steps in the process:
- Boot a temporary instance using the base AMI defined in the HCL template file.
- Provision the instance using configuration management tools like Ansible, Chef, or a simple automated script to configure the example into the desired state.
- Create a new machine image from the temporary running instance and shut down the temporary instance after the AMI is registered.
Create Jenkins Master EC2 AMI
Step 1: Verify Packer is available on our local machine. Just type packer
in our terminal.
Step 2: Create an HCL template file named jenkins-master.pkr.hcl
and fill it with the following code.
jenkins-master.pkr.hcl
packer
packer {
required_plugins {
amazon = {
source = "github.com/hashicorp/amazon"
version = "~> 1"
}
}
}
source "amazon-ebs" "jenkins-master" {
ami_description = "Amazon Linux Image with Jenkins Server"
ami_name = "jenkins-master-{{timestamp}}"
instance_type = "${var.instance_type}"
profile = "${var.aws_profile}"
region = "${var.region}"
source_ami_filter {
filters = {
name = "amzn2-ami-hvm*"
root-device-type = "ebs"
virtualization-type = "hvm"
}
most_recent = true
owners = ["amazon"]
}
ssh_username = "ec2-user"
tags = {
"Name" = "Jenkins Master"
"Environment" = "SandBox"
"OS_Version" = "Amazon Linux 2"
"Release" = "Latest"
"Created-by" = "Packer"
}
}
build {
name = "jenkins-master"
sources = ["source.amazon-ebs.jenkins-master"]
provisioner "shell" {
execute_command = "sudo -E -S sh '{{ .Path }}'"
script = "./setup-jenkins-master.sh"
}
}
This file consists of 3 main blocks:
-
Packer
block that we can define our required plugins for AMI creation -
Source
block that we can define our source AMI (base AMI) -
Build
block that we can define severalprovisioner
or what packer need to run to create a custom AMI
We can get the Source AMI using the source_ami_filter
attribute. So that Packer will automatically populate the source_ami
based on the filtering criteria that we are defined on the template. If multiple AMIs meet all of the filtering criteria provided in source_ami_filter
, the most_recent
attribute will select the newest Amazon Linux AMI.
The build
block has a provisioner
stage that is responsible for installing and configuring all needed dependencies. Packer fully supports multiple modern configuration management tools such as Ansible, Chef, or even Bash scripts are also supported.
We can also see that several arguments need to pass a variable that can be overridden at the Packer runtime. So, let's create the variables file.
Step 3: Create the variables file and call it variables.pkr.hcl
that defined our variable on our main packer template.
variables.pkr.hcl
variable "aws_profile" {
type = string
default = "default"
}
variable "instance_type" {
type = string
default = "t3.micro"
}
variable "region" {
type = string
default = "us-east-1"
}
Step 4: We also need to create a bash script called setup-jenkins-master.sh
for installing and configuring all dependencies in provisioner
stage
setup-jenkins-master.sh
bash
#!/bin/bash
echo "Installing Amazon Linux extras"
amazon-linux-extras install epel -y
echo "Install Jenkins stable release"
yum remove -y java
amazon-linux-extras install java-openjdk11 -y
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
yum install -y jenkins
chkconfig jenkins on
systemctl start jenkins
systemctl status jenkins
journalctl -u jenkins
The script is straightforward, it will install Amazon Linux Extras, Java Development Kit (JDK), and also Jenkins. Once the Jenkins package is installed with the Yum package manager, the script configures Jenkins to start automatically if the machine has been restarted with the chkconfig
command.
Step 5: We can also create an environment variable file that will override the default value on the variables.pkr.hcl
file. This step is optional, but if you want, you can create a file with the pattern *.auto.pkrvars.hcl
, for example jenkins-master.auto.pkrvars.hcl
jenkins-master.auto.pkrvars.hcl
packer
aws_profile = "packer" # change with your aws profile name
instance_type = "t3.micro"
region = "ap-southeast-3"
Step 6: We need to create a packer
user with a custom IAM policy to create an AMI. For this, let's create a file called packer-iam-policy.json
I assume that you are familiar with configuring AWS CLI. You can also refer to this AWS documentation.
packer-iam-policy.json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action" : [
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CopyImage",
"ec2:CreateImage",
"ec2:CreateKeypair",
"ec2:CreateSecurityGroup",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteKeyPair",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSnapshot",
"ec2:DeleteVolume",
"ec2:DeregisterImage",
"ec2:DescribeImageAttribute",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSnapshots",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:GetPasswordData",
"ec2:ModifyImageAttribute",
"ec2:ModifyInstanceAttribute",
"ec2:ModifySnapshotAttribute",
"ec2:RegisterImage",
"ec2:RunInstances",
"ec2:StopInstances",
"ec2:TerminateInstances"
],
"Resource" : "*"
}]
}
We can use this shell command to create a packer
user using AWS CLI command and attach the packer-iam-policy
to this user
shell
aws iam create-user --user-name packer
aws iam put-user-policy --user-name packer --policy-name packer-iam-policy --policy-document file://packer-iam-policy.json
aws iam create-access-key --user-name packer
Take note of the AccessKeyId
and SecretAccessKey
because the packer will need this key to access AWS resources.
Step 7: With a properly configured IAM user, it is time to build our first image. We need to run packer init .
for the first time. It will give the output like this.
Step 8: After initializing, run the packer build .
command to start the Packer Build.
Packer will create temporary Key Pair, Security Group, EC2 Instance, based on the configuration specified in the HCL template file and then execute the bash script on the deployed instance.
Packer Spin-Up Temporary EC2 Instance
At the end of running the packer build .
command, Packer will automatically clean up the temporary resource (Key Pair, Security Group, EC2 Instance) and write outputs of the artifacts created as part of the build. Artifacts are the results of a build and are typically represented by the AMI ID.
Packer Automatically Terminate Our Temporary EC2 Instance
Step 9: We can verify the newly created AMI on the EC2 Console. Go to EC2
and then click on AMIs
. Filter on the Owned by me
.
Launch Our Jenkins EC2 Using the New AMI
Yeah!!! Finally, our Jenkins AMI has been created. Let’s test it out and see if Jenkins has been properly installed. For this, we can test Launch Instance via Console, but in this article I will use Terraform.
Step 1: Verify Terraform is available on our local machine. Just type terraform
in our terminal.
Step 2: Create a file named main.tf
, and fill it with the following code.
main.tf
terraform
provider "aws" {
region = var.region
profile = var.aws_profile
}
data "aws_ami" "packer_image" {
most_recent = true
filter {
name = "tag:Created-by"
values = ["Packer"]
}
filter {
name = "tag:Name"
values = [var.app_name]
}
owners = ["self"]
}
resource "aws_security_group" "allow_jenkins" {
name = "allow_jenkins"
description = "Allow inbound traffic to jenkins 8080"
ingress {
description = "Jenkins Inbound Traffic"
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = var.allowed_ip
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "allow_jenkins"
}
}
resource "aws_iam_role" "jenkins_role" {
name = "jenkins-iam-role"
assume_role_policy = jsonencode({
"Version" = "2012-10-17",
"Statement" = [
{
"Action" = "sts:AssumeRole",
"Effect" = "Allow",
"Sid" = "",
"Principal" = {
"Service" = "ec2.amazonaws.com"
}
}
]
})
}
resource "aws_iam_role_policy_attachment" "AmazonSSMManagedInstanceCore" {
policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
role = aws_iam_role.jenkins_role.name
}
resource "aws_iam_instance_profile" "jenkins_instance_profile" {
name = "jenkins-instance-profile"
role = aws_iam_role.jenkins_role.name
}
resource "aws_instance" "jenkins_ami" {
ami = data.aws_ami.packer_image.id
instance_type = var.instance_type
iam_instance_profile = aws_iam_instance_profile.jenkins_instance_profile.name
vpc_security_group_ids = [aws_security_group.allow_jenkins.id]
tags = {
"Name" = var.app_name
}
}
This Terraform script will create a Security Group
to open Inbound Access on port 8080 (default Jenkins Port) from the specific CIDR (we can change it on the Terraform Variable). It will also create an IAM Role
with SSM Policy because we will access our EC2 Using SSM Session Manager
(We don't open SSH port). Lastly, it will create an EC2 Instance
in which the AMI will be queried based on the data block we filtered based on the new AMI created by Packer
.
Step 3: We need to create a variable file to define our variable block that will be used on the main template.
So, let's create variables.tf
, and fill it with the following code.
variables.tf
terraform
variable "region" {
type = string
description = "AWS Region"
default = "us-east-1"
}
variable "app_name" {
type = string
description = "Application Name"
default = "Jenkins Master"
}
variable "instance_type" {
type = string
description = "EC2 Instance Type"
default = "t3.small"
}
variable "aws_profile" {
type = string
description = "Custom AWS Profile"
default = "default"
}
variable "allowed_ip" {
type = list(string)
description = "List of Allow IP Address to Access EC2"
default = ["0.0.0.0/0"]
}
Step 4: We can also create the outputs file, so the Terraform will print the output value once the provisioning process is done. This step is not mandatory, but it is still worth having when we deal with huge Terraform scripts with modules. :)
So, let's create outputs.tf
, and fill it with the following code.
outputs.tf
terraform
output "public_ip" {
value = aws_instance.jenkins_ami.public_ip
}
output "public_dns" {
value = aws_instance.jenkins_ami.public_dns
}
output "jenkins_url" {
value = "http://${aws_instance.jenkins_ami.public_dns}:8080"
}
Step 5: Last but not least, we can also create an environment variable file that will override the default value on the variables.tf
file. We can name it terraform.tfvars
, so Terraform will automatically detect the new value of variables.
terraform.tfvars
terraform
region = "ap-southeast-3"
instance_type = "t3.micro"
aws_profile = "kobokan-aer" # change with your aws profile name
allowed_ip = ["10.10.10.10/32"] # change with your IP
Step 6: Finally, we can try to run our Terraform scripts. For the first, we need to run terraform init
to initilizing the backend and install the required plugins.
Terraform will create this file and directory if the initializing runs successfully.
Step 7: After initializing, run the terraform apply -auto-approve
to start resource provisioning.
In the real project, always run the
terraform plan
first to see the list of resources Terraform will create.
We can see that Terraform will automatically pull the latest AMI created by Packer.
At the end of the terraform apply
, we can verify the Jenkins App will be automatically installed on our EC2. Simply copy the jenkins_url
output on our favorite Browser.
Jenkins automatically installed and running
Step 8: If you want, you can test to use the Jenkins. Just copy the Initial Admin Password on the EC2. Click on the Jenkins Master EC2 -> Connect -> Session Manager -> Connect
Step 9: Copy this password on the Jenkins UI, and we only need to follow the steps that Jenkins provides will be straightforward.
Yeayy!!! we successfully Set Up our Jenkins Master on the EC2 Using the AMI from Packer. Feel free to play around with that.
Clean Up
To clean up our resources, simply run the following shell command on our local terminal.
shell
aws ec2 deregister-image --image-id <your-ami-id>
aws ec2 delete-snapshot --snapshot-id <your-snapshot-id>
# On the Terraform Working Directory
terraform destroy -auto-approve
Conclusion
Now that we can use Jenkins AMI to create Jenkins instances. Terraform will pull the latest AMI created by Packer. We can still improve it by integrating it into the pipeline to automatically create AMI on the Packer workflow and provision the EC2 using Terraform. But it is totally AWSome for now :)
Top comments (0)