DEV Community

Cover image for Build AWS AMI with HashiCorp Packer using GitHub Actions
Manish R Warang
Manish R Warang

Posted on • Originally published at blog.g33kzone.com on

Build AWS AMI with HashiCorp Packer using GitHub Actions

#packer #aws #githubactions #github

Security best practices recommend the latest base images (e.g. AWS AMI ) for spinning VM on the cloud. Up to date software patches reduce the risk of any security breach. This emphasizes a strong need to set up an Image Factory to automate the image creation process with the latest software versions.

Also, every organization has a standard list of software to be installed for a given image. Only the essential software must be part of the list to reduce the blast surface from a security standpoint. This list of software can either be installed once the VM is created or when the VM is in the process of creation. However, both these techniques are time-consuming and hence not recommended.

It would be preferred to bake this software list in the base image itself. HashiCorp Packer is an open-source tool that specializes in building automated machine images for multiple platforms from a single source configuration.

HashiCorp Packer Installation

Refer to HashiCorp documentation for Packer installation based on your hardware OS.

GitHub Actions

For this demo, we will use GitHub Actions to create CI/CD pipeline to automate this workflow and eventually push the baked image (AMI) in AWS. It is a platform to automate tasks within the software development lifecycle. It's an event-driven framework, which means we can carry series of commands for a given event or can be scheduled for one-off or repetitive tasks. (e.g. Execute a Test Suite on Pull Request creation, Adding labels to issues, Lint checks, etc.)

Actions are defined in YAML files, which allows pipeline workflow to be triggered using any GitHub events like on creation of Pull Requests, on code commits, and much more.

Prerequisites

  • AWS User with Programmatic access
    • AWS Access Key ID
    • AWS Secret Access Key
  • AWS IAM Privileges to create EC2 Instance (create, modify and delete EC2 instances). Refer documentation for the full list of IAM permissions required to run the amazon-ebs builder.

In this post, we will bake Open JDK (Java 8) in our Ubuntu base Image and push it into AWS. Packer configurations can be written in HCL (.pkr.hcl file extension) and JSON (.pkr.json) formats. We will use the HCL language for this demo.

Packer Flow.png

Reference GitHub repository - pkr-aws-ubuntu-java

Code Time

Let us start writing Packer configuration. (I am using a Linux machine for this demo)

Packer Configuration

Create Project folder pkr-aws-ubuntu-java

mkdir pkr-aws-ubuntu-java && cd $_
Enter fullscreen mode Exit fullscreen mode

Create a file named aws-demo.pkr.hcl

touch aws-demo.pkr.hcl
Enter fullscreen mode Exit fullscreen mode

Open your favorite IDE (e.g. VSCode). Copy the below code in aws-demo.pkr.hcl file.

packer {
  required_plugins {
    amazon = {
      version = ">= 0.0.2"
      source = "github.com/hashicorp/amazon"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The packer {} block contains Packer settings, including a required Packer version. The required_plugins block in the Packer block, specifies the plugin required by the template to build your image. The plugin block contains a version and source attribute.

Source block

The source block configures a specific builder plugin, which is then invoked by the build block. Source blocks use builders and communicators to define virtualization type, image launch type, etc.

Copy the following code to aws-demo.pkr.hcl file.

variable "ami_prefix" {
  type = string
  default = "packer-aws-ubuntu-java"
}

locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}

source "amazon-ebs" "ubuntu_java" {
  ami_name = "${var.ami_prefix}-${local.timestamp}"
  instance_type = "t2.micro"
  region = "us-east-1"
  source_ami_filter {
    filters = {
      name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners = ["099720109477"]
  }
  ssh_username = "ubuntu"
}
Enter fullscreen mode Exit fullscreen mode

Variable ami_prefix is used to define the AMI image. Local variable timestamp helps ensure uniqueness to AMI name.

The amazon-ebs builder launches the source AMI, runs provisioners within this instance, then repackages it into an EBS-backed AMI. This builder configuration launches a t2.micro AMI in the us-east-1 region using an ubuntu:xenial AMI as the base image.

It creates the AMI named packer-aws-ubuntu-java+timestamp. AMI names must be unique else it will throw an error.

It also uses the SSH communicator - by specifying the ssh_username attribute. Packer is then able to SSH into EC2 instance using a temporary keypair and security group to provision your instances.

Build Block

The build block defines what Packer should do with the EC2 instance after it launches.

build {
  name = "packer-ubuntu"
  sources = [
    "source.amazon-ebs.ubuntu_java"
  ]

  provisioner "shell" {

    inline = [
      "echo Install Open JDK 8 - START",
      "sleep 10",
      "sudo apt-get update",
      "sudo apt-get install -y openjdk-8-jdk",
      "echo Install Open JDK 8 - SUCCESS",
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

provisioner block helps automate modifications to your base image. It leverages shell scripts, file uploads, and integrations with modern configuration management tools such as Ansible, Chef, etc.

The above provisioner defines a shell provisioner and installs Open JDK 8 in the base image.

The final file aws-demo.pkr.hcl should look as below.

packer {
  required_plugins {
    amazon = {
      version = ">= 0.0.2"
      source = "github.com/hashicorp/amazon"
    }
  }
}

variable "ami_prefix" {
  type = string
  default = "packer-aws-ubuntu-java"
}

locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}

source "amazon-ebs" "ubuntu_java" {
  ami_name = "${var.ami_prefix}-${local.timestamp}"
  instance_type = "t2.micro"
  region = "us-east-1"
  source_ami_filter {
    filters = {
      name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

build {
  name = "packer-ubuntu"
  sources = [
    "source.amazon-ebs.ubuntu_java"
  ]

  provisioner "shell" {

    inline = [
      "echo Install Open JDK 8 - START",
      "sleep 10",
      "sudo apt-get update",
      "sudo apt-get install -y openjdk-8-jdk",
      "echo Install Open JDK 8 - SUCCESS",
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

GitHub Actions

Create a new file in the .github/workflows directory named github-actions-packer.yml

Folder Structure.png

We will schedule this workflow to run in the wee hours - let's say 04:00 Hrs in the morning.

name - The name of your workflow. GitHub displays the names of your workflows on your repository's actions page - "AWS AMI using Packer Config"

name: AWS AMI using Packer Config
Enter fullscreen mode Exit fullscreen mode

on - (Required) The name of the GitHub event that triggers the workflow. We have configured to trigger the workflow on schedule.

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron: '0 4 * * *'
Enter fullscreen mode Exit fullscreen mode

jobs - A workflow run is made up of one or more jobs. These jobs can run in parallel or sequentially. Each job executes in a runner environment specified by runs-on.

job name - The name of the job displayed on GitHub.

runs-on - (Required) Determines the type of machine to run the job on. The machine can be either a GitHub-hosted runner or a self-hosted runner. Available GitHub-hosted runner types are: windows-latest / windows-2019 / windows-2016 / ubuntu-latest / ubuntu-20.04 etc.

jobs:
  packer:
    runs-on: ubuntu-latest
    name: packer
Enter fullscreen mode Exit fullscreen mode

steps - Sequence of tasks called steps within a Job. They can execute commands, set up tasks, or run actions in your repository, a public repository, or action published in a Docker registry.

The first step is to check out the source code in the runner environment.

Checkout V2- This action checks out your repository under $GITHUB_WORKSPACE, so your workflow can access it.

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2
Enter fullscreen mode Exit fullscreen mode

To ensure access to the AWS Cloud environment we need to configure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the runner environment. The values for these variables will be configured as GitHub Secrets in the below section.

Configure AWS Credentials - This action configures AWS credential and region environment variables for use in other GitHub Actions.

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          # aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }} 
          # if you have/need it
          aws-region: us-east-1
Enter fullscreen mode Exit fullscreen mode

Init initializes the Packer configuration used in the GitHub action workflow.

      # Initialize Packer templates
      - name: Initialize Packer Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: init
Enter fullscreen mode Exit fullscreen mode

Validate checks whether the configuration has been properly written. It will throw an error otherwise.

      # validate templates
      - name: Validate Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: validate
          arguments: -syntax-only
          target: aws-demo.pkr.hcl
Enter fullscreen mode Exit fullscreen mode

Build executes the Packer configuration.

      # build artifact
      - name: Build Artifact
        uses: hashicorp/packer-github-actions@master
        with:
          command: build
          arguments: "-color=false -on-error=abort"
          target: aws-demo.pkr.hcl
        env:
          PACKER_LOG: 1
Enter fullscreen mode Exit fullscreen mode

The complete file github-actions-packer.yml will look as below.

name: AWS AMI using Packer Config

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron: '0 4 * * *'

jobs:
  packer:
    runs-on: ubuntu-latest
    name: packer

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          # aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }} 
          # if you have/need it
          aws-region: us-east-1

      # Initialize Packer templates
      - name: Initialize Packer Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: init

      # validate templates
      - name: Validate Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: validate
          arguments: -syntax-only
          target: aws-demo.pkr.hcl

      # build artifact
      - name: Build Artifact
        uses: hashicorp/packer-github-actions@master
        with:
          command: build
          arguments: "-color=false -on-error=abort"
          target: aws-demo.pkr.hcl
        env:
          PACKER_LOG: 1
Enter fullscreen mode Exit fullscreen mode

The source code is ready and can be pushed to the GitHub repository. As configured, the workflow will be triggered at 04:00 hrs in the morning.

Packer Actions 1.png

Packer Actions 2.png

Packer Actions 3.png

Packer Actions 4.png

Top comments (0)

Read next

suin profile image

How to Host Helm Charts on GitHub Container Registry

suin -

0x3d_site profile image

The Only Open-Source FAQ Guide You'll Ever Need—Straight From the 0x3d Collection

0x3d Site -

jatinmehrotra profile image

EKS Auto Mode Unlocked for Existing Clusters with Terraform

Jatin Mehrotra -

osahenru profile image

Using dj-rest-auth to integrate GitHub authentication in your Django project

Osahenru -