DEV Community

loading...
Cover image for Building A Ubuntu VirtualBox Image on AWS EC2 With Packer

Building A Ubuntu VirtualBox Image on AWS EC2 With Packer

Ash Wu
Gopher, Rails, devOps, 🐈Crazy cat person, 🏹holiday archer. Creator of @golangtw
・3 min read

Building a virtualbox image on AWS EC2 with packer - this sounds straightforward, how hard could it be?

Indeed, it's relatively simple because there are existing tools and tutorials our there, but I still ran into a few hiccups during my experiment and I think it's worth nothing that down.

Why

Some background first. I need to build a VirtualBox image as a part of the release. There are few different approaches. In the past I did this with Vagrant and VirtualBox in my home-made Jenkins CI. So naturally I would dig in this way.

But now we're using CircleCI and it's hard to get VirtualBox running inside CircleCI environment.

I also checked AWS, AWS doesn't support nested virtualization unless you're using an expensive bare-metal machine. GCP supports nested virtualization but at that point I decided not to mess with it.

Chances are that we may need to build a AWS AMI too in the future, so after some digging I decide to use Packer with its amazon-ebs builder to build an AMI and then utilize aws-cli to export AMI to VMDK.

Packer

Using Packer to build an AMI is quite easy. Here's an example hcl file:

variable "ami_name" {
  type = string
  default = "${env("CUSTOM_AMI_NAME")}"
}
variable "region" {
  type    = string
  default = "ap-northeast-1"
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "${var.ami_name}"
  instance_type = "c4.2xlarge"
  ssh_pty = true
  region        = "${var.region}"
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-bionic-18.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  launch_block_device_mappings {
    device_name = "/dev/sda1"
    volume_size = 40
    volume_type = "gp2"
    delete_on_termination = true
  }
  ssh_username = "ubuntu"
}

build {
  sources = ["source.amazon-ebs.ubuntu"]

  provisioner "shell" {
    script = "./server_setup.sh"
  }
}
Enter fullscreen mode Exit fullscreen mode

There are few things we need to pay attention here in the bootstrap script.

First, you'll want to wait until cloud-init is finished to proceed your script. Otherwise there will be random failures.

# Wait for cloud-init to finish
while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done
Enter fullscreen mode Exit fullscreen mode

Second, if you build the AMI and export to VMDK, import it in your VirtualBox now, you'll find that there's no network interface in your virtual machine. Because those AMIs on AWS are optimized for cloud environments, we need to tweak a little bit after your bootstrap is finished.

# Remove cloud-init to speed up the boot in virtualbox
sudo touch /etc/cloud/cloud-init.disabled
sudo apt remove cloud-init --yes
sudo rm -rf /etc/cloud/; sudo rm -rf /var/lib/cloud/

# Setup network in virtualbox
sudo rm /etc/netplan/50-cloud-init.yaml
sudo bash -c "cat > /etc/netplan/config.yaml" <<'EOF'
network:
  version: 2
  renderer: networkd
  ethernets:
    id0:
      match:
        name: en*
      dhcp4: yes
EOF
sudo bash -c "echo '@reboot /usr/sbin/netplan apply' | crontab -"
Enter fullscreen mode Exit fullscreen mode

Finally, you can convert AMI to VMDK on S3 when packer finished the building process.

export CUSTOM_AMI_NAME=your-ami-name
packer build example.pkr.hcl

AMI_ID=$(aws ec2 describe-images --owners self --filters "Name=name,Values=${CUSTOM_AMI_NAME}" | grep ImageId | cut -f4 -d'"')

aws ec2 export-image --image-id ${AMI_ID} --disk-image-format VMDK --s3-export-location S3Bucket=bucketname,S3Prefix=prefixname/

Enter fullscreen mode Exit fullscreen mode

Discussion (0)