Running AWS CodeBuild locally on a MacBook-M1

by Udaybhaskar Sarma Seetamraju
[e-mail]
Dec 31 2023

Highest-level Context

If you are into “Shift-Left” (whether re: Testing, Security, or Replicating-problems-on-developer-laptop, etc ..), then this article is for you.

For the very first time that you switch to an M1-chipset based MacBooks (from intel-chip based MacBooks) .. Productivity is significantly impacted when doing development/testing/troubleshooting “locally” on your laptop. Out-of-scope of this article is supporting those switching from Windoze.

Towards enabling up to 5x developer-productivity by allowing developers to robustly SIMULATE the Cloud-environment on a laptop — I have the following series of articles re: M1-chipset based MacBooks:

1. (This) Running AWS CodeBuild locally on MacBook-M1.
   • Running Containers based on older Ubuntu 20.04 (released in the year 2020) as well as on the newer Ubuntu 22.04 (released in the year 2022)
   • Running Containers based on arm64-based Linux
2. Running AWS Glue locally on MacBook-M1. Various scenarios covered like: you do Not have “aws credentials” on your Laptop (forcing you to mock all the AWS API calls like S3 GET, Glue-Catalog queries, etc..)
3. New Security-related Best-Practices when creating arm64/aarch64 Docker-Images on a MacBook-M1.

Summary

Aiming for very simple set of commands, based on bash-shell scripts --> to start a CodeBuild’s build locally on your MacBook-M1 laptop.

You may ask: Why even bother with arm64/aarch64 docker images, especially when we can set the following ENV-Variables and successfully emulate x86/amd64 chipsets on MacBook-M1?

export BUILDPLATFORM="linux/amd64"
export DOCKER_DEFAULT_PLATFORM="${BUILDPLATFORM}"
export TARGETPLATFORM="${DOCKER_DEFAULT_PLATFORM}"

My response: You should aim to have your software work on linux/arm64 containers, which invariably is cheapest compute on cloud.

Next, I have tips on how to significantly raise your productivity, in debugging/developing your buildspec.yaml, even if your company denies your AWS CLI credentials.

Yes, this article is based on AWS own official Documentation.

But .. ..

those AWS instructions (link above) are a very complex for anything other than AmazonLinux2/2023 on x86, with “gotchas” and challenges of using Ubuntu-vs-AmazonLinux on a MacBook-M1 Laptop.

Based on very simple CLI-arguments, the bash-scripts offered in this article will help automatically download these images (if missing) and execute a build using the buildspec file in current-working directory.

Very-Advanced User: Short summary

1. You need 2 container-images are required to start a CodeBuild’s build locally on your laptop.
   1. amazon/aws-codebuild-local:latest
(This is the “Engine” a.k.a. AWS-CodeBuild’s Agent/Platform on your laptop)

   2. If AmazonLinux2/2023 is your Build O/S: pull the image
 public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:5.0
   3. If Ubuntu 20.04 or 22.04 is your Build-env O/S, then you’ll need locally custom-build the following Docker-Images from AWS official source-code:
      • aws/codebuild/standard:5.0
      • aws/codebuild/standard:7.0
      • No, you can NOT find these images anywhere!

2. Optionally, instead of image #2 above, if you want aarch64-chipset:
   1. then download the image:
 public.ecr.aws/codebuild/local-builds:aarch64
   2. For a full list of all possible images (with Go/Java/Python/..): 
 aws codebuild list-curated-environment-images

Problem Statements

1. Want to use Ubuntu as the O/S for CodeBuild projects, and use it to test/debug locally on my MacBook-M1.
2. Rather than wait 10+ minutes to find a stupid-mistake in the buildspec.yaml file, can I run buildspec.yaml file locally on my laptop, to test everything in it?
3. How to proactively ensure the CodeBuild project’s build will work on all chip-architectures - without having to scramble later? How to explicitly utilize x86_64/amd64/arm64/aarch64 architectures locally on laptop?
4. Company does Not allow Laptops to have AWS-Credentials (in ~/.aws/credentials file);
My buildspec.yaml file uses Secrets, Parameter-Store, etc.. as well as AWS CLI commands.
Even so, how can I EFFICIENTLY test/debug the buildspec.yaml file locally on my laptop?

Ready to use scripts

Please note: For Ubuntu based CodeBuild, read the sub-section (below) titled “CodeBuild projects using Ubuntu”.

I have bash-shell scripts that require just one CLI-argument (either AmazonLinux2 or UBUNTU).

To get started .. ..

1. git clone https://gitlab.com/tosarma/macbook-m1 

   • You’ll notice 2 bash-scripts in AWS-CodeBuild/ sub-folder:
     • “LOCAL-aws-codebuild-runner.sh” and
     • “LOCAL-create-aws-codebuild-standard-image.sh”

2. On your bash-terminal, run the 1st script above using the full-path to it.
   • FYI: The 1st script will automatically run the 2nd one as needed.

FYI — Running the 1st script assumes you have a buildspec.yaml file in the current-folder. Or a “.yml” file.
Note: If you have a LOCAL-buildspec.yaml file in the current-folder, then it is used instead.
Re: this “LOCAL-” file, immediately please read the following sub-section “No AWS Credentials on your laptop?”.

Note: Only for “AmazonLinux2” cli-argument, the __ FIRST __ time you run the above script it’ll take up to 5-minutes to download about 30+/- images from AWS ECR-Repo.

No Bash? Want Python instead?

Just replace the “.sh” with “.py” — in the script names (above).
And, of course, you must insert “python3” at the very beginning of the CLI (this is a platform-independent advice).

WARNING: Without the benefit of “docker cli”, you get ZERO visibility into the progress of docker-activity. This is due to use of un-friendly Docker’s Python APIs, because of which the python-code _ WILL _ _ HANG _ for very long time!

To repeat:

• “LOCAL-aws-codebuild-runner.py” will hang with NO output, for roughly 2-to-5 minutes (depending on how much CPU and MEMORY you have allocated to the Docker-Desktop, as well as speed of your internet connection)
• “LOCAL-create-aws-codebuild-standard-image.py” will hang with NO output .. ..
  • .. for AmazonLinux2 for roughly 2-to-5 minutes (depending on how much CPU and MEMORY you have allocated to the Docker-Desktop, as well as speed of your internet connection)
  • .. for Ubuntu (see full details below) for a minimum of 2+ H O U R S !! (If you have minimal CPU allocated for Docker-Desktop, it will as long as 4+ HOURS)

Important - Note these:

1. I only tested using Python3.11; No other Python version tested.
2. PRE-REQUISITES:
   • pip3 install docker
   • pip3 install GitPython

---

---

Using Ubuntu 20.04

Warning: As of 2024-January, only ‘x86_64/amd64’ chipset-architecture supported (on MacBook-M1) for “Ubuntu” O/S.

Question: Who would use Ubuntu (instead of AmazonLinux) - for their CodeBuild projects?

Answer: Ubuntu has great trouble-free support for installing EXACT versions of software, whether Google-Chrome (for headless testing) or older versions of NodeJS or Python, etc.. ..

Warning: Only for “Ubuntu” O/S CodeBuild builds, you _ MUST _ turn-OFF (that is, un-check the checkbox) as shown in screenshot below.
Note: For fix many __ OTHER __ issues while running Docker-containers on MacBook-M1, you are REQUIRED to turn-ON this checkbox.
So, please pay attention to conflicting configurations (within Docker-Desktop on MacBook-M1)!

Warning: Only for “Ubuntu” O/S, the __ FIRST __ time you run the above script it’ll takes minimum 2 hours — to re-create Ubuntu Image runtime from scratch (after downloading the AWS Source-code).

Why? Because, unfortunately, AWS does Not offer these ready-to-use images for Ubuntu O/S to download.

To get started, here are the simple commands to run!

cd <Your-own-project>

export BUILDPLATFORM=linux/amd64
export DOCKER_DEFAULT_PLATFORM="${BUILDPLATFORM}"
export TARGETPLATFORM="${DOCKER_DEFAULT_PLATFORM}"

<Path-to-Git-Cloned-folder>/AWS-CodeBuild/LOCAL-aws-codebuild-runner.sh  UBUNTU

Let me know of any issues with the above.

Issues & Errors?

See Appendix, for resolving the errors.
Example: See Appendix sub-section titled “Docker-Desktop settings for Ubuntu-on-x86 images”.

Installing Chromium Headless on Ubuntu

See install-Chromium-latest-on-ubuntu20.04.sh
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads

Installing Old Node.JS on Ubuntu

Edit and run install-NodeJS-latest-on-ubuntu20.04.sh
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads

Installing Old Python on Ubuntu

Edit and run install-python-latest-on-ubuntu20.04.sh
under https://gitlab.com/tosarma/macbook-m1/-/tree/main/AWS-CodeBuild/software-install-scripts?ref_type=heads

Ubuntu 22.04

To switch to the newer Ubuntu 22.04 (released in the year 2022) ..

1. Edit script “./AWS-CodeBuild/LOCAL-create-aws-codebuild-standard-image.sh”.
2. UN-comment the line # 21 (to use “aws/codebuild/standard:7.0”)
3. Comment out the next line # 22. (Disable “5.0”)
4. Must run “docker system prune --all --force —volumes”.
5. Finally, follow the instructions in above section titled “CodeBuild projects using Ubuntu 20.04”

No AWS Credentials on your laptop?

For security-reasons, many companies are denying developers the AWS-credentials for AWS-CLI use.
If your buildspec.yaml file has AWS-Secrets (quite common!) or if it runs AWS-CLI inside (example: to get Stack-outputs), etc .. ..
.. then, you have a showstopper in locally testing/debugging your buildspec.yaml locally on your laptop.

My best-practice is to:

• Create a 2nd (new) file named “LOCAL-buildspec.yaml” (as described below) +
• Create a 3rd (new) file called “.env” file to along with it.

WARNING: Do _ NOT _ git-commit the “.env” file, as per global practices.
Immediately add the “.env” file to your “.gitignore” file.

This new “LOCAL-buildspec.yaml” file will:

1. Not have entries for Secrets and Parameter-Store entries
2. Not have all AWS-CLI commands
3. Not have any CDK or other commands.
4. Must “source” the .env file, in the “Install or Pre-Build” phases as:

. .env

NOTE: Re: the .env file:-

• You will ensure the .env file sets all bash-variables that’ll contain the values for the Secrets, Parameter-Store entries, etc. ..
• You will ensure .env file also provides all the values expected from the AWS-CLI commands (that were removed).

Finally, just run:

cd <your-own-project>

<Path-to-Git-Cloned-folder>/AWS-CodeBuild/LOCAL-aws-codebuild-runner.sh

Open Questions, Concerns and Challenges

WARNING: Do _ NOT _ git commit the .env file, as per global practices.
Immediately add the .env file to your .gitignore file.

If you mistakenly commit it, confess honestly and immediately to your corporate security team, and work on fixing the security vulnerability.

FYI - CodeBuild for Ubuntu O/S on aarch64 & amd64 chipsets are Not yet supported.
No timetables available.

Note: If you mistakenly or consciously run “docker system prune”, the Ubuntu O/S option will take 2+ hrs again (to re-create images from AWS official source-code)!
Warning: Do __ Not __ rely on “save” and “import” of the docker-image.
The Following failed for me.

 docker save                   --output ~/aws_codebuild_standard_5.0.tar   aws/codebuild/standard:5.0  
 docker import --platform linux/x86_64  ~/aws_codebuild_standard_5.0.tar   aws/codebuild/standard:5.0 

FYI: Above “.tar” file is typically 10GB in size and takes about 2-3 minutes to be “saved”.

APPENDIX

Docker-Desktop settings for Ubuntu-on-x86 images

Why?
As of 2023, per https://github.com/moby/moby/issues/44578 Docker-Desktop’s containerd-integration can NOT interact with images that don't have the default platform.
And .. aarch64 is the default-platform on MacBook-M1, while we’re seeking amd64 for Ubuntu.

Docker-Desktop settings for aarch64-chipset

See screenshot in previous sub-section.
Turn _ ON _ the setting titled “Use containerd for pulling and storing images”!
Yup! Doing just the opposite!
I can’t help explain this crazy conflicting instructions.
As of 2023, this is a Docker-on-MacBook issue.

Running out of Disk-space or Memory?

Screenshot below shows the recommended “high” settings.
After building images, you can reduce:

• “CPU” can be lowered to “2”.
• “Memory” can be lowered to “4GB”.

FYI only - To run on a MacBook-M1, many amd64 emulated containers like Neo4j v4.x will frequently fail, unless you provide Docker with a minimum of 5+ cpus and 8GB of RAM!