Converting dev environments to Apple Silicon

Or "How I learned to stop worrying and love the Apple Silicon bomb"

By now, if you are a developer working on Macs, you're probably in the process of figuring out whether everything you have will be able to run on an Apple Silicon computer. Yep, the hype is real, and it's everywhere. Of course, if you believe the hype, the ARM chips powering the new Mac can do everything from make you coffee without a second thought or even write your code for you ;) To be fair, and this is not a review of the Apple Silicon Macs, I do have to say the machine is quite nice.

The reality is that Apple will switch to ARM chips, and as devs, we need to be prepared. So this past weekend, faced with a shortage of Intel Macbook Pros for our new devs, we sat down to make it all work for our Supermetrics developers.

I’ll walk you through the differences we found and what we had to do to convert, hoping that others will find it helpful.

The following is a summary of what we had to do in the barest of bones:

• prepare for slightly different paths on the M1 and some slightly different libraries
• get the latest version of Docker
• use buildx to compile the containers on the various environments (Mac Intel, Arm64)

   

Step 1: Preparing for the new local environment

Supermetrics develops locally on as close to a duplicate of our production systems as possible. Each of our devs has a standard setup and a command-line tool that enables any dev to start, stop and debug their local setup. The tool also allows the clearing of caches, manipulating logs, and so on. While not perfect, it does the trick.

While our internal tools sort of worked “out of the box”, we first had to get the M1 machine simply running all the libraries we use. Luckily, homebrew is already functional for Apple Silicon. If you have any scripts that assume where brew is installed, be warned that the default is different from a Mac Intel: /usr/local for Mac Intel, /opt/homebrew on the Apple Silicon.

For all our bash scripts, we needed to detect both the system and the chip now. Some of our devs are on Linux boxes as well :).

For reference, here's the bash that we used:

#!/usr/bin/env bash

function check_system() {

 unameOut="$(uname -s)"
 local op_sys=""
 case "${unameOut}" in
     Linux*)     op_sys=Linux;;
     Darwin*)    op_sys=Mac;;
     *)          op_sys="UNKNOWN:${unameOut}"
 esac

 echo $op_sys
}

function check_chip() {
 unameMOut="$(uname -m)"
 local op_chip=""
 case "${unameMOut}" in
     arm64*)     op_chip=Arm;;
     x86_64*)    op_chip=Intel;;
     *)          op_chip="UNKNOWN:${unameMOut}"
 esac

 echo $op_chip
}

check_system
check_chip

There were some minor changes to our installation scripts — making sure that all the libraries can be installed at the same version — but by and large, the community has come through with fixes to almost everything we use.

 

Step 2: Checking out Docker

As of June 2021, Docker Desktop runs great on Apple Silicon. So go get it.

Next step, fire up all our containers and hope that everything works... and boom. Nope. Well... ok, sort of.

Here's the important thing, yes your Intel AMD64 images will run on Apple Arm64, and they’ll do that in emulation. But hey, wasn't the whole point of doing this to try and avoid emulation.

Our servers are custom compiled versions of PHP, Python, and a host of other libraries. So, of course, it's time to compile. But here's a description of our local systems.

In short,

• Redis and RedisInsight
• MySQL (5.6, 5.7, 8.0)
• Jwilder/Nginx-proxy (every dev shop should use it — a future blog post maybe!)
• Various custom compiled debug versions of our production systems

You can run all containers by specifying the platform to emulate — but our goal was to allow our tools to seamlessly call docker-compose without having to worry if they were running on a Linux, Mac Intel, or a Mac Apple Silicon System.

In addition, MySQL 5.6 and MySQL 5.7 official images are not provided as Arm64 images :(

 

Step 3: Compiling all the containers

To make the whole thing seamless, the best way is to recompile the images you can to Arm64 versions or make sure that the source has Arm64 versions.

To get everything working, here’s what we did (exact commands follow):

1. Where previously we would reference the official image directly in our docker-compose files (like mysql:5.6), we now recompile even official images into our container registry (Google Container Registry – GCR). For example, local_mysql was an image that we created and tagged into three different versions: 5.6, 5.7 and 8.0 but also stored as emulated Arm64.

   FROM --platform=linux/amd64 mysql:5.6

1. We then needed to compile the AMD64 version on an AMD64 machine (Linux or Mac Intel) and the Arm64 version on an Apple Silicon Mac and upload it to our registry from each one with the correct platform. This is an important note since the compilation process cannot be emulated, it needs to be executed on the appropriate underlying platform.

2. Where we could, we re-compiled or referenced the correct upstream image. A note: if you are recompiling something like PHP, you do need to compile on the correct platform. So you should compile it on Apple Silicon for the Arm64 (an Arm64 Linux box on Google Cloud should also work). It is possible that you could compile it under emulation on an AMD64, but because I had two machines — I used both and didn’t bother to try emulation or a cloud Arm64 server.

3. Because it's completely annoying to have to do this again and again (we do build our local images a lot), we wrote an internal tool to do the builds for us on Apple Silicon and Intel Macs for the future.

Using Buildx

While I hope this article stays relevant, please refer to the correct Buildx documentation. To use the new docker buildx commands, you will need to have set up a builder. I've referenced the Google Container Registry (gcr.io) as that's where we host our containers, but, of course, ECR and other container registries will work.

Create your builder

docker buildx create --name my_local_builder --platform linux/amd64,linux/arm64
docker buildx use my_local_builder

You should now be able to see the builder running.

docker buildx inspect --bootstrap

Building images

Building is straightforward — if you want to build multiple images on one machine (some images you will be able to) simply, add all the platforms (comma separated) rather than the single used here.

# no_cache_string = --no-cache if you want to ignore the docker build cache
# build_this_platform = linux/arm64 or linux/amd64 assuming your builder is set for those
# repo_image = your repository image
# build_docker_tag = your image tag
# use_docker_file = use -f if you want to use something other than the default `Dockerfile`


docker buildx build ${no_cache_string} --platform ${build_this_platform} -t ${repo_image}:${build_docker_tag} --push -f ${use_docker_file} .

In our specific case — to keep the images separate as we built them on different platforms and pushed up to the repository — we appended -arm64 and -amd64 to the version tag.

Building a manifest

Once you have pushed my_local_debug_image:1.1-amd64 as linux/amd64 and my_local_debug_image:1.1-arm64 as linux/arm64, you need to create a manifest that points to both platforms.

When a developer calls for my_local_debug_image:1.1, the registry will serve the correct platform image.

docker buildx imagetools create -t eu.gcr.io/myproject/my_local_debug_image:1.1 eu.gcr.io/myproject/my_local_debug_image:1.1-arm64 eu.gcr.io/myproject/my_local_debug_image:1.1-amd64

 

Final words

Please note, much of this can likely be done in emulation, but your mileage may vary. My buildx would crash on certain compilations, and it was easier and better to simply use an Arm64 machine like the Apple Silicon machine.

I was heartily disappointed that specific libraries we use, like Snowflake ODBC or Oracle ODBC drivers, did not actually have native Arm64 libraries. I'm sure by the end of the year this will be fixed, but still disappointing.

Personally, I do like the Apple Silicon machines, and I'm heartily hoping for an Arm64 future :)

---

Learn more about Supermetrics as a workplace at supermetrics.com/careers/engineering.

Comments

[Jean-Michel Garnier]

Thanks Duleepa for this article. Very intersting.

By chance, do you know if these Apple silicon Docker images for mysql 5.* are available somewhere in a public registry?

Also would you mind adding some examples to illustrate your whole workflow? I am a kind of docker noob and I am not sure how to build and compile mysql-5.7 on Apple Silicon for the Arm64 ...

Any chance you open source your internal tool ;)?

[Duleepa Wijayawardhana]

So with MySQL 5.7 on a docker your dockerfile would be something like:

FROM --platform=linux/amd64 mysql:5.7

And then build on the Apple Silicon with buildx as above

[Duleepa Wijayawardhana]

Hey that's a good idea, I'll see if I can clean up our internal tool. Let me look up the mysql 5.7 stuff and get back to you. Sorry about the delay I never got a note on this!

[Jean-Michel Garnier]

Thanks Duleepa for taking the time to reply :)