DEV Community

Cover image for Run Docker and Kubernetes on your Apple Silicon in an Enterprise Environment
Oliver Brinker
Oliver Brinker

Posted on

Run Docker and Kubernetes on your Apple Silicon in an Enterprise Environment

#docker #colima #zscaler #kind

In many companies secure web gateways from providers like Zscaler, Cloudflare and Palo Alto Networks are applied between the computers of the employees and the Internet. These systems act as a “man-in-the-middle” route, inspecting the traffic between users and the Internet to filter URLs, protect against malware and control application access. These systems, operated mainly in the cloud, are decrypting and re-encrypting the HTTPS traffic and presenting a certificate to the client signed by a provider-controlled Certified Authority (CA).

Certificate Details

While the Operating System and most applications on the company computer are configured to trust the CA, other apps like developer tools might experience errors in validating these certificates.

I had such issues when trying to pull a container image from Docker Hub using the Docker client in combination wit Colima and further when deploying some basic app to my local Kubernetes cluster using kind.

...tls: failed to verify certificate: x509: certificate signed by unknown authority...
Enter fullscreen mode Exit fullscreen mode

So, let's fix this issue and get Docker and kind up and running to load images from any registry 🧑‍💻🛠️

Setup Colima and Docker tools

There are a bunch of options to run containers locally on macOS. In addition to the dominant Docker Desktop, there are other excellent tools like OrbStack, Podman/Podman Desktop and even a solution from Apple starting with macOS 26 (Tahoe).

Because only the Docker engine is available under an Open Source license (Apache license, 2.0) and Docker Desktop is the only way to get the Docker engine under macOS (and Windows), I decided to give Colima a chance. Not only because Docker Desktop needs a commercial license in an enterprise context, but also because I like to work from the terminal and don't need most of the stuff offered by Docker Desktop.

Some of my colleagues recommended Colima and after a quick review the project looked very promising to me. The installation should be straight forward and can be found in the Colima docs. I used Homebrew to install Colima on my system.

> brew install colima
Enter fullscreen mode Exit fullscreen mode

After a successful installation of Colima, at least the Docker CLI is needed additionally to start Colima for the first time. I would recommend to install some further Docker tools.

> brew install docker docker-compose docker-credential-helper docker-buildx`
Enter fullscreen mode Exit fullscreen mode

With the Docker CLI installed, Colima can be started from the terminal.

> colima start
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0001] creating and starting ...         context=vm
INFO[0002] downloading disk image ...        context=vm
INFO[0072] provisioning ...                  context=docker
INFO[0074] starting ...                      context=docker
INFO[0075] done
Enter fullscreen mode Exit fullscreen mode

During the first start Colima downloads an VM image which executes a Docker (and containerd) daemon. Colima supports various command line parameters to check the status of the running instance...

> colima status
INFO[0000] colima is running using macOS Virtualization.Framework 
INFO[0000] arch: aarch64                                
INFO[0000] runtime: docker                              
INFO[0000] mountType: virtiofs                          
INFO[0000] docker socket: unix:///Users/<user>/.colima/default/docker.sock 
INFO[0000] containerd socket: unix:///Users/<user>/.colima/default/containerd.sock
Enter fullscreen mode Exit fullscreen mode

...and to list the initially created VM.

> colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK      RUNTIME    ADDRESS
default    Running    aarch64    2       2GiB      100GiB    docker
Enter fullscreen mode Exit fullscreen mode

Later we will see some more commands to work with Colima, but first let's pull and start a simple container image from Docker Hub.

docker run hello-world
Enter fullscreen mode Exit fullscreen mode

Ouch 😨 We expect to see the output of the hello-world container, but instead we got an error message.

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
docker: failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://docker-images-prod.6aa30f8b08e16409b46e0173d6de2f56.r2.cloudflarestorage.com/registry-v2/docker/registry/v2/blobs/sha256/ca/ca9905c726f06de3cb54aaa54d4d1eade5403594e3fbfb050ccc970fd0212983/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=f1baa2dd9b876aeb89efebbfc9e5d5f4%2F20260224%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20260224T194609Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=b51009b9434113a49b28b68bf577746a790239c7c8ba6b8e9a2e2aa2573f9870": tls: failed to verify certificate: x509: certificate signed by unknown authority

Run 'docker run --help' for more information
Enter fullscreen mode Exit fullscreen mode

I know this type of error message all too well, because I've seen it several times during my developer career, e.g. when from my code requesting a resource over HTTPS, which uses a self-signed certificate. It's a safety net to avoid communication to a potentially unsafe system.

IMHO, to ignore such an error and to allow insecure connections by configuration is not an option. The answer should be to make the certificate of the authority known to Docker/Colima.

An easy way is to copy the certificate of the root CA to the Docker settings. In my case it is the certificate of the Zscaler root CA. This is required, because Colima does not load the certificate from the macOS system keychain. Instead Colima copies the content of the directory ~/.docker/certs.d into the VM at startup.

So, create the directory certs.d, copy the certificate file into it and restart Colima.

> mkdir -p ~/.docker/certs.d
> copy zscaler.crt ~/.docker/certs.d
> colima restart
INFO[0000] stopping colima                              
INFO[0000] stopping ...                                  context=docker
INFO[0001] stopping ...                       context=vm
INFO[0004] done                                       
INFO[0007] starting colima                              
INFO[0007] runtime: docker                              
INFO[0010] starting ...                       context=vm
INFO[0018] provisioning ...                   context=docker
INFO[0019] starting ...                       context=docker
INFO[0021] done
Enter fullscreen mode Exit fullscreen mode

Bamm, now the hello-world image can be pulled from Docker Hub and the container is created and executed 🎉

> docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
198f93fd5094: Pull complete 
95ce02e4a4f1: Download complete 
Digest: sha256:ef54e839ef541993b4e87f25e752f7cf4238fa55f017957c2eb44077083d7a6a
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly...
Enter fullscreen mode Exit fullscreen mode

What happened? As mentioned already above, Colima copies the content of the ~/.docker/certs.d folder into the VM at startup. We can check this by SSHing into the VM and list the available files and folders.

> colima ssh
lima@colima:/Users/<user>$ ll .docker/certs.d/
total 4
drwxr-xr-x 3 lima lima   96 Feb 24 21:47 ./
drwxr-xr-x 5 lima lima  160 Feb 24 21:47 ../
-rw-r--r-- 1 lima lima 1732 Feb 24 21:47 zscaler.crt
Enter fullscreen mode Exit fullscreen mode

If you have the feeling to see the same content as in the home directory of your host system, you are right 👍 You can also find the config.json file in the ~/.docker folder containing your settings for the Docker client.

Setup kind

After having Colima for Docker up and running, the next surprise was waiting when deploying a first app to a local kind cluster. I use kind as a local Kubernetes cluster for development and testing.

Wait, is this really a surprise? Not really, because applications running inside containers do not inherit the trust settings from the host system and this is also the case for kind. Kind runs Kubernetes components in Docker (kind = Kubernetes in Docker). It means that we need to spend some effort to make the kind containers trust the certificates presented by Zscaler.

First let's install kind using Homebrew.

> brew install kind
Enter fullscreen mode Exit fullscreen mode

Afterwards a cluster must be created for kind.

> kind create cluster 
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.35.0) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
Enter fullscreen mode Exit fullscreen mode

This downloads the required container image for kind - of course with the help of the fix of the certificate issue described in the previous chapter 😉 - and starts the kind cluster.

> docker images
IMAGE                                                                                  ID             DISK USAGE   CONTENT SIZE   EXTRA
kindest/node@sha256:452d707d4862f52530247495d180205e029056831160e22870e37e3f6c1ac31f   452d707d4862       1.31GB          363MB    U   
> docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                       NAMES
01cfe1e8ffb7   kindest/node:v1.35.0   "/usr/local/bin/entr…"   17 minutes ago   Up 17 minutes   127.0.0.1:53740->6443/tcp   kind-control-plane
> kind get clusters
kind
> kind get nodes
kind-control-plane
Enter fullscreen mode Exit fullscreen mode

Assuming kubectl is installed on your system, you are able to interact with your cluster.

> kubectl get nodes -o wide
NAME                 STATUS   ROLES           AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION      CONTAINER-RUNTIME
kind-control-plane   Ready    control-plane   8m32s   v1.35.0   172.18.0.2    <none>        Debian GNU/Linux 12 (bookworm)   6.8.0-100-generic   containerd://2.2.0
> kubectl get namespaces
NAME                 STATUS   AGE
default              Active   8m45s
kube-node-lease      Active   8m45s
kube-public          Active   8m45s
kube-system          Active   8m45s
local-path-storage   Active   8m42s
Enter fullscreen mode Exit fullscreen mode

Next, let's create a namespace and deploy an Apache HTTP server by applying the YAML manifest below with kubectl.

apiVersion: v1
kind: Namespace
metadata:
  name: apache
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: apache
  namespace: apache
  labels:
    app: apache
spec:
  replicas: 3
  selector:
    matchLabels:
      app: apache
  template:
    metadata:
      labels:
        app: apache
    spec:
      containers:
      - name: apache
        image: httpd:2.4.66-alpine3.23
        ports:
        - containerPort: 80
Enter fullscreen mode Exit fullscreen mode

Ok, namespace and deployment were created. Now, let's check the status of the three pods.

> kubectl get pods
NAME                      READY   STATUS         RESTARTS   AGE
apache-6d954c6ddf-4bbdl   0/1     ErrImagePull   0          14s
apache-6d954c6ddf-98plg   0/1     ErrImagePull   0          14s
apache-6d954c6ddf-g6g8l   0/1     ErrImagePull   0          14s
Enter fullscreen mode Exit fullscreen mode

The pods are not ready, because the image cannot be loaded. We already know this kind of error message 😩

> kubectl describe pod apache-6d954c6ddf-4bbdl
...
Warning  Failed     49s                  kubelet            Failed to pull image "docker.io/library/httpd:2.4.66-alpine3.23": failed to pull and unpack image "docker.io/library/httpd:2.4.66-alpine3.23": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://docker-images-prod.6aa30f8b08e16409b46e0173d6de2f56.r2.cloudflarestorage.com/registry-v2/docker/registry/v2/blobs/sha256/01/01f7e8ffbe696febf182e228b900ed95d731b7abfea0867a4be2440e8cb2f406/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=f1baa2dd9b876aeb89efebbfc9e5d5f4%2F20260225%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20260225T102347Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=789241fa858366a7b2861f9a768eeaf34b6b02f02ceb52cee7e1d516454126c9": tls: failed to verify certificate: x509: certificate signed by unknown authority
...
Enter fullscreen mode Exit fullscreen mode

Well it means that we have to bring the certificate of the Zscaler CA into the kind container, which in turn runs the components of the Kubernetes cluster. As we have seen already in the outputs above, kind is running as a Docker container with the aid of Colima. This container must be enhanced with the required trust settings.

> docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED        STATUS        PORTS                       NAMES
01cfe1e8ffb7   kindest/node:v1.35.0   "/usr/local/bin/entr…"   21 hours ago   Up 21 hours   127.0.0.1:53740->6443/tcp   kind-control-plane
Enter fullscreen mode Exit fullscreen mode

We can create a kind cluster using a configuration file. This approach allows to start the kind container(s) with the needed certificate file of the CA. So, let's delete the previously created kind cluster and create a new one using a configuration file.

> kind delete cluster
Deleting cluster "kind" ...
Deleted nodes: ["kind-control-plane"]
Enter fullscreen mode Exit fullscreen mode

Afterwards the kind cluster is removed and the kind-control-plane container is stopped and deleted. In the next step we create the configuration for kind in a file called kind-config.yaml and add a mount point to load the CA certificate into the container. Further I will create a separate worker node. This is not needed, but shows the possibilities you get with kind to create a local cluster consisting of several nodes and to support local high availability (more).

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - hostPath: <path_on_host_system>/zscaler.crt
    containerPath: /usr/local/share/ca-certificates/zscaler.crt
- role: worker
  extraMounts:
  - hostPath: <path_on_host_system>/zscaler.crt
    containerPath: /usr/local/share/ca-certificates/zscaler.crt
Enter fullscreen mode Exit fullscreen mode

With the configuration file the kind cluster can be set up again.

> kind create cluster --config="kind-config.yaml"
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.35.0) 🖼
 ✓ Preparing nodes 📦 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
 ✓ Joining worker nodes 🚜 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a nice day! 👋
> kind get clusters
kind
> kind get nodes
kind-control-plane
kind-worker
> docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                       NAMES
c22c6fff01b4   kindest/node:v1.35.0   "/usr/local/bin/entr…"   59 seconds ago   Up 58 seconds   127.0.0.1:54564->6443/tcp   kind-control-plane
54f5d47f27b1   kindest/node:v1.35.0   "/usr/local/bin/entr…"   59 seconds ago   Up 58 seconds                               kind-worker
Enter fullscreen mode Exit fullscreen mode

As we can see the cluster is up with a control plane node and a worker node and with 'docker exec' we are able to check the volume mount with the CA certificate.

> docker exec -it kind-control-plane ls -la /usr/local/share/ca-certificates/zscaler.crt
-rw-r--r-- 1 root root 1732 Feb  1 15:56 /usr/local/share/ca-certificates/zscaler.crt
Enter fullscreen mode Exit fullscreen mode

Well, but two last steps are required, because the CA certificate is just available to the Linux Operating System in the container, but it has not been added to the list of trusted certs. Additionally, the containerd daemon within the kind containers must be restarted to make the updated trusted certificates available to it.

> docker exec -it kind-control-plane /usr/sbin/update-ca-certificates
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
> docker exec -it kind-worker /usr/sbin/update-ca-certificates
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
> docker exec -it kind-control-plane systemctl restart containerd
> docker exec -it kind-worker systemctl restart containerd
Enter fullscreen mode Exit fullscreen mode

Because we have two nodes and have to call the commands for both of the nodes, a corresponding shell script would be helpful here, especially when working with even more control plane and worker nodes.

After applying the above Kubernetes deployment of the Apache HTTP server again, the three pods should be started after kind or rather containerd has successfully pulled the container image from Docker Hub 🚀🙌

> kubectl apply -f install-apache.yaml
namespace/apache created
deployment.apps/apache created
> kubectl get pods
NAME                      READY   STATUS    RESTARTS   AGE
apache-6d954c6ddf-kd59s   1/1     Running   0          37s
apache-6d954c6ddf-mp84x   1/1     Running   0          37s
apache-6d954c6ddf-rr47p   1/1     Running   0          37s
Enter fullscreen mode Exit fullscreen mode

Hope this helps to get your Docker with Colima and Kubernetes with kind running in your enterprise environment 🛟

I'll take a look soon, if it would be possible to build custom container images containing the needed cert settings. So, stay tuned ✌️

References

https://github.com/abiosoft/colima
https://github.com/abiosoft/colima/issues/131
https://docs.ddev.com/en/stable/users/usage/networking/
https://kind.sigs.k8s.io/
https://github.com/kubernetes-sigs/kind/issues/1010#issuecomment-546238916
https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/1159454#1159454

Top comments (0)