You can run kubectl from a docker container with Docker Desktop

#nuvolaris #docker #kubernets #serverless

In the process of setting up the development environment for Nuvolaris, I decided to switch from virtual machines to containers, leveraging the devcontainer feature of VSCode. Then I built a Docker image that is a perfect fit for our needs: there everything is in it to build our project.

But, I almost went on panic when I realized that, since a key component is an operator, and a cli that interacts with Kubernetes, we need to be able to access a Kubernetes cluster for development FROM a container.

In my initial plans, I would have setup a cluster Kubernetes in the virtual machine itself. But now, that everything is in a container? I actually felt initially very smart having the idea of setting up a Kubernetes cluster using kind. It is a tool that can build a Kubernetes cluster just using docker. But my plans went immediately awry because kind expects to be able to access to docker using localhost, and this is not the case within a container.

Even if I was able to access to docker from docker with a non-root user, inside the container there is not the proxy to localhost that is available outside a container. It is probably possible to configure kind to work anyway, but my first attempt failed.

Then I went in deep investigation mode, and I discovered that with Docker Desktop all the services are available using the domain docker.internal. And this domain is available also outside of the container. Then I thought that maybe it is possible to access to the Kubernetes that Docker itself provides from inside a container. maybe changing the configuration. Actually the situation is much better!

When you enable Kubernetes with Docker Desktop it creates a configuration file .kube/config that actually uses the domain kubernetes.docker.internalto talk to Kubernetes from the outside.

So I just copied the generated kubeconfig inside the container:

docker cp $HOME/.kube/conf container:/home/nuvolairs/.kube/config

and kubectl worked from inside the container, talking to Docker's Kubernetes!!!

Yes, this is a feature already available in Docker Desktop but not documented (or at least I was unable to find specific documentation for it).

I discovered it by myself and it saved my day and a week of efforts building the development environment that is now perfectly usable for our needs.