Alex Ortiz

Posted on Oct 26, 2019 • Updated on Nov 16, 2019

docker learn #03: Learnings from Silicon Valley

#docker #containers #devops

I went to Silicon Valley last weekend for the Silicon Valley Code Camp SVCC 2019, held at PayPal's global headquarters. Here's me looking mesmerized at one of the many very awesome talks:

From "Silicon Valley Code Camp Saturday" by Peter Kellner

As you can imagine, at a conference full of topics related to software development, engineering, operations, security, and cloud architecture, Docker came up quite a bit, so I learned a lot because of it 🎉. Here's some of what I learned on this awesome work trip.

Everything I Learned about Docker in Silicon Valley Last Weekend

First, here were the sessions I attended. Where Docker came up, I've tagged it with docker:

Knowing Linux and Docker Security for Kubernetes Security by Sunil Sabat docker
Deploy deep learning models as a web microservice in minutes by Shruthi Swaminathan docker
Getting Started with The Go Programming Language ( golang ) by Todd McLeod docker
Introduction to Kubernetes and OpenShift by Dave Nugent docker
ACID Transactions with MongoDB by Nuri Halperin
Powering DesignOps at PayPal by Stan Carrico
How Humans Learn and Why It Matters by Ted Young
Serverless Swift - for Effective Backend as a Service by Marek Sadowski docker
Serverless containers - a match made in heaven by Martin Omander docker
Building Reliable and Resilient Services by Manoj Agarwal
Master Dynamic Programming by Hien Luu

And now for some insights! The stuff below isn't sorted by importance or by the lecture I learned or heard it in.

Container Adoption Skyrocketed in the Last Three Years; May Double in the Next Three

The 2019 Container Adoption Survey predicts that container technology will be a $4.3B market by 2022, double what it is today ($2.1B). To put that further into context, Dave Nugent explained that 87% of IT Admins who responded to that survey this year are running containers, and 90% of those containers are in production. Of the IT Admins who responded, 65% are using two or more orchestration tools, such as Kubernetes. (Just three years ago, 72% of respondents had said that no, they were not using orchestration services. By last year, the numbers had climbed only a little, with only 34% of respondents saying they were using Kubernetes. So those figures from 2019 are impressive).

On that note, if you know containers, you know how popular Google's container orchestration technology Kubernetes is. On the Kubernetes repo on GitHub, you can see that it has 84,000+ commits, 2,200+ open issues, and a whopping 30,000 issues that are closed.

In other words, containers and their orchestrators are on very heavy rotation.

A Little Go History, and its Choice for both Docker and Kubernetes

From Todd McLeod's presentation, I learned how Golang was invented. In 2005, Google assessed every programming language on the planet but didn't find any that met their needs at the time. For context, multicore computers were a new thing, so there weren't yet any programming languages capable of natively benefitting from such computers, i.e., using multiple cores to benefit from parallelism.

Google wanted three things:

1) ease of programming
2) efficient execution
3) efficient compilation

After not finding a language that could achieve all three, and wishing to utilize a language that could take advantage of multicore processing, Google asked Ken Thompson (of B, C, Unix, and UTF-8), Robert G (of HotSpot and JVM), and Rob Pike (also of Unix, UTF-8) to create such a language, and Go was the result.

Kubernetes and Go both coming from Google, I get why Go was the obvious choice to write Kubernetes. What's interesting is that Docker is also written in Go. Perhaps Go was chosen for Docker because of Go's reputation as a security-conscious, type-safe, and fast language, or it could have been a coincidence.

Alternatives to Docker?

I noticed that Docker had the strongest word-of-mouth of any container platform throughout the conference. There are other container runtime engines, such as containerd and CoreOS's rkt. But Docker is by far the most popular container runtime today.

Even in the enterprise space, Docker is the main contender currently. Both Google via Kubernetes and IBM via Red Hat OpenShift make heavy use of Docker's container infrastructure. But it will be interesting to see if any viable alternatives to Docker emerge in the coming years, in the enterprise or beyond. One space to watch may be Google's relatively new offering, KNative ("Kubernetes meets Serverless"), which is pitched as having the best of containers plus the best of FaaS and PaaS serverless architecture. I don't yet know enough about the history of Linux containers, about why Docker became so popular, and about what Docker did better than anyone else at a technical level and a community level to become so attractive and widely adopted, but in tech, nothing lasts forever. Even top-tier, hyper-popular incumbent technologies get disrupted eventually, and sometimes quickly.

What would it take? This Elon Musk quote is pretty relevant on this point:

"If you're entering anything where there's an existing marketplace, against large, entrenched competitors, then your product or service needs to be much better than theirs. It can't be a little bit better, because then you put yourself in the shoes of the consumer... you're always going to buy the trusted brand unless there's a big difference."

I'm loving learning docker but am also intrigued about larger questions of what makes containers so interesting, and so interesting right now, and what new technologies, new pressures, and new solution trends could mean for containers in the future. Lots to learn!

To Understand Docker Security, Understand Linux Security

As I heard in Sunil Sabat's lecture, the security principles behind Docker (and Kubernetes) actually build on the sound security foundations known to those familiar with the Linux ecosystem. The best practices applied to Linux kernels transfer nicely to Linux containers and Kubernetes clusters & pods.

The building blocks are things like

resource limits on CPU and memory usage and requests
kernel namespaces
control groups (cgroups) for runtime security and to enforce isolation
mandatory access control (MAC)
security profiles, policies, and features like AppArmor, SELinux, and seccomp

All pretty advanced stuff, but also all relevant for the "container supply chain" that extends fro Linux to Docker to Kubernetes and into cloud platforms like AWS or GCP.

Some More TidBits

There's lots more that I learned about at the conference, but here is a roundup of other miscellany:

Cool new concept / term overheard: pre-warmed containers. This is relevant in situations when you want to avoid long "cold-start times" for container images that haven't been run for weeks
With Google Cloud Run, which is a managed platform for stateless containers, it's possible to get Google to run containers for you, and you can later choose to do your own container engineering ops
I heard a cool example of using a container to run an open-source PDF conversion app to automatically convert Word documents into PDFs on GCP
Martin Omander of Google says he recently got a COBOL hello world app running in a container :)
With FaaS/PaaS platforms like IBM OpenWhisk, you can "run anything in Docker as function"
More from IBM: I saw a presentation on how its Model Asset Exchange (MAX) and Data Asset Exchange (DAX) can be used to a) find open-source deep learning algorithms, b) find open-source data sets, and then c) use Docker containers to deploy lightweight neural networks for data analysis. In other words, you can run machine learning models as microservices using Docker. Wow!

There was more, but that's all I can squeeze in for this week. But not before some bonus Q&A!

This Week's Docker Answer Roundup

As I learn, I'm answering questions. Here are a few I answered this week.

On the purpose of Docker

Q: What is the purpose of Docker?

Here's my answer:

The purpose of Docker is to help you create and deploy Linux containers easily, portably, and reliably. For some technology architectures and use cases, running apps on a local machine or a virtual machine makes a lot of sense, but for situations when you want to run stateless applications or tasks that can be containerized quickly and also deployed across dozens, hundreds, or even thousands of users, Linux containers are sometimes the way to go. When that’s the case, Docker gives you a mechanism to create those containers (using Docker images).

On what Dockerfiles are for

Q: What is Dockerfile for in Docker?

Here's my answer:

A Dockerfile is a plaintext document in which you list the instructions Docker needs to use to create a Docker image. From that Docker image, docker containers can later be made and run. (A container is a single process, task, or application.)

The instructions in the Dockerfile include instructions like

which base image (for example, which Linux operating system) your application runs on;
which commands to run or application dependencies to install
files or directories that must be copied to the docker container in order for your app to work correctly
which ports need to be exposed (if applicable) for your application to work
and lots more

All of this information in the Dockerfile is then used by your Docker host (server or daemon) to create a docker image that can be used and reused to build any number of containers.

On the difference between two similar-looking docker commands

Q: What are the differences between the ‘docker run’ and the ‘docker create’?

Here's my answer:

The instruction docker run is really a combo command of two instructions in one: docker create + docker start:

docker create builds you a container from a docker image
docker start instantiates the container

There will be situations in which you want to create a container without actually starting it. If so, you’d use docker create, and later on you’d use docker start.

Here’s an example.

Let’s say you create three linux containers based on the latest version of alpine. Suppose you name them container1, container2, and container3, so that you can refer to them with these custom names rather than with their default names or container IDs.

You would use the following commands:

docker create --name="container1" alpine:latest sleep 600
docker create --name="container2" alpine:latest sleep 600
docker create --name="container3" alpine:latest sleep 600

Here are the components of these instructions:

docker create is the instruction to create the container
--name="custom_name" specifies a key-value pair that names the container
alpine:latest is the docker image from which the container instance is made
sleep 600 is a Linux command (explained below)

Note: Alpine is a lightweight Linux operating system, not an application or process, so if you used docker create --name="container1" alpine:latest, without the sleep command, then Docker would create an alpine container but immediately exit it, because there’s nothing happening inside that container. So for this example’s sake, we keep our three alpine containers running for 10 minutes by using the Linux sleep command set to 600 seconds.

Docker would thus create three alpine containers for you, each remaining active for 600 seconds, i.e., ten minutes. But the three containers would not yet be running.

If you checked this, by typing docker ps -a into the docker command line, your three containers would show up with a STATUS of “Created”.

Now comes the important part. Since you’ve used the docker create command, you could then selectively start one of these containers at will. For example, container1:

docker start container1

Now, container1 would show up with a STATUS of “Up”, and container2 and container3 would continue to show up with a STATUS of “Created”, as before.

From there, you can start, stop, or restart a container as you please. (Of course, in this example, after ten minutes, Docker will exit these containers, per the Sleep instructions from above.)

In summary, docker run creates and immediately starts a container, whereas docker create lets you create a container without starting it. This gives you the flexibility to start and immediately run containers when you need to while also being able to create containers that you don’t. For example, if you wanted to run a command inside a container that had already been created and was already running, it wouldn’t make sense to use the command docker run, as that command always creates a new (additional) container. You would simply use docker create and then docker start instead.

Some Helpful Resources

Here are a few items I found helpful this past week:

Online Playground: Play With Docker
Video: VMWare, Kubernetes in 5 mins
Book: William E Shotts Jr., The Linux Command Line: A Complete Introduction

And with that, I say farewell to another week of docker learn. See you next time!

Hit the follow button, and be sure to stay up to date on all the awesome work my teams are doing at Educative.io and in our EdPresso Community. They are awesome and you should follow them on Twitter at @educativeinc and @edpressoc, respectively.