Vahid Aghajani

Posted on Oct 12 Originally published at software-engineer-blog.com

Docker Compose vs an Orchestrator — Compose Is a Command, an Orchestrator Is a Loop

#docker #devops #programming #softwareengineering

📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram

Originally published on software-engineer-blog.com.

Docker Compose versus an orchestrator. Both start your containers from one file, so many people think the orchestrator is only a bigger version of Compose. It is not. Compose is a command. An orchestrator is a loop.

This article follows one service that wants three copies, and one machine that dies at three in the morning. It reads one Compose file, one small Python model and one Kubernetes file, line by line. At the end you know what the restart line protects, what it cannot protect, and what an orchestrator costs you.

The server dies at 03:00. Who starts your containers again?

You run one service as three containers on one server. A container is one packaged, running copy of your program.

One Compose file describes all three copies, and one command starts them. For months this works.

time	what happens
for months	3 copies run on one server
03:00	the server loses power
03:00	3 copies become 0
	the file still says 3

But a file cannot do anything. And the command that read the file finished months ago.

So here is the question:

When the machine dies, who starts your containers again?

What this article covers (and what it does not)

The subject is DevOps: the work of running software on servers. This article covers one decision inside it: Compose on one machine, or an orchestrator on many machines.

An orchestrator is a program that keeps your containers running across a group of machines.

Left alone on purpose: what Docker and Kubernetes are, as two tools, and container images. Both have their own episodes.

Where does this live? It is built into your tools. You will not write an orchestrator yourself. Kubernetes, Docker Swarm and Nomad are orchestrators, and they are already built. What you touch is small:

a file that says how many copies you want;
the number of machines;
the failure you must plan for.

This decision matters the first time one machine is not enough, or one machine is not safe enough.

A light switch and a thermostat

Think of two things on a wall.

The first thing is a light switch. You press it once, and the lamp turns on. After that, the switch does nothing. If the lamp breaks at night, the switch does not know, and the room stays dark until you come back.

The second thing is a thermostat. You give it a target: twenty-one degrees. It measures the room again and again. When a window opens and the room gets cold, the thermostat turns the heater on. Nobody told it to do that.

	a light switch	a thermostat
what you give it	one press	a target: 21 degrees
what it does after that	nothing	measures the room, again and again
when something breaks at night	the room stays dark until you come	it turns the heater on by itself
in this article	Compose: a command	an orchestrator: a loop

Compose is the light switch. An orchestrator is the thermostat.

The Compose file, line by line

Here is the Compose file. It is real, and it is short.

services:
  web:
    image: registry.example.com/web:1.4
    restart: unless-stopped
    deploy:
      replicas: 3

# then, on the one machine, you run:
#   docker compose up -d

The service is called web.
The image line says which packaged program to run.
The restart line tells the Docker engine to start a container again if that container stops. The Docker engine is the program on the machine that runs containers.
The replicas line says three. A replica is one running copy.

Now you run one command: docker compose up.

step	what the command does
1	reads this file, once
2	asks the engine on this one machine for three containers
3	is finished

The whole system, drawn once

Part	What it is	What it does
The file	one YAML file, one number	holds the number you want: 3
The command	`docker compose up`, one process	reads the file once, starts 3 copies, then exits
Node A	1 server, 1 container engine	runs the copies

A node is one machine that runs containers.

Two arrows join the parts. The file goes to the command: read once. The command goes to node A: start 3. Both arrows point one way only. Nothing flows back, and after the command exits nothing runs in the middle any more.

So remember one number for this whole article.


wanted	3
running	3
machines	1

The restart line: what it protects, and what it cannot

But the file has a restart line. Is that not enough? Test it.

The first failure: one container crashes. The engine on node A sees it, and starts that container again in a few seconds. Running: three again. Good. (The Linux processes episode shows the same idea with systemd.)

The second failure: node A itself dies. The engine was on node A, so the engine is dead too.

time	what happens	running
before	3 copies on node A	3
a crash	one container crashes, the engine restarts it	2, then 3
03:00	node A dies, and the engine dies with it	0
03:40	a person wakes up and fixes it	3

That is forty minutes with zero copies.

So the restart line protects a container. Here is the second question:

What protects you from a dead machine?

The answer: a loop that never exits

An orchestrator answers that question with a loop.

This code is a small model of what Kubernetes, Swarm and Nomad do inside. You do not copy it into your service. It only helps you reason about them.

DESIRED = 3          # copies of web we asked for
ROOM = 2             # copies one node can hold

def reconcile(nodes):
    alive = {n: c for n, c in nodes.items() if c is not None}   # keep only the nodes that are alive
    missing = DESIRED - sum(alive.values())                     # desired, minus the copies that run
    while missing > 0:
        free = [n for n in sorted(alive) if alive[n] < ROOM]    # nodes that still have room
        if not free:
            break                                               # no room: those copies stay pending
        target = min(free, key=lambda n: alive[n])              # the node with the fewest copies
        alive[target] += 1                                      # start one copy there
        missing -= 1
    return alive, missing

while True:
    state, pending = reconcile(read_nodes())
    time.sleep(5)                                               # again every five seconds, forever

DESIRED is three: the copies we asked for.
ROOM is two: one node can hold two copies.
The reconcile function first keeps only the nodes that are alive.
Then it counts what is missing: desired, minus the copies that run.
While something is missing, it lists the nodes that still have room.
If no node has room, it stops, and those copies stay pending.
Otherwise it picks the node with the fewest copies, and starts one copy there.
At the bottom, the function runs again every five seconds, forever.

The same picture, with one change

In the middle there is now a control loop, and it never exits.

Part	What it is	What it does
The file	one YAML file, one number	holds the number you want: 3
The control loop	the orchestrator, a service that never exits	reads the status of every node and starts what is missing
Nodes A, B, C	3 servers, 1 container engine each	run one copy each

You send the file once, and the loop keeps the number. The arrows between the loop and the nodes now go both ways: status comes up, start commands go down.

Now node A dies, the same machine as before.

step	what the loop does	node A	node B	node C	running
before		1	1	1	3
1	asks every node for its status; node A does not answer	dead	1	1	2
2	counts: desired 3, running 2, missing 1	dead	1	1	2
3	node B and node C each hold one copy, so it picks node B	dead	2	1	3

Running: three again.

No person woke up. And two copies answered users during the whole time.

What you write: a file, not the loop

So what do you write? Not the loop. You write a file, and it is very close to the Compose file.

This file is for Kubernetes, and it is called a Deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels: {app: web}
  template:
    metadata:
      labels: {app: web}
    spec:
      containers:
        - name: web
          image: registry.example.com/web:1.4
          resources:
            requests:
              memory: 2Gi

replicas: 3. That is the same number as before.
The image is the same too.
The memory line is new. It says that one copy needs two gigabytes. The orchestrator uses that line to decide which node has room.

You send this file once, and the control loop keeps it.

Now the cost in time. Kubernetes does not replace the copies at once. By default, it waits about five minutes for a silent node to come back.

The catch: the loop cannot create machines

And here is the catch.

The loop can start containers. It cannot create machines. So the room must exist before the failure.

The check is one line.

def survives(nodes, room, desired, failures): return (nodes - failures) * room >= desired

survives(3, 2, 3, failures=1)   # (3 - 1) * 2 = 4    4 >= 3    True
survives(3, 2, 3, failures=2)   # (3 - 2) * 2 = 2    2 <  3    False
# False: 2 copies run, 1 copy stays pending

Take the nodes, subtract the failures, and multiply by the room on each node. The result must be at least the desired number.

nodes	failures	room that is left	desired	result
3	1	2 nodes × 2 = 4	3	4 is at least 3: we survive
3	2	1 node × 2 = 2	3	2 is less than 3: 2 copies run, 1 stays pending

You pay for spare room, or the loop has nowhere to put a copy.

The bill, because it is real

So which one do you choose?

	Compose	an orchestrator
what it is	a command that runs once	a loop that never exits
machines for the work	1	3 in our example, with spare room
machines for the control loop	0	usually 3 of its own
a container crashes	the engine restarts it	the loop repairs it
a machine dies	0 copies until a person comes	no outage: 2 copies keep answering, and the third is started again
what you learn and run	one file, one command	more machines and more to learn

Why three machines for the control loop? Because the control loop must survive a dead machine too. Three can lose one, because two of three still agree.

And what you pay for it:

what you gained	what you pay
A dead machine causes no outage.	Three nodes with spare room, not one.
No person has to wake up at 03:00.	A control plane of three machines, and more to learn.
The number in the file is kept, not only started.	A default wait of about five minutes in Kubernetes before a silent node is replaced.

Compose is right for your laptop, for tests, and for a small product that can accept a short outage.

Choose the orchestrator when one machine is not enough, or when forty minutes of zero is not acceptable.

For your next interview

What is the difference between Docker Compose and an orchestrator?

Docker Compose is a command. It reads one file and starts your containers on one machine, one time. An orchestrator, such as Kubernetes, is a control loop over many machines. It compares what you want with what is running, again and again, and it repairs the difference.

Three words to remember:

Desired state: the number of copies you asked for.
Reconcile: compare desired with running, and start what is missing.
Headroom: the spare room on other nodes that a repair needs.

The one-line verdict

Compose is a command that reads your file once on one machine; an orchestrator is a loop that keeps the number, so stay on Compose while a short outage is acceptable, and move to an orchestrator when one machine is not enough or not safe enough, and pay for the spare room first.

One thing to go and look at tonight: open your Compose file and count the machines it runs on. If the answer is one, decide how many minutes of zero copies your product can accept.

References and further reading

On the Compose file and the command

Docker, Compose Deploy Specification (Docker documentation) — replicas specifies the number of containers that should be running at any given time: docs.docker.com — Compose Deploy Specification
Docker, docker compose up (Docker documentation) — the command builds, creates and starts the containers for a service, and with --detach it leaves them running in the background: docs.docker.com — docker compose up

On the restart line

Docker, Start containers automatically (Docker documentation) — restart policies control whether a container starts again automatically when it exits, and unless-stopped is one of them: docs.docker.com — Start containers automatically

On the control loop and the desired state

The Kubernetes Authors, Controllers (Kubernetes documentation) — a control loop is a loop that never ends and regulates the state of a system; the page uses the thermostat as its example, and each controller moves the current state closer to the desired state: kubernetes.io — Controllers
Burns, Grant, Oppenheimer, Brewer and Wilkes, Borg, Omega, and Kubernetes (ACM Queue, 2016) — the reconciliation controller loop, which compares a desired state with the observed state, is shared by all three systems and makes them robust to failures: research.google — Borg, Omega, and Kubernetes

On the Deployment file, the memory line and the five-minute wait

The Kubernetes Authors, Resource Management for Pods and Containers (Kubernetes documentation) — the scheduler uses the resource request of a container to decide which node to place it on, and a Pod that fits on no node stays unscheduled until a place is found: kubernetes.io — Resource Management for Pods and Containers
The Kubernetes Authors, Taints and Tolerations (Kubernetes documentation) — Kubernetes adds a toleration of 300 seconds for a node that is not ready or unreachable, so Pods stay bound to that node for 5 minutes after the problem is detected: kubernetes.io — Taints and Tolerations

On the three machines of the control loop

Docker, Raft consensus in swarm mode (Docker documentation) — Raft tolerates up to (N-1)/2 failures and needs a majority of (N/2)+1 members to agree, so three managers can lose one: docs.docker.com — Raft consensus in swarm mode

If a reference you would expect is missing, say so in the comments and I will add it.

Watch the full episode: Docker Compose vs an orchestrator — Explained in Detail on YouTube

DEV Community