DEV Community: penumbra23

Container Runtime in Rust - Part II

penumbra23 — Wed, 06 Oct 2021 16:13:20 +0000

Part I of the series described the filesystem layout and how the runtime jails the container process inside the root filesystem of the container.
Part II dives a bit deeper into the implementation and shows how the runtime creates the child process and how they communicate until the point where the user-defined process kicks in. It will also describe how a pseudo-terminal is set up and show the importance of Unix sockets.
By the end of this part, we should have a basic runtime that's interoperable with Docker.

Clone

Part 0 briefly explained the clone syscall. It's like fork/vfork, but with more options to control the child process. Actually, some implementations of fork propagate the call to clone().

Besides controlling which parts of the execution context get shared from the parent process, the clone call offers the possibility for us to create a separate memory block for the child's stack. The nix implementation of clone has the following signature:

pub fn clone(
 cb: CloneCb<'_>, 
 stack: &mut [u8], 
 flags: CloneFlags, 
 signal: Option<c_int>
) -> Result<Pid>

The signal argument, if specified, is sent back to the parent when the child process terminates.

Here is the code snippet depicting the create command and the parent-child relation:

pub fn create(create: Create) {
    let container_id = create.id;
    let root = create.root;
    let bundle = create.bundle;

    // Load config.json specification file
    let spec = match Spec::try_from(Path::new(&bundle).join("config.json").as_path()) {
        Ok(spec) => spec,
        Err(err) => {
            error!("{}", err);
            exit(1);
        }
    };

    const STACK_SIZE: usize = 4 * 1024 * 1024; // 4 MB
    let ref mut stack: [u8; STACK_SIZE] = [0; STACK_SIZE];

    // Take namespaces from config.json
    let spec_namespaces = spec.linux.namespaces.into_iter()
        .map(|ns| to_flags(ns))
        .reduce(|a, b| a | b);

    let clone_flags = match spec_namespaces {
        Some(flags) => flags,
        None => CloneFlags::empty(),
    };

    let child = clone(Box::new(child_fun), stack, clone_flags, None);

    // Parent process
}

One problem that arises from viewing the above is the communication between parent and child. In the case of spawning a thread, one could say that creating an in-memory channel would solve the issue (Rust has excellent support for multithreading and thread-safe multi-producer, single-consumer queue). In our example, this is not the case because it's creating (or better say cloning) a new process with its separate memory space.

Inter-process communication (IPC) is a set of techniques that allows processes to communicate with each other. The two most widely used are:

shared memory
sockets

In our example, we'll use Unix sockets (AF_UNIX) to establish a "client-server" channel between the parent and the child processes. The container process is going to bind to that Unix socket and listen for incoming connections from the parent process. Both processes will use the socket connection to inform each other when different parts of the execution pass or fail. The socket connection also comes in handy when the start command is being invoked, to inform the container process to start the user-defined program. The following diagram describes the "protocol" better:

Unix Sockets

For those unfamiliar with Unix (domain) Sockets, this Linux feature will hopefully be mind-blowing (at least for me it was). Unix sockets are an inter-process communication mechanism that establishes a two-way data exchange channel between processes running on the same machine. One can think of them as TCP/IP sockets that don't use the network stack to send and receive data, but a file on the filesystem.

In the case of the container runtime, Unix sockets offer a bi-directional data exchange for the runtime parent and child process. That exchange channel is essential for the container runtime! What if something breaks down in the child process? How can the parent process continue? Or how does the child know when the start command gets invoked?

For these purposes, the container runtime implements IPC channels. Those are bidirectional channels using Unix domain sockets. One process acts as a "server" and the other processes (known as "clients") connect to the server process.

To shorten up the story, here's a rough idea of how that Rust code might look:

pub struct IpcChannel {
    fd: i32,
    sock_path: String,
    _client: Option<i32>,
}

impl IpcChannel {
    pub fn new(path: &String) -> Result<IpcChannel> {
        let socket_raw_fd = socket(
            AddressFamily::Unix,
            SockType::SeqPacket,
            SockFlag::SOCK_CLOEXEC,
            None,
        )
        .map_err(|_| Error {
            msg: "unable to create IPC socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        let sockaddr = SockAddr::new_unix(Path::new(path)).map_err(|_| Error {
            msg: "unable to create unix socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        bind(socket_raw_fd, &sockaddr).map_err(|_| Error {
            msg: "unable to bind IPC socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        listen(socket_raw_fd, 10).map_err(|_| Error {
            msg: "unable to listen IPC socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;
        Ok(IpcChannel {
            fd: socket_raw_fd,
            sock_path: path.clone(),
            _client: None,
        })
    }

    pub fn connect(path: &String) -> Result<IpcChannel> {
        let socket_raw_fd = socket(
            AddressFamily::Unix,
            SockType::SeqPacket,
            SockFlag::SOCK_CLOEXEC,
            None,
        )
        .map_err(|_| Error {
            msg: "unable to create IPC socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        let sockaddr = SockAddr::new_unix(Path::new(path)).map_err(|_| Error {
            msg: "unable to create unix socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        connect(socket_raw_fd, &sockaddr).map_err(|_| Error {
            msg: "unable to connect to unix socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        Ok(IpcChannel {
            fd: socket_raw_fd,
            sock_path: path.clone(),
            _client: None,
        })
    }

    pub fn accept(&mut self) -> Result<()> {
        let child_socket_fd = nix::sys::socket::accept(self.fd).map_err(|_| Error {
            msg: "unable to accept incoming socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        self._client = Some(child_socket_fd);
        Ok(())
    }

    pub fn send(&self, msg: &str) -> Result<()> {
        let fd = match self._client {
            Some(fd) => fd,
            None => self.fd,
        };

        write(fd, msg.as_bytes()).map_err(|err| Error {
            msg: format!("unable to write to unix socket {}", err),
            err_type: ErrorType::Runtime,
        })?;

        Ok(())
    }

    pub fn recv(&self) -> Result<String> {
        let fd = match self._client {
            Some(fd) => fd,
            None => self.fd,
        };
        let mut buf = [0; 1024];
        let num = read(fd, &mut buf).unwrap();

        match std::str::from_utf8(&buf[0..num]) {
            Ok(str) => Ok(str.trim().to_string()),
            Err(_) => Err(Error {
                msg: "error while converting byte to string {}".to_string(),
                err_type: ErrorType::Runtime,
            }),
        }
    }

    pub fn close(&self) -> Result<()> {
        close(self.fd).map_err(|_| Error {
            msg: "error closing socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        std::fs::remove_file(&self.sock_path).map_err(|_| Error {
            msg: "error removing socket".to_string(),
            err_type: ErrorType::Runtime,
        })?;

        Ok(())
    }
}

The server calls the new method and binds to the .sock file. Then it calls accept and waits for incoming connections. On the other hand, the client just calls connect with the same .sock file and after that, the server and client can exchange messages. In the end, both processes call close and the communication is finished.

Note that I've used SOCK_SEQPACKET sockets, because the messages come in-order, it's connection-based and the message gets flushed all at once (opposite to SOCK_STREAM).

Terminal

To have a nice interaction with the container once it starts, the runtime should be able to provide a terminal interface if the user requested a terminal.

When running a Docker command like this one:

docker run alpine ping 8.8.8.8

you will see the output of the ping command sending ICMP requests to Google's DNS. The output of the ping command is piped through Docker, but when we want to stop the command (by using Ctrl+C) nothing happens. That's because when pressing the SIGINT key combination, the signal is sent to Docker which isn't passing the command to the actual container process.

On the other side, when running:

docker run -it alpine ping 8.8.8.8

and pressing Ctrl+C, the command terminates immediately, as if it runs on the host machine. Why is that?
That's because in the first example the container process doesn't have a terminal instantiated, therefore the user nor Docker can't forward the signal to the container via tty.

Luckily for us, the -t option sets the terminal: true flag inside the config.json file. After that, it's the container runtime's responsibility to create a so-called "pseudo-terminal" (pty).

To simplify things, a PTY is a (master-slave) pair of communication devices that act like a real terminal. Any command sent to the master gets forwarded to the slave end, from text input to process signals. PTYs are a very important and used feature of the Linux kernel (ssh uses it!).

Now it's simple:

if terminal: true the container runtime creates a PTY
the slave descriptor goes to the child process
the master descriptor goes to the calling process (in this case Docker)

But how does the child process send the master descriptor to Docker?

Sigh… This was a real PITA to find out and the solution was outside the scope of the OCI runtime spec. runc developed a solution for which the steps are described here. TLDR; our friend Unix sockets came to help. Docker creates a Unix domain socket and passes it to the container runtime as console-socket argument. After the container runtime creates the PTY, it sends the master end to that same Unix socket with SCM_RIGHTS.

Conclusion

Finally, we have a ready-to-test OCI container runtime!

This part explained the clone syscall and how it detaches the execution context from the parent process. It also has a flexible API so that we can specify the new stack for the process.
Unix domain sockets play a big role here because they sync the whole parent-child communication and handle potential scenes when errors show up, on both sides.

Part II rounds up the Container Runtime in Rust series. The whole source code for the experimental container runtime can be found on this Github repo. Feel free to ask questions or point interesting things out in the implementation.

Container Runtime in Rust — Part I

penumbra23 — Sat, 18 Sep 2021 17:46:38 +0000

Intro

In Part 0 of the series, we’ve seen how processes can get a restricted view of the resources they see. This part will explain how a container runtime prepares and creates an isolated environment for the container process.

A prerequisite for this part is a basic knowledge of how the Linux filesystem works, what are inodes, symbolic links, and mount points.

The full source code for this post can be found here.

First, let’s start with the OCI spec.

Operations

At the time of writing, the OCI spec defines a minimum of five standard operations: create, start, state, delete and kill. Having that in mind, using the clap library we can generate a nice CLI interface in no time. It should look something like this:

let matches = App::new("Container Runtime")
    .subcommand(
        SubCommand::with_name("create")
            .arg(Arg::with_name("bundle").required(true))
            .arg(Arg::with_name("id").required(true)),
    )
    .subcommand(SubCommand::with_name("start").arg(Arg::with_name("id").required(true)))
    .subcommand(
        SubCommand::with_name("kill")
            .arg(Arg::with_name("id").required(true))
            .arg(Arg::with_name("signal")),
    )
    .subcommand(SubCommand::with_name("delete").arg(Arg::with_name("id").required(true)))
    .subcommand(SubCommand::with_name("state").arg(Arg::with_name("id").required(true)))
    .get_matches();

We’ll give the most attention to the create and start command because those are the two most important commands when running the docker run command.

The bundle directory contains the config.json file that holds all the metadata for creating the container:

ociVersion - version of the OCI spec
process - the user-defined process that the container executes (shell, database, web app, gRPC service, etc.) with the necessary args and environment variables
root - path to the subdirectory for the container root
hostname of the container
mounts - list of mount points inside the container

In addition, the OCI spec contains a platform-specific section enabling custom settings based on the platform where the container is being run. Because we are looking only at Linux containers, the linux section will be of use to us.

The create command is supplied with the container ID and the bundle path. Its purpose is to initialize the container process, mount all necessary subdirectories, “jail” the container inside the root.path folder, update all system variables inside the container (env, hostname, user, group), execute a couple of hooks (we’ll look into that later), assign the unique ID to the container itself, and wait until the start command gets fired. After the create command finishes, the container is in status Created and the user process MUST wait for the start command to start the actual container process.
Everything seems straightforward regarding the implementation, but the “jail” -ing part can be a bit confusing. How is it done?

chroot

Chroot is a syscall that changes the root directory of the calling process. It takes the new root path as an argument, which can be an absolute or relative path. The chroot command from the terminal does the same thing, except it takes an additional argument, namely the process that’s going to be executed inside the changed root.
Before we take a look at an example, first we need to prepare the new rootfs. Unfortunately, the binaries used in the jail must reside inside the chroot-ed directory (obviously), so we need to have a premade rootfs. Luckily, we can use our host OS binaries and mount bind the already existing files and end up with a structure like this:

Don’t worry if your listing differs, just be sure to have bash and ls inside the bin directory.

Let’s take a look at the chroot command (run with sudo):

As we can see, listing directories outside our root (ls ..) lists the jailed root and it seems that we can’t see anything outside. Also, listing the bin and lib directories gives the same result as with the above example.
One can say “that’s how containers are jailed” and jump right onto building a container from scratch. But, things aren’t that easy… Chroot doesn’t change the filesystem nor the mount points that the process sees. It just changes the view for the process’ root, but everything remains the same. And, breaking this jail is fairly easy as described here.

pivot_root

pivot_root on the other hand does exactly what we need. Given a new root and subdirectory of the current root, it moves the current root to the subdirectory and mounts the new root as the root mount point. In this way, it changes the physically mounted folder of the root directory. Later on, we can unmount the “old” root and just leave the newly created root mount point.
We’re going to take a look at an example.

NOTE: pivot_root changes the root mount point and can potentially make a mess out of your filesystem, so be sure to follow the steps below.

First, we need a real rootfs filesystem. We can’t use the above example because we mount bind the host binaries. We need an independent directory that can live on its own. For that, we are going to use Docker to export a fresh rootfs from an Alpine container. Then we’re going to use unshare (remember our friend from Part 0) to create a new mount namespace. Then we’re going to pivot the roots inside our container. It should look like this:

docker export simply copies files from the container to the host system into a tar archive. After exporting the rootfs from the Alpine image, we bind mount the directory to itself, why? Because by the specification of the pivot_root syscall, new_root must be a path to a mount point that’s different than “/”.

After preparing the container root, we need to create a new mount namespace to have it differently than our host environment, so that pivot_root doesn’t change anything on the host mount namespace. We create a temporary folder to hold the old root, pivot the root, unmount (or unlink with umount -l) the oldroot and, to finalize the swap, remove the oldroot folder.

Voila! Now we have a bash process running inside the jailed container folder.

In Rust code, mounting the rootfs folder with the nix crate would look something like this:

pub fn mount_rootfs(rootfs: &Path) -> Result<(), Box<dyn Error>> {
    mount(
        None::<&str>,
        "/",
        None::<&str>,
        MsFlags::MS_PRIVATE | MsFlags::MS_REC,
        None::<&str>,
    )?;

    mount::<Path, Path, str, str>(
        Some(&rootfs),
        &rootfs,
        None::<&str>,
        MsFlags::MS_BIND | MsFlags::MS_REC,
        None::<&str>,
    )?;

    Ok(())
}

The first mount changes the mount propagation of the root mount point to private (shared mount propagation isn’t allowed with pivot_root, for obvious reasons).

The code for the whole pivoting procedure should look something like this:

pub fn pivot_rootfs(rootfs: &Path) -> Result<(), Box<dyn Error>> {
    chdir(rootfs)?;

    std::fs::create_dir_all(rootfs.join("oldroot"))?;

    pivot_root(rootfs.as_os_str(), rootfs.join("oldroot").as_os_str())?;

    umount2("./oldroot", MntFlags::MNT_DETACH)?;

    std::fs::remove_dir_all("./oldroot")?;

    chdir("/")?;
    Ok(())
}

Note that both mount_rootfs and pivot_rootfs are called in the newly created mount namespace.

Special links & mounts

The OCI runtime spec defines a set of special symlinks. These symbolic links are used to pass the stdin, stdout, and stderr streams from the container engine (Docker, containerd) to the runtime and vice versa. It simply binds the container’s standard streams to the outside file descriptors of the container process. The container runtime needs to establish these symlinks before pivot_root.

The OCI runtime spec defines a set of filesystems that need to be mounted inside the container. While extracting some config.json files from images like Alpine, Ubuntu, Debian, the /dev/pts and /dev/shm mount points are present inside the mounts section of the runtime config spec.

Two important filesystems that need more attention are proc and sysfs.

The proc filesystem mounts to the /proc directory and acts as an interface for the internal structures of the kernel. For each process it holds a /proc/[PID] subdirectory that keeps the file descriptors, cpu and memory usage, mount information, page tables and many other things. For example, one can’t inspect the current mount points (with the mount command) until this fs isn’t created.

The exact command for mounting the proc fs is:

mount -t proc proc /proc

The sysfs filesystem is a pseudo-fs-like proc that provides an interface to the internal kernel objects. Opposite to the proc filesystem, it holds system-wide information like metadata of block and char devices, bus info, drivers, control groups, kernel info, and other global variables. Mounting the sysfs is the same as with proc:

mount -t sysfs sysfs /sys

Both proc and sysfs need to be mounted after pivot_root, when the new root mount point is created.

Devices

In Linux, everything is considered as a file. Hard drives, peripherals, even processes are fully described through file descriptors. Devices aren’t an exception either. Diskettes, CDROMs, serial ports, any device you attach should appear inside the /dev subdirectory under the root directory. Devices have types and the majority of devices are either block (store data of some kind) or character (stream or transfer data to/from) devices. Terminals, pseudo-random number generators and even the /dev/null file is considered a device too.

The OCI specification defines required devices for each container and the config.json contains a devices list under the linux section. The container runtime is responsible for creating these devices inside the container root directory. The syscall for creating devices is mknod. This syscall (also command inside the terminal) accepts 4 required parameters:

pathname - full path to the file location
type - block, character or other device types
major & minor - unique identifiers for the device

For example, char device with major minor numbers 1, 8 is the random device representing the pseudo-random number generator. Whenever your app requests a random number, this device gets a request.
We can easily generate the special devices with nix’s mknod function or, in case of binding to the host’s device (which is covered by the OCI spec) use the mount bind option.

Conclusion

We’ve seen how chroot changes the view of the root directory for the current process and how pivot_root swaps the root mount points, creating a logical isolation of the filesystem. We’ve also seen how to create standard container devices and that different containers can request special devices inside the mounts section of the config.json file.

Knowing how unshare and pivot_root work gives us the ability to manually create Linux containers in our terminal. In the next parts, we’ll dive a bit deeper into the implementation. Specifically on cloning the child process and preparations for starting the container command.

Container Runtime in Rust - Part 0

penumbra23 — Thu, 09 Sep 2021 17:42:03 +0000

Intro

From a DevOps perspective, I can only say what a time to be alive! With the rise of containers all the pain we had with setting up VMs, backup/restore procedures, dynamic provisioning because of scaling because of increased usage volume because of unoptimized architectural setup because of, etc. Everything is gone (well, almost everything from the above list) with a single docker-compose file or a convenient one-liner like helm install my-app.
But, what are containers essentially, and how does Docker work under the hood?

This will be a series describing my journey writing a small, experimental container runtime in what other language than the one perfect for the job — Rust!

Part 0 describes the system features of the Linux OS that Docker and other tools utilize to start a process that is isolated and protected or, in other words, a container. The whole series is based on Linux containers, so any occurrence of the word container is interchangeable with “Linux container”.

The full source code for this post can be found here.

Container

When being asked to describe, with as few words as possible, what a container is, most developers feel disappointed. Described with one tech word, it’s a forked or cloned process. It has a dedicated PID, it’s owned by a user and a group, you can list it with the ps command and send signals to it (yeah, signal #9 too). It’s nothing more than that, nothing super fancy bleeding edge around it, just an old-fashioned process.

But how is it isolated from the rest of the system?

The answer is namespaces.

Namespaces provide the logical isolation of resources for processes running in different sets of namespaces. There are different types of namespaces, for example, MOUNT namespace for all mount points that the current process can see, NETWORK namespaces for the network interfaces and traffic rules, PID for the process tree, and so on. Two processes running in different PID namespaces don’t see the same process tree. A separate NETWORK namespace gets its own network stack, routing table, firewalls and a loopback interface. Two processes with different network namespaces that bind to their respective loopback devices are bound to a separate logical interface so that traffic doesn’t interfere between them.

The MOUNT namespace contains the list of mount points a process can see. When first cloning from a mount namespace (the CLONE_NEWNS flag) all mount points are copied from the parent to the child namespace. Any additional mount point created in the child isn’t propagated to the parent mount namespace. Also, when the child process unmounts any mount point, it’s only being affected inside his mount namespace.

Root (“/”) isn’t an exception either. The root mount point doesn’t have to be (mostly isn’t) the same directory for different mount namespaces. When running a container, Docker (specifically containerd) creates a directory for each container’s root mount point. Before the actual container runs the user-defined process, it’s the container runtime’s responsibility to mount that directory as the container’s root. In that way, each container has its own root mount point that’s separated from the rest of the filesystem and all other containers running on the host.

Each process has a /proc/PID/ns subdirectory on the host containing symlinks for each namespace they belong to (pid, net, user, cgroup, …). If two processes belong to the same namespace, their symlinks will be the same.

Let’s see what namespaces a simple sleep command has (on my machine of course):

Running another process in the same shell and inspecting it’s namespaces gives the same symlinks as the above:

The two processes are run without any extra commands or setup, they have the same namespace symlinks, therefore they belong to the same namespaces.

Linux provides the UNSHARE syscall from the sched.h library to change the process’ execution context and allow it to create and enter new namespaces. After UNSHARE gets called with the specific bit mask (flags combining which namespaces to “unshare” from the execution context), the running process gets detached from the root namespaces into it’s own set of namespaces. Unfortunately, it’s not enough to call UNSHARE and expect to have an isolated container (for example, a parent process that “unshares” the PID namespace, still runs in the root PID namespace, but any child process created afterwards will enter the newly created PID namespace). Usually after the container runtime calls UNSHARE it gets followed by a fork/vfork call to create the actual container process.

CLONE syscall is a far better option and the implementation in Rust’s nix package feels more robust and fine-grained. It gives the ability to specify the namespace flags as in UNSHARE, forks a child process and creates the stack for the child.

SETNS syscall (NSENTER command) gives the option to change the given namespace to an already existing namespace by it’s file descriptor. For example, to fork a shell process that enters the mount namespace of process with PID 15, enter in a new shell (with root permissions):

nsenter --mount=/proc/15/ns/mnt /bin/sh

Enough theory! To confirm everything said above, let’s jump onto a Docker-based example. We’ll run an alpine container, list the process on the host system, inspect it’s namespaces and docker exec into it with SETNS.

Let’s run a long running sleep container:

After inspecting the container’s PID, let’s see the namespace symlinks under /proc/PID/ns:

Interesting! Some namespaces are the same as with the above sleep command that we’ve run in our terminal (specifically the user and cgroup namespaces), but most of them are different, which proves that Docker separates containers in different namespaces.

Now, let’s try to exec a shell inside the container to inspect the filesystem:

Starting to make sense? Well, almost…

What’s going on here is that we’ve run NSENTER to start a shell process inside the mount namespace of the container process. The container’s mount namespace is different from the root one because the root directory lists a bit different tree structure than on my WSL instance. Additionally, on my Debian instance, printing the Linux distro info (/etc/os-release) inside the container shows an Alpine Linux distro. So we conclude:

docker exec -it <CONTAINER_ID> <CMD>
equals to:
nsenter -a -t <CONTAINER_PID> <CMD>

nsenter -a enters all namespace of process with PID specified in the -t arg

Docker

Most readers are already familiar with the Docker client-server model, so it’s unnecessary to explain how the CLI invokes commands on the dockerd service running in the background. Let’s open the front hood and see what’s going underneath.

In the last example, we’ve seen that the docker run command forks a process in a separate namespace set. What’s actually happening is that Docker (again, more specifically containerd, but let’s keep it simple for now) calls the underlying container runtime to create the specified namespaces, prepare the container environment and execute some special commands needed before the actual user-defined command starts.

But if that’s the responsibility of the runtime, what do we use Docker for?
Docker prepares everything before container creation, with the 2 most important parts:

config.json
container root directory

These two parts (together with other things that we will discover in this series) are called the container bundle.

The config.json file has the complete layout for the whole container lifecycle, from container start to container deletion. It holds the path for the container root directory, a list of namespaces that need to be unshared, resource limits for the container process, hooks that need to be executed at a specific point in time, and many other settings.

The container root directory is the directory mentioned in the mount namespace section. That’s the subdirectory somewhere on the host system that will be the root directory for the container. The user-defined process MUST NOT know that there’s a whole different world outside the container root directory, which basically “cages” (most literature refers to it as “jails”) the user process inside the container root directory.

Besides these two most important things, Docker does other preparations too (for example, pulls the image layers from the remote repository, sets up the networking interfaces if the container has networking enabled, and so on).

OCI Specification

Standardization is a crucial part of any software integration. From writing REST APIs or gRPC services to designing low-level network protocols, everything starts with a well-defined document describing what needs to be (un)expected behavior and the minimal contract that an implementation needs to fulfill.

The Open Container Initiative (OCI) created and still maintains the OCI runtime specification that can be found here. It’s a spec that still evolves and adds new features that a container runtime can or may execute while launching a container process.

I won’t go into the details of the spec, as it’s described very well in the document, but at least here is what it is in a nutshell.

An OCI-compliant container runtime is a CLI binary that implements the following commands:

create <id> <bundle_path>
start <id>
state <id>
kill <id> <signal>
delete <id>

As with any other emerging spec, the OCI runtime spec describes the minimum features for creating a container. Popular container runtime implementations like runc or crun have additional arguments that help setting up the PID file of the process, root folder for the container state, or the socket file for the container terminal.

But don’t worry about these parts. In the upcoming articles in this series, we’ll dive deeper into it with some Rust code.

DEV Community: penumbra23

Container Runtime in Rust - Part II

Clone

Unix Sockets

Terminal

Conclusion

Container Runtime in Rust — Part I

Intro

Operations

chroot

pivot_root

Special links & mounts

Devices

Conclusion

Container Runtime in Rust - Part 0

Intro

Container

Docker

OCI Specification

References