In the previous post, we built the node image and wrote the agent, all done by executing commands on the shell with docker build and run, but the whole process should be automated by our program, this is what this article will focus on, on creating node on command, listing them and deleting them.
Before we begin I had some minor fixes in the code: rename the command variables in cmd/pod.go because we are going to use the same name in node, change the agent pods routes to pods and not pod as BP in REST, and moved the GenerateNewID function to pkg/util.go. Some big change was in the Dockerfile, I am relying on the environment that is running the commands to have locally the node image built, so I made a Makefile to build the main program, the agent, and the node image.
Makefile:
default: build
build: build-agent build-node-image
go build -o bin/main main.go
build-node-image:
sudo docker build -t own-kube-node .
build-agent:
go build -o bin/agent pkg/agent/agent.go
As a result, we can rely on the binary for the agent is already built when we build out the image so I removed the build from the Dockerfile:
# Dockerfile for node image
FROM ubuntu
WORKDIR /agent
RUN apt-get update && apt-get install -y wget containerd
COPY bin/agent .
EXPOSE 10250
ENTRYPOINT [ "./agent" ]
Let's start with our node functionality, Like the pod, we have related files to node functionality in pkg/node.
We start on pkg/node/constraints.go to define some constraints for our work:
package node
import "github.com/jonatan5524/own-kubernetes/pkg/agent/api"
const (
NODE_NAME = "node"
NODE_IMAGE = "own-kube-node"
NODE_PORT = api.PORT + "/tcp"
NODE_PORT_HOST_IP = "0.0.0.0"
MEMORY_LIMIT = 2.9e+9 // 2900MB
CPU_LIMIT = 2
NODE_HOST_MIN_PORT = 10250
NODE_HOST_MAX_PORT = 10300
)
All the constraints will be used in the following files and all the values are taken from the last article when we ran the docker container node.
In the pkg/node.go we define a Node struct:
type Node struct {
Id string
Port string
}
Id - generated id for the node
Port - generate unused port for the node
In this file we also define function for creating a new node using the docker sdk:
func NewNode(cli *client.Client, ctx context.Context) (*Node, error) {
exists, err := isNodeImageExists(cli, ctx)
if err != nil {
return nil, err
} else if exists == false {
return nil, fmt.Errorf("node image: %s not exists locally, need to build the image", NODE_IMAGE)
}
id := pkg.GenerateNewID(NODE_NAME)
config := &container.Config{
Image: NODE_IMAGE,
}
hostConfig := &container.HostConfig{
PortBindings: nat.PortMap{
nat.Port(fmt.Sprintf("%s/tcp", api.PORT)): []nat.PortBinding{
{
HostIP: NODE_PORT_HOST_IP,
HostPort: "0",
},
},
},
Resources: container.Resources{
Memory: MEMORY_LIMIT,
CPUShares: CPU_LIMIT,
},
Privileged: true,
}
_, err = cli.ContainerCreate(ctx, config, hostConfig, nil, nil, id)
if err != nil {
return nil, err
}
return &Node{Id: id}, nil
}
We first check if the node image exists, then we generate an id, and create some config for the container:
- node image
- port forwarding from the port inside the container (the agent API) to outside of the container. The HostPort is 0 because Linux assigns a random open port when a service assigns a 0 port.
- memory limit
- CPU limit
- privileged All the configurations are equivalent to the command we ran in the previous article:
sudo docker run -it --memory="2900MB" --cpus="2" -p 10250:10250 --privileged --name test --rm containerd_test
Then we create the container with the config and generated Id and return a new Node struct.
The isNodeImageExists function implementation:
func isNodeImageExists(cli *client.Client, ctx context.Context) (bool, error) {
images, err := cli.ImageList(ctx, types.ImageListOptions{})
if err != nil {
return false, err
}
for _, image := range images {
if strings.Contains(image.RepoTags[0], NODE_IMAGE) {
return true, nil
}
}
return false, nil
}
The function List the existing images, go through each one, and check if its RepoTag contains an own-kube-node
Let's move to pkg/node/service.go, the first function we will go through will be the initDockerConnection which initialized the docker client and context:
func initDockerConnection() (*client.Client, context.Context, error) {
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return nil, nil, err
}
return cli, ctx, nil
}
Next, is the NewNodeAndRun function which creates new node and starts the container:
func NewNodeAndRun() (*Node, error) {
cli, ctx, err := initDockerConnection()
if err != nil {
return nil, err
}
defer cli.Close()
node, err := NewNode(cli, ctx)
if err != nil {
return nil, err
}
if err := cli.ContainerStart(ctx, node.Id, types.ContainerStartOptions{}); err != nil {
return nil, err
}
log.Printf("node created: %s\n", node.Id)
log.Printf("starting node\n")
container, err := cli.ContainerInspect(ctx, node.Id)
if err != nil {
return nil, err
}
// get linux generated port
node.Port = container.NetworkSettings.Ports[nat.Port(fmt.Sprintf("%s/tcp", api.PORT))][0].HostPort
log.Printf("node assign port: %s\n", node.Port)
return node, nil
}
As you can see in this function, we are initializing the connection to the docker daemon, creating a new node and container, starting the container, and then because we assign port 0 we need to retrieve the assigned port by the system.
The next function is ListRunningNodes:
func ListRunningNodes() ([]string, error) {
cli, ctx, err := initDockerConnection()
if err != nil {
return []string{}, err
}
defer cli.Close()
runningNodes := []string{}
filter := filters.NewArgs(filters.KeyValuePair{Key: "ancestor", Value: NODE_IMAGE})
containers, err := cli.ContainerList(ctx, types.ContainerListOptions{Filters: filter})
if err != nil {
return runningNodes, err
}
for _, container := range containers {
runningNodes = append(runningNodes, container.Names[0])
}
return runningNodes, nil
}
The function initializes the connection, calls the ContainerList method with the ancestor filter which means the image we want to filter, and then runs over each container to create a list of names.
The last function is KillNode function which simply stops the container and removes it:
func KillNode(name string) (string, error) {
cli, ctx, err := initDockerConnection()
if err != nil {
return "", err
}
defer cli.Close()
if err := cli.ContainerStop(ctx, name, nil); err != nil {
return "", err
}
removeOptions := types.ContainerRemoveOptions{
RemoveVolumes: true,
Force: true,
}
if err := cli.ContainerRemove(ctx, name, removeOptions); err != nil {
return "", err
}
return name, nil
}
The last part we need to complete is cmd/node.go its structure is the same as cmd/pod.go, I'm not going to add this file here as this article is too long, you can see it in the source code.
we can see that everything works on the terminal:
❯ make
go build -o bin/agent pkg/agent/agent.go
sudo docker build -t own-kube-node .
...
❯ sudo ./bin/main node create
2022/10/12 19:18:26 node created: node-7daf80ec-0a46-4977-952c-66deb81e32f7
2022/10/12 19:18:26 starting node
2022/10/12 19:18:26 node assign port: 49167
❯ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2066e8e0b933 own-kube-node "./agent" 5 seconds ago Up 4 seconds 0.0.0.0:49167->10250/tcp node-7daf80ec-0a46-4977-952c-66deb81e32f
❯ curl -X POST localhost:49167/pods -H 'Content-Type: application/json' -d '{"name": "redis", "image registry": "docker.io/library/redis:alpine"}'
{"image registry":"docker.io/library/redis:alpine","name":"redis-a1cfce57-5db9-4894-a8cf-c277444e1b0c"}
And that's it, we have a node created by command using an API call that can also create for us a pod inside the node!
In the next article, we will cover some Linux network and get working communication with our pods and nodes.
As always, the source code can be found here, the changes were in pkg/node, Dockerfile, Makefile, cmd/node.
Top comments (0)