DEV Community: Liem Le Hoang Duc

HashiCorp Nomad Introduction

Liem Le Hoang Duc — Wed, 01 Apr 2020 11:42:42 +0000

Oh, yeah???

Daniele Polencic

@danielepolencic

Are you running your own bare-metal Kubernetes cluster?

I feel you.

19:53 PM - 14 Sep 2019

37 130

P/s: I'm not telling to use HashiCorpNomad and abandoned K8S, but at least, take a look at it, you may think God is not abandoned you, no more!

TL;DR

Nomad is the project from HashiCorp, a company that help me, and a lot of other guys, working with Cloud more easily.
It was a workload orchestrator.
It support container application, legacy application, Java, Batch Job, QEMU.
It's easy to operate and maintain.
It had less magic than k8s, actually, no magics.
It's may be what you want from the very beginning but other guys just bark about K8S and told you to use K8S.

Use cases

You're working with legacy system, which is not ready to one-step move to Cloud.
You're not familiar with Orchestrator and Containerized application.
You're not familiar with Cloud, or you don't want to learn K8S.
You cannot use K8S because you don't have an OPs team, and your country don't have a K8S provider which you trusted.
You want to try out something differences than K8S.

Facts

In facts, some company combine K8S and Nomad together to handle their workload. Learn about it here

Terms

Nomad Server

This one will operate and checking the state of system. For example:
"Is this app in state that user defined?" If not, reschedule it!
"I need 1GB of RAM for my app, can you handle it?" Agent-X have enough memory, Ok, I will run your app on it!

To start a nomad server:
nomad agent -server -bind=0.0.0.0 -data-dir=/tmp/nomad-server

Nomad Agent

This one will listen to Server request and reporting the state of application to Server, if the state of the application is abnormal, Server will request Agent to stop/start/restart the application.

To start a nomad agent:
nomad agent -agent -data-dir=/tmp/nomad-agent

Notes:

On local machine, I usually use nomad agent -dev to bootstrap nomad cluster. Think about K8S 🙃

Job

Is the configuration which is defined the expected state, which you want your app into.

A job file usually have following structure:

job
  \_ group
        \_ task

Example:

Group

Is a collection of application which will be scheduled on the same Agent.
Application in the same group can be shared volume and network (using Consul Connect).
Pro tips: Using group as your unit of scaling.
Bonus For people come from K8S, using group as your Pod will be good enough. Combine group with difference type of Job will provide things similar to Deployment, DaemonSet.
Example:

Task

Is a unit of work which was your application. Say container, binary, Java app.
Pro tips: Naming your task follow your job will help you debug and managing/monitoring application more easily and friendly. Eg: Your job name is "example", then your nginx should named "example-nginx".
Example:

Service

Note: This is a bit out of Nomad, becuz you will need Consul to let this work. But, who use Nomad without Consul?
After you start your application, say, an API and you want to populate its address to other services over Consul, you can use this stanza instead of self register inside your code.
Example:

Conclusion:

That enough for the introduction. As a developer, you can get use of Nomad right away.
In the next post, I will show you how to create a website using Docker, Traefik and Nomad.

For people who dont have patience. Come here with a full source code how to deploy a web using Nomad, Traefik and Docker as well as how to bootstrap Nomad Cluster using Ansible.

https://github.com/liemle3893/nomad-getting-started

Docker basics

Liem Le Hoang Duc — Wed, 01 Apr 2020 10:46:50 +0000

I'm here to share and well as to note some basics about Docker.
I'm Java developer, so I will mapping the concept from Java to Docker. If something confused you, please add comment and I will answer/update accordingly.

Terms

Docker Machine

The host/machine which is run docker server.
If you run Docker on Mac you will have a Virtual Machine which is, in this case, a Docker Machine

Image

Similar to Java, you packaged your app into Jar/War file and then your server will run it.
Similar to Docker, Image is a type of package that contains all of your things extras some OS files (/usr/bin, etc...).

Container

From Java point, you have Jar/War file, then you run it by java -jar your-jar.jar. The Java Runtime will starting to run your bytecode and you have a process.
Same for Docker, when you type docker run your-image, the docker runtime (currently default is runc) will start to run your image and create a process, which is, in docker terms - container.

Tag

Tagging or Versioning is always important.
When I was using Java, I just created my Jar/War and then Upload it to repository, say, version 1.0.0-RELEASE then the Ops guys will download that file from Repository and Run it.
In Docker, I have an image, I tag it, and then I push it into repository.
Tag is just the old wine in new bottle.

Volume

In Java, most of the time you will read/write file directly on Host Machine/Server and usually, that file will remain until you delete it.
In Docker, every time start a new container, the file system is fresh, none of your files from previous session will remain. This is a so-called "immutable". To solve that problem, Docker have a concept called volume.
To bind a volume inside Docker, simply add option --volume or -v for short.

Frequency used commands

`docker run`

Start a container

$ docker run --rm -it -P -v /tmp:/tmp/host ubuntu:18.04

In above command:
--rm mean: after this container finish execute, clean it!
it mean: Interactive shell. Without this, if your application require you to input, you cannot do it.
-P mean: If this container EXPOSE ports, please expose all of those ports randomly on DockerMachine. To bind specific port, 1323 for example. -p 11323:1323. Where 11323 is the port on Host Machine, and 1323 is port expose by container.
-v /tmp:/tmp/host mean: bind "/tmp" from host volume to "/tmp/host" in container.
Other useful options:
-d: Run container in detached mode. You shell will be freed immediately
--name: Name your container, then use can interact with its.

`docker logs`

$ docker run --name my-nginx --rm -p 38080:80 -d nginx:latest

Then curl 127.0.0.1:38080

Finally, to see access log of Nginx. docker logs my-nginx
Above command will print full logs of my-nginx container. To limit and follow limited line of logs, use docker logs --follow --tail 10 my-nginx

`docker exec`

$ docker exec -it my-nginx /bin/bash

This command is very useful to debug our code.
"Is my code created file as I expected?"
"How do I know if my code have all it dependencies? ldd my-binary!"
Using docker exec give you the same power as ssh to your server.

`docker pull`

$ docker pull containous/whoami

Will download containous/whoami:latest image to your machine.
To break down docker run, like we do in git.
git clone = git init + git remote add + git pull
docker run = docker pull + docker container create + docker container start

After testing and learning, now it's time to create our own docker image.

`docker build`

Here my file template for almost every project (Golang)

ARG BASE_IMAGE=gcr.io/distroless/base
FROM golang:1.14 as builder
COPY . /work
WORKDIR /work

RUN CGO_ENABLED=0 go build -a -ldflags '-extldflags "-static"' -o app main.go


FROM $BASE_IMAGE
LABEL maintainer="contact@liemlhd.com"
# Copy Built App
COPY --from=builder --chown=65532:65532 /work/app /app/app
USER nonroot
WORKDIR /app

EXPOSE 1323

ENTRYPOINT ["./app"]

And then

$ # To build production image
$ docker build -t saboteurkid/go-demo:latest -f Dockerfile .
$ # To build test/dev image
$ docker build -t saboteurkid/go-demo:latest-debug \
 --build-arg BASE_IMAGE=saboteurkid/ubuntu:18.04-base \
 -f Dockerfile .

`docker push`

Finally, Push you Image to the Repository

$ docker push saboteurkid/go-demo:latest
$ docker push saboteurkid/go-demo:latest-debug

Some others useful commands:

docker tag: this is useful when you want to rename a tag to difference name. Say I want to fork a saboteurkid/go-demo:1.0.0 version from saboteurkid/go-demo:latest. Simply type:

$ docker tag saboteurkid/go-demo:latest saboteurkid/go-demo:1.0.0

docker commit: this is useful when you want to commit your changes after add some changes to your container. Example:

$ docker exec my-nginx cp /etc/hosts /tmp/hosts
$ docker commit my-nginx saboteurkid/nginx-with-text-file
$ docker run --rm -it -d --name nginx-with-file saboteurkid/nginx-with-text-file
$ docker exec nginx-with-file cat /tmp/file.txt | head -n1 
$ # Will show: `127.0.0.1 localhost`