A Crystal Story: A container that sails!

Previously on ...

We started building a Simple Static File Server. Then we added options so that we could set the folder and port.

And all of this using the Crystal language!

And now ...

... it's time to containerize our great, awesome, no words to describe its greatness file server using Docker 🐳

First things first

What is a container?

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another [...] Containers isolate software from its environment and ensure that it works uniformly despite differences for instance between development and staging. - Docker.com

Wait, so what is Docker? 🐳

A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application [...] Container images become containers at runtime and in the case of Docker containers - images become containers when they run on Docker Engine. - Docker.com

Perfect! Now we know the basics! And if you want, there are more to read about containers

Second things second

maybe we should start writing some meaningful titles(?) 🤔

Preparing our server for containerization 👨‍🏭

We are going to make some (minimal) changes to our server. Some of them are a must-do for containerization and others are only cosmetic changes (that will become really useful).

- Ctrl+C

Our server will handle Ctrl+C, so that when running in the container, we could stop it with this key combination.

# Handle Ctrl+C and kill signal.
# Needed for hosting this process in a docker
# as the entry point command
Signal::INT.trap { puts "Caught Ctrl+C..."; exit }
Signal::TERM.trap { puts "Caught kill..."; exit }

- Default path and port

We are going to change the default path from ./public to /www. And the port from 8081 to 80

# Default values
path = "/www"
port = 80

- Bye bye 127.0.0.1

And the last modification is the TCP address. From 127.0.0.1 to 0.0.0.0
Here is a great article explaining Docker Networking and why we need to make this change.

address = server.bind_tcp "0.0.0.0", port

- Our soon-to-be-dockerized server

And here is the final result:

# file_server.cr
require "http"
require "option_parser"

# Handle Ctrl+C and kill signal.
# Needed for hosting this process in a docker
# as the entry point command
Signal::INT.trap { puts "Caught Ctrl+C..."; exit }
Signal::TERM.trap { puts "Caught kill..."; exit }

# Default values
path = "/www"
port = 80

OptionParser.parse do |parser|
  parser.banner = "A Simple Static File Server!"

  parser.on "-f PATH", "--files=PATH", "Files path (default: #{path})" do |files_path|
    path = files_path
  end
  parser.on "-p PORT", "--port=PORT", "Port to listen (default: #{port})" do |server_port|
    port = server_port.to_i
  end
  parser.on "-h", "--help", "Show help" do
    puts parser
    exit
  end
end

server = HTTP::Server.new([
  HTTP::LogHandler.new,
  HTTP::ErrorHandler.new,
  HTTP::StaticFileHandler.new(path),
])

address = server.bind_tcp "0.0.0.0", port
puts "Listening on http://#{address} and serving files in path #{path}"
server.listen

Great! And more thing:

- Compiling our server

This step is here just to show how to compile our application. When creating our Docker image, we are going to need our compiled application with all its dependencies.

Building for production

For building our file server we are going to use the crystal build command

$ crystal build --release ./file_server.cr

And now we have our binary file:

$ ls
file_server.cr
file_server         <-- here it is!
file_server.dwarf

Some info about our new binary file:

$ file file_server
file_server: Mach-O 64-bit executable x86_64

Note: On Linux we may use the same file command.

Dependencies

Our file server have dependencies and they need to be included in the soon to be created docker image. Let's see which libraries our application is using:

$ otool -L file_server
file_server:
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
    /usr/local/opt/libevent/lib/libevent-2.1.7.dylib (compatibility version 8.0.0, current version 8.0.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

Note: On a Linux OS we may use ldd ./file_server.

Perfect! Now we may proceed to sail ("sail" you know ... because of Docker ... and the whale ... and the containers ... I think it would be right to use sea analogies 🌊)

Building the Docker image 🏗⛵️

Ok, we are at the docks building our next to sail docker image, using the following Dockerfile:

# Dockerfile
FROM crystallang/crystal:latest

ADD . /src
WORKDIR /src
RUN crystal build --release ./file_server.cr

RUN ldd ./file_server | tr -s '[:blank:]' '\n' | grep '^/' | \
   xargs -I % sh -c 'mkdir -p $(dirname deps%); cp % deps%;'

FROM scratch
COPY --from=0 /src/deps /
COPY --from=0 /src/file_server /file_server

EXPOSE 80

ENTRYPOINT ["/file_server"]

wait wait ... subtitles: ON, please!

Let's begin saying that we are using multi-stage builds.

The first stage is based on the crystallang/crystal:latest image:

FROM crystallang/crystal:latest
...

because we need to build --release our application:

...
RUN crystal build --release ./file_server.cr
...

and list the dependencies, like we did before, but using ldd because it is an Ubuntu based image:

...
RUN ldd ./file_server | tr -s '[:blank:]' '\n' | grep '^/' | \
   xargs -I % sh -c 'mkdir -p $(dirname deps%); cp % deps%;'
...

The second stage is based on the Docker Official Image scratch:

...
FROM scratch
...

resulting in a really small docker image 🤓 that contains our file server binary and dependencies.

Actually building the image

Well, now we need to actually build the image:

$ docker build -t "file_server:0.1.0" .

The build starts:

$ docker build -t "file_server:0.1.0" .
Sending build context to Docker daemon  2.222MB
Step 1/10 : FROM crystallang/crystal:latest
 ---> e9906ad8c49f
...

And a few minutes later ⏰ it finishes:

...
Step 10/10 : ENTRYPOINT ["/file_server"]
 ---> Using cache
 ---> 11f647222892
Successfully built 11f647222892
Successfully tagged file_server:0.1.0

And here it is: our brand new image! 🤩

$ docker images
REPOSITORY    TAG     IMAGE ID      CREATED        SIZE
file_server  0.1.0  11f647222892  2 minutes ago   8.98MB

with a total size of only 8.98MB! Great!

Running the container ⛵️💨

Let's run the new image (resulting in the creation of a new container):

$ docker run --rm -it -v ${PWD}:/www -p 8080:80 file_server:0.1.0

Note: we are mapping the container's port 80 to local port 8080

The application (now running inside the container) replies:

Listening on http://0.0.0.0:80 and serving files in path /www

Browsing to http://localhost:8080 it will list the files in the current folder:

• file_server.dwarf
• Dockerfile
• file_server.cr
• file_server

And that's it! Our dockerized file server has sailed! 🎉

Farewell and see you later. Summing up

• We prepared our file server for containerization.
• We build --release our file server and list its dependencies.
• We created a custom docker image using multi-stage builds.
• Finally we ran the container!

And so we have reached the end of this post and also the end of this trilogy (I know, it's sad 😢, but don't you worry we can read it again 😃)

Hope you enjoyed it! Until next Crystal sailing!

Thanks, thanks, thanks to:
@bcardiff , @diegoliberman and @petti for reviewing this post and improving the code and text!! 👏👏👏

Photo by Julius_Silver on pixabay.com

Comments

[Szikszai Gusztáv]

Very nice walk through, thank you! I never had the time to figure out how to copy the dependencies like this, and didn't know about multistage builds either.

[Franciscello]

Hi! Thanks for the comment! And the copy dependencies magic trick was borned in the gist list-deps.cr 🤓🎩

[David YOTEAU]

Great job!
why not use the --static argument instead of copying dependencies?

[Franciscello]

Hey David! Thanks for the comment and good observation! While writing this post I also thought that when using --static I wouldn't need to copy the dependencies. But then @bcardiff sent me this reference:

Building fully statical linked executables is currently only supported on Alpine Linux.
crystal-lang.org/reference/using_t...

And here is another good reference about statically compiling dependencies: crystal-lang.org/2020/02/02/alpine...