DEV Community: Jérémie Drouet

Build metrics and budgets with git-metrics

Jérémie Drouet — Wed, 07 Aug 2024 16:30:14 +0000

Build metrics and budgets with git-metrics

Monitoring metrics for a project can be challenging. Deciding what to track, determining the cardinality of each tag, naming the metrics, and choosing what should be split into a tag or kept as a metric are all complex tasks that require experience.

But monitoring the run of your project is not the only thing we need to monitor; the development phase is also important. Code coverage is a good example of how we monitor the build of a project. Many projects implement rules to prevent code coverage from decreasing. In the frontend world, there is a growing trend to track metrics such as page size and load time. This kind of monitoring might seem less complicated, but determining where and how to store the metrics is more challenging.

In recent years, I've been working on several projects where I wanted to put in place such boundaries. When looking at how to implement that, I always ended up using multiple external services or deploying applications to store those metrics that I would have to keep running all the time. This didn't match my requirements. I wanted to keep that cheap in terms of resource usage and low in terms of maintenance. I don't want to have a VPS running 24/7 just to push some metrics.

With global warming already well established, and human activity being the principal reason for it, we should rethink the way we build software. We cannot keep increasing exponentially the resource usage of our systems. Therefore, we need to put in place boundaries on the projects we build.

State of the Art

For open-source projects, many SaaS platforms offer free tiers for monitoring. For tracking code coverage, you can use Codecov or Coveralls. For tracking complexity, CodeClimate is a good option. These platforms integrate well with GitHub repositories.

For closed-source projects, these SaaS platforms offer paid subscriptions, which may require budget considerations. Alternatively, if your DevOps team has the bandwidth, you can set up tools like SonarQube for in-house monitoring.

However, be prepared for potential vendor lock-in, which can make switching services or losing historical data problematic.

Alternative Approach

Shouldn’t the metrics related to the code be attached to the code itself?

Some projects, such as typescript-action, version code coverage alongside the code. This approach isn't perfect, as it requires developers to update the metrics for each commit, or you need a CI job to keep them in sync. However, keeping metrics directly in Git is appealing as it provides easy access and avoids reliance on external services. Switching from GitHub to GitLab, for instance, would be as simple as pushing the repository.

To address synchronization issues, instead of writing these metrics with the code, they can be kept next to it. Git offers a feature called notes, which allows attaching textual data to a commit reference.

Solution Implementation

This is where git-metrics comes into play.

Basic Usage

With git-metrics, you can attach, replace, or remove metrics for a given commit with a simple command:

$ git-metrics add my-metric-name --tag "foo: bar" 1024.0

To show the metrics attached to a given commit:

$ git metrics show <commit-sha>
my-metric-name{foo="bar"} 1024.0

To display a list of all commits, or within a commit range:

$ git metrics diff HEAD~2..HEAD
- my-metric-name{foo="bar"} 512.0
+ my-metric-name{foo="bar"} 1024.0 (+200.00 %)

These commands provide a solid foundation, but there are additional features worth exploring.

Extra features

Budget management

Tracking metrics is useful, but the goal is to improve the codebase or project. One way to ensure this is by blocking commits that don’t meet certain metric criteria, a feature already used by many external services.

With git-metrics, you can configure rules by adding a .git-metrics.toml file to your repository:

# fails when increase size is above 10%
[[metrics.binary-size.rules]]
type = "max-increase"
ratio = 0.1

# fails when the size increases by more that 1MB
[[metrics.binary-size.rules]]
type = "max-increase"
value = 1048576.0

# fails when binary size is more than 10MB
[[metrics.binary-size.rules]]
type = "max"
value = 10485760.0

# fails when the code coverage goes below 80%
[[metrics."coverage.lines.percentage".rules]]
type = "min"
value = 0.8

# fails when the code coverage decreases of more than 5%
[[metrics."coverage.lines.percentage".rules]]
type = "max-decrease"
ratio = 0.05

Running git-metrics check from your CI after adding your metrics will inform you if your changes meet the specified budget.

Importing from other file formats

Manually adding metrics with the add command can be cumbersome, especially for derived metrics. For instance, after computing code coverage, you might end up with an lcov file, which needs to be processed to extract useful metrics. git-metrics can handle this:

$ git-metrics import lcov ./lcov.info

This command adds summary metrics such as coverage.lines.count, coverage.functions.hit, or coverage.branches.percentage.

Currently, git-metrics supports only lcov files, but more formats will be supported as the tool evolves.

Conclusion

git-metrics is in its early stages but already provides essential commands for tracking metrics, creating budgets, and blocking contributions that don’t meet the budget criteria, all within your git repository.

Similar to the mentioned SaaS solutions, git-metrics offers GitHub actions for installation, tracking, and checking metrics. GitLab support is expected soon.

Article also available here

Mailer sender microservice

Jérémie Drouet — Mon, 26 Oct 2020 15:51:06 +0000

I've been working for many companies until now, banks, service companies, small startups or big startups, and sending emails was always a challenge.

Why?

First there is the problem of output. It's pretty hard to have a mail that renders on mobile as well as in Outlook.

Then comes the implementation where the language that is used is not always the same, the architecture is different between projects or the developers don't have time to spend on the mailing feature.

It's quite easy to create a fully featured implementation in a big monolith, but the email service doesn't usually need to be scaled as much as the rest. So separating it as a microservice seems to be the right way to do it.

What are the solutions?

As well as one doesn't reimplement a database (or more an ORM) each time it's needed, using a microservice to build, interpolate and send emails seems like the smart move.

Of course, you could go for a SaaS, pay some money to get access to lots of features that you probably don't need and be tied to their solution.

Or you could go for a fully open source solution, providing everything you need, it's probably not worth the time to reimplement everything. And that's where Catapulte comes into play.

What is Catapulte?

Catapulte is a micro-service, implemented in Rust, and built on top of MRML, the Rust implementation of the nice MJML library.

It comes with the possibility of declaring templates, using variables and interpolating them before sending the email.

Why should you use Catapulte?

Mainly because it makes it easier to build an email template, bundle it and push it to production. And if it's not easy enough, it's possible to couple it with Jolimail to allow anyone to edit your email templates from their browser and have a dynamic preview.

The Second advantage is that it doesn't use much RAM (3Mo on the demo instance) and is really low in CPU consumption. It's mainly distributed through a lightweight docker image that allows you to run it almost everywhere (a laptop, an old computer, a server, a Raspberry PI, in the cloud, ...). And because it's a layer of abstraction in front of the SMTP server, it offers the possibility to start multiple instances with different SMTP server and load balance between them.

To finish up, it's open source, you don't have to maintain it (although you're welcome to contribute), so it costs almost nothing.

How can you use Catapulte?

If you want to try it, there is an demo instance of the Jolimail suite available.

In order to use it locally, it's also pretty straightforward, your will need to have Docker installed.

docker run -d \
  --name catapulte \
  -e SMTP_HOSTNAME=localhost \
  -e SMTP_PORT=25 \
  -e SMTP_USERNAME=optional \
  -e SMTP_PASSWORD=optional \
  -e TEMPLATE_PROVIDER=local \
  -e TEMPLATE_ROOT=/templates \
  -p 3000:3000 \
  -v /path/to/your/templates:/templates:ro \
  jdrouet/catapulte:canary

And voila, it's ready to send emails using curl.

curl -X POST -v \
  -H "Content-Type: application/json" \
  --data '{"from":"alice@example.com","to":"bob@example.com","params":{"some":"data"}}' \
  http://localhost:3000/templates/the-name-of-your-template

Conclusion

Sending emails properly is more time consuming than complicated to implement. It's now your choice if you want to reinvent the wheel or to use an open source solution that comes with many advantages. Spending time on your business features is probably more important.

Hacktoberfest, the good kick

Jérémie Drouet — Sun, 18 Oct 2020 14:56:03 +0000

Jolimail Overview

Before joining Docker and then Datadog, I used to build applications for clients. Each application had to have a way to send transactional emails and the available solutions for that were never awesome, nor installable on premise.

Last year I decided to make a big move from the JS environment to Rust and decided to implement the nice MJML Framework in Rust.

From that, came Catapulte which is a microservice that only generate emails, interpolates variables and sends the email.

But the solution given by Catapulte was not what I expected: you still had to develop the template by hand and put it in the Catapulte container you run in production.

From that, was born Jolimail which allows you to edit the Mjml template in the browser and have a direct preview. And the big advantage of the Rust implementation is that everything is running in the browser. As opposed to MJML that cannot run in the browser and has to make a backend call each time you have to render your template.

The other advantage is, although we don't have a WYSIWYG editor yet, that anybody can edit a template in Jolimail, make several different versions, but only publish the one they want when they're ready.

My Hacktoberfest Experience

As of today, the whole suite is working, you can host a Jolimail instance and a Catapulte instance, and send emails.

The main features to come will be to implement a WYSIWYG editor and another microservice to have the confirmation of reception for the emails.

What I Learned From Hacktoberfest

You've to make a checklist and stick to it. When you work on your own project, you can be easily distracted. You don't have a constraint (maybe time) on what you've to do, so you easily digress. So, to be sure you don't go out of the way, make a checklist and go!

Building multi arch Docker images without using qemu

Jérémie Drouet — Wed, 23 Sep 2020 12:09:01 +0000

Lately I've been doing lots of Rust with MRML, Catapulte and Jolimail, and one of the pain points of Rust is that the compiling process is pretty slow. When, on top of that, you want to build it for amd64, i386, arm64 and arm32v7 because your application is open source and supposed to work everywhere, it becomes a bit tricky to make it build.

Candidly, I initially tried to build it using docker and Qemu for multiple architectures but I finished with some weird errors.

After a bit of chatting with my previous colleagues from Docker, I was encouraged to try to cross compile. Cross compiling is really great when you don't have a lib dependency on the system. But Catapulte and Jolimail are based on actix which depends on OpenSSL, which is supposed to be compiled for the architecture. In short, to cross compile your rust code you first have to cross compile OpenSSL. If you have other libraries, well, cross compile them.

So I did it for Jolimail, it cross compiles, but deep down, I don't like this approach for several reason. First, I've to handle the build of OpenSSL. If there is a new version, I've to update everything and not only rebuild my image. Then, it adds an unwanted complexity to my Dockerfile. So I decided to find another way to do this.

After a bit of digging and a good hint from a nice user on the docker slack community (Billy), I finally did my first use of docker buildx create --append. If you're not familiar with docker buildx and multi arch images, I encourage you to take a look at the article I wrote, back when I was at Docker.

When you are on an amd64 machine and you want to make a multi arch image, the initial buildx solution is to use Qemu to emulate the target environment. But you can also, when you create a buildx context, specify a context target for a specific platform.

Like lots of my peers, I bought every version of the raspberry pi, I even have several of each version, but I don't really have time to properly use them. So I started one (3b+ I guess) and installed HypriotOS on it (take a look, install it, test it, it's great). I then created a context to be able to build on the raspberry pi from my local machine.

docker context create rpi --docker "host=ssh://pirate@black-pearl"

When this is done, I am able to target the raspberry pi by doing DOCKER_CONTEXT=rpi docker pull jdrouet/kaamelott-quote.

Then the trick is to tell buildx that, when it needs to build the image for arm32v7, it should use the context rpi.
First, I use docker buildx ls to get the name of the current builder. Then, I append the context to this builder with the following command.

docker buildx create --append --name you-builder-name --platform linux/arm/v7 rpi

And that's it! I can now build your multi arch image.

docker buildx build --tag jdrouet/catapulte:some-tag --platform linux/amd64,linux/i386,linux/arm/v7 .

Multi arch images with docker

Jérémie Drouet — Wed, 27 Nov 2019 10:40:06 +0000

Most of the time, when I start a project, I like to try to install it everywhere to test it. Since I have a cluster of Raspberry PIs, I love to install my projects on it and see how they evolve, where they could break and fix the issues.

To do so, I try to get as close a possible to my production configuration. So I usually put in place a CI/CD that test, build and deploy my projects. But when it comes to build my Docker Images for ARMv7 on a regular CI, it tends to become a bit complicated. You need to have Qemu available on the CI to emulate the target platform.

# depending on the CI
docker run --rm --privileged hypriot/qemu-register
# or
docker run --rm --privileged multiarch/qemu-user-static:register --reset

When this is done, I also want to try the image on my laptop. So, on the CI, we have to build for different architectures with the same Dockerfile.

One way to do it would be to have and argument in our Dockerfile to specify the platform.

ARG BASE_ARCH=amd64
FROM ${BASE_ARCH}/alpine

Once this is done, I "just" have to build several times on my favorite CI, create a manifest and annotate each image in a final stage.

docker build --build-arg BASE_ARCH=amd64 -t my_image:amd64-latest .
docker build --build-arg BASE_ARCH=arm32v7 -t my_image:arm32v7-latest .
docker manifest create my_image:latest my_image:amd64-latest my_image:arm32v7-latest
docker manifest annotate --arch amd64 my_image:latest my_image:amd64-latest
docker manifest annotate --arch armv7 my_image:latest my_image:arm32v7-latest

Ok, you can say it, it's pretty verbose.

Luckily, the Docker Team (relatively) recently released buildx in Tech Preview. What this does is that it will build all the images, do the manifest and annotate everything in only 1 command.

First we have to remove the argument from our dockerfile because it became useless. If we create an image from a multiarch image, buildx will manage the rest.

docker buildx build --platform linux/arm/v7,linux/amd64 -t my_image:latest .

At this point, I can use the image on any of the 2 platforms without having to use different images.

Ok, now this is nice but buildx is only available in experimental and requires Docker Engine 18.09+. And depending on the CI that is going to be used, the installation will more or less differ. So I prepared a repository containing an example of multi arch build with Github Actions, Circle CI, Travis CI and Gitlab CI.