DEV Community: Patrick Elsen

How to make use of the GitLab CI for Rust Projects

Patrick Elsen — Mon, 17 Oct 2022 13:26:04 +0000

GitLab is an open-source-core collaborative development platform that has project planning and continuous integration features built in. I like it, because it is very easy to use, and yet powerful. The free hosted version gives you a lot of awesome features, but you can even self-host GitLab, as it is open-source and permissively licensed. In this series, I'm going to be showing you a few examples on how you can set it up so that you get the most out of it. Hopefully you will find it useful.

In this article, I will show you how you can use GitLab to host your Rust project, complete with a working CI setup that will run your tests, build the project and publish documentation on every commit. If you like open-source software and don't like complicated CI systems, this article is for you.

You can skip this article and go straight to the repository if you just want to see a minimal example.

Setting up the Project

I'm assuming you already have a GitLab account. If you do not, you should create one, they are free and you will get a lot of free perks with it, such as free access to some amount of CI runner minutes. You will also need to set up your SSH keys so you can push to it.

We will need to create a new project to walk you through how to set up the CI to get the most out of it. When you log in to GitLab, you are typically greeted by your list of projects. You can either press the big blue New Project button or press the little plus in the navigation bar up top and select New project/repository.

Once you have found either one of these buttons, they should take you to the new project page. Here, GitLab will ask you if you want to create a blank project or from a template. You can select Create blank project.

Once you've selected that, it will ask you for a name for the project, and some other options. You can put in any name you like, I'm naming this one Rust GitLab Example. Notice that it will fill in the URL for you, but you can also customize it. To make it easier to type, the URL should use lowercase characters and dashes rather than spaces. I have also deselected the Initialize repository with README option, because I'll write one myself.

When you are happy with your selection, hit the Create project button and it should take you to the new, empty project page.

Now, you've successfully setup a new GitLab project. In the next section, I'll show you how to put some Rust code into it and setup the CI to do it's magic.

Creating a Rust Project

Next up, we need new Rust project that we can push to GitLab. If you're doing this with your own Rust project, you'll obviously already have some code to push. I don't, so I'm going to write some placeholder code. As usual, start out by using Cargo to generate a new project:



$ cargo new rust-gitlab-example

Next, I add clap as a dependency to let me parse command-line arguments. As usual, cargo add does this quickly.



$ cargo add clap --features derive

Then, I add some placeholder code. As I said, this is just an example to showcase the CI capabilities, feel free to use your own Rust project instead. I drop this into src/main.rs:



//! # Rust GitLab Example
//!
//! This implements a little command-line utility which will print a greeting for
//! a specified name.
use clap::Parser;

/// Command-line options
#[derive(Parser, Clone, Debug)]
pub struct Options {
    /// Name to use for greeting
    #[clap(long, short, default_value = "Human")]
    name: String,
}

/// Given a name, generate a greeting.
fn greeting(name: &str) -> String {
    format!("Hello {name}!")
}

#[test]
fn greeting_works() {
    assert_eq!(greeting("Mark"), "Hello Mark!".to_string());
}

fn main() {
    let options = Options::parse();
    println!("{}", greeting(&options.name));
}

You can verify that everything works by running the unit tests:



$ cargo test
running 1 test
test greeting_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

We can also make sure that the binary works, by running it with cargo run.



$ cargo run
Hello, Human!
$ cargo run -- --name Patrick
Hello, Patrick!

At this point, we should have some code that runs. Bonus points for having documentation comments and some unit tests. These are important because in the next section we'll be adding a CI pipeline job that will run the unit tests and publish the documentation.

Writing CI Configuration

Now that we have some example code, we have to tell the GitLab CI system what it should do. We do this by creating a .gitlab-ci.yml file in the repository with a configuration. This is fortunately not very difficult, but before I explain how this is done, let me quickly walk you through the basic concepts.

The CI system uses the concepts of jobs and stages. Jobs belong to stages. Stages (and with them, all jobs that are part of them) run sequentially, in the order that you define them. All of the jobs that belong to a stage run in parallel, meaning at the same time. A run of all of the jobs and stages is called a pipeline. If any of the jobs in a stage fails, the entire stage is marked as failing and the CI pipeline run is marked as a failure. The entire pipeline is launched whenever you push new code, tags, branches, create merge requests, or launch it manually.

For advanced users, GitLab also offers the ability to declare your CI jobs as a Directed Acyclic Graph to have your jobs run faster, which can be useful when you have a lot of them. You can also mark jobs as being allowed to fail, or only run conditionally (for example only when files in a certain folder have changed, which is useful when you're stuck with a big monorepo). This article barely scratches the surface of what the CI system can do, check out the documentation if you want to know more.

For the jobs, GitLab CI makes use of Docker. Every job launches some docker container (that we can specify) and runs some commands inside of that container. If we create any new files within a job that we want to keep or that we want future jobs to have access to, we can create artifacts, which are files or directories that are created as the output of a job and that future jobs can use as inputs. So to recap:

Name	Description
Job	A few commands that run in a Docker container. Can produce some output in the form of artifacts. If any of the commands do not exit successfully, the job is considered failed.
Stage	A collection of jobs that run in parallel. If any of the jobs fail, the entire stage is considered failed.
Pipeline	A collection of stages that run sequentially. If any of the stages fail, the pipeline run is considered failed. Is triggered by pushing commits, tags, branches, creating merge requests or can be triggered manually.

Defining Stages

For this example, we want to define three stages:

test, where we run unit tests,
build, where we compile the code,
publish, where we publish the binaries and documentation.

You can start out by creating an empty .gitlab-ci.yml file and adding the list of stages to it:



stages:
  - test
  - build
  - publish

Defining Jobs

Next, we have to define the jobs. We can start by creating the tests job, which will run the unit tests. This job does not produce any output, so we do not declare any artifacts here. We declare that this job is part of the tests stage. We also declare that we want it to use the rust Docker image. If any of the steps in the script section fails, the entire job is treated as a failure. The cargo test command will return a non-success status code on test failure, so that will do what we want. This is what we add to our .gitlab-ci.yml file:



tests:
  stage: test
  image: rust
  script:
    - cargo test

Looks easy, right?

Next, we want to define the jobs where we actually build the binary, and while we're at it we'll also define a job where we build the documentation. These jobs are both part of the build stage, so that they will run in parallel. One difference here to the tests job is that both of these jobs generate artifacts. The build job will export the generated binary as an artifact, while the documentation job will export the entire documentation folder. You can export multiple files or folders from a single job by adding more paths.

Job artifacts are files or folders that are created by jobs. They are stored and can either be used as dependencies for future jobs, or downloaded using the API.



build:amd64:
  stage: build
  image: rust
  script:
    - cargo build --release
  artifacts:
    paths:
      - target/release/rust-gitlab-example

rustdoc:
  stage: build
  image: rust
  script:
    - cargo doc
  artifacts:
    paths:
      - target/doc

Finally, we want to publish the built binary and documentation. We can use the GitLab Pages feature to accomplish that. This is a feature of GitLab where they will host static files for you, free of charge. To make use of that, you need to create a CI job called pages and it must export an artifact folder called public. Whatever is in that folder, will then be publicly hosted at xfbs.gitlab.io/rust-gitlab-example.

GitLab Pages lets you host static content by declaring a CI job called pages. By default, any artifacts in the public/ folder from that CI job will be hosted at yourusername.gitlab.io/projectname. You can add a custom domain to host it at a more friendly domain through the UI. If your project is private, by default this hosted page is protected too, so it is also useful for publishing project-internal or confidential documentation.

Note in this stage that we are using a different Docker container (the alpine container) because we do not need to build any Rust. We also declare some dependencies, this means that CI will fetch the artifacts from those jobs and extract them so that they are accessible in this job. In the job itself, we really just copy things to the right place and don't do anything else.

One important note about this job: this job is only set to run when pushing to the master branch. This is very important, because the published documentation should reflect what is on the master branch, and if this were to be omitted, this job would also run when creating private branches or merge requests.



pages:
  stage: publish
  image: alpine
  dependencies:
    - build:amd64
    - rustdoc
  script:
    - mkdir -p public
    - mv target/doc public/doc
    - mv target/release/rust-gitlab-example
  artifacts:
    paths:
      - public
  only:
    - master

You can find the full .gitlab-ci.yml file here.

Pushing it to GitLab

Now, saving this file and pushing it up to GitLab, we can see that our pipeline is being correctly detected. In the left navigation, there is a little rocket icon labelled CI/CD, clicking this will get you to the list of CI pipeline runs, and clicking on the latest one will look something like this:

You can click on any of the pipeline jobs to see the output of the script that was running. For example, if we want to see if there was any compiler warnings or issues when running the unit tests, we can click on the tests job.

During the pages job, we've published the documentation that we generated and we've published the binaries. We can check that this is published:

These things are available here, if you want to check them out yourself:

Binary: xfbs.gitlab.io/rust-gitlab-example/rust-gitlab-example-amd64
Documentation: xfbs.gitlab.io/rust-gitlab-example/doc

Conclusion

The GitLab repository that I created in this article is available here.

GitLab makes it very easy to create repositories, use off-the-shelf Docker containers to put together custom CI jobs. We were able to add CI capabilities including publishing binaries and documentation with only a handful of lines of configuration. The fact that it simply uses regular Docker containers also makes it easier to locally debug CI configuration.

GitLab Pages makes hosting static things pleasant and is perfect for things like nightly builds, test coverage reports, or documentation. Some people even use it to host blogs or personal websites. For example, passgen.it, my password generator with a Regex-like syntax, is hosted by GitLab Pages with a custom domain, and the releases (binaries, tarballs, debian packages) are generated by the CI on every commit for all platforms.

Generating prime numbers with Python and Rust

Patrick Elsen — Tue, 11 Oct 2022 13:58:53 +0000

The other day, I helped a friend with some Project Euler problems. Those are little programming puzzles that involve maths and coding. I am really quite fond of them, and I recommend you to try some if you want to hone your skills and you are not afraid of trying something a bit difficult.

One thing that comes up in some of these challenges is needing prime numbers. As the challenges are all about writing code, it means needing to implement a prime number generator.

A prime number is a natural number that is only divisible by 1 and itself. Some examples are: 2, 3, 7, 11, 13, 17, and 19.

There are quite a few strategies that can be used to generate them, and in fact quite a few more efficient than the one I'm going to discuss here. The point of this is not to find the best strategy, but rather to see how it can be implemented well and optimized.

Generating prime numbers up to one million

This is perhaps the most naive solution for generating prime numbers up to on million in Python. In this solution, we simply iterate through numbers up to one million, and then try to see if they are divisible by anything other than 1 and itself.

primes = []
for candidate in range(1000000):
    is_prime = True
    for divisor in range(2, candidate - 1):
        if candidate % divisor == 0:
            is_prime = False
            break
    if is_prime:
        primes.append(candidate)

This solution is incredibly inefficient, it takes forever to finish. We can do a lot better than this. There is three optimizations that we can implement:

We don't even need to check even numbers, because we know that they are divisible by two. So we start our primes list with [2] and then skip over all even candidate numbers in the first place. This means less candidates to try.
All positive natural numbers are either prime or the product of (zero) or more primes. This means that instead of checking all candidate primes for divisibility by [2..candidate-1], we can simply check them for divisibility by the primes we have already found. This means less division tests.
We only need to check for divisibility by primes up to sqrt(candidate), because for a candidate c to be divisible by a, it means there exists a solution for the formula $\cdot b$ . If a is larger than the square root, b must be smaller than the square root. For this reason, it is sufficient to check for divisors up to the square root. This also means that we need less division tests.

With these optimisations in place, our code looks like this:

from math import sqrt

primes = [2]
maximum = 1000000

for num in range(3, maximum, 2):
    is_prime = True
    square_root = sqrt(num)
    for prime in primes:
        if num % prime == 0:
            is_prime = False
            break
        if prime > square_root:
            break
    if is_prime:
        primes.append(num)

The runtime of this solution improved to only taking about 940ms on my machine, demonstrating that these optimisations have massively improved the implementation.

Implementing it in Rust

I will translate the algorithm into Rust here to show you what it would look like and how well it performs.

I have to say that this is kind of cheating, because Rust is a compiled language, which means that the compiler has optimization opportunities that an interpreted language such as Python does not have.

Creating a new, empty Rust project with cargo new primes, we can translate the Python version into Rust, only making use of iterators to sift through the list of primes to try dividing.

fn main() {
    let mut primes = vec![2];
    let maximum: u64 = 1000000;

    for candidate in 3..maximum {
        let square_root = (candidate as f64).sqrt() as u64 + 1;
        let is_prime = primes
            .iter()
            .take_while(|p| p <= &&square_root)
            .all(|p| candidate % p != 0);
        if is_prime {
            primes.push(candidate);
        }
    }
}

What we do here is very similar to what we did in the Python version:

We start out with an array of primes, pre-filled with the first prime,
We iterate through the numbers 3 up to our maximum,
For every candidate, we determine the square root. If none of the primes up to the square root of the candidate are a divisor, we consider the candidate to be prime and append it to the list.

One difference to note here: running this in release mode (using cargo run --release) is so fast that it only takes about 30ms to generate one million primes. In the time that Python can generate on million primes, Rust can generate 20 million primes. This is to be expected, because Rust can take advantage of compilation.

Introducing Parallelism

These solutions so far have been lacking in one aspect: they have not been able to take advantage of multiple CPU cores. How hard would it be to take the current Rust solution and introduce some parallelism?

The rayon crate in Rust is designed to help with exactly that: making it easy to write code in a multi-threaded way. I have never worked with it, so let's give it a go.

We can get started by adding rayon to the list of dependencies:

cargo add rayon

Now, rayon gives us some interesting primitives to work with. For example, any Rust code that uses iterators can be transformed into running in parallel by replacing .iter() with .par_iter(). Behind the scenes, rayon handles starting a thread pool, dividing up the work, and waiting for the threads to finish.

For example, we could replace the iterator-based code that we use to check if a number is divisible by some primes:

use rayon::prelude::*;

let is_prime = primes
    .par_iter()
    .filter(|c| c < &&square_root)
    .all(|p| candidate % p != 0);

However, doing this is actually bad for performance. This iteration walks the list of primes linearly, and as such it is actually very fast already (good locality, friendly for CPU caches). Breaking this down to run on multiple threads does more harm than good, and brings our runtime from 30ms up to 17s. Can we find a better optimization opportunity?

We have to try to break down the algorithm into bigger chunks. So rather than breaking down the loop where we test individual divisors, we can break it down into testing different candidate primes in multiple threads.

One thing we must fix before doing that is dealing with mutability. Our list of primes is used for two purposes: we use it to check candidate numbers for being prime by checking divisibility (read-only usage) and we append new primes to the end of it (read-write usage). The issue here is that when we work multi-threaded, it becomes problematic to use the same data in both a read-only and read-write manner. The easy solution to that is splitting that up into two separate variables: the primes array is now only used read-only, and we introduce a new new_primes array that we append new primes to. Occasionally we append the new_primes array onto primes.

Here is what all of this looks like together. Note that I've had to import rayon's prelude (importing that gives us access to all of rayon's iterator adapters) and we are using par_extend(), which will extend an array in parallel. So in this implementation, we will test different candidates for primality at the same time.

use rayon::prelude::*;

fn main() {
    let mut primes = vec![2];
    let maximum: u64 = 1_000_000;

    loop {
        let mut new_primes = vec![];
        let last_prime = *primes.last().unwrap();
        if last_prime > maximum {
            break;
        }

        new_primes.par_extend((last_prime..last_prime*2)
            .into_par_iter()
            .filter(|candidate| {
                let square_root = (*candidate as f64).sqrt() as u64 + 1;
                primes
                    .iter()
                    .take_while(|p| p <= &&square_root)
                    .all(|p| candidate % p != 0)
            })
        );

        primes.append(&mut new_primes);
    }
}

This implementation cuts down the time it takes to compute primes up to one million down to about 11ms on my machine.

Results

Here are the raw results from the rest runs:

Algorithm	Language	Time (relative change)
Naive (1 million)	Python	24m7s
Better (1 million)	Python	940ms (153936% faster)
Better (1 million)	Rust	30ms (3133% faster)
Parallel (1 million)	Rust	11ms (273% faster)

We have learned that it is quite easy to generate prime numbers in both Python and Rust. Generating them quickly takes a little bit of experimentation. Rayon helps a lot in making things multi-threaded, it's almost magic. But we still had to identify the best places to introduce parallelism, and we had to change the algorithm slightly.

Just for fun, I decided to see how many primes I could generate with the Rayon implementation. After running it for 2m25s, it went up to the prime 5222275627, which I thought was quite impressive. Running it for longer, about 30m (close to how long the naive Python implementation took to generate primes up to one million), it was able to come up with the prime 41778204911.

It was a lot of fun to figure out how to generate primes quickly. I hope that you learned something, maybe you will find a use-case for rayon yourself. It was a pleasure to work with.

Writing a toy DNS Server in Rust using Trust DNS

Patrick Elsen — Sun, 02 Oct 2022 21:35:13 +0000

Ever wondered how you can write a DNS server in Rust? No? Well, too bad, I'm telling you anyways. But don't worry, this is going to be a fun one.

Before we actually get started, I must perhaps first explain what DNS is. If you really want to understand it, I think you should check out Julia Evans' DNS Zine post, she does a far better job at explaining it than I ever could. But if you don't have enough time, here's the TL;DR:

DNS is the protocol that is used to answer questions like What's the IP address of google.com? Usually, when you open a website in your browser, you use domain names. In https://dev.to, the dev.to is a domain name. Other domain names you might know are youtube.com or lobste.rs. But in order for your browser (or any software on your computer, for that matter) to establish a connection, it needs to know which IP address to talk to. An IP address is something like 172.217.16.206: basically just a few numbers that are enough to send your request to the right place. Very similar to a phone number or a physical address. Using the DNS protocol, your browser sends a request to get the IP addresses for some domain name, and then it gets a response with the answers.

If you are using Linux or macOS (or you have WSL2 enabled on your Windows 10 machine), you can use the DNS protocol to look up the IP addresses for dev.to using the dig tool:

$ dig +short dev.to
151.101.194.217
151.101.2.217
151.101.66.217
151.101.130.217

In this example, we are request the IPv4 address records of dev.to, but you can use the DNS protocol to query other stuff as well. We'll get to that in a bit.

Setting up a new Rust project

I'm writing this blog post as I'm writing this DNS server, and if you want to you can follow along, but all of the code also lives on xfbs/dnsfun if you just want to clone it and try it out for yourself.

First, we must setup a new Rust project. Rust uses cargo, a package manager, build system, documentation generation tool, and so much more. If you haven't set up Rust, check out the instructions at rustup to get started. We use the cargo new subcommand to create a new Rust project.

cargo new dnsfun

What this does is three things:

Creates the folder dnsfun
Creates a default code file at dnsfun/src/main.rs
Creates a default config at dnsfun/Cargo.toml

Adding dependencies

Once the project exists, we have to add dependencies. Usually, I add them one-by-one, when I need them. But for this project, I already know what I'm going to use, so I add the ones I know I will need upfront.

Normally, we would add these dependencies manually to the Crates.toml file, but cargo has a helpful subcommand to help us do that.

cargo add clap --features derive,env
cargo add tokio --features macros,rt-multi-thread,net
cargo add trust-dns-server
cargo add tracing
cargo add tracing-subscriber
cargo add async-trait
cargo add anyhow
cargo add thiserror

What this does is add the dependencies to the [dependencies] section in the Cargo.toml file. The dependencies are:

tokio, a very powerful async library,
clap, a library used to parse command-line options,
trust-dns-server, a library that implements a DNS server,
tracing, which lets us do logging and tracing of what's going on,
tracing-subscriber, which lets us print logs and traces to the standard output,
async-trait, which helps us create async trait methods,
anyhow, a package that provides a generic error type,
thiserror, a package that lets us easily define our own error types.

One thing you may have noticed is the features thing — in Rust, dependencies often have optional features that are only built if requested. If you read the documentation for a crate, it will tell you which features you need to enable to use certain things. This just helps keep compile times low.

Adding boilerplate code

With these dependencies in place, we can start with a very small boilerplate. Rust is not very verbose, it does not need some intricate setup or folder structure. Since we do want to use async, we need to initialize an async runtime, and because we want some amount of logging we need to initialize the tracing_subscriber crate. This is accomplished like this in the src/main.rs file.

#[tokio::main]
async fn main() {
    tracing_subscriber::fmt::init();
}

The #[tokio::main] is a macro from the tokio crate that automatically generates the code needed to start the runtime. We can use this because we turned on the macros feature of tokio.

Adding command-line options

There's some configuration options that the DNS server needs that we don't want to hard-code. So instead, we use command-line options to set them at runtime. We could have also used a configuration file, but that seemed overkill. So we create a new file, src/options.rs, with this content:

use clap::Parser;
use std::net::SocketAddr;

#[derive(Parser, Clone, Debug)]
pub struct Options {
    /// UDP socket to listen on.
    #[clap(long, short, default_value = "0.0.0.0:1053", env = "DNSFUN_UDP")]
    pub udp: Vec<SocketAddr>,

    /// TCP socket to listen on.
    #[clap(long, short, env = "DNSFUN_TCP")]
    pub tcp: Vec<SocketAddr>,

    /// Domain name
    #[clap(long, short, default_value = "dnsfun.dev", env = "DNSFUN_DOMAIN")]
    pub domain: String,
}

For explanation, the DNS protocol usually uses either UDP or TCP. Typically, UDP is preferred, but because there is a fixed maximum size for UDP packets, TCP can be used as fallback for when the response would be too large. We've declared three command-line options here:

--udp <socket> sets the sockets (address and port to listen on) for UDP packets. If nothing is specified, it defaults to 0.0.0.0:1053.
--tcp <socket> sets the sockets to listen for TCP connections. There is no default specified, so by default it won't listen to TCP connections.
--domain <domain> sets the domain that we want to serve DNS requests for, it defaults to dnsfun.dev.

The clap crate with the derive feature is very neat: we have a full declarative command-line parser in only 20 or so lines of Rust. This is all made possible because of the #[derive(Parser)] — that turns a Rust struct into a definition for command-line options. Any member of the field becomes a flag. If you add a field that is Option<String>, it becomes an optional flag. If you have a field that is a Vec<String>, you can pass it multiple times.

Creating a dummy Request Handler

In order for us to serve DNS requests, we need some code that generates the response for each request. The crate that we are using, trust-dns-server, calls this a RequestHandler.

We start by creating our own dummy implementation of a RequestHandler, just so we can get something to compile, and then we will get back to it and actually fill out the code.

We'll put this into a separate module, just to keep things somewhat tidy. So, we create a src/handler.rs with the following content:

use crate::Options;
use trust_dns_server::server::{Request, RequestHandler, ResponseHandler, ResponseInfo};

/// DNS Request Handler
#[derive(Clone, Debug)]
pub struct Handler {}

impl Handler {
    /// Create new handler from command-line options.
    pub fn from_options(_options: &Options) -> Self {
        Handler {}
    }
}

#[async_trait::async_trait]
impl RequestHandler for Handler {
    async fn handle_request<R: ResponseHandler>(
        &self,
        _request: &Request,
        _response: R,
    ) -> ResponseInfo {
        todo!()
    }
}

I'll walk you through what is going on here. First, we are importing a few things, such as the Options struct that holds the command-line options and some types we need for the DNS server.

Next, we define a struct called Handler. With the #[derive(Clone, Debug)], we are asking the Rust compiler to implement Clone for us, which gives us a .clone() method, and to implement Debug, which generates code to let us pretty-print it for debugging purposes.

In the impl Handler block, we define a function from_options to create a Handler struct from some parsed command-line options. If you are used to classes, this is roughly equivalent to a constructor.

Finally, we implement the RequestHandler trait for our Handler struct. To implement a trait, we must to define all of the functions that the trait requires. If you are used to classes, you can think of traits like interfaces or abstract base classes. Any struct you create can implement as many traits as you like. You can also create your own traits and implement them for any structs you like, the system is very flexible.

You might be a bit confused here — why does our implementation of the trait look so different from the trait definition in the documentation for RequestHandler? What's with all those weird lifetimes? There is some macro magic going on here. On that documentation page, if you click on source in the top right, you will see that we've implemented the trait exactly like how it was specified, but the docs show the version of the trait that's been automatically converted into a non-async method.

Right now, you cannot use async methods in trait methods in Rust, but the #[async-trait::async-trait] macro makes that possible still. This is a known quirk in the compiler, and the compiler itself will tell you to use async-trait instead. Eventually, async methods in traits will be stabilized, but thanks to the powerful macro system we can polyfill that ourselves.

In the handle_request method, you can see that it is generic over R: it accepts anything that implements ResponseHandler. This is because sending out the response depends on the protocol being used (DNS can work over UDP, TCP, TLS, HTTPS, etc). So the actual response handler type will depend on which protocol the request came in on. Also, you can see that we've put a todo!() where the implementation should be, this means that it will just crash because it is not implemented.

Launching DNS Server

Now that we have a dummy RequestHandler in place, we can get back to the main method and actually start the DNS server. To do so, we change the src/main.rs to import the handler struct and create a ServerFuture with the right listeners and then await it:

use anyhow::Result;
use clap::Parser;
use handler::Handler;
use options::Options;
use std::time::Duration;
use tokio::net::{TcpListener, UdpSocket};
use trust_dns_server::ServerFuture;

mod handler;
mod options;

/// Timeout for TCP connections.
const TCP_TIMEOUT: Duration = Duration::from_secs(10);

#[tokio::main]
async fn main() -> Result<()> {
    tracing_subscriber::fmt::init();
    let options = Options::parse();
    let handler = Handler::from_options(&options);

    // create DNS server
    let mut server = ServerFuture::new(handler);

    // register UDP listeners
    for udp in &options.udp {
        server.register_socket(UdpSocket::bind(udp).await?);
    }

    // register TCP listeners
    for tcp in &options.tcp {
        server.register_listener(TcpListener::bind(&tcp).await?, TCP_TIMEOUT);
    }

    // run DNS server
    server.block_until_done().await?;

    Ok(())
}

The first thing you might spot here is that we've defined a constant, TCP_TIMEOUT. This timeout is to prevent an attacker from creating a ton of TCP connections and then just letting them sit idle. It's good practice to give these constants a name so that we can add a documentation comment (starts with three slashes rather than two).

You can see how we are constructing a ServerFuture, this is the DNS server. If you look at the documentation for the new method, you can see that it needs a handler which implements RequestHandler. We have previously implemented that for our own Handler. So when we create the ServerFuture, we have to pass it an instance of our Handler. This trait system makes software very composable: the trust-dns-server crate hasn't hardcoded the responder, but they let us put our own logic into it simply by implementing the right trait.

We're also finally using the command-line options by registering the appropriate listeners.

Something you might notice is that there is .await in some places but not in others. Any time we call an async functions, we have to .await it to get the result. But not all functions we call need to be async: only those functions that might block at some point. For example, opening a UdpSocket might block, because we have to wait for the operating system to give us a response, so that needs to be async. But creating a Handler struct from command-line options cannot block, because we have all the data we need (no need to wait for anything from disk). If this async stuff doesn't make sense to you right now, I might make a future blog post explaining it some more.

Running the DNS server

At this point, we can already launch our DNS server. It is functional, except that any time we actually try to respond to a DNS packet, we will crash because of the todo!(). But that's okay, let's try it out!

We can use cargo run -- <options> to run it. The double dashes are needed to separate options for cargo run (before them) from options for the program (after them).

First, we can check out the automatically generated command-line options that we get from clap. If we pass the --help argument to our program, we will get a help text:

$ cargo run -- --help
Usage: dnsfun [OPTIONS]

Options:
  -u, --udp <UDP>        UDP socket to listen on [env: DNSFUN_UDP=] [default: 0.0.0.0:1053]
  -t, --tcp <TCP>        TCP socket to listen on [env: DNSFUN_TCP=]
  -d, --domain <DOMAIN>  Domain name [env: DNSFUN_DOMAIN=] [default: dnsfun.dev]
  -h, --help             Print help information

Sweet, it tells us what the flags are, what they mean (this is parsed from the documentation string in the Options struct), any default values and environment values that can be set.

To run our DNS server, we need two terminals. In the one terminal, we run the server itself. To get a better idea of what is going on, we can set the RUST_LOG environment value to trace, which turns on verbose logging of the tracing-subscriber. In the other terminal, we use dig to make a request to our DNS server. Here's what it looks like:

$ RUST_LOG=trace cargo run
2022-10-02T08:32:31.013539Z TRACE mio::poll: registering event source with poller: token=Token(0), interests=READABLE | WRITABLE    
2022-10-02T08:32:31.013657Z DEBUG trust_dns_server::server::server_future: registering udp: PollEvented { io: Some(UdpSocket { addr: 0.0.0.0:1053, fd: 6 }) }
2022-10-02T08:32:34.149745Z DEBUG trust_dns_server::server::server_future: received udp request from: 127.0.0.1:51997
2022-10-02T08:32:34.150111Z TRACE trust_dns_proto::rr::record_data: reading OPT
2022-10-02T08:32:34.150249Z DEBUG trust_dns_server::server::server_future: request:30324 src:UDP://127.0.0.1#51997 type:QUERY dnssec:false QUERY:test.com.:A:IN qflags:RD,AD
thread 'tokio-runtime-worker' panicked at 'not yet implemented', src/handler.rs:27:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

In the other terminal:

$ dig @127.0.0.1 -p 1053 test.com

So far, we can receive UDP DNS requests! But we obviously can't respond to them. We've gotten command-line parsing essentially for free with clap, gotten logging basically for free with tracing_subscriber, and a DNS server implementation for free from trust-dns-server. Now comes the interesting part, where we fill in our request handler.

Handling Requests, for real this time

Now, we have to implement our handle_request method to do something useful. There's a lot of funny things we could implement, but to keep this blog post somewhat short, there's three things I want to implement:

myip.dnsfun.dev should return the IP address of whomever is querying it.
counter.dnsfun.dev should return how many requests it has served so far.
<name>.hello.dnsfun.dev should return a greeting for the name.

This part is perhaps the hardest part, the DNS protocol offers a lot of terminology and possibilities that might be confusing. If you feel confused at times, don't worry — so did I. Here's the entire diff of this step.

The very first thing I need to do to make this possible is that we need to add some more fields to the Handler struct. To have a counter that is usable, we need some kind of atomic counter (it has to be atomic, because this whole thing is multithreaded). We will also need some help to route the requests for different subdomains.

/// DNS Request Handler
#[derive(Clone, Debug)]
pub struct Handler {
    /// Request counter, incremented on every successful request.
    pub counter: Arc<AtomicU64>,
    /// Domain to serve DNS responses for (requests for other domains are silently ignored).
    pub root_zone: LowerName,
    /// Zone name for counter (counter.dnsfun.dev)
    pub counter_zone: LowerName,
    /// Zone name for myip (myip.dnsfun.dev)
    pub myip_zone: LowerName,
    /// Zone name for hello (hello.dnsfun.dev)
    pub hello_zone: LowerName,
}

I've decided to use an Arc<AtomicU64> for my atomic counter. This is a variable that I can atomically increment and load from multiple threads. I also store the zone names in here, this is so I can easily check for a query if it matches anything I care about.

The RequestHandler trait wants the function to return a ResponseInfo, but there are cases where some error happens and we don't actually send a response. For this reason, I moved the actual logic into a different method, called do_handle_request, which I call in this handler. If that returns an error, I log it and fake a ResponseInfo.

#[async_trait::async_trait]
impl RequestHandler for Handler {
    async fn handle_request<R: ResponseHandler>(
        &self,
        request: &Request,
        response: R,
    ) -> ResponseInfo {
        // try to handle request
        match self.do_handle_request(request, response).await {
            Ok(info) => info,
            Err(error) => {
                error!("Error in RequestHandler: {error}");
                let mut header = Header::new();
                header.set_response_code(ResponseCode::ServFail);
                header.into()
            }
        }
    }
}

Next, in the do_handle_response method, which is just a private method of the Handler struct (similar to a private class method in OOP languages), I have to take a look at the request we got and decide what to do about it.

impl Handler {
    /// Handle request, returning ResponseInfo if response was successfully sent, or an error.
    async fn do_handle_request<R: ResponseHandler>(
        &self,
        request: &Request,
        response: R,
    ) -> Result<ResponseInfo, Error> {
        // make sure the request is a query
        if request.op_code() != OpCode::Query {
            return Err(Error::InvalidOpCode(request.op_code()));
        }

        // make sure the message type is a query
        if request.message_type() != MessageType::Query {
            return Err(Error::InvalidMessageType(request.message_type()));
        }

        match request.query().name() {
            name if self.myip_zone.zone_of(name) => {
                self.do_handle_request_myip(request, response).await
            }
            name if self.counter_zone.zone_of(name) => {
                self.do_handle_request_counter(request, response).await
            }
            name if self.hello_zone.zone_of(name) => {
                self.do_handle_request_hello(request, response).await
            }
            name if self.root_zone.zone_of(name) => {
                self.do_handle_request_default(request, response).await
            }
            name => Err(Error::InvalidZone(name.clone())),
        }
    }
}

What you can see here is that I'm making sure that we are serving a query (it wouldn't make sense for us to respond to anything else). I have separate handlers for the different zones (subdomains) that I want to service, and I just try to find a match. For example, a lookup for myip.dnsfun.dev would match the myip_zone, so that would call the do_handle_request_myip handler. A lookup for unknown.dnsfun.dev would match the root_zone, so it would call the do_handle_request_default handler. If we get a request for google.com, which doesn't even match the root zone (dnsfun.dev), when we don't even issue a response at all.

impl Handler {
    /// Handle requests for myip.{domain}.
    async fn do_handle_request_myip<R: ResponseHandler>(
        &self,
        request: &Request,
        mut responder: R,
    ) -> Result<ResponseInfo, Error> {
        self.counter.fetch_add(1, Ordering::SeqCst);
        let builder = MessageResponseBuilder::from_message_request(request);
        let mut header = Header::response_from_request(request.header());
        header.set_authoritative(true);
        let rdata = match request.src().ip() {
            IpAddr::V4(ipv4) => RData::A(ipv4),
            IpAddr::V6(ipv6) => RData::AAAA(ipv6),
        };
        let records = vec![Record::from_rdata(request.query().name().into(), 60, rdata)];
        let response = builder.build(header, records.iter(), &[], &[], &[]);
        Ok(responder.send_response(response).await?)
    }
}

Looking at the do_handle_response_myip handler, it shouldn't be too complicated: we increment the counter, then we build a response, and we use the source IP of the request to determine what to respond with.

impl Handler {
    /// Handle requests for counter.{domain}.
    async fn do_handle_request_counter<R: ResponseHandler>(
        &self,
        request: &Request,
        mut responder: R,
    ) -> Result<ResponseInfo, Error> {
        let counter = self.counter.fetch_add(1, Ordering::SeqCst);
        let builder = MessageResponseBuilder::from_message_request(request);
        let mut header = Header::response_from_request(request.header());
        header.set_authoritative(true);
        let rdata = RData::TXT(TXT::new(vec![counter.to_string()]));
        let records = vec![Record::from_rdata(request.query().name().into(), 60, rdata)];
        let response = builder.build(header, records.iter(), &[], &[], &[]);
        Ok(responder.send_response(response).await?)
    }
}

The do_handle_request_counter is similarly simple: we increment the counter, which returns the old value of it. Then we take that value, convert it to text and create a response with a text record.

impl Handler {
    /// Handle requests for *.hello.{domain}.
    async fn do_handle_request_hello<R: ResponseHandler>(
        &self,
        request: &Request,
        mut responder: R,
    ) -> Result<ResponseInfo, Error> {
        self.counter.fetch_add(1, Ordering::SeqCst);
        let builder = MessageResponseBuilder::from_message_request(request);
        let mut header = Header::response_from_request(request.header());
        header.set_authoritative(true);
        let name: &Name = request.query().name().borrow();
        let zone_parts = (name.num_labels() - self.hello_zone.num_labels() - 1) as usize;
        let name = name
            .iter()
            .enumerate()
            .filter(|(i, _)| i <= &zone_parts)
            .fold(String::from("hello,"), |a, (_, b)| {
                a + " " + &String::from_utf8_lossy(b)
            });
        let rdata = RData::TXT(TXT::new(vec![name]));
        let records = vec![Record::from_rdata(request.query().name().into(), 60, rdata)];
        let response = builder.build(header, records.iter(), &[], &[], &[]);
        Ok(responder.send_response(response).await?)
    }
}

Finally, the do_handle_request_hello is a bit complicated, because we have to extract the name part from the query. For example, when you make a query for patrick.hello.dnsfun.dev, we return hello, patrick. So part of this complexity here is taking the query, splitting it into parts like ["patrick", "hello", "dnsfun", "dev"], and then chopping the last parts off, leaving ["patrick"], and then creating a string from that. That is what the iterators are doing here.

To get some nice error messages, we also create our own error type. This is something that thiserror makes very easy, we can just create an enum and give it some information on how the errors should be displayed, and it takes care of the rest.

#[derive(thiserror::Error, Debug)]
pub enum Error {
    #[error("Invalid OpCode {0:}")]
    InvalidOpCode(OpCode),
    #[error("Invalid MessageType {0:}")]
    InvalidMessageType(MessageType),
    #[error("Invalid Zone {0:}")]
    InvalidZone(LowerName),
    #[error("IO error: {0:}")]
    Io(#[from] std::io::Error),
}

Trying out the DNS server

Again, we can run the server locally using cargo run. Running queries against it with dig shows that all of the features are working as expected:

$ dig +short @127.0.0.1 -p 1053 patrick.hello.dnsfun.dev
"hello, patrick"
$ dig +short @127.0.0.1 -p 1053 elon.musk.hello.dnsfun.dev
"hello, elon musk"
$ dig +short @127.0.0.1 -p 1053 counter.dnsfun.dev
"3"
$ dig +short @127.0.0.1 -p 1053 counter.dnsfun.dev
"4"
$ dig +short @127.0.0.1 -p 1053 myip.dnsfun.dev
127.0.0.1

Containerizing the DNS Server

Now that we have built it, we need some way of deploying it. Since we are using Rust, we can simply build and deploy the binary, because it has no dependencies. But in the real world, that is not always the case. Maybe it needs some data at runtime, or it needs some specific libraries. Containerizing makes it possible to bundle it, along with all of the runtime data it needs, and deploy it anywhere.

We are using Docker to do this, so all we need to do is write a small Dockerfile to drop into the repository that tells Docker how to build the container for this service. The diff for this step is here.

FROM rust AS builder

# copy code files
COPY Cargo.toml Cargo.lock /code/
COPY /src/ /code/src/

# build code
WORKDIR /code
RUN cargo build --release

# runtime container
FROM debian:11 AS runtime

# set default logging, can be overridden
ENV RUST_LOG=info

# copy binary
COPY --from=builder /code/target/release/dnsfun /usr/local/bin/dnsfun

# set entrypoint
ENTRYPOINT ["/usr/local/bin/dnsfun"]

This Dockerfile works in two stages: in the first stage, we use a rust image to build the service, resulting in a binary. In the next stage, we copy this binary into an empty debian container. Just for fun, we also add this to our Cargo.toml to make sure that the resulting binary is small:

[profile.release]
opt-level = "z"
lto = "thin"
debug = false
strip = true

What this does is tell the compiler to optimize for small file size, perform some thin link-time optimisation, and strip debug symbols from the resulting binary. This results in a 2MB binary, there is certainly still a lot of room for making it smaller but this is good enough.

Automating the build

Normally, I prefer to use GitLab and their CI system, but for this project we are using GitHub. What we want to achieve is that this project be compiled, the Docker container created and pushed to the Docker Hub automatically on every push. We can do this with GitHub Actions.

We don't even have to come up with something on our own. We can just copy the sample configuration from the actions-rs project to build and test the Rust code automatically, and the sample config for Docker.

The configuration files need to be placed inside the .github/workflows/ folder. You can see here what they look like.

Deploying DNS-Fun

I went ahead and purchased the domain dnsfun.dev from my registrar for 19.99€, and I purchased a VPS from my cloud provider for about 4.50€ per month. I then logged in to that VPS and ran something like this:

$ apt update
$ apt install docker.io apparmor-utils
$ docker run --name dnsfun -p 53:1053/udp xfbs/dnsfun --udp 0.0.0.0:1053

I had to do a few steps to get this VPS to be it's own name server -- I won't show you how it's done because it's different for every provider. But it's not too hard, you have to set some glue records and then update the nameserver settings. Piece of cake.

Now, you can make requests to it! You can either make requests through your normal DNS resolver, which means that the responses are cached:

$ dig +short counter.dnsfun.dev TXT
"2140997"
$ dig +short mike.hello.dnsfun.dev TXT
"hello, mike"

Or, you can make request directly to it. The IP address of the server is 159.69.35.198.

$ dig +short @159.69.35.198 counter.dnsfun.dev TXT
"2141001"
$ dig +short @159.69.35.198 myip.dnsfun.dev
123.23.45.67

I'm going to leave this thing running. Feel free to play around with it as you please. You can even try to break it, too. If you discover any issues, please open up an issue on GitHub. If you have any fun ideas on what to implement, create a pull request. I will accept all of them and deploy them.

Conclusion

Rust is certainly not the easiest language, but I like the expressiveness of it. The trait system makes building composable software easy. The macro system, although it feels a little bit like black magic, feels like fresh air compared to some other languages and helps keep code clean. I'm very impressed with the ecosystem that Rust has to offer.

I hope you enjoyed reading this blog post, it did turn out to be quite long. I wanted to walk you through the entire process of building and deploying this, without skipping over any parts.

Passgen: A password generator that uses a regex-like syntax to create secure passwords of any shape.

Patrick Elsen — Sun, 14 Aug 2022 13:54:29 +0000

I have a bit of a love-hate relationship with passwords. On the one hand, they are necessary for authenticating with services, and for decrypting drives. On the other hand, losing them means losing data or access to accounts. Choosing simpler passwords or reusing them across accounts makes dealing with them easier, but compromises their security.

Ideally, passwords should be as long as possible¹. While previously, the advice was to use password policies (such as enforcing the use of special characters), current best practice is to avoid that², and instead to encourage long passwords. Popular computer comic XKCD #936 notes that long word-based passwords have more strength and memorability than special-character-based passwords:

Based on this, I wanted to set out to build a tool that could help with generating random, strong passwords with different shapes. I wanted it to be very flexible, so that you can easily create passwords with different shapes (like, 16 alphanumeric characters, or 4 english words).

Regular Expressions

I'm also a very happy used of regular expressions. This is a language that is often used for filtering or searching text-based content. For example, the regular expression [a-z]{8} will match on any input that consists of exactly eight lowercase latin characters. If you are not familiar with regular expressions, here is a table that showcases their syntax:

Description	Regex
The literal string `abc`	`abc`
Either an `a`, a `b`, or a `c`	`[abc]`
Any lowercase latin character	`[a-z]`
16 lowercase latin characters	`[a-z]{16}`

Normally, regular expressions are used in code to validate input. For example, you might use a regular expression like [a-z0-9.]{3,16}@yourcompany.com to validate if something is a valid company email address.

What if I could build a tool that you can give a regular expression, and it evaluates it in reverse, spitting out a string that matches it rather than determining if a given string matches?

Out of that idea, passgen was born. It started out as a small single-file C project, and evolved into something I consider actually useful. It supports a small subset of the regular expression syntax and will produce a random string matching that. For example:



# generate a password with eight lower-case latin characters
$ passgen "[a-z]{8}"
nsaaemni
# generate a password with twelve alphanumeric latin characters
$ passgen "[a-zA-Z0-9]{12}"
Lp7FBOldLhJC

Passgen is fully unicode-aware, meaning that you can generate passwords for other languages, too. Here's some examples of that in action:



# password with german characters
$ passgen "[a-zßäüöA-ZÄÜÖ0-9]{8}"
6Mu6cüYI
# password with sixteen japanese characters
$ passgen "[\u{3040}-\u{309F}]{16}"
しがぢゐめまゔょぎっきす゚だぃぷ

Wordlists

One of the reasons I built Passgen was because I wanted to create more memorable passwords for myself. Using completely random letters makes for passwords that are short, but not memorable, as the XKCD earlier points out. So passgen has the ability to read wordlists and pick random words from these.

Example	Pattern
Random word from wordlist	`\w{english}`
Six random word from wordlist, separated by a hyphen	`(\w{english}-){5}\w{english}`
Six random word from wordlist, separated by a hyphen, followed by a random number	`(\w{english}-){5}\w{english}[0-9]{1,2}`

For example, if you are on Debian or Ubuntu, and you have the wamerican package installed, you will have a wordlist available at /usr/share/dict/american-english. If you are on another platform, there are likely similar preinstalled wordlists you can use. Using the --wordlist command-line flag lets you load this wordlist and then reference it in the pattern.



# pick 6 random words
$ passgen --wordlist english:/usr/share/dict/words "(\w{english}-){5}\w{english}"
vichyssoise-outstretching-requisite-weekended-homogenizes-calypsos
# pick 6 random words, and a one or two digit number
$ passgen --wordlist english:/usr/share/dict/words "(\w{english}-){5}\w{english}[0-9]{1,2}"
Karamazov-tasselling-pianos-shirr-devoutly-bullrings0

Markov Chain

While I think it is a good idea to use random words from a wordlist, I experimented with ways to come up with passwords that are similarly memorable (because they are pronounceable), but not neccessarily straight from a wordlist.

The idea here is using a Markov Chain to "learn" which letters frequently occur together from a dictionary file, and then generating new, random words from that. The outcome is words that are pronounceable, but have ideally more entropy, and thus are harder to crack.

To use this feature, use \m{wordlist} rather than \w{wordlist} in the pattern, it's really as simple as that!



# generate five words from english markov chain
$ passgen --wordlist english:/usr/share/dict/words "(\m{english}-){4}\m{english}"
dam-bloodling-tempouthed-distnuminest-muchee
# generate four words from english markov chain and a number
$ passgen --wordlist english:/usr/share/dict/words "(\m{english}-){3}\m{english}[0-9]{1,2}"
lariticitude's-lieu's-gadorayons-flers44

Complexity

Anytime passgen has a choice between multiple options, it knows how many options there are and it can thus keep track of how "random" your password is. This means that it can tell you how many bits of randomness your password contains, which can tell you something about how difficult it would be for an attacker to crack your password, if the attacker knew the pattern that you used to generate it. For example, if your password has 80 bits of entropy, it would take on average $2^{80}=1208925819614629174706176$ attempts to guess it.

Use the --complexity flag to tell passgen to output it's estimate of the complexity. Here's some examples:



# eight lowercase alphabetic characters
$ passgen --complexity "[a-z]{8}"
entropy: 37.603518 bits
qhziaohr
# twelve alphanumeric characters
$ passgen --complexity "[a-zA-Z0-9]{12}"
entropy: 71.450356 bits
IoSNTUNU3M1Q
# six random dictionary words
$ passgen --complexity --wordlist english:/usr/share/dict/words "(\w{english}-){5}\w{english}"
entropy: 100.025099 bits
smirked-Ila-ounces-circumstance's-bedpan-process
# six random markov chain words
$ passgen --complexity --wordlist english:/usr/share/dict/words "(\m{english}-){5}\m{english}"
entropy: 125.801760 bits
confabriquing-mas-Onei-scrite-elaw-inast

It is important to keep in mind that this feature is not perfect. If you give passgen a pattern like (a|a){5}, which technically has zero bits of entropy (it always produces aaaaa), passgen still calculates a nonzero complexity because passgen makes choices (between the first a and the second a).

Randomness

Passgen will preferentially use the Kernel's secure random number generator as a source of randomness, which is getrandom() on Linux and arc4random_buf() on macOS. It falls back to using /dev/urandom. However, the source of random data is configurable, it can also use a seeded pseudorandom number generator, but at this time only the insecure xorshift algorithm is implemented for that.

Trivia

Passgen uses libseccomp³ on amd64 platforms to sandbox itself, restricting it to only a small amount of allowed syscalls.

It can also output as JSON by passing the --json option.

Try it out

If you want to try it out, the source code is available here, and the releases page has signed releases for a bunch of different architectures and platforms. Please be so kind as to file an issue if something does not work, as I don't have a way of testing all of the builds currently.

If you are on Debian or Ubuntu, the .deb releases will be useful for you, because they let you install (and remove) passgen with APT easily.

Summary

The website has an overview of the syntax that is available to use, but here is a quick summary.

Example	Pattern
Generate the literal string `abc`	`passgen "abc"`
Pick a character from `a`, `b`, `c`	`passgen "[abc]"`
Pick a character from the range `a-z`	`passgen "[a-z]"`
Repeat `a` four times	`passgen "a{4}"`
Pick four lowercase characters	`passgen "[a-z]{4}"`

To generate passwords, you have a number of constructs available to use:

Verbatim characters. The pattern abc generates the "password" abc. They are simply passed through.
Characters sets, enclosed in brackets. You can either list all allowed characters or use ranges of characters, which are inclusive. Examples: [abc], [a-zA-Z0-9], [0123456789abcdef].
Groups, enclosed in paretheses. Groups let you list one or more possible patterns. Examples: (word) generates word, (this|that) generates this or that, ([a-z]|[0-9]) generates a lowercase latin character ([a-z]) or a number ([0-9]).
Repeat. Using braces lets you repeat whatever was before. You can either specify a fixed number or a lower and an upper bound, separated by a comma. Examples: [abc]{3}, [a-z]{9,12}, (this|that){2}.

If this syntax is too complicated for you, the passgen binary also comes with some pattern presets, that you can use with the -p flag. These are named after the software that generates passwords matching these patterns.



# generated by older versions of safari
$ passgen -p apple1
asG-mIQ-7jz-LAT
# generated by current safari
$ passgen -p apple2
lHODYM-MgdmfG-llo5kD
# generated by firefox
$ passgen -p firefox
iBpDBlTg8bX051H

Development

If you are a real nerd, you may want to compile it for yourself. You need some dependencies:

CMake
C compiler (gcc or clang)
Git

With those dependencies, you should be able to build it by cloning the repository, fetching the submodules (this is needed for libutf8proc⁴).



git clone https://gitlab.com/xfbs/passgen
git submodule update --init
mkdir passgen/build
cd passgen/build
cmake ..
make -j8

If you want to understand how pieces of it work, there is also some documentation available that is built off the master branch.

Tests

Passgen has a number of unit tests built-in, you can run these with make test. You can also run these tests under valgrind if you have it installed, this will let you make sure that there are no memory leaks.



$ valgrind ./src/test/passgen-test -v

Support

Your support is appreciated. If you run passgen, and something doesn't work, please let me know. You can file an issue in the repository and I will try to fix it.

If you enjoy passgen, and you find bugs, have suggestions, or want to help get it into any package repositories, please don't hesitate to reach out to me, I appreciate any help. I hope that it can be useful to other people.

Minimum length of the passwords should be enforced by the application. Passwords shorter than 8 characters are considered to be weak. https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html ↩
https://www.troyhunt.com/passwords-evolved-authentication-guidance-for-the-modern-era/ ↩
https://github.com/seccomp/libseccomp ↩
utf8proc, a UTF-8 decoding library. https://github.com/JuliaStrings/utf8proc ↩