DEV Community: Minh Nguyen (he/him)

How to use MailHog in Lando

Minh Nguyen (he/him) — Mon, 08 Feb 2021 05:00:00 +0000

Photo by Charlotte Harrison on Unsplash.

If you’ve ever needed a service to capture e-mails, MailHog is a pretty good choice. MailHog contains an SMTP server which you send messages to, simulating your production SMTP service, and provides a HTTP interface so you can see what’s being sent.

It isn’t a substitute for your production mail service or e-mail client, so you’ll still need to plug it all in for testing, and make sure to check in a variety of e-mail clients. This is good if you need to debug why e-mail clients are rendering your messages strangely and need to experiment.

With that in mind, I’ll show you how to add a MailHog service in Lando via the Landofile .lando.yml.

Prerequisites

You know how to run Lando
You have an application which sends messages via SMTP

Setting up your Landofile

In your Landofile (.lando.yml), add a new section for services. If your Landofile already has one, you can append to it instead. Below I define a new service called mail:

# .lando.yml
# ...the rest of the YAML config sits above here

services:
  mail:
    type: mailhog

We’re almost ready to go, but we need to provide information about which services need to be able to access MailHog. To do that, define a key labelled hogfrom, and list all our services. In this case, appserver is responsible for sending our message:

# .lando.yml
# ...the rest of the YAML config sits above here  

services:
  mail:
    type: mailhog
    hogfrom:
      - appserver

Run lando rebuild, which should rebuild your stack, including the new MailHog service. In the output where you see a list of URLs you can access your app from, you should now see a localhost URL that you can open to access the MailHog interface.

Configuring your web app

In your web app, you will need to change the SMTP configuration so it points at MailHog. In Drupal, this can be done by overriding the system configuration in settings.local.php. This may differ depending on your specific application, so I can’t help with that.

In terms of credentials and URLs, here is what you can use to connect to MailHog’s SMTP server:

Hostname: mail—same as the name we gave the service above
Port: 1025
Username: as desired
Password: as desired

Some questions

Couldn’t I just run MailHog outside of Lando or Docker?

Sure, but Lando exists as a way to lock down those tools so your colleagues don’t have to figure out how to install MailHog, or ask you how to install it.

Wasn’t MailHog development suspended?

It was for some time, but development picked up again and a new release 1.0.1 was put out August 12 2020. I was a bit surprised as well, since the last official release 1.0.0 was in 2017.

How to install ClamAV in Lando for Drupal

Minh Nguyen (he/him) — Tue, 02 Feb 2021 23:00:00 +0000

Photo by N Bandaru on Unsplash.

ClamAV is an antivirus software service which allows servers to scan for viruses. While it is notable for use in scanning for viruses on mail servers, we can also use it in our web applications to scan file uploads.

I’m doing some local development on a Drupal based site, which makes use of ClamAV through a contributed module to scan files uploaded to the site.

Having the module enabled but not having ClamAV on the system throws an error:

You could just uninstall the ClamAV module and work without it, but that introduces a potential gap in your test coverage—the behaviour of the site operates differently without the module installed, which could lead to issues that you don’t discover until later.

Prerequisites

You know how to run Lando
Your application is configured to scan file uploads e.g. in Drupal, the ClamAV module

Setting up your Landofile

In your Landofile (.lando.yml), we will use the services section to modify the main application service—in our case labelled appserver to install ClamAV:

# .lando.yml
# ...the rest of the YAML config sits above here

services:
  appserver:
    build_as_root:
      - apt-get update -y
      - apt-get install clamav clamav-daemon -y
      - echo "TCPSocket 3310" >> /etc/clamav/clamd.conf
      - freshclam
      - update-rc.d clamav-daemon enable
    run_as_root:
      - service clamav-daemon start

This sets up the commands to run that installs ClamAV, sets up the ability to connect to the service on port 3310, update the database definitions necessary to scan files, and then enable the ClamAV service.

Note here that we are using apt-get to install the packages as the base container images that Lando uses are from Debian. You may need to adjust if your base distribution is different.

Once the changes are made, rebuild the service with lando rebuild. This will relaunch your application, install ClamAV into your local container, and start it.

I can now load up my environment and confirm that ClamAV is successfully loaded, and that file uploads work:

Furthermore, we can use the EICAR test file to confirm that it does block viruses from being uploaded:

Possible future changes

I could set out these changes in a custom service, or alternatively use a prebuilt container for ClamAV. Of course, the current hosting environment assumes ClamAV running on localhost on a TCP port so it’s not practical to modify it for that yet.

Hopefully this has been a useful guide in how to have ClamAV set up for your Drupal site. If you’re not running Drupal, hopefully you’ll find it useful if you’re integrating ClamAV for your own applications and infrastructure.

Annotating my pages with microformats2 and h-entry

Minh Nguyen (he/him) — Tue, 06 Oct 2020 12:44:33 +0000

h-entry is an open microformat for datestamped content on the web. I was looking at ways to make my site a bit more open to syndication, and also as an alternative to RSS/Atom feeds. I haven’t seen major browsers pick up on these formats, but the main focus are IndieWeb readers, and I’m all for them.

Now, please note that this post isn’t much of a guide to implementing h-entry or other parts of the microformats2 spec on your site, as it is a summary of I went about implementing it for this website.

What it looks like

{
  "items": [
    {
      "type": [
        "h-entry"
      ],
      "properties": {
        "name": [
          "My blog post"
        ],
        "author": [
          {
            "value": "John Smith"
          }
        ],
        "published": [
          "2020-01-01 12:00:00"
        ],
        "summary": [
          "A short summary of my blog post, and what you can do."
        ],
        "content": [
          {
            "value": "Starting paragraph of my blog post.",
            "html": "<p>Starting paragraph of my blog post.</p>"
          }
        ]
      }
    }
  ]
}

In short, there’s basically all of the information about the post, in a JSON object that could be easily be used as part of a different app. With that in mind, how does a parser go about doing this?

Elements of h-entry

h-entry itself is a what is known as a root class name. We can use it to define a h-entry, and it has a series of properties that the parser will determine, based on knowing that. Other root class names include:

h-adr
h-event
h-geo
h-item
h-product
h-recipe

You can find a more detailed list of these on the microformats2 page, along with examples and detailed descriptions.

Back to h-entry, we can establish this simply by naming it as a class:

<article class="h-entry">
...
</article>

and that’s it, our article is now parseable as it is, though it might not be very useful by itself. This is where we add some core h-entry properties!

h-entry properties

There’s quite an amount of these, so I’ll only go with the properties that I used for this site.

p-name

This represents the entry name or title. For the title, we just need to add this class to the element that contains it:

<h1 class="p-name">{post.frontmatter.title}</h1>

e-content

The full content of the entry in question i.e. the full body of the blog post:

<section class="e-content">...</section>

dt-published

The datestamp of when the entry was published:

<time class="dt-published" datetime="2020-09-29">29 September, 2020</time>

Implementing h-entry within a Gatsby content type

Let’s get topical! For this, I’m using Gatsby as my site generator, so in order to get this working for my blog post, I need to modify the template for the blog post’s render function (located in /src/templates/blog-post.js), which returns JSX as follows:

export default ({data}) => {
  const post = data.markdownRemark
  return (
    <Layout>
      <SEO title={post.frontmatter.title} description={post.excerpt} />
      <section className="h-entry">
        <h1 style={{marginBottom: `0.5rem`}} className="p-name">{post.frontmatter.title}</h1>
        { post.frontmatter.draft ? (
          `Draft`
        ) : (
          <time className="dt-published" dateTime={post.frontmatter.htmldate}>{post.frontmatter.date}</time>
        )}
        <section className="e-content" dangerouslySetInnerHTML={{ __html: post.html }} />
      </section>
    </Layout>
  )
}

You can see that the 2 section, the h1, and the time elements are updated as above to have the microformat2 elements defined via the className attribute. While I was able to get it working on these generic HTML elements, I wasn’t able to get it working for the time element, which was previously a separate element called <DateTime>. I couldn’t figure it out, so I reverted to using the base element that it represented.

Running NixOS in Production

Minh Nguyen (he/him) — Thu, 17 Sep 2020 08:16:00 +0000

NixOS logo created by Tim Cuthbertson (@timbertson), licenced under CC-BY 4.0.

I’ve moved some items from my previous site infrastructure over to a VPS powered running NixOS in the past weekend. What made me do it? There were a few reasons that came to mind for it.

The first is that I’m sort of done running on OpenVZ—the performance is pretty bare compared to other options out there. There’s only a few choices for operating systems to run on it. Ubuntu 16.04 (Xenial) is okay to work with, but on the limited resources, things always took a lot longer than they should have, whether it was running Ghost or trying to build this site on Gatsby.

Another reason for doing this is that I don’t remember a lot of the things I did setting up the site. Could I tell you I installed Certbot? Nope. Could I recite the hundred word set of compiler arguments to make I compiled nginx with Pagespeed support? Nope. Would I know how to update PHP from 7.2 once it reaches end of life? Nope—I’d have to look that one up (and I already have to do that in my day to day work!).

I could have picked any other OS to operate, like Debian or CentOS. But I ended up on NixOS for the following reasons:

I wanted to learn more about NixOS, and how to run through application deployment and management to see how it stacks up against other methods.
The idea of declaring my system configuration as code I could deploy was interesting to me, since I spend a lot of time running through package installation and configuration in my day-to-day—making that process easier would definitely give me time to do other things.

Some of the details

For some context, I have Nix installed on my Arch Linux system (there’s a useful guide to install it on their wiki). From there, it was a matter of putting together a few pieces of configuration. The build script itself is powered by Elixir—mainly a result of the fact I couldn’t get the Haskell script in the guide I was following to work.

The VPS is an Amazon EC2 instance, which was created using the instructions provided by NixOS to launch a provided Amazon Machine Image (AMI).

The first stop is server.nix, which is built like this:

let
  nixos = import <nixpkgs/nixos> {
    configuration = import ./configuration.nix;
  };
in
  nixos.system

configuration.nix contains the bulk of the configuration which is set up on the system:

{config, pkgs, ...}:
let
  unstable = import <nixos-unstable> {};
in
{
  imports = [<nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ./users.nix ./firewall.nix ./webserver.nix];
  ec2.hvm = true;
  networking.hostName = "mnguyen-nix-demo";
  environment.systemPackages = [
    unstable.caddy2
    pkgs.fish
    pkgs.htop
    pkgs.mosh
    pkgs.vim
  ];

  programs.fish.enable = true;

  # sudo without requiring password
  security.sudo.wheelNeedsPassword = false;

  ## Enable BBR module
  boot.kernelModules = ["tcp_bbr"];

  ## Network hardening and performance
  boot.kernel.sysctl = {
    # Disable magic SysRq key
    "kernel.sysrq" = 0;
    # Ignore ICMP broadcasts to avoid participating in Smurf attacks
    "net.ipv4.icmp_echo_ignore_broadcasts" = 1;
    # Ignore bad ICMP errors
    "net.ipv4.icmp_ignore_bogus_error_responses" = 1;
    # Reverse-path filter for spoof protection
    "net.ipv4.conf.default.rp_filter" = 1;
    "net.ipv4.conf.all.rp_filter" = 1;
    # SYN flood protection
    "net.ipv4.tcp_syncookies" = 1;
    # Do not accept ICMP redirects (prevent MITM attacks)
    "net.ipv4.conf.all.accept_redirects" = 0;
    "net.ipv4.conf.default.accept_redirects" = 0;
    "net.ipv4.conf.all.secure_redirects" = 0;
    "net.ipv4.conf.default.secure_redirects" = 0;
    "net.ipv6.conf.all.accept_redirects" = 0;
    "net.ipv6.conf.default.accept_redirects" = 0;
    # Do not send ICMP redirects (we are not a router)
    "net.ipv4.conf.all.send_redirects" = 0;
    # Do not accept IP source route packets (we are not a router)
    "net.ipv4.conf.all.accept_source_route" = 0;
    "net.ipv6.conf.all.accept_source_route" = 0;
    # Protect against tcp time-wait assassination hazards
    "net.ipv4.tcp_rfc1337" = 1;
    # TCP Fast Open (TFO)
    "net.ipv4.tcp_fastopen" = 3;
    ## Bufferbloat mitigations
    # Requires >= 4.9 & kernel module
    "net.ipv4.tcp_congestion_control" = "bbr";
    # Requires >= 4.19
    "net.core.default_qdisc" = "cake";
  };

  # disable passwordless SSH
  services.openssh.passwordAuthentication = false;

  # Let trusted users upload unsigned packages
  nix.trustedUsers = ["@wheel"];

  # Clean up packages after a while
  nix.gc = {
    automatic = true;
    dates = "weekly UTC";
  };

  # Disable reinitialisation of AMI on restart or power cycle
  systemd.services.amazon-init.enable = false;

  swapDevices = [
    {
      device = "/swapfile";
      priority = 10;
      size = 1024;
    }
  ];

  systemd.services.fathom = {
    description = "Fathom Server";
    requires = ["network.target"];
    after = ["network.target"];
    wantedBy = ["multi-user.target"];

    serviceConfig = {
      Type = "simple";
      User = "minh";
      Restart = "on-failure";
      RestartSec = 3;
      WorkingDirectory = "/var/lib/fathom";
      ExecStart = "/home/minh/bin/fathom --config=/etc/fathom.env server";
    };
  };
}

imports pulls in specific NixOS configuration for EC2 which it’s running on, and I’ve started work separating out configuration to separate files, ./users.nix, ./firewall.nix, and ./webserver.nix.

Packages

You can see near the top of configuration.nix there’s a variable called environment.systemPackages. This lists all the additional software packages I want installed. In this case, I have fish, htop, mosh, and vim installed from the standard repo, as well as Caddy 2 from the unstable NixOS branch.

Swap

I don’t have any swap on the system, so the variable swapdevices is set to create a swap file at /swapfile. The size is defined in megabytes, so this file is 1 gigabyte in size.

Firewall

NixOS comes with a firewall setup out of the box to only be open for SSH on port 22. To change it up, we can set networking.firewall.allowedTCPPorts as a list of ports we want to be open.

I also want to move the default SSH port to something else, this is defined by services.openssh.listenAddresses.

In the end, my firewall.nix file looks like this:

{ config, pkgs, ...}:
{
  # SSHD Port reassignment
  services.openssh.listenAddresses = [
    { addr = "0.0.0.0"; port = 37586; }
  ];

  # Allowed TCP range
  networking.firewall.allowedTCPPorts = [80 443 37586];

  # Allow Mosh connections
  networking.firewall.allowedUDPPortRanges = [{ from = 60000; to = 60010; }];
}

Caddy server

Moving on, I set up the web server within webserver.nix:

{config, pkgs, ...}:
let
  unstable = import <nixos-unstable> {};
  caddyDir = "/var/lib/caddy";
  caddyConfig = pkgs.writeText "Caddyfile"
    ''{
  storage file_system {
    root /var/lib/caddy
  }
}
mnguyen.io {
  root * /srv/www/mnguyen.io
  file_server
  header / {
    X-Content-Type-Options "nosniff"
    X-Frame-Options "sameorigin"
    Referrer-Policy "no-referrer-when-downgrade"
    X-UA-Compatible "IE=edge,chrome=1"
    X-XSS-Protection "1; mode=block"
    Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    Content-Security-Policy "default-src 'self'; connect-src https://analytics.mnguyen.io 'self'; font-src 'self' data:; img-src https://analytics.mnguyen.io 'self' data:; script-src https://analytics.mnguyen.io 'self' 'unsafe-inline'; style-src 'unsafe-inline'; worker-src 'self'; prefetch-src 'self'; report-uri https://mnguyen.report-uri.com/r/d/csp/enforce; report-to https://mnguyen.report-uri.com/r/d/csp/enforce"
  }
}

www.mnguyen.io {
  redir https://mnguyen.io{uri}
}

analytics.mnguyen.io {
  reverse_proxy localhost:9000
}
'';
in
{
  systemd.services.caddy = {
    description = "Caddy web server";
    after = ["network-online.target"];
    wants = ["network-online.target"];
    wantedBy = ["multi-user.target"];
    serviceConfig = {
      User = "caddy";
      Group = "caddy";
      ExecStart = ''
        ${unstable.caddy2}/bin/caddy run --config ${caddyConfig} --adapter caddyfile
      '';
      ExecReload = ''
        ${unstable.caddy2}/bin/caddy reload --config ${caddyConfig} --adapter caddyfile
      '';
      TimeoutStopSec = "5s";
      LimitNOFILE = 1048576;
      LimitNPROC = 512;
      PrivateTmp = true;
      ProtectSystem = "full";
      AmbientCapabilities = "cap_net_bind_service";
    };
  };

  users.users.caddy = {
    group = "caddy";
    uid = config.ids.uids.caddy;
    home = caddyDir;
    createHome = true;
    extraGroups = ["users"];
  };

  users.groups.caddy.gid = config.ids.uids.caddy;
}

What this does is set up the Caddyfile that Caddy 2 will be using, as well as the systemd service which sets up the command, sets the capabilities of the service so it can run at lower port numbers (below 1024 to be precise), and sets up a dedicated caddy user which it will run under.

Users

Speaking of which, users in NixOS can be defined in a few lines. This is what can go into users.nix:

{ config, pkgs, ...}:
{
  users.users.minh = {
    isNormalUser = true;
    extraGroups = ["wheel"];
    shell = pkgs.fish;

    # password = "my secure password";
    # hashedPassword = "$6$qTK.7QsrnONOr$ZsAfPlnEPLtpiO9j1qp/POkDga2LtK1UOD0nrG497CegYEq5e.E6iHf5tDqwfLViBSWEsw8sn5t885p6HyRgS1";

    openssh.authorizedKeys.keys = [
      "ssh-rsa PUBLICKEYHERE"
    ];
  };
}

The flag isNormalUser tells NixOS to give us a home directory at /home/minh. I then set up sudo access by adding this user to the wheel group, set the shell to fish, and set the allowed SSH key that is allowed to access this user on the system—I cannot sign in with a password unless it’s explicitly set.

If I did want to set a password, I could use the insecure option password which is a string, or slightly more secure passwordHash (which uses a hash generated from the command mkpasswd -m sha-512, detailed on the passwordHash options page)

Fathom analytics

I pulled the last public available release of Fathom Lite from their repository, and created a systemd service file to run it. I was worried for a bit that it wouldn’t run, as there were a lot of discussions going on the community forums about having to patch or otherwise emulate a standard Linux system to make binaries work.

Thankfully, Fathom isn’t one of those programs, and it worked flawlessly without any other work. It’s not packaged for NixOS, so this is probably the least NixOS part of this system.

Building NixOS

There are 2 commands to build our NixOS packages and send it to our remote system, then there’s 2 other commands to switch profiles and activate our new setup:

nix-build: which builds the NixOS packages
nix-copy-closure: sends the NixOS packages to our remote
nix-env: Set the Nix profile to our uploaded packages
switch-to-configuration: Switch the system configuration to our new configuration

These all come together in an Elixir script:

server_raw = File.read!("server_address.txt")
server_processed = String.replace(server_raw, "\n", "")

defmodule Build do
  def upload_to_system(path, address) do
    {path_str, _status} = path
    fixed_path_str = String.replace(path_str, "\n", "")
    System.cmd("nix-copy-closure", ["--to", "--use-substitutes", address, fixed_path_str])
    Build.activate_nix(path_str, address)
  end

  def activate_nix(path, address) do
    profile = "/nix/var/nix/profiles/system"
    System.cmd("ssh", ["#{address}", "sudo nix-env --profile #{profile} --set #{path}"])
    System.cmd("ssh", ["#{address}", "sudo #{profile}/bin/switch-to-configuration switch"])
  end
end

System.cmd("nix-build", ["server.nix", "--no-out-link"])
  |> Build.upload_to_system(server_processed)

Running the script runs all of those commands sequentially, and executes all the above. One thing I note about this is that the script hangs if there’s an issue with any of the configuration files, which could lead to a server that’s unresponsive. You may need to power cycle your remote instance if that happens.

Concluding remarks

I learned quite a bit about NixOS, but like most things involved quite a bit of trial and error. Thankfully, having this sort of configuration means I can easily rebuild the system (such as when I found myself unable to connect via SSH when I changed the port but didn’t realise the firewall needed to be changed as well).

If you’re interested in NixOS, I’d encourage you to take a look.

Useful links

In case you wanted to delve further into some of the resources I looked up while I was getting this set up:

Deploying NixOS to Amazon EC2 by Type Classes: This was very useful starting out to understand how to build NixOS locally, and how to ship it off to a different installation.
Running a local dns and web server by José Luis Lafuente: I found the section to define the custom systemd service very useful (just a heads up you want to set home for the user in that config to the Caddy data directory—mine was /var/lib/caddy)

DEV Community: Minh Nguyen (he/him)

How to use MailHog in Lando

Prerequisites

Setting up your Landofile

Configuring your web app

Some questions

Couldn’t I just run MailHog outside of Lando or Docker?

Wasn’t MailHog development suspended?

How to install ClamAV in Lando for Drupal

Prerequisites

Setting up your Landofile

Possible future changes

Annotating my pages with microformats2 and h-entry

What it looks like

Elements of h-entry

h-entry properties

p-name

e-content

dt-published

Implementing h-entry within a Gatsby content type

Further reading

Running NixOS in Production

Some of the details

Packages

Swap

Firewall

Caddy server

Users

Fathom analytics

Building NixOS

Concluding remarks

Useful links