DEV Community: Mat Jones

Introducing Yaclt (Yet Another ChangeLog Tool)

Mat Jones — Sun, 08 Aug 2021 20:08:07 +0000

The Problem

Several months ago, I set out to solve a pain point I was running into at work — manually writing release notes by copy and pasting entries from Merge Request descriptions. I couldn't find an existing tool that quite worked the way we wanted it to work. We didn't necessarily want to use git commit messages as our changelog entries, which eliminates a lot of the other changelog tools out there.

We wanted to have explicit control over the changelog entries, and we wanted our developers to be intentional and thoughtful about the messages they add to the changelog. This manifested itself in the following workflow, supported by a set of bash scripts I set up:

Each Merge Request contains a changelog entry in a single, self-contained markdown file within a changelogs/ directory (a file can contain multiple lines, each line being an individual changelog entry)
- These changelogs are strictly validated against a format pattern
- This is enforced in a step of the CI build
When tagging a release, we first run a script which:
- Iterates through all files under changelogs/
- Copies each entry within each file into the global CHANGELOG.md file
- Deletes all individual entry files under changelogs/

After a few weeks of testing and tweaking the scripts which helped us automate this workflow, it turned out it worked really, really well for us.

These scripts, however, lived directly in one repository, so the release notes process for our other projects remained a manual process. This prompted an evolution of the tooling such that it could be shared between projects.

The Result

Enter Yaclt, or: Yet Another ChangeLog Tool. Yaclt is the next evolution of this set of bash scripts; it's a proper command line tool, written in TypeScript, and highly configurable to meet the specific needs of a project, or even a specific developer.

Each command is configurable with many options, and any option can also be specified in a configuration file. The configuration file supports yaml, JSON, or Javascript format; if you're using a Javascript configuration file, any option can be a function which returns the option value. This can be useful for things like computing the next release number from git history, or using a package.json version as the release number.

The tool also has some built-in features like parsing issue numbers from git branch names, given a format to match against, using the first line of your most recent commit message as the changelog message, opening generated changelogs in $EDITOR, and more.

Generating and validating changelog entries is super simple:

And so is collecting the entries into the global CHANGELOG.md to prepare for a release:

This project is still a work in progress, and there are some great ideas for enhancements in the backlog, but it should be usable for your basic changelog workflow. Please open an issue if you run into any problems or pain points (or even better, submit a Pull Request)!

Stars on the repository are greatly appreciated!

mrjones2014 / yaclt

Yet Another Change Log Tool

What is yaclt? It's Yet Another ChangeLog Tool.

yaclt is a file-based changelog generator. The idea is basically the following:

For each pull request/merge request, include a changelog file generated by this tool -- it gets stored in source control until the next release
To prepare a release, use the prepare-release command, which will
- Gather all the individual changelog entries into a global CHANGELOG.md (or whatever file you've specified)
- Delete all the individual changelog files

The changelog entry format and the global changelog file template are both Handlebars templates, and you can use the extra helpers from handlebars-helpers.

Help for each command can be found in COMMANDS.md or by running the command with the --help argument.

Dependencies

yaclt requires that git exists on your $PATH and that node 12+ is installed.

Configuration

All command line flags and arguments can be specified in…

View on GitHub

Storing Dotfiles in a Git Repo

Mat Jones — Sat, 22 May 2021 17:54:21 +0000

Everyone has seen those dotfiles repositories on GitHub. There's lots of different ways to manage them, but the method I use now requires no extra tooling (other than git and a command line shell of your choice), no symbolic links to get files into the right locations, can be triggered from any directory on disk, and is easy to replicate on a new system.

Basically what we're going to do is set up a git repository at ~/.cfg or ~/.dotfiles or any directory within your home directory of your choosing (although you probably don't want to use ~/.config since that is the default value of $XDG_CONFIG_HOME), and a shell alias to help manage and control it.

Creating the Repo

If you're setting this up the first time, there's a few steps you'll need to take to set up. First, create the repository:

git init --bare $HOME/.dotfiles

This creates a "bare" git repository at ~/.dotfiles. Now we'll set up an alias to interact with it from any directory on disk. Add the following alias to your ~/.bashrc or ~/.zshrc or ~/.config/fish/config.fish file, then source the file:

# make sure the --git-dir is the same as the
# directory where you created the repo above.
alias config="git --git-dir=$HOME/.dotfiles --work-tree=$HOME"

The --work-tree=$HOME option sets the directory that the repository tracks to your home directory. Now, since there's probably more files in your home directory that you don't want in the repo than files you do want in the repo, you should configure the repo to not show untracked files by default. We can do that by setting a repository-local configuration option.

config config --local status.showUntrackedFiles no

Tracking Files

To track files in our new ~/.dotfiles repo, we just need to add them. From any directory on disk, you can run the following command to add your ~/.bashrc or ~/.zshrc or ~/.config/fish/config.fish file to your new dotfiles repo:

config add ~/.bashrc
config add ~/.zshrc
config add ~/.config/fish/config.fish

config commit -m "Add .bashrc/.zshrc/config.fish file"
config push

Installing on a New System

Of course, the main point of doing this is to easily sync your config across new machines. We can easily do this with a small bash script to initialize the system's dotfiles from your git repository.

#!/usr/bin/env bash

git clone --bare git@github.com:mrjones2014/dotfiles.git $HOME/.dotfiles

# define config alias locally since the dotfiles
# aren't installed on the system yet
function config {
   git --git-dir=$HOME/.dotfiles/ --work-tree=$HOME $@
}

# create a directory to backup existing dotfiles to
mkdir -p .dotfiles-backup
config checkout
if [ $? = 0 ]; then
  echo "Checked out dotfiles from git@github.com:mrjones2014/dotfiles.git";
  else
    echo "Moving existing dotfiles to ~/.dotfiles-backup";
    config checkout 2>&1 | egrep "\s+\." | awk {'print $1'} | xargs -I{} mv {} .dotfiles-backup/{}
fi

# checkout dotfiles from repo
config checkout
config config status.showUntrackedFiles no

Now, installing your dotfiles on a new system is as simple as running:

curl https://raw.githubusercontent.com/mrjones2014/dotfiles/master/scripts/config-init | bash

That's Really It

That's really all there is to it. Now you can easily add and track changes to dotfiles via your new config shell alias.

config add ~/.config/something/somefile
config commit -m "add somefile"
config push

Comment below with your dotfiles repo links! Feel free to browse mine for inspiration.

mrjones2014 / dotfiles

❄️ My dotfiles for NixOS and macOS as a Nix flake. Neovim, Fish shell, Wezterm, etc.

Dotfiles

These are my NixOS and macOS dotfiles, packaged as a Nix flake, using home-manager.

For info on how to set up my Nix flake, see the setup instructions.

Why a Nix Flake

Nix is an incredibly complex piece of software, but despite that, I believe that Nix is the only sensible way to manage software today. Using Nix, and particularly a Nix flake, offers a few unique benefits:

Reproducibility: Nix environments are described by text files (*.nix files), and as long as you stay within the guard rails, the environment should be deterministic.
Isolation: Packages can have access to their dependencies without those dependencies cluttering up the global environment — this also means different packages can depend on different versions of the same dependency without conflicts.
Rollbacks: Totally screw up your environment by accident? Just roll back to a previous generation.
Immutability…

View on GitHub

How I Built My Personal Website

Mat Jones — Fri, 22 Jan 2021 23:05:15 +0000

I had a lot of fun building my personal site, because I used it as an opportunity to learn some new tech. Let's take a tour of what's in it.

Core

At it's core, the site is powered by Next.js and React. I used Next.js because the content is mostly static, which is a use-case Next excels at. Next enables Server Side Rendering (SSR) by default, but also supports Server Side Generation (SSG), which generates static HTML pages at build time. You can read more about the differences between SSR and SSG in this great article by the Vercel team.

vercel / next.js

The React Framework

Code Style

In the project, I used React functional components with hooks almost exclusively because I find I can write much cleaner and more concise components using the hooks API vs. traditional class-based components; the only exceptions being _app.tsx and _document.tsx, which extend base classes from the Next framework. A _document.tsx file allows you to customize the base HTML template that the React app is injected into, while _app.tsx allows you to use shared layouts, add an Error Boundary, import global stylesheets, and more.

Build Customization

Markdown Copy Content

I've also added @mdx-js/loader and @next/mdx to my build configuration, which allows me to import markdown (*.md) files as React components. Using this technique allows me to keep some of my copy text (such as the "about" page) in markdown rather than having to manually write out the JSX for it, which is just simpler and easier to maintain.

Working with SVGs

I'm using SVGR to transform my SVG files into React components, rather than loading them via <img src="/my-image.svg"/>. This means my SVGs will be loaded with the markup, rather than being loaded asynchronously, which can help eliminate Cumulative Layout Shift (CLS) issues. It's as simple as customizing your next.config.js file to use the @svgr/webpack loader for *.svg file types, then importing them in your components, a la:

import SvgAvatar from "/avatar.svg";

const MyComponent: React.FC = () => (
  <MyComponent>
    <SvgAvatar/>
  </MyComponent>
);

DEV Blog

I previously wrote another article about how we can use the DEV API to load your articles in order to embed them in your own website.

How to Embed Your Dev.to Blog in Your Personal Website

Mat Jones ・ Jan 18 ・ 5 min read

I combined this technique with Next's getStaticProps method to load all my DEV articles at build time, and statically generate the blog pages for my site. To keep the pages up-to-date, I configured the GitHub Action that builds and deploys my site to run periodically. I configured it to run every 30 minutes, however in practice it seems like for some reason it actually runs closer to once every hour. If your CI/CD pipeline is actually time-sensitive, I recommend not using GitHub Actions.

Design

UI Framework

On a recommendation from a friend, I decided to learn Blueprint.js and use it to build my website. After building lots of components using it, tweaking it, and styling it, I think it has become my new favorite React UI framework.

Right out of the box, everything looks amazing, cohesive, and enterprise-quality. Their component library is pretty extensive, and includes just about everything you'd want in your base toolkit, from simple stuff like navbars and buttons, to layout things like cards, to more complex components like interactive menus, progress bars and drawers. It's pretty easy to customize and compose the built-in components to make more complex designs. My only complaint is that there are a select few places I had to use the dreaded !important CSS directive to override some specific styles from Blueprint's stylesheets.

Stylesheets

My go-to language for stylesheets has been SCSS for a long time, because, well, let's face it, writing vanilla CSS just sucks.

One of the greatest features of CSS preprocessors like Sass is the ability to write nested styles; if your markup can be nested, then why can't your stylesheets?!

You can also easily re-use bits of common styles via mixins, and even run compile-time logical operations to further customize the generated stylesheets. It's incredibly powerful and extensible.

Deployment

My site is hosted statically on GitHub Pages, configured to deploy from the gh-pages branch of the repository. Deployment happens automatically on push, periodically, and can be manually triggered via a GitHub Actions workflow. The date of the most recent build is injected into the app at compile time via an environment variable in next.config.js.

How to Embed Your Dev.to Blog in Your Personal Website

Mat Jones — Mon, 18 Jan 2021 21:57:36 +0000

I recently switched my blog over from Medium to DEV, one of the primary reasons being I wanted an easy way to show my full blog on my personal website, as well as on the platform I'm using.

In Medium's desperate quest for monetization, they've made it extremely difficult to embed your blog on your own website. There are several projects out there on GitHub that attempt to help do this, but none of them seem to work very well, and they're all somewhat lacking in the styles department.

DEV has an open API, making it much easier to embed your blog in your own website, with full custom markup and styles, so you can easily make it look exactly how you want. In this article, we'll take a look at just how easy this is to get set up.

We're going to be using TypeScript, but if you're not familiar with it or just aren't using it, you should still be able to follow along, just excluding the TypeScript interfaces.

API

I'm going to be using Superagent as my HTTP client, because I like the functional and declarative API, but you can easily swap this out for something else like Axios, or the browser Fetch API. The full API documentation for DEV can be found here.

Let's start by setting up some interfaces to type our responses with (if you're using vanilla JavaScript you can skip this step).

dev-to-article-meta.ts

export default interface DevToArticleMeta {
  type_of: string;
  id: number;
  title: string;
  description: string;
  readable_publish_date: string;
  slug: string;
  path: string;
  url: string;
  comments_count: string;
  public_reactions_count: string;
  collection_id?: number;
  published_timestamp: string;
  positive_reactions_count: string;
  cover_image: string;
  social_image: string;
  tag_list: Array<string>;
}

dev-to-article.ts

import DevToArticleMeta from "interfaces/dev-to-article-meta";

export default interface DevToArticle extends DevToArticleMeta {
  body_html: string;
  body_markdown: string;
}

Now let's set up a service to handle fetching our data. We'll need two methods:

async fetchArticles(): Promise<Array<DevToArticleMeta>>
async getArticle(slug: string): Promise<DevToArticle>

dev-to-service.ts

import DevToArticle from "interfaces/dev-to-article";
import DevToArticleMeta from "interfaces/dev-to-article-meta";
import superagent from "superagent";

// setup API endpoints and queries
const DEV_TO_USERNAME = "matjones"; // swap this for your username
const ARTICLES_API = "https://dev.to/api/articles";

// helper method to type responses
const parseResponse = <T>(response: any): T =>
  (typeof response === "string" ? JSON.parse(response) : response) as T;

const fetchArticles = async () => {
  // GET the endpoint
  const response = await superagent.get(ARTICLES_API)
          // and add the username query parameter
          .query({ username: DEV_TO_USERNAME });
  return parseResponse(response.body);
};

const getArticle = async (slug: string) => {
  // build the API endpoint URL, `slug` is the `slug`
  //property of one of your articles,
  // e.g. "protecting-your-privacy-online-3bmc"
  const endpoint = `${ARTICLES_API}/${DEV_TO_USERNAME}/${slug}`;
  const response = await superagent.get(endpoint);
  return parseResponse<DevToArticle>(response.body);
};

export const DevToService = {
  fetchArticles,
  getArticle,
};

And that's pretty much it for our API layer. Nice and simple. 😎

The fetchArticles() function will fetch all of your published articles. If you want to implement pagination or a maximum number of results, you can add the page and per_page query parameters via the .query() method chained off of superagent.get().

Integrating Into a React App

While this is technically all you need to get the data needed to render your articles, we can make it a little easier to work with in a React app by adding some custom React hooks.

STOP: If you're not familiar with React hooks, go read up on hooks first, then come back. I'll wait.

We'll make a really simple hook for each of our service methods. Since useEffect callbacks can't be async themselves, we're going to be using an IIFE, but note that you could also use the trusty old Promise.then() syntax as well.

use-dev-to-articles.ts

import DevToArticleMeta from "interfaces/dev-to-article-meta";
import { useEffect, useState } from "react";
import { DevToService } from "utils/dev-to-service";

/**
* Fetch all of my published dev.to articles
* @param onError a callback which is invoked if the request fails
*/
export default function useDevToArticles(onError?: () => void) {
  const [loading, setLoading] = useState(true);
  const [articles, setArticles] = useState<Array<DevToArticleMeta>>([]);

  useEffect(() => {
    (async () => {
      try {
        setArticles(await DevToService.fetchArticles());
      } catch (e) {
        onError?.();
      }

      setLoading(false);
    })();
  }, [onError]);

  // return the array of articles, and the loading indicator
  return { articles, loading };
}

use-dev-to-article.ts

import DevToArticle from "interfaces/dev-to-article";
import { useState, useEffect } from "react";
import { DevToService } from "utils/dev-to-service";

/**
* Get a specific article given the article's slug.
* @param slug the slug of the article to retrieve.
* @param onError a callback which is invoked if the request fails
*/
export default function useDevToArticle(slug: string, onError?: () => void) {
  const [loading, setLoading] = useState(true);
  const [article, setArticle] = useState<DevToArticle>();

  useEffect(() => {
    // this bit may not be necessary for you; I needed
    // it because I'm using Next.js server side rendering
    // so slug is `undefined` on the initial render
    // since I'm getting it from a route parameter
    // e.g. /blog/:slug
    if (slug == null || slug.length === 0) {
      return;
    }

    (async () => {
      try {
        setArticle(await DevToService.getArticle(slug));
      } catch (e) {
        onError?.();
      }

      setLoading(false);
    })();
  }, [onError, slug]);
  // return the article, and the loading indicator
  return { article, loading };
}

Rendering an Article's Body

There are two ways you can render an article's body; using the body_html property or the body_markdown property. If your article does not contain any HTML elements (you're only using Markdown) and does not contain any DEV Liquid tags, you can use the body_markdown property and use something like react-markdown to render it.

However, if you're using Liquid tags, or you have HTML elements in your markdown (for example, I often use the <figcaption> element to add captions to images and code snippets), you'll need to render the body_html property. In React, this can be done like this:

<div dangerouslySetInnerHTML={{ __html: article.body_html }}/>

Notice the dangerouslySetInnerHTML syntax; it is intentionally ugly to look at and difficult to type quickly, because doing this could potentially open your app to cross-site-scripting vulnerabilities. In general, you should only use dangerouslySetInnerHTML with trusted inputs (e.g. inputs that you control, not unknown end users).

Results

Using this approach, I was able to easily embed my DEV blog into my personal website. Clicking the comment or thumbs up icon opens the article on dev.to so that you can add reactions or comments.

The /blog page on my personal website.

A specific blog article on my personal website.

The Ultimate Work From Home Programming Setup

Mat Jones — Sat, 16 Jan 2021 14:59:25 +0000

When COVID-19 first broke out, it forced a massive shift in the workplace; the only way to survive as a business was to have employees work from home. And it seems like work from home policies are here to stay, at least in the tech industry. The company I work for only had an “as needed” work from home policy before the pandemic, but now we will have an option to work from home full time, even after the office reopens, an option which I think many will take advantage of.

So, since we’re going to be working from home for the foreseeable future, I decided to upgrade my home office setup. I’ll take you through each piece of hardware, and why I like it for programming.

Monitors

Monitors are arguably the most important pieces of hardware in a programmer’s home office. As a software engineer, I usually need to have a lot of applications open at once, so maximizing screen real estate was my main objective when selecting monitors.

My main monitor is a 49" Samsung CHG90 QLED 1080P Curved Super Ultrawide 32:9 Gaming Monitor on it’s stock stand. It might seem excessive, but it’s a truly incredible experience for programming. Typically, I use the right two-thirds of the monitor for my IDE, and I can fit 2–3 open code editors side by side, and the left one-third is for my web browser, running the web app I’m working on. The one caveat is that native window management doesn’t handle this very well, so I use a third-party window management software called Divvy, which allows you to use keyboard shortcuts to snap windows to user defined areas of the screen.

My secondary monitor is a 24" ViewSonic 1440P VX2478 series, mounted vertically (portrait mode) on a VIVO Single Monitor Desk Mount. I have this monitor mounted on the left side of my desk, and use it almost exclusively for Slack. It’s great to be able to see a lot of the previous messages in portrait mode, and it keeps slack out of my way on my main monitors.

My final monitor is a 35" Sceptre C355W-3440UN 1440P Curved Ultrawide mounted above my main monitor on a VIVO Single Monitor Extra Tall Desk Mount. It’s a great, affordable ultrawide option that competes with much more expensive models from other manufacturers like LG or Dell. I use the left half of this monitor for ClickUp, the project management and agile board software we use at work, and the right half for [Spotify](https://www.spotify.com/us/\) and any other miscellaneous software I may need to use throughout the course of the day.

My monitor orientation setup.

With curved ultrawide monitors, you typically want to sit farther away from them than a typical flat 16:9 monitor in order to get a wider field of view, so I have all of these monitors sitting on an IKEA LINNMON / ADILS corner desk, which gives me a lot of depth space to push the monitors back into the corner.

Peripherals

Let’s start with the obvious; a great mechanical keyboard is a necessity for a programmer. Right now, I’m using a DURGOD Taurus K320 Mechanical Keyboard with Cherry MX Clear switches, and so far it’s the best, most satisfying mechanical keyboard I’ve ever used. Don’t be turned off by the relatively low price point; it has an excellent build quality, with virtually no key wobble, stabilizers for the long keys (enter, shift, backspace, etc.), a super solid frame with zero flex, and extremely high quality plastics for all the parts. I really like Cherry MX Clear switches with it, because it provides an extremely stiff keypress (though this is just based on preference) while still not being obnoxiously loud, like Cherry MX Blues.

For my mouse, I use a Logitech G502. It has removable weights so you can configure it to your preferred weight (I prefer a heavier mouse), but the main attraction here is really the scroll wheel. The G502 has a button that can toggle the heavy, weighted scroll wheel between behaving like any other scroll wheel and being completely free spinning (check out the video below to see what I mean). It’s amazing for scrolling through huge code files that are hundreds of lines long. I’ve also considered picking up a Logitech MX Master 3, which has the same scroll wheel feature, but also has a thumb wheel for horizontal scrolling, which seems interesting.

I also recommend getting an oversized mouse+keyboard mat, like the Logitech G840 XL. It may seem excessive, but it’s really nice to have a soft surface to put both your mouse and keyboard on, especially since it helps to prevent your keyboard from sliding like it might on a harder surface.

If you’re a programmer, software engineer, or tech industry professional of any kind, chances are you might be working from home for a long time, or indefinitely. If that’s the case, it might be time to upgrade your home office.

Code Comments Are Stupid

Mat Jones — Sat, 16 Jan 2021 14:49:49 +0000

Code comments are lame.

Code comments once served a critical purpose; adding context or explanation to a snippet of code. But this was back when programming languages were mostly esoteric, and the barrier to entry for writing code was extremely high.

Today, though, we have extremely expressive programming languages, like C# (especially since the addition of LINQ), Python, Kotlin, etc. that make code comments virtually obsolete. With a highly expressive language, nearly all comments become unnecessary, because it’s easy to write self-documenting code; the code is easy enough to understand at a glance that you don’t need comments anymore.

“You should always write your code as if comments didn’t exist.” — Jeff Atwood (source)

I’ll be writing most of my code examples in Kotlin because it’s one of my favorite languages, and I haven’t gotten to use it recently, but all these examples are applicable to any sufficiently expressive language.

Implementation Comments

Most of the time, comments in the implementation of a bit of code can be made unnecessary just by naming things better, and keeping your functions and methods small and focused on a single task. Let’s consider a simple example.

We’re working on a web application, and we want to implement a method to validate that a password meets certain required criteria. For this example, let’s say our criteria are:

Must contain at least one uppercase letter and at least one lowercase letter
Must contain one number
Must not include whitespace
Must contain one of the following special characters: ! # $ % ^ & *
Must be between 8 and 20 digits

We could do it in very few lines of code with a complex regular expression:

/**
 * Validate that the given password matches our password criteria.
 * @param password the password to check against our criteria
 * @return         true if the password matches all criteria, false otherwise
 */
fun isPasswordValid(password: String): Boolean {
    // Pattern matches that:
    // - Must contain at least one uppercase letter and at least one lowercase letter
    // - Must contain one number
    // - Must not include whitespace
    // - Must contain one of the following special characters: ! # $ % ^ & * 
    // - Must be between 8 and 20 digits
    var pattern = """^(?=.*[A-Z])(?=.*[a-z])(?=.*[0-9])(?=.*[!#$%^&*])[\S]{8,20}$""".toRegex();
    return password.matches(pattern);
}

fun main() {
    println(isPasswordValid("Passw0rd!")); // true
    println(isPasswordValid("P assw0rd!")) // false; contains whitespace
    println(isPasswordValid("Password!")); // false; no numeric digits
    println(isPasswordValid("Passw0rd")); // false; no special characters
    println(isPasswordValid("password!")); // false; no uppercase letter
    println(isPasswordValid("PASSW0RD!")); // false, no lowercase letter
    println(isPasswordValid("Pswrd!")); // false; too short
    println(isPasswordValid("UltraL0ngSecurePassw0rd!")); // false; too long
}

A great way to make enemies is to make sure nobody else can understand your code.

Without those comments inside of the isPasswordValid function, it’s really cryptic and almost impossible to tell what it’s doing without taking a ton of time to figure out that really complicated regular expression.

By splitting each password criterion into a distinct unit of work, the code and the regular expressions used in the code both become a lot clearer and easier for a reader to grok with just a glance.

fun containsUppercaseLetter(str: String) = str.matches("""(?=.*[A-Z]).*""".toRegex());
fun containsLowercaseLetter(str: String) = str.matches("""(?=.*[a-z]).*""".toRegex());
fun containsNumericDigit(str: String) = str.matches("""(?=.*[0-9]).*""".toRegex());
fun doesNotContainWhitespace(str: String) = !str.matches("""(?=.*[\s]).*""".toRegex());
fun containsSpecialCharacter(str: String) = str.matches("""(?=.*[!#$%^&*\.]).*""".toRegex());
fun lengthIsInRangeInclusive(str: String) = str.length in 8..20;

/**
 * Validate that the given password matches our password criteria.
 * @param password the password to check against our criteria
 * @return         true if the password matches all criteria, false otherwise
 */
fun isPasswordValid(password: String): Boolean {
    return lengthIsInRangeInclusive(password) &&
            containsUppercaseLetter(password) &&
            containsLowercaseLetter(password) &&
            containsNumericDigit(password) &&
            doesNotContainWhitespace(password) &&
            containsSpecialCharacter(password);
}

fun main() {
    println(isPasswordValid("Passw0rd!")); // true
    println(isPasswordValid("P assw0rd!")) // false; contains whitespace
    println(isPasswordValid("Password!")); // false; no numeric digits
    println(isPasswordValid("Passw0rd")); // false; no special characters
    println(isPasswordValid("password!")); // false; no uppercase letter
    println(isPasswordValid("PASSW0RD!")); // false, no lowercase letter
    println(isPasswordValid("Pswrd!")); // false; too short
    println(isPasswordValid("UltraL0ngSecurePassw0rd!")); // false; too long
}

A few more lines of code can go a long way for clarity.

The code is only a few more lines long, but is so much clearer at a glance that we don’t even need that huge comment block anymore. We’ve converted a comment telling us what the code is doing into code that tells us itself what it’s doing.

Redundant Comments

This is the most useless (and annoying) type of code comment. These are the type of comments where you can just remove the comment without even changing the code, and it’s still just as clear what the code is doing.

Let’s say we’re working on a command line tool grab some data from an API and write it to a file. We’ll need to ask the user for a file path to write the data to.

fun callApi(): ApiResult {
    // implementation omitted
}

fun writeJsonToFile(json: String, filePath: String) {
    // implementation omitted
}

// Call the API and get the results
var apiResult = callApi();

// Get the desired file path to save the results to
print("Enter file path to save results:");
var filePath = readLine()!!;

// make sure it's a JSON file
if (!filePath.endsWith(".json")) {
    filePath = "${filePath}.json";
}

// write the data to the json file
writeJsonToFile(apiResult.toJson(), filePath);

Seems good… or not?

There’s a lot cluttering up this code, though. It may seem a bit complicated, but that’s just because of the comments. In situations like this, removing the comments can actually make the code less intimidating to a reader.

When you see a function defined as writeJsonToFile(json: String, filePath: String), with no comments or explanation, what do you think it does? You can probably immediately understand that “it writes some JSON data to a file”. We can remove every single comment, and the code is still just as clear, readable, and understandable.

fun callApi(): ApiResult {
    // implementation omitted
}

fun writeJsonToFile(json: String, filePath: String) {
    // implementation omitted
}

var apiResult = callApi();

print("Enter file path to save results: ");
var filePath = readLine()!!;

if (!filePath.endsWith(".json")) {
    filePath = "${filePath}.json";
}

writeJsonToFile(apiResult.toJson(), filePath);

Its still just as easy to understand this code without the comments.

We didn’t change any of the code here, we just removed all the comments. Yet, it still seems less intimidating on your first read, right? Appropriately named variables and functions make all the comments completely redundant, so the comments were just clutter.

Exceptions to the Rule

Documentation Comments

The main exception to this is tooling-specific documentation comments, such as JSDoc and C# XML Documentation Comments. These types of comments provide popup API documentation via IntelliSense (or another autocomplete mechanism) in most IDEs and editors, like VS Code, Rider, and others.

Documentation comments also usually hook into documentation generator tools, like DocFX or Sandcastle, which can automatically generate HTML documentation web pages from your documentation comments.

Clarification Comments

Sometimes, there is an obvious solution that seems simpler or better, but there is a specific reason the developer chose not to use that solution. I’ve found this to be extremely common in JavaScript. Let’s take a look at an example. We’ll use TypeScript to help make things a bit clearer.

const isFinite = (value: number | Number): boolean => {
  // don't use the global window.isFinite(value)
  // because it returns true for null/undefined
  return Number.isFinite(value);
};

...

export const NumberUtils = {
  isFinite,
  ...
};

Finally, an appropriate place for a comment.

In this example, the comment is warranted, because it prevents future developers from introducing bugs without realizing it by changing return Number.isFinite(value); to just return isFinite(value);

Another common example is when you need to prevent bugs caused by Internet Explorer having different implementations of built-in functions than every other browser, because reasons. Let’s look at another example in TypeScript.

const addEntry<T>(set: Set<T>, value: T): Set<T> {
  // Don't directly return the value of `set.add`
  // because it's not chainable in IE 11
  set.add(value);
  return set;
};

...

export const SetUtils = {
  addEntry,
  ...
};

Internet Explorer sucks.

Once again, in this example, the comment prevents a future developer from unknowingly introducing Internet Explorer-specific bugs by changing this implementation to just return set.add(value);

Modern programming languages have become so expressive that code comments are virtually obsolete, except in specific circumstances. The time has come to replace code comments with highly expressive, self-documenting code.

Strongly Typed Error Handling in TypeScript

Mat Jones — Sat, 16 Jan 2021 14:42:46 +0000

TypeScript is a great language. TypeScript takes JavaScript and makes it actually good. If there’s one glaring weakness, it’s the inability to use strongly typed catch blocks. However, this is mostly due to a design flaw in the JavaScript language; in JavaScript, you can throw anything, not just Error types.

Justification

Consider the following, completely valid TypeScript code:

try {
  const x: number = 7;
  throw x;
} catch (e) {
  console.log(typeof e); // "number"
  console.log(e); // 7
}

You can quite literally throw anything.

It’s easy to see why this can be a problem. One use-case where this is less than ideal is consuming a web API from TypeScript. It’s quite common for non-success HTTP status codes (500 Internal Server Error, 404 Not Found, etc.) to be thrown as an error by API consumer code.

Usage

All models are Immutable.js Record classes.

/*
Let's say the API returns a 500 error, with an error
result JSON object:
-- Status: 500
-- Data:
{
  "errors": [
    {
      "key": "UserAlreadyExists",
      "message": "User with specified email already exists.",
      "type": "Validation"
    }
  ]
  "resultObject": undefined
}
*/

const createUserApi = ServiceFactory.create(UserRecord, "/api/users/");

const createUser = async (user: UserRecord): Promise<UserRecord | undefined> => {
  try {
    const result = await createUserApi(user);
    return result.resultObject;
  } catch (e) {
    // if the thing that was thrown is a ResultRecord, show a toast message for each error
    if (e instanceof ResultRecord) {
      e.errors.forEach((error: ResultErrorRecord) => toast.error(error.message));
      return undefined.
    }

    // otherwise, it could be anything, so just show a generic error message
    toast.error("There was an issue creating the user.");
    return undefined;
  } finally {
    // maybe we need to turn off a loading indicator here
  }
};

You can see in this example that handling errors natively in TypeScript is… quite sloppy. The “maybe monad” common pattern to more generically handle errors and control flow. Basically, what we want to do is create an abstraction that can strongly type thrown errors to a specified type that you know is likely to be thrown. In our case, we want to be able to handle errors from a strongly typed ResultRecord with ResultErrorRecords inside it.

What if we could take the example above, and represent the same logic but with less code and strong typing in the catch block? In the following example, one of result or error will be non-null, but not both.

/*
Let's say the API returns the same response as before.
result JSON object:
-- Status: 500
-- Data:
{
  "errors": [
    {
      "key": "UserAlreadyExists",
      "message": "User with specified email already exists.",
      "type": "Validation"
    }
  ]
  "resultObject": undefined
}
*/

const createUserApi = ServiceFactory.create(UserRecord, "/api/users/");

const createUser = async (user: UserRecord): Promise<UserRecord | undefined> =>
  await Do.try(async () => {
    const result = await createUserApi(user);
    return result.resultObject;
  }).catch((result?: ResultRecord<UserRecord>, error?: any) => {
    // if result is not null, it will have errors; show toast errors
    result?.errors?.forEach((e: ResultErrorRecord) => toast.error(e.message));
    // otherwise, unknown error; show a generic toast message
    error != null && toast.error("There was an issue creating the user.");
  }).finally(() => {
    // maybe we need to turn off a loading indicator here
  }).getAwaiter();

Automatic typecasting? Nice 😎

This pattern gives us a more functional approach to error handling, gives us strongly typed errors, and works really, really nicely when used in combination with React hooks.

const LoginForm: React.FunctionComponent = () => {
  const { create: createUserApi } = UserService.useCreate();
  const history = useHistory();
  const [user, setUser] = useState(new UserRecord());
  const [errors, setErrors] = useState<Array<ResultErrorRecord>>([]);
  const [loading, setLoading] = useState(false);

  const createUser = useCallback(
    async () => await Do.try(async () => {
      setErrors([]);
      setLoading(true);
      const result = await createUserApi(user);
      history.push(RouteUtils.getUrl(siteMap.myProfile, { userId: result.resultObject!.id! }));
    }).catch((result?: ResultRecord<UserRecord>, error?: any) => {
      setErrors(result?.errors ?? []);
      error != null && toast.error("There was an issue signing up.");
    }).finally(() => setLoading(false))
      .getAwaiter(),
    [createUserApi, user]
  );

  return (
    <div className="c-login-form">
      <InputFormField
        disabled={loading}
        label="Email"
        required={true}
        type={InputTypes.Email}
        value={user.email}
        onChange={e => setUser(user.with({ email: e.target.value })}
        errors={errors.filter(e => e.key === "SignUp.Email")}
      />
      <InputFormField
        disabled={loading}
        label="Password"
        required={true}
        type={InputTypes.Password}
        value={user.password}
        onChange={e => setUser(user.with({ password: e.target.value })}
        errors={errors.filter(e => e.key === "SignUp.Password")}
      />
      <Button onClick={createUser}>Sign Up</Button>
      {loading && (<LoadingSpinner/>)}
    </div>
  );
};

Strong typing is better than weak typing.

Clean, concise, and strongly typed error handling in just 46 lines of code, including the UI.

Implementation

So how does this fancy-schmancy Do.try work under the hood? By adding an abstraction on top of regular old Promises. Let’s break it down.

First, let’s define some utility types we’re going to need:

/**
 * Represents an asynchronous method reference.
 */
type AsyncWorkload<T> = () => Promise<T>;

/**
 * Handler for a typed error ResultRecord, or any type if a Javascript error occurred.
 */
type CatchResultHandler<T> = (result?: ResultRecord<T>, error?: any) => void;

/**
 * Handler for Do.try().finally(); Runs whether an error occurred or not.
 */
type FinallyHandler = () => void;

Strongly typed handlers for strongly typed errors.

Next, let’s take a look at our constructor:

class Do<TResourceType, TReturnVal = void> {
  private promise: Promise<TReturnVal>;

  private constructor(workload: AsyncWorkload<TReturnVal>) {
    this.promise = workload();
  }
}

Private constructor? 🤔

That private constructor is no mistake. You’ll notice in the previous snippets, usage of this pattern starts with Do.try; that’s because try is a static factory method that returns an instance of Do. The private constructor can only be called internally to the class, by the try method. The implementation of try is very straightforward:

class Do<TResourceType, TReturnVal = void> {
  ... 
  public static try<TResourceType, TReturnVal = void>(
    workload: AsyncWorkload<TReturnVal>
  ): Do<TResourceType, TReturnVal> {
    return new Do<TResourceType, TReturnVal>(workload);
  }
}

You’ll never know unless you… try.

The finally method is just as straightforward, with one important caveat:

class Do<TResourceType, TReturnVal = void> {
  ...
  public finally(
    finallyHandler: FinallyHandler
  ): Do<TResourceType, TReturnVal> {
    this.promise = this.promise.finally(finallyHandler);
    return this;
  }
}

Pay special attention to the return value.

Notice the return value, return this; This allows for method chaining, i.e. Do.try(workload).catch(catchHandler).finally(finallyHandler); In this code, catch and finally are both called on the same instance of Do which is returned from Do.try.

There’s also a getAwaiter method, which allows us to await for the result. All we need to do is return the internal promise.

class Do<TResourceType, TReturnVal = void> {
  ...
  public async getAwaiter(): Promise<TReturnVal> {
    return this.promise;
  }
}

This just allows us to await the result of the entire method chain.

Now let’s get to the interesting part; the catch method. Inside the catch method, we’re going to type guard the thrown object; if the thrown object is a ResultRecord instance, we cast it as such and pass it as the catch handler’s first argument; otherwise, it’s some unknown error, so we pass it as the catch handler’s second argument. We also need to cast the promise back to a Promise<TReturnVal> because of the return type of Promise.catch, but the promise is still a valid Promise<TReturnVal>.

class Do<TResourceType, TReturnVal = void> {
  public catch(
    errorHandler: CatchResultHandler<TResourceType>
  ): Do<TResourceType, TReturnVal> {
    this.promise = this.promise.catch((err: any) => {
      // check if thrown object is a ResultRecord
      if (err instanceof ResultRecord) {
        // pass the ResultRecord as the first parameter
        errorHandler(err, undefined);
        return;
      }

      // otherwise, pass the generic error as the second parameter
      errorHandler(undefined, err);
    }) as Promise<TReturnVal>;

    // notice again, we are returning this to allow method chaining
    return this;
  }
}

We just need to cast the promise back to a `Promise` because of the return type of `Promise.catch`, but rest assured, the promise is still a valid `Promise`.

And there you have a basic implementation of a “maybe monad”. While the implementation here is an opinionated one, offering strongly typed error handling for ResultRecord errors, you could easily implement the same thing for virtually any type you want to use to wrap up your errors, just as long as you’re able to implement a type guard for it.

Taking It Further

I think strongly typed error handling speaks enough for itself, but we can take it even further. This pattern enables an extremely powerful utility, and I think it’s the strongest argument for using it: default behavior. We can extend our Do class to have a global configuration, allowing us to define default behavior which is applied to every instance of Do across the entire application.

All we need to do is add a static configuration mechanism, and implement a check for our configuration inside the constructor:

interface DoTryConfig {
  /**
   * A default handler that will always run on error, if configured,
   * even if a `catch()` does not exist in the call chain.
   * This is useful for adding default error handling in the
   * development environment, such as `console.error(err)`.
   */
  defaultErrorHandler?: CatchResultHandler<any>;
}

class Do<TResourceType, TReturnVal = void> {
  ...
  private static config: DoTryConfig = {
    defaultErrorHandler: undefined,
  };

  private constructor(workload: AsyncWorkload<TReturnVal>) {
    this.promise = workload().catch((err: any) => {
      // check for defaultErrorHandler from config
      if (err instanceof ResultRecord) {
        Do.config.defaultErrorHandler?.(err, undefined);
        throw err; // rethrow so it doesn't interrupt call chain
      }

      Do.config.defaultErrorHandler?.(undefined, err);
      throw err; // rethrow so it doesn't interrupt call chain
    });
  }

  /**
   * Sets the global configuration object for class {Do}
   * @param config the {DoTryConfig} object to set
   */
  public static configure(config: DoTryConfig): void {
    Do.config = config;
  }
}

Default behavior is epic.

So what does it look like to apply default behavior? Let’s contrive an example.

We’re working on a large scale React application, and in order to aid debugging errors during development, we want to always log errors to the console in the development environment. Well, with the configuration mechanism we just added, it becomes trivially easy to add this default behavior. Just open up your index.ts app entrypoint and add the handler:

// index.ts

EnvironmentUtils.runIfDevelopment(() => {
  Do.configure({
    defaultErrorHandler: (result?: ResultRecord<any>, error?: any) => {
      result != null && console.error(result);
      error != null && console.error(error);
    }
  });
});

Log errors to the console by default.

You could use the same configuration mechanism to add default behavior to the try or finally portions of the call chain as well.

The syntax is quite nice to read and easy to understand at a glace, but with the added bonus of having strongly typed errors, and optional default behavior.

What do you think? Are you going to try “maybe monads” or the Do.try pattern in your next TypeScript project?

Protecting Your Privacy Online

Mat Jones — Sat, 16 Jan 2021 14:29:23 +0000

Do you know who’s tracking you?

If you’re like most people, you use Google Chrome as your default browser, you use Google as your default search engine, and you don’t often think about online privacy much further than that. Why should huge tech companies make fortunes at the direct expense of their users’ privacy?

There are a lot of ways you can protect yourself and your privacy online, if you care enough to make the effort.

Browser

If you use Google Chrome as your default browser, the browser itself is spying on you. Google wants to collect your user data so badly, they won’t even let you use DuckDuckGo as your default search engine in most countries anymore.

Last week, it was uncovered that the browser setting in Chrome to clear site data and cookies on exit was exempting Google and YouTube from these requests. Reportedly, it was a bug, but as a software engineer who works daily with web technologies, I find it extremely hard to believe that it really was a bug, and not just a sneaky, unethical feature that Google got caught on.

If you value your privacy at all, I highly recommend you switch to Mozilla Firefox. Firefox is built with privacy in mind, and the browser code is open source. On top of that, Firefox has better performance than Google Chrome.

Search Engine

Please, stop using Google. Not only does Google track everything you search for and everything you click on in search results, but they also manipulate search results for their own gain and to suppress political opposition. Further, Google tracks what you do after you leave Google, too. Google trackers exist on 75% of the top million websites.

DuckDuckGo is a privacy-focused search engine that never tracks you, never collects or sells user data, and strives to provide unbiased results. DuckDuckGo also offers a browser extension and a mobile browser to help protect yourself everywhere on the web.

Facebook

My advice: just delete your Facebook account. Facebook is one of the worst offenders, if not the worst offender. Same goes for any Facebook-owned companies, such as Instagram and WhatsApp.

In 2016, Facebook leaked millions of users’ private data to British consulting firm Cambridge Analytica without the users’ consent. This data was personally identifiable, and primarily used for targeted political advertising.

In 2019, Facebook ran a program to trick people into giving up all their data willingly by paying users to install a custom Facebook VPN app called Onavo, which allowed Facebook to just collect all meaningful data passing through the device, whether Facebook-related or not. This was a violation of Apple’s developer rules, so Onavo was quickly removed from the app store.

If you still aren’t ready to give up on Facebook, at least install the Facebook Container extension for Firefox, built by Mozilla. Facebook Container makes Facebook run inside a special sandboxed container, making it hard for Facebook to track you to external websites or use 3rd party cookies.

If you’re looking for a secure and private messaging alternative to WhatsApp, I recommend Telegram or Signal.

Browser Extensions

If you’re looking to take privacy more seriously, there are several browser extensions I use and recommend for blocking trackers, ads, and other malware. All of these extensions are available for both Chrome and Firefox. They’re also all open source, meaning anyone can view or submit patches for the code. Open source is generally considered good for security. In software development, Linus’s Law states:

Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.

Open source projects of sufficient size and traction have potentially thousands of contributors and testers actively working to find and resolve bugs and vulnerabilities.

DuckDuckGo Privacy Essentials

Download

Tired of being tracked online? We can help. At DuckDuckGo, we believe online privacy should be simple.

Blocks hidden trackers
Enforce HTTPS encrypted connections where possible
Gives each website a “Privacy Grade” (A-F) so you can see how protected you are at a glance
Option to send standardized Global Privacy Control signal to websites you visit
Open source

uBlock Origin

Download

uBlock Origin is not an “ad blocker”, it’s a wide-spectrum content blocker with CPU and memory efficiency as a primary feature.

Filters and blocks content based on block lists
Add additional popular community block lists
Block specific elements on a page manually or automatically
Add custom global or local filtering and blocking rules for content and/or Javascript
Open source

Privacy Badger

Download

Privacy Badger automatically learns to block invisible trackers. Instead of keeping lists of what to block, Privacy Badger automatically discovers trackers based on their behavior.

Sends standardized Global Privacy Control and Do Not Track signals to websites you visit
Blocks scripts that do not respect the Global Privacy Control or Do Not Track signals
Open source

HTTPS Everywhere

Download

Many sites on the web offer some limited support for encryption over HTTPS, but make it difficult to use. For instance, they may default to unencrypted HTTP, or fill encrypted pages with links that go back to the unencrypted site. The HTTPS Everywhere extension fixes these problems by rewriting all requests to these sites to HTTPS.

Attempts to upgrade insecure HTTP connections to secure HTTPS connections
If HTTP→HTTPS upgrade is not possible, blocks the insecure HTTP connection by default
Open source

ClearURLs

Download

This extension will automatically remove tracking elements from URLs to help protect your privacy when browsing through the Internet.

Automatically removes URL tracking parameters like Google Analytics parameters and Amazon tracking parameters
Uses over 250 URL matching rules, matches a huge set of trackers
Open source

DNS

Real stats from my Technitium DNS server.

If you want to take your privacy one step further, you can prevent your Internet Service Provider (ISP) from detecting every single site you request by running your own custom, encrypted DNS server. I personally use and recommend Technitium DNS, a free and open source DNS server.

Technitium allows you to enforce that all DNS queries are encrypted, either via DNS-over-TLS or DNS-over-HTTPS, which prevents your ISP from being able to see all of your DNS requests unencrypted. Additionally, you can block ads and malware that may have slipped through your other defenses by using a DNS Sinkhole. Basically, if a domain is requested which is in any of your configured block lists, Technitium just resolves the IP of the domain to an invalid IP address, 0.0.0.0.

Top domains blocked over the last one week by my Technitium DNS server

VPN

Always use protection.

If you want to truly be fully anonymous online, you’ll need to use a 3rd party VPN service. However, not all VPNs are created equal; you need to make sure you choose a VPN provider with a strict no-logging policy. I use and can recommend NordVPN.

NordVPN has a strict no-logging policy, and this policy has been verified by independent auditor PricewaterhouseCoopers AG, one of the Big Four auditing firms. NordVPN operates within the jurisdiction of Panama, which has no legislation regarding mandatory data retention, unlike the US and EU; this is a good thing for a VPN provider. Such a strict no-logging policy could not exist legally under US or EU jurisdiction. You can even pay for your NordVPN subscription with cryptocurrency, without ever leaving a trace of your real identity.

The Onion Router (Tor)

If you want to go Edward Snowden level private, NordVPN also has a feature called Onion Over VPN. When connected to an Onion Over VPN server, all your internet traffic gets routed through a VPN server, then through a Tor proxy, and only then reaches the Internet. You can, of course, just use Tor directly via the Tor Browser without a VPN at all, but this still opens you up to some vulnerabilities.

If you use Tor without a VPN, your ISP will know you are using Tor to access the internet, though this alone doesn’t compromise the actual data being sent and received over Tor. However, your IP address can still be compromised if there is a snooper, or multiple snoopers, in the network.

Tor node servers are volunteer-operated; anyone, including authorities or other bad actors, can set up and operate a Tor node. While no single node is able to see the full network request or response, someone operating multiple servers which your request/response is routed through — especially entry or exist nodes — can discover your real IP address with a little work. For this reason, I still recommend using a VPN provider with Tor. That way, if your IP address is compromised, it’s just a VPN IP address and no real identifying information is compromised.

“I have nothing to hide. Why should I care?”

If you’re asking this, Edward Snowden is disappointed.

Privacy Is Your Right

Privacy is a right, but it’s not one we’ve always had; in fact, people in several authoritarian nations around the world, people still don’t have the right to privacy. And it’s one of the most important rights, right behind the right to free speech. Those who came before us fought so that we could enjoy the right to privacy, and dismissing that is grossly ignorant of history and its importance.

Your Data Has Value

Google is among the top 5 richest companies in the world when ranked by market cap. But the lion’s share of Google products are free; Gmail, Google Calendar, Google Drive, Google Photos, Google Meet, YouTube, etc. So how does Google make so much money? Simple. They sell your personal user data to third parties. Why should a global mega-corporation be allowed to profit freely and massively from your personal user data?

You Have No Idea How Much Is Out There

Most of the time, apathy about online privacy is actually a symptom of ignorance. Studies show that even those who are initially unbothered by online privacy issues become concerned when confronted with the amount and nature of data collection.

If a stranger asked you to to fill out a 150 question form regarding personal information, with the express intention to sell that data to 3rd parties, would you consent?

Honorable Mentions

There are several other privacy tools that I haven’t covered here, but are definitely worth an honorable mention.

Password Managers

Everyone should be using a password manager. Reusing a password for multiple accounts is a security risk. A decent password manager can automatically detect signup forms in your browser and offer to generate a secure, strong password for you, sync your passwords between devices, and warn you if you have any weak, compromised, or reused passwords. I personally use a 1Password family account at home and a team account at work, and I can confidently recommend 1Password, but Dashlane is another great option. If you’re looking for a free and completely offline solution, you can check out KeePassXC.

PrivacyTool.io

Privacy-respecting search engine to find more privacy tools
Privacy-focused services and software
Non-profit and open source

DeleteMe

Automatically opt-out of the sale of your personal information by over 40 of the biggest data brokers
Provides DIY guides to manually opt-out for free

haveibeenpwned.com

Historical records of data breaches
Check if any accounts associated with your email address have been compromised