DEV Community: Adam Coster

Custom JavaScript/Typescript Errors: Why and how to use them

Adam Coster — Wed, 20 Dec 2023 19:27:47 +0000

Custom error classes are great:

You can add your own fields, like error codes and such
You can simplify your error-handling logic while also making it more robust
You can add logging and other useful functionality right into the error class

This post includes a short discussion about how and why to create custom JavaScript errors, plus JavaScript (and Typescript) templates and Visual Studio Code snippets to make it easy to create your own.

How to write a custom JavaScript Error class

Creating a custom error class in JavaScript is pretty straight-forward: you just need to extend the base Error class:

class MyError extends Error {}

That's really all it takes! Then you can create and throw MyError instances just as you would regular Errors:

throw new MyError("Oh no!");

The one additional thing I always add to my custom error classes is a call to the static Error.captureStackTrace, which is available on V8-based runtimes.

Passing your error's constructor to Error.captureStackTrace makes it so that your constructor code won't show up in the stack trace. This isn't essential, but it makes things a little less noisy during debugging.

You can take this a step further: when you have an assertion function that throws your custom error, you can pass that function into Error.captureStackTrace instead of your error's constructor. This makes your assertion-caused traces easier to read by starting the trace right where the assertion was called.

(Since this API is only available in V8, you'll want to wrap it in an if or use optional chaining (the ?. operator) for compatibility.)

export class MyError extends Error {
    constructor(message, asserter=undefined) {
    // (Now that we're overriding the parent `Error` 
    //  constructor, we have to call `super`!)
        super(message);
        Error.captureStackTrace?.(this, asserter || this.constructor);
    }
}

Why write custom Error classes?

JavaScript runtimes provide a base Error class, plus some more-specific classes that extend that base class (like SyntaxError and TypeError).

You can differentiate between these kinds of errors using the instanceof operator:

try{
    // do something naughty...
}
catch(err){
    // There's no way to know ahead of time what caused `err`, so
    // you can use `instanceof` to help narrow down what happened.
    if(err instanceof SyntaxError) {
        // do whatever makes sense in that case
    }
    else if(err instanceof TypeError){/* ...*/ }
    // etc etc
}

The more information you can get from the caught error, the easier it is to handle it properly without creating even more problems.

To that end, custom error classes let you control exactly what information is available during error handling, and can provide type information to reduce the chances that you create new errors during error-handling. Plus, you can add all kinds of bells and whistles to make them even more useful.

Case study: `ENOENT` in Node's `fs` module

In Node, if you try to fs.readFileSync() a non-existent file, an error is thrown. If you catch that error and inspect it, you'll see that it's a regular old Error instance (i.e. not from a child class) but has had several new fields added to it: errno, syscall, code, and path.

When try/catching code that might throw such an error, you have to handle it like this:

import fs from 'fs';
try{
    fs.readFileSync('fake-file.nope');
}
catch(err){
    if(err instanceof Error && 'code' in err && err.code === 'ENOENT'){
        // Then this was a non-existent file error, handle accordingly!
    }
}

Even once you've narrowed down that the caught error has err.code === ENOENT, that doesn't tell us anything else about the error instance we've caught. We have to do the same kinds of laborious checks for any other field we want to check.

So not only is it clunky, it's also easy to create more errors from inside your catch block!

However, if Node instead used a custom FileError class (for example), we'd only need to do one check and then our editor would immediately know about all other non-standard error fields provided by the custom class:

import {readFileSync, FileError} from './fs-wrapper-with-custom-errors';
try{
    readFileSync('fake-file.nope');
}
catch(err){
    if(err instanceof FileError){
        // Then we already know that this instance will have the
        // `errno`, `syscall`, `code`, and `path` fields, so we can
        // get right into the handling logic!
    }
}

Case Study: Uncaught errors in Express.js

The super-popular Node.js server library, Express, has a powerful feature: you can add a route whose whole job is to handle errors thrown inside your other routes.

Used in combination with custom error classes, you can simplify a lot of otherwise-cumbersome error handling. For example, instead of having every route have its own logic for handling 404s, via its own try/catch blocks, you can use a custom error to put all of that logic in one place:

class NotFoundError extends Error {}
function assertFound(thing:any): asserts thing {
    if(!thing){ throw NotFoundError(); }
}

app.get('/some-route', (req,res)=>{
    const something = getThingIfItExists();
    assertFound(something); // throws if not found!
    res.send(something);
});

app.use((err, req, res, next)=>{
    if(err instanceof NotFoundError){
        return res.sendStatus(404);
    }
});

Time to make your own!

Now that we've covered the hows and whys of custom JavaScript Error classes, we just need it to be easy. To that end, the remaining sections provide templates and VSCode snippets for making custom Error classes in JavaScript and Typescript.

Custom Error Class Template (JavaScript)

Here's the JavaScript template, complete with full JSDocs so you can still get type support from your IDE. You can paste this into your code, find-replace "My" with whatever name you want, and you're good to go (though using the VSCode snippet would be less tedious!).

export class MyError extends Error {
    /**
     * @param {string} message The error message
     * @param {Function} [asserter] The assertion function that threw the error. Removes stack-trace noise if provided.
     */
    constructor(message, asserter) {
        super(message);
        Error.captureStackTrace?.(this, asserter || this.constructor);
    }
}

/**
 * @param {any} err
 * @returns {err is MyError}
 */
export function isMyError(err) {
    return err instanceof MyError;
}

/**
 * Assert a condition and throw a MyError if the condition is not met
 * @param {any} condition The condition to assert
 * @param {string} message The error message
 * @param {any} [cause] The cause of the error
 * @returns {asserts condition}
 */
export function assertMyClaim(condition, message, cause) {
    if (!condition) {
        const err = new MyError(message, assertMyClaim);
        if (cause) {
            err.cause = cause;
        }
        throw err;
    }
}

And a VSCode snippet to make it really easy:

{
  // ...your other snippets
  "Create custom JavaScript Error": {
        "scope": "javascript",
        "prefix": "error-custom",
        "body": [
            "export class ${1:Custom}Error extends Error {",
            "\t/**",
            "\t * @param {string} message The error message",
            "\t * @param {Function} [asserter] The assertion function that threw the error. Removes stack-trace noise if provided.",
            "\t */",
            "\tconstructor(message, asserter) {",
            "\t\tsuper(message);",
            "\t\tError.captureStackTrace?.(this, asserter || this.constructor);",
            "\t}",
            "}",
            "",
            "/**",
            " * @param {any} err",
            " * @returns {err is ${1:Custom}Error}",
            " */",
            "export function is${1:Custom}Error(err) {",
            "\treturn err instanceof ${1:Custom}Error;",
            "}",
            "",
            "/**",
            " * Assert a condition and throw a ${1:Custom}Error if the condition is not met",
            " * @param {any} condition - The condition to assert",
            " * @param {string} message - The error message",
            " * @param {any} [cause] - The cause of the error",
            " * @returns {asserts condition}",
            " */",
            "export function assert${1:Custom}Claim(condition, message, cause) {",
            "\tif (!condition) {",
            "\t\tconst err = new ${1:Custom}Error(message, assert${1:Custom}Claim);",
            "\t\tif(cause){",
            "\t\t\terr.cause = cause;",
            "\t\t}",
            "\t\tthrow err;",
            "\t}",
            "}"
        ],
        "description": "Create a custom JavaScript error."
    }
}

Custom Error Class Template (Typescript)

The Typescript version of the template is exactly the same -- it just moves type annotations from JSDocs into Typescript proper.

export class MyError extends Error {
    /**
     * @param asserter The assertion function that threw the error. Removes stack-trace noise if provided.
     */
    constructor(message: string, asserter?: Function) {
        super(message);
        Error.captureStackTrace?.(this, asserter || this.constructor);
    }
}

export function isMyError(err: any): err is MyError {
    return err instanceof MyError;
}

/**
 * @param cause The cause of the error, e.g. a different, caught error
 */
export function assertMyClaim(
    condition: any,
    message: string,
    cause?: any,
): asserts condition {
    if (!condition) {
        const err = new MyError(message, assertMyClaim);
        if (cause) {
            err.cause = cause;
        }
        throw err;
    }
}

And again, you could copy-paste this into your code and find-replace "My" with the name you want. But it'll be a lot easier to just add the following snippet to VSCode:

{
  // ... your other snippets
  "Create custom Typescript Error": {
    "scope": "typescript",
    "prefix": "error-custom",
    "body": [
      "export class ${1:Custom}Error extends Error {",
      "\t/**",
      "\t * @param asserter The assertion function that threw the error. Removes stack-trace noise if provided.",
      "\t */",
      "\tconstructor(message: string, asserter?: Function) {",
      "\t\tsuper(message);",
      "\t\tError.captureStackTrace?.(this, asserter || this.constructor);",
      "\t}",
      "}",
      "",
      "export function is${1:Custom}Error(err: any): err is ${1:Custom}Error {",
      "\treturn err instanceof ${1:Custom}Error;",
      "}",
      "",
      "/**",
      " * @param cause The cause of the error, e.g. a different, caught error",
      " */",
      "export function assert${1:Custom}Claim(",
      "\tcondition: any,",
      "\tmessage: string,",
      "\tcause?: any,",
      "): asserts condition {",
      "\tif (!condition) {",
      "\t\tconst err = new ${1:Custom}Error(message, assert${1:Custom}Claim);",
      "\t\tif (cause) {",
      "\t\t\terr.cause = cause;",
      "\t\t}",
      "\t\tthrow err;",
      "\t}",
      "}"
    ],
    "description": "Create a custom Typescript Error."
    }
}

HTTP Guide

Adam Coster — Mon, 03 Jul 2023 21:14:52 +0000

HTTP (Hyper Text Transfer Protocol) is the main way you get information from other people's computers (a.k.a. "the cloud", a.k.a. "the Internet"). It's how browsers fetch web pages, and how automated systems talk to web services.

Here we'll be talking about HTTP in a practical way, under the assumption that you don't care about the technical details and need to know just enough to be empowered to make use of HTTP in your automation toolkit.

There are lots of good sources out there for more technical details if you want to dig deeper, but this should be a solid start.

Client & Server

There's no way around some of the HTTP jargon since you'll see them out in the wild and need to know what they mean. Two of the biggies are "Client" and "Server" and their near-synonyms.

Client: An entity that wants to talk to some other entity (the Server), and thus initiates "requests" to it. The word "request" is perfect since Clients don't get to make demands. They ask politely, and the Server decides what happens. "Client-side" is an oft-used adjective to describe things happening on the Client. Examples of clients: browsers, games, desktop apps that use a web service, apps on your phone, etc.
Server: The entity that a Client makes requests to, and that might send responses back (if it feels like it). HTTP servers are software applications that know how to listen for, parse, and respond to HTTP requests from clients. "Server-side" is an oft-used adjective to describe things happening on the server.
Local: You'll sometimes see "local" or "locally" used as a synonym for "client". Seen in phrases like, "the server doesn't process everything for you, you'll have to finish up locally."
Remote: You'll sometimes see "remote" (meaning "distant", as in "remote control") used as a synonym for "server". This creates symmetry with "local" referring to the client, and makes sense since usually the servers we care about are over there somewhere (remote) while the clients we care about are right here in front of us (local).

The directionality here is important: HTTP requests are initiated by the client and the server sends responses back. Servers do not send requests to Clients (by definition). Not over HTTP, anyway.

⚠ These terms are overloaded so it can get confusing. People often use "the client" to refer to the computer or even the person initiating the HTTP request, even though the real client is the application running on that computer. Similarly, you'll often see "the server" used to refer to the computer instead of the server application running on that computer. To add even more confusion, you can run client and server applications on the same machine -- in that case the terms "local" and "remote" can still be used but now make less obvious sense.

⚠ Client-Server relationships exist for all kinds of other "protocols" (ways of sending and receiving data over the Internet, one of which being HTTP), but here we're just talking about HTTP.

Requests & Responses

HTTP is all about "Client Requests" and subsequent "Server Responses". Skipping the technical bits, the flow of an HTTP loop looks like this:

Client composes a request for some particular goal
Client sends that request to a Server
Server receives that request and parses its contents (if it wants to)
Server composes a response based on the contents of the request (if it feels so inclined)
Server sends the Response to the Client. (Server may instead choose to not respond at all!)
Client receives the Response and parses its contents
Client performs some task based on the Response contents

For example, let's take an imagined Weather App (the Client) and its Server:

App gets launched by the user and uses the device's GPS to find the user's location, clock to get the time, and file system to load the last retrieved forecast. It sees that it's been a while since App fetched updated weather info for this location, so it creates a request to get the updated forecast (the request includes the user's location, and any other info it wants to send to the server)
App sends the request to the Weather server
Server receives the request and parses its contents
Server checks the request for information about the user and sees that they have purchased the $9.99/month "In-Depth-Weather Plan". So it fetches to-the-minute forecast data for the provided location and creates a response that includes this data in the body. (Maybe it also stores all the device and user information sent to it, so that the company selling the app can sell that data later... but that's a different topic!)
Server sends the response to App
App receives the response and parses it.
App sees that the data is the In-Depth type, so it runs local code to render a per-minute graph on the user's screen.

Anatomy

What does it mean to "compose" a request or response? What do these things look like?

While there are nuances and technical details, you can think of both HTTP requests and responses as boring old text files with three main sections:

Start Line (for requests) or Status Line (for responses). This line provides enough information to indicate that the rest of the content consists of an HTTP request/response, plus some summary info.
Headers. These are Key: Value pairs the contain metadata about the request/client or response/server. Request headers often include things like what filetypes the client can understand, the preferred human languages of the client user, and credentials for a logged-in user. Both Request and Response headers will contain information about the Body section (if there is one), such as its type, size, and encoding.
Body (Optional). Some sort of data being sent to (requests) or received from (responses) the server. For example, the content of a PNG you're uploading or an HTML document you're downloading. Many of the Headers are used to provide information about this data.

The biggest difference between requests and responses is that first line, the rest is minor details. So let's see how those first lines compare!

Request "Start Line": Consists of an HTTP method (more on that shortly, a URL, and the protocol type and version: METHOD URL HTTP/VERSION. For example, if I'm uploading an image to myimagehost.example.com/images, my Start Line might look like this: POST /images HTTP/1.1.
Response "Status Line": Consists of the protocol type and version, the status code (more on that shortly), and a message about the status: HTTP/VERSION CODE MESSAGE. This line is used to summarize the outcome of the request in a generic way. For example, a successful image upload might result in a response with status line HTTP/1.1 201 Created. Failing the upload because I'm not allowed might result in the status line HTTP/1.1 403 Access Denied.

HTTP Methods

An "HTTP Method" is one of a handful of keywords (like POST or GET) that tell the server what kind of action you are trying to take. When describing a request, people often include the method. So they'll say things like "Send a GET Request to example.com/rad-webpage".

While there are standards and best-practices for which Method to use for any given purpose, in the real world it's sorta all over the place. There is no Internet Police to enforce how HTTP Methods are used, and there are all kinds of reasons why any given website might use them in a bespoke way. (And not always good reasons!)

I've summarized the most common Methods below, including what they're supposed to be for versus how they're actually used.

GET is for, well, getting things. It's the method your browser uses when you click a link or type a URL into the address bar. GET requests are not supposed to include a Body (but the Internet is a silly place). If you want to fetch a webpage, download an image, or "get" anything from the web, you'll probably use a GET request. In practice, you'll sometimes find GET requests used to change things on the server as well — in those cases the content of the URL should give you a hint about what will happen when you send the request.
POST is for posting (uploading) things. When you fill out a form on a website and hit that submit button your browser sends a POST request to the server with all the form data. POSTs are supposed to be for adding new things to a server. In other words, if you sent a POST to upload an image twice in a row, there should be two copies of that image uploaded to the server now. Historically, GET and POST were the only HTTP methods supported out of the box by browsers, so POST tends to get used for all kinds of purposes. It's pretty common to find services that only use GET and POST requests.
DELETE is for, you guessed it, deleting things. DELETE requests are not supposed to contain a body — the URL is supposed to point to the thing you want to delete.
PATCH is for modifying a subset of something. For example, if you want to update the author of a blog post, you might need to send a PATCH request indicating that fact. In practice, people often use PUT or POST for this purpose instead.
PUT is for completely replacing something that already exists. It's like PATCH, except you replace the whole thing instead of modifying part of it. Since that's a subtle distinction, and because the world is chaos, you'll often find PATCH or POST used instead.

HTTP Status Codes

An "HTTP Status Code" is intended to give a rough idea of why a request succeeded or failed, and therefore hint at what might be found in the body (if anything).

As with HTTP Methods, there are standards and best-practices for which Status Codes to use under any given scenario. But, also as with Methods, in the real world it's a lot more chaotic than that. I've summarized the most common status codes below, and what you might see out in the wild:

200 OK. This is the server giving you a thumbs-up and saying that it understood what you were trying to do. So if you got a 200 then you got what you wanted, right? NOPE! There's nuance here, on top of the fact that the person who wrote the server logic might have just chosen to send a confusing status code. Some servers only reply with 200 codes and provide details in the body! An example of the nuance is: what if your request was a search, and there were no results? The server could send back a 200 with an empty list of results, to indicate that the search ran without error but found nothing. But it could also send a 404 status without any results to indicate the same thing.
201 Created. This is supposed to mean that you uploaded something new (like an image) and that thing now exists. You'll often see 200 used instead.
204 No Comment. A 204 indicates that the server did what you asked, but doesn't have anything to send you back (no response body). You'll often see 200 used instead, and even see 204s that do include a body!
2XX There are other 200-level codes that mean various things, but they aren't very common. You can usually assume that things happened as intended if you get a 200-299 status code.
3XX Are various forms of "Go look over there instead!" (a.k.a. "redirects"). You'll get these if something's URL changed but the server knows about the old URL, or if the client already has the thing it's trying to get and that thing hasn't changed on the server.
400 Whoopsy! All 4XX status are some form of "You messed something up, you silly client!" 400 is the most generic one. It's often used as a catch-all for every kind of client error, so you'll typically need to look for an explanation in the response body or headers.
401 Bad Credentials. You should only get this if your request included access credentials (like a password) but they are somehow invalid (expired or malformed, typically), or if you are trying to connect to a password-protected server without a server password.
403 Access Denied. This is the server flipping you the bird and saying, "Nope, not today, Satan!" You'll get this for protected actions if you have insufficient permission. It's similar to 401, and sometimes used instead, but is supposed to mean "There's nothing wrong with your credentials (or lack thereof) but you are not allowed to do this thing you're trying to do."
404 Not Found. The old classic. You've probably heard of it. It's supposed to mean that the action you wanted to take on a thing is impossible because the thing doesn't exist. It is also used for the more subtle, "That thing doesn't exist as far as you're concerned". For example, if I request something that does exist, but my user isn't allowed to access that thing, then I could either get a 403 ("you aren't allowed to see that") or a 404 ("as far as you're concerned, that thing doesn't exist").

Bringing it all together

In practice, you will not be sending or receiving raw requests. You'll be using tools that end up doing that for you, and that abstract away the details and make it easier for you to add all the information required for sending a request or processing a response. Still, it's useful to see what the raw stuff looks like because it's simpler than you'd expect and it will help you understand what your tools are doing.

Let's look at some (fake) examples of raw requests and responses that make use of all the stuff we talked about above. We'll use the Weather App and its Server from before:

The user boots up the App, which checks to see if the user has purchased any IAPs by requesting that info from the App's Server (via the URL http://fake-weather-app.example.com/purchases). The request might look like this:

  GET /purchases HTTP/1.1
  Host: fake-weather-app.example.com
  Accept: application/json
  Authentication: Basic SomeCredentialsProvingUserIdentity
  Agent: Fake Weather App/v11.0.23 Android/Google Play

The Server receives the request, checks the Authentication to make sure it's legit, adds info to its logs, and then finally checks its database for purchases made by this user. It doesn't find any! It then sends a response that might look like this (learn about JSON):

  HTTP/1.1 200 OK
  Content-Type: application/json; utf-8

  {"purchases":[]}

Thus ends the first request-response cycle! The App sees that the user hasn't made any purchases, and then prompts them to visit the in-app store to see the options. The users does this, and is intrigued 🤔
The App user purchases the "In-Depth Forecast" IAP bundle, which eventually results in that purchase's info being POSTed to the App's Server with the following request:

  POST /purchases HTTP/1.1
  Host: fake-weather-app.example.com
  Content-Type: application/json: utf-8
  Accept: application/json
  Authentication: Basic SomeCredentialsProvingUserIdentity
  Agent: Fake Weather App/v11.0.23 Android/Google Play

  {
    "new_purchases": [
      {
        "iap_id": "blahblahblah",
        "paid_amount": 29.99,
        "receipt": "RandomLookingStuffProvingThePurchaseHappened"
      }
    ]
  }

The App's Server receives this, checks the token, adds this purchase to the database, and responds:

  HTTP/1.1 201 OK
  Content-Type: application/json; utf-8

  {
    "purchases":[
      {
        "iap_id": "blahblahblah"
      }
    ]
  }

The App receives this, parses it, and since it trusts its Server it flips a switch for that bundle so that now every time it checks to see if the user owns that bundle, the answer is yes.

In real life, you'll be using tools with varying complexity and features, like libraries in your favorite programming language or applications built specifically for sending raw requests and receiving raw responses.

Tools like these will let you separately handle bodies and headers, automatically parse out status codes from responses, and automatically parse response bodies. All the parts will be there, and you'll need to know what to do with all the parts, but you'll get lots of help from your tool-friends.

If you want to dig deeper, I recommend using Postman to send requests to real websites and see what they send back, and using your browser's developer tools to see what your browser sends and receives when you visit websites.

JSON Guide

Adam Coster — Mon, 03 Jul 2023 21:11:09 +0000

JSON (JavaScript Object Notation) is a super popular way of representing complex data for several reasons:

It's precise and unambiguous (so computers can read it)
It's not too verbose (so humans can read it)
It's simple (so humans can learn it)

How does JSON work?

Data Types & Format

JSON can be used to represent data structures composed of the following kinds of data subtypes:

Number: A number! Just like you'd expect
Boolean: true or false
String: Some text, always wrapped in double-quotes, like "hello"
Null: null, which is a sort of "empty" value. Interpretation depends on context, but it usually indicates the absence of something.
Array: A list of data (of any JSON-compatible type), separated by commas with the whole thing wrapped in square brackets, e.g. ["hello", -10.3, null, ["another array"], {"a key":"for an object"}, false]
Object: A collection of keys (always strings) and corresponding values (any JSON data type), represented as comma-separated key-value pairs, with colons between each key and its value, all surrounded by a pair of curly brackets, e.g. {"this is a key": "this is a value", "next key":["value can be arrays",10], "yes":true}

Looking at those examples, you might be thinking, "Whoa, that's really hard to read." You're not wrong!

JSON allows you to use however much whitespace you want between separators, values, and brackets. If we use good spacing plus a dash of syntax highlighting, it gets a lot better. Here's that last example again but nicely formatted:

{
    "this is a key": "this is a value",
    "next key": [
        "value can be arrays",
        10
    ],
    "yes": true
}

JSON is extremely strict. If you forget a comma (or have one too many), or use single-quotes instead of double-quotes for your strings, it'll be considered invalid and most parsers will throw errors.

JSON Interpretation

Well-written JSON describes itself, but it's pretty rare that something is so self-descriptive that you don't need to look for documentation or do experiments.

Typically, JSON "Objects" are used to represent instances of things that have properties. For example, a "user" may have a name, age, email address, password, and a jillion other possible properties. You might represent all that with an Object like so:

{
    "name": "Vlad",
    "email": "vlad@impaler.blood",
    "age": 300,
    "isVampire": true
}

An Array is typically used to contain a set of related values, though that isn't strictly required. A common use case is getting search results from a web service, where you'll end up with more than one result for the same sort of thing.

For example, let's say you fetch the list of users from a service. That list might look something like:

[
    {
        "name": "Adam",
        "email": "adam@example.com"
    },
    {
        "name": "Vlad",
        "email": "vlad@impaler.blood"
    },
    {
        "name": "Susan",
        "email": "susan@example.com"
    }
]

While you'll frequently see things like lists of objects, it is rare that you'll get them like in the above example without any metadata (data about data).

Typically, if you're getting JSON data from some source it will be an Object whose root keys are for both metadata and the actual data itself. For example:

{
    // metadata...
    "page": 1,
    "total_pages": 5,
    "total_returned": 3,
    "search_results":
        // actual data we wanted...
        [
            {
                "name": "Adam",
                "email": "adam@example.com"
            },
            {
                "name": "Vlad",
                "email": "vlad@impaler.blood"
            },
            {
                "name": "Susan",
                "email": "susan@example.com"
            }
        ]
}

Interpreting JSON data is all about understanding where it came from, looking for any standardization of the outputs from that source, knowing what you were trying to obtain, and then figuring out what all of the keys and values mean. It can be tricky business, especially for deeply nested data (objects of arrays of arrays of objects of objects... it can get pretty gnarly!).

Special Characters, Escape!

I noted above that strings are always wrapped in double-quotes. What if you want to put a double-quote inside of a string?

There are other special cases, too. What if you want to put a newline or a tab in a string?

Double-quotes are "special characters." They're special because they don't simply indicate a text character, they have a function. They are providing instructions to parsers.

For example, a boring old non-special R means "I'm the letter 'R'" while the special " means, "A string is starting or ending!" See that difference? The " is functional.

Newlines are a related, but different problem. How do you represent a newline in text? You might think that you can just hit the ENTER key and be done with it. Not a bad guess, but it doesn't work that way for JSON! If you slap a newline into a string via the ENTER key, JSON gets stuck at that newline and assumes you forgot an end-quote, and then throws errors. Oops.

To represent newlines without using newlines, you have to make a regular character special. It's sorta the opposite of the double-quotes case, where we made a special character more regular.

There is a special character you can use inside JSON strings that basically toggles special-ness. It says, "If the next character is already a special character, make it a regular character. But if it's not a special character, make it one!"

We call this special character the "escape character". Why? I guess because it lets you escape from the normal state of the world? Who knows. The point is that the escape character inverts the specialness of the next character. JSON's escape character is \.

Side note: This whole "escape character" thing is quite common in text programming and data representation, and the \ character is probably the most common character used for this purpose.

So if you want to use a double-quote inside a string, you need to differentiate it from a double-quote that would end the string. You do this by escaping it: \".

If you want to stick a newline inside a string, instead of hitting that ENTER key you invoke the special form of the letter n: \n.

What if you want to use a literal \ character inside a string? You guessed it: \\.

(There are other special characters in JSON, but mostly you'll only care about double-quotes, newlines, and tabs.)

For example:

{
    "name": "Adam",
    "email": "me@example.com",
    "bio": "Adam is writing stuff about JSON.\nSee that \"slash-n\" right back there? That was a NEWLINE! \"What about quotes?\" you ask?\n\tWell, you just saw an escaped quote with a newline and tab character after it!"
}

JSON Limitations

While JSON is super popular on the web and elsewhere, it has some significant limitations that make it less than ideal for some use cases:

It only has one kind of "number". In programming, we often want to differentiate between integer ("whole") numbers and floating-point ("decimal") numbers. JSON doesn't have that distinction, so you have to be careful about how you use numbers.
There are other data types that we very often want to use, like dates and binary, that JSON does not support. Those have to be stored as strings or numbers and converted back and forth as needed. This can turn into quite the headache.
It's more verbose than is strictly necessary. For example, since the keys in JSON Objects are always strings, why do we need to wrap them in double-quotes?
It does not support comments, in the programming sense. In programming, a "comment" is a non-functional bit of text that can be used to provide information to the reader without impacting interpretation.

There are some extensions to plain JSON, like JSON5 and JSONC, and some libraries that add support for other data types, but those are not natively supported by most software.

JSON Tools

If you're trying to read or write JSON, in either case you're going to want some decent tools to catch problems early. It's incredibly easy to miss a special character (or forget to escape one), especially if you don't have good indentation and syntax highlighting.

VSCode is always a great go-to for dealing with JSON. You can edit JSON files directly with it, or create a new tab Ctrl + N and paste some JSON into it (then set the language to JSON for that tab). You'll get syntax highlighting out of the box, and VSCode will highlight errors and try to autocomplete brackets and quotes for you! You can even get auto-formatters (also called "prettifiers") that will fix all the indentation for you, so you can dump a wall of spaceless nightmare-JSON and then let robots make it readable for you.

If you don't have VSCode ready to go for JSON, or are otherwise in a pinch, there are tons of online JSON tools out there.

The "Can We Not" Principle

Adam Coster — Mon, 03 Jul 2023 21:08:41 +0000

One of the most important pillars I use to guide my decisions is the "Can we not?" principle. It's a simple idea: if I can avoid doing something, I should.

Sure, you do have to do some things to, say, build and sell a product or manage a household. But things you've done are also the source of future problems. Code we write has to be maintained. Features we build have to be supported.

The more we do, the more we have to do. And this just keeps adding up -- what is an organization or household but an ever-growing collection of things that have to be maintained?

Conversely, any time you don't do something you now have one fewer thing to deal with tomorrow. That is, if that thing you didn't do today is also something you don't have to do tomorrow.

Some things can't be avoided. But far more things can be avoided than we typically think. It's always worth asking the question: "Can we not?"

Configure pnpm for the best possible developer experience

Adam Coster — Mon, 29 May 2023 18:57:28 +0000

I've spent a lot of time with the big-three Node.js package managers (npm, pnpm, and yarn), and I've consistently found pnpm to deliver the best overall experience. Of particular interest to me is that it works great with "monorepos" (many packages in one git repository).

While pnpm's defaults are solid, there are a bunch of tweaks that make things even better. After a lot of trial and error I've found the settings that make my development experience great.

The first sections of this post are for people new to pnpm. For those already familiar, you can jump right to my configuration recommendations.

Differences from npm

pnpm is largely a drop-in replacement for npm: just plop in a package.json file and start using pnpm <whatever> commands!

There are some differences that are good to be aware of:

Monorepos are specified using a pnpm-workspace.yaml file instead of the "workspaces" field in package.json that npm and yarn use.
You can run all of your package.json scripts via pnpm <script-name> (npm lets you do that for some scripts, but others require the npm run subcommand)
You can also run your installed binaries with pnpm <command>. For example, you might want to run local versions of Typescript (pnpm tsc) or eslint (pnpm eslint). npm uses a separate command (npx) to run binaries.
pnpm separates the installing the dependencies listed in your package.json (via the command pnpm install) from adding new dependencies (via the command pnpm add). npm uses npm install for both.
To add a local dependency within a monorepo, in your package.json "dependencies" field you'll prefix your local dependencies' version-range strings with workspace: (a.k.a. the "workspace protocol"). When publishing, pnpm resolves those prefixed versions to their actual versions in the published packages. For example, if I have package a depending on b in my monorepo, the package.json for a would look like:
```
"dependencies":{
  "b": "workspace:*" // (where '*' means "any version")
}
```

There are many other differences, but the above are the most likely to trip you up when switching from npm.

Installing pnpm

I recommend using corepack since that keeps things nice and automagic, but there are also other ways.

Basic setup for monorepos

There is only one thing you need to do to enable pnpm for your monorepo: add a pnpm-workspace.yaml file alongside your root package.json. It should look like this:

# The only field in this config file!
packages:
  # Add paths to each package's folder, or glob patterns
  - 'packages/*'

That's it! Now you're ready to use pnpm. BUT, with some additional configuration you can get a much slicker dev experience.

Switching from npm

While the switch from npm to pnpm is pretty smooth, there are some changes you'll have to make:

Nuke all node_modules folders with rm -rf **/node_modules. pnpm stores dependencies completely differently and will need to download everything fresh.
In your package.json files, update all local dependency listings to use the workspace protocol. For example, {dependencies:{myLocalPackage:"^1.2.3"}} would become {dependencies:{myLocalPackage:"workspace:*"}}
Find all instances of the words "npm" and "npx" across all projects and replace them with pnpm. You can use a regex search like \bnp[mx]\b and restrict the search to appropriate filepaths to do this quickly and with minimal error.

Update your VSCode settings.json:

//...
"search.exclude": {
  // Avoid polluting search results with lockfile content
  "pnpm-lock.yaml": true,
},
// Ensure VSCode uses pnpm instead of npm
"eslint.packageManager": "pnpm",
"npm.packageManager": "pnpm",
// For those using file-nesting, nest the new files. E.g.:
"explorer.fileNesting.patterns":{
  "package.json": "pnpm-workspace.yaml, pnpm-lock.yaml"
}

Get the best developer experience

The pnpm experience is solid from the jump, but you can make it GREAT by digging into the configuration options. The following are my recommendations for how to configure pnpm in a monorepo context:

The `.npmrc` file

Just like npm, pnpm reads configuration information from .npmrc files. These files use key=value pairs to set config options.

Notably, any configuration settings in your repo-root .npmrc file will also apply to all of the packages within your monorepo! This is super useful, since it allows you to centralize your pnpm settings. You can even override settings for a specific package by adding a different .npmrc alongside that package's package.json.

Create a .npmrc file alongside your root package.json file, and then set the following (see the docs for all options):

use-node-version=18.16.0 (or whatever version you want). This setting causes pnpm to use the listed Node version for its operations, including when you run your package.json scripts or run node itself (if you do so via pnpm node instead of calling node directly). pnpm will even install that version for you if it isn't already available! You can also change this setting for any specific package, allowing you to ensure that your packages are always using the node version you want without you ever having to think about it.
strict-peer-dependencies=false Peer dependencies are a nightmare. I always set this to false to make them less so.
publish-branch=main (or whatever your main git branch is). This prevents pnpm publish from running if you aren't on the specified branch, which can save you some headaches.
save-prefix="". This makes it so that exact versions are used when you run pnpm add. This is a good default since it's the safest, and you can always change the behavior for specific dependencies by manually editing their version (e.g. by changing "1.1.3" to "^1.1.3").
link-workspace-packages=false. Your monorepo packages can depend on each other, but how does pnpm decide if you mean the local version of a dependency (allowing symlinking right to the source) versus a specific past version (requiring downloading separate files)? pnpm provides a few ways to handle this, but setting this option to false is the most predictable and explicit. And that means fewer surprises! This option tells pnpm to only resolve local deps to the local files when the workspace: protocol is used, and to otherwise download published versions.
save-workspace-protocol=rolling. This dictates how pnpm add decides whether or not to include the workspace: prefix and whether or not the version will be specified. Whatever default you choose, you can easily override a dependency by manually editing its version string in your package.json dependencies. There's significant nuance in which option to choose for defaults that make sense for your use case:
- false: Never use the workspace: prefix. In combination with link-workspace-packages=false newly-added deps will always resolve to published, static artifacts (instead of local ones). This yields the fewest surprises, but doesn't let you take full advantage of having all of your packages in one repo.
- true: Always include the workspace: prefix and the version of the dep at the time of pnpm add, for local dependencies. This is the most balanced choice if you have really good versioning and testing tooling, since you can still do rapid development across dependency boundaries but will get alerted when a dep gets bumped too far.
- rolling: Always include the workspace: prefix but not the specific version. This provides the smoothest development experience, since you can freely move between projects and see the consequences on their dependents instantly, without having to deal with semver range conflicts. However, you need robust versioning and testing tooling to stay out of trouble since it's easy to publish something without also publishing its updated dependency, among other issues. This case is best reserved for monorepos where all projects always get version-bumped at the same time, and automatically, if there have been any changes to them.

Manage dependency versions with syncpack and depcheck

pnpm helps to reduce downloads and the size of dependencies on disk by symlinking shared dependencies. However, when your shared dependencies use incompatible versions you lose a lot of those benefits. This is a particular problem when you use the more conservative settings recommended above, like forcing exact versions of dependencies instead of ranges.

To mitigate this, I recommend using syncpack. Syncpack looks at all of your package.json files to find all common dependencies, and ensures they're all at the same version. If you need more nuance than that, it also has a lot of configuration options.

Add syncpack fix-mismatches to the "postinstall" script in your package.json to ensure that your dependency versions are being frequently synced.

Finally, over time you're likely to end up with some stray dependencies that are no longer in use by your packages. depcheck is a great tool for this, and it also has a ton of configuration options.

JavaScript wants your functions to have names

Adam Coster — Mon, 29 May 2023 18:54:34 +0000

JavaScript function names can be quite helpful for debugging and logging, but how does JavaScript determine what a function's name is?

JavaScript functions have a `name` property

The first thing to note, since it's not at all obvious, is that JavaScript functions are special kinds of objects with a few default properties:

function myFunction(){};
console.log(Object.getOwnPropertyNames(myFunction));
// [ 'length', 'name', 'prototype' ]

One of those properties is the name field. That's where your function's name is stored.

Function names are readonly

If we take a named function and look at its name property descriptor, we see that it is readonly:

function myFunction(){};
console.log(Object.getOwnPropertyDescriptor(myFunction, 'name'));
// {
//   value: 'myFunction',
//   writable: false,
//   enumerable: false,
//   configurable: true
// }
myFunction.name = 'anotherName'; // TypeError: Cannot assign to read only property 'name' of function 'function myFunction(){}

Notice that the name property is configurable by default. So while we can't directly re-assign the name using an assignment operation, we can still use Object.defineProperty to rename a function:

function myFunction(){};
Object.defineProperty(myFunction, 'name', {writable: true, value: 'myRenamedFunction'});
console.log(myFunction.name); // myRenamedFunction

JavaScript wants to name your functions

You can explicitly name a function by using the function myFunctionName(){} syntax. But when you don't use that syntax, e.g. with arrow functions or with "anonymous" function(){} declarations, JavaScript will still try to find a name to give to your function.

If you've declared your function on the right-hand-side of an assignment, the name of the variable on the left is used as the name:

const myFunction = function(){};
console.log(myFunction.name); // myFunction

const myArrowFunction = () => {};
console.log(myArrowFunction.name); // myArrowFunction

const myObject = {
  myMethod: function(){},
  myArrowMethod: () => {}
};
console.log(myObject.myMethod.name); // myMethod
console.log(myObject.myArrowMethod.name); // myArrowMethod

Do "anonymous" JavaScript functions even exist?

No, not really. If you create an "anonymous" function that also isn't directly assigned to a named variable, you can get an "anonymous" function:

const anonymousFunctionGenerator = ()=>()=>{};
const myAnonymousFunction = anonymousFunctionGenerator();
console.log(myAnonymousFunction); // [Function (anonymous)]

BUT! The function does still have a name, it's just a sneaky one: an empty string!

const anonymousFunctionGenerator = ()=>()=>{};
const myAnonymousFunction = anonymousFunctionGenerator();
console.log(Object.getOwnPropertyDescriptor(myAnonymousFunction, 'name'));
// { value: '', writable: false, enumerable: false, configurable: true }

CommonJS and ESM import/export compatibility, by example

Adam Coster — Thu, 10 Mar 2022 00:56:35 +0000

Node's CommonJS vs. ECMAScript ("ESM") divide is probably the source of most of my quality of life frustrations as a fullstack Typescript/Node/Javascript programmer.

I can often go for weeks at a time before running into new incompatibility problems, so then each time I have to remind myself how interoperability works between them. Well, this time I made a tiny, simple demo so that next time I can just refer to it. And now you can, too!

Short summary of the CommonJS/ESM distinction and problem:

CommonJS uses the require('./file.js') syntax for importing other modules and the module.exports = syntax for exporting stuff from modules
ESM uses the import {stuff} from './file.js' syntax for importing and the export stuff syntax for exports
CommonJS files can use the .cjs extension to tell Node that they are in CommonJS
ESM files can use the .mjs extension to tell Node that they are in ESM
CommonJS imports are synchronous
ESM imports are asynchronous (which also allows for top-level await)
CommonJS works in Node but does not work in browsers
ESM is supported by all modern browsers and the latest versions of Node, but does not work at all in Node versions below 12
Tons of the core JavaScript ecosystem tooling was developed in Node and Node only recently supported ESM, so a huge fraction of existing Node projects are in CommonJS

So that's our situation. Now, to the problem at hand: If you are using ESM, can you import CommonJS? What about the other way around?

In short, YES! But with considerations.

Sample export modules

Let's start with some importable modules. One in CommonJS, the other in ESM:

/**
 * @file `exporter.mjs`
 * (An ESM module exporting a default and named entity.)
 */

export function namedMjsExport() {}

export default function defaultMjsExport() {}

/**
 * @file `exporter.cjs`
 * (A CommonJS module exporting a default and named entity.)
 */

module.exports = function defaultCjsExport() {};

module.exports.namedCjsExport = function namedCjsExport() {};

Importing from ESM and CommonJS to ESM

What does it look like to import both of those modules into another ESM module? Simple! If you are importing into an ESM module, it looks the same either way:

/**
 * @file `importer.mjs`
 *
 * An ESM module that imports stuff
 */

import defaultCjsExport, { namedCjsExport } from "./exporter.cjs";
import defaultMjsExport, { namedMjsExport } from "./exporter.mjs";

console.log({
  title: "Importing into an ESM module.",
  defaultCjsExport,
  namedCjsExport,
  defaultMjsExport,
  namedMjsExport,
});

And after we run that script via node importer.mjs (Node v16):

{
  title: 'Importing into an ESM module.',
  defaultCjsExport: [Function: defaultCjsExport] {
    namedCjsExport: [Function: namedCjsExport]
  },
  namedCjsExport: [Function: namedCjsExport],
  defaultMjsExport: [Function: defaultMjsExport],
  namedMjsExport: [Function: namedMjsExport]
}

Perfect! If we're using ESM we can basically treat all code as if it's also ESM. (There are some nuances, but we can usually get away with ignoring them.)

Importing from ESM and CommonJS to CommonJS

So importing into ESM is no biggie, are we so lucky with importing into CommonJS?

NOPE!

Since require() is synchronous, you can't use it to import ESM modules at all! In CommonJS you have to use require syntax for other CommonJS modules and an import() function (distinct from the import keyword used in ESM!), a function that returns a promise, to import ESM.

Let's take a look:

/**
 * @file `importer.cjs`
 *
 * From a require-style Node script, import cjs and mjs modules.
 */

/**
 * Import a module by `require()`ing it. If that results in
 * an error, return the error code.
 */
function requireModule(modulePath, exportName) {
  try {
    const imported = require(modulePath);
    return exportName ? imported[exportName] : imported;
  } catch (err) {
    return err.code;
  }
}

/**
 * CommonJS does not have top-level `await`, so we can wrap
 * everything in an `async` IIFE to make our lives a little easier.
 */
(async function () {
  console.log({
    title: "Importing into a CommonJS module",

    // CJS<-CJS and MJS<-CJS are equivalent
    defaultCjsExport: requireModule("./exporter.cjs"),
    namedCjsExport: requireModule("./exporter.cjs", "namedCjsExport"),

    // Cannot `require` an ESM module
    defaultMjsExportUsingRequire: requireModule("./exporter.mjs"),
    namedMjsExportUsingRequire: requireModule(
      "./exporter.mjs",
      "namedMjsExport"
    ),

    defaultMjsExport: (await import("./exporter.mjs")).default,
    namedMjsExport: (await import("./exporter.mjs")).namedMjsExport,
  });
})();

And the output of node importer.cjs:

{
  title: 'Importing into a CommonJS module',
  defaultCjsExport: [Function: defaultCjsExport] {
    namedCjsExport: [Function: namedCjsExport]
  },
  namedCjsExport: [Function: namedCjsExport],
  defaultMjsExportUsingRequire: 'ERR_REQUIRE_ESM',
  namedMjsExportUsingRequire: 'ERR_REQUIRE_ESM',
  defaultMjsExport: [Function: defaultMjsExport],
  namedMjsExport: [Function: namedMjsExport]
}

Oh, wow, look at how much more code we needed and how careful we need to be!

Advice

I've been all-in on ESM for a while now. It's a better developer experience and is clearly what we'll be using in the future. But it comes with headaches because so much of the Node ecosystem is still in CommonJS, and you should think carefully before going all-in.

Don't forget about the file extensions! Modern Node handles the .mjs and .cjs extensions, so if you need to use one module type in one place and another somewhere else, feel free to mix it up! This also works in Typescript (v4.5+) with the .mts and .cts extensions.
(But also note that some tools don't know about those extensions...)
Tools written in CommonJS (i.e. most existing Node-based tools) usually handle ESM poorly. Even extremely popular projects. If you want to guarantee that you can use a tool with your code, you may want to stick with CommonJS.
If you will mostly be importing other packages into your project (versus having yours imported into others), ESM will let you not have to worry much about what kind of modules you're importing.
ESM spec requires that import paths be valid paths, meaning you need the file extension and everything (CommonJS doesn't require that). Node has an option to skip out on that requirement for ESM modules, if you want to keep it old-school: node --es-module-specifier-resolution=node your-dope-module.mjs
If you do decide to go all-in on ESM in Node, be ready to do a lot of very annoying troubleshooting!

✂ Save time and make fewer mistakes with Snippets

Adam Coster — Tue, 01 Jun 2021 12:32:00 +0000

Last week I was sitting down to write a custom JavaScript error and became immediately annoyed. I've had to do that just often enough that it felt like I was being a robot, but not so often that I could do it without looking back at references.

I thought, "I guess this is what 'snippets' are for, right? I should figure out how those work."

After having done that... I'm now even more annoyed. Because snippets are amazing, VSCode (and other editors) already have a ton of useful built-in ones, and I've been using VSCode for years without even trying them out (despite definitely knowing that they existed).

Why hadn't I looked into them before!? Especially as a person who obsesses over automations and process improvement!

In short, I misunderstood snippets. I just sorta assumed that snippets were a library of boilerplate code that you could search and then insert into your new code. *Yawn*. That would mean having to search for the snippet I needed, then click around and edit the stuff that just inserted... How tedious! So I figured it was faster to write things from scratch.

I had made that assumption and then didn't take the two minutes it would have taken to look them up. The first few sentences of VSCode's snippet docs were all I would have needed to read.

Whoops!

It turns out that VSCode snippets have a spectacular feature that makes them so great, and that I simply didn't realize might exist: placeholders.

You can set up a snippet to have a bunch of placeholders, and then once you've plopped the snippet into your code you can hit tab to bounce from one placeholder to the next.

And! If two placeholders have the same name, they get filled out at the same time! The snippet can also dictate where your cursor should end up after you've filled the placeholders! It can even put in clipboard or selected text!

And! You can create per-language, per-project, and user-wide libraries! So you can sync your own favorite snippets to use everywhere you go, while also making sure that your project collaborators have access to whatever snippets everyone needs to make the work easier.

So much power!

Now I'm finally using the built-ins (they're particularly nice for for loops) and I've started to build my own little libraries.

Here's that dang custom error code as a Typescript snippet:

/** @file MyCustom.code-snippets */
{
  "Custom Error": {
    "scope": "typescript",
    // The prefix is the what gets used for
    // auto-complete, so when I type "error"
    // in VSCode (in a Typescript file) one
    // of the auto-complete options is this very
    // snippet.
    "prefix": "error-custom",
    "body": [
      // Each entry in the array is treated as a separate
      // line, and VSCode will use the correct newline
      // characters based on your system and VSCode settings.
      "export class ${1:name} extends Error {",
      // The use of the tab character tells VSCode how things
      // should be *relatively* indented, so that it can
      // correctly auto-indent based on the code location
      // you put the snippet into.
      "\tconstructor(message:string) {",
      "\t\tsuper(message);",
      // Here I'm referencing the first placeholder from
      // above, so that when I'm filling it out above it
      // shows up down here as well!
      "\t\tthis.name = '$1';",
      "\t\tError.captureStackTrace(this, this.constructor);",
      "\t}",
      "}",
      "",
      "export function assert(claim:any, message = 'Assertion failed'): asserts claim {",
      "\tif(!claim){throw new $1(message);}",
      "}"
    ]
  }
}

That delightful little snippet drops in a custom error class, plus an assertion to go along with it to throw that very same error. Here's an example of what it looks like fully-populated:

export class MyCustomError extends Error {
  constructor(message: string) {
    super(message);
    this.name = 'MyCustomError';
    Error.captureStackTrace(this, this.constructor);
  }
}

export function assert(
  claim: any,
  message = 'Assertion failed',
): asserts claim {
  if (!claim) {
    throw new MyCustomError(message);
  }
}

And here's a Markdown snippet that converts some selected text into a link to its definition (this functionality is quite specific to a project I'm working on):

{
  "Add class to term reference": {
    "scope": "markdown",
    "prefix": "term",
    "body": [
      "[${TM_SELECTED_TEXT}](#term:${TM_SELECTED_TEXT/(.*)/${1:/downcase}/})"
    ]
  }
}

That snippet makes it so that if I select the word Something and then start typing term, I'll overwrite the original word and then see VSCode's auto-complete suggestions. When I choose the above snippet, VSCode remembers that I had selected Something before overwriting it with the snippet prefix, and so it inserts: [Something](#term:something)

Neat!

Now that I'm done kicking myself for all the wasted past efforts of not using snippets, I'm extremely excited about building out my new snippet libraries and trying out those created by others.

Don't be like me. Use snippets.

If you've found any VSCode snippet libraries you really like, or have your own snippet you're particularly pumped about, please share!

This post was adapted from an issue of the DevChat.

🚀 Super-charge your email with your own domain, routing, aliases, and catch-alls via Google Workspace

Adam Coster — Sun, 16 May 2021 23:32:58 +0000

There's a good chance you're still using an email address you set up in college, or even in high school. Or a personal email address that you wish was more professional.

Changing your email address, like changing your phone number, is a pain. You've probably done it as rarely as possible. Maybe you just accumulated new ones, and find yourself constantly juggling accounts.

But what if you could change your email address at any moment with the slightest effort? Or have different addresses for work and play? Plus another for your side hustle? Or one for each of your 10 side hustles? Even separate addresses for every single service you sign up for, to make your services harder to hack into and let you see exactly who is selling your information?

What about making addresses for family members, collaborators, or employees? What if you could create email accounts for other people with your own domain, all addresses that you still own and control so that no important data will ever be lost no matter who comes and goes?

And what if that was all centralized, and easily changeable at your whim without any stress?

This article is about how you can do all of that in Gmail, through Google Workspace. Whether it's for yourself, your family, or your business, having full control over your email is a powerful thing.

Unfortunately, there's no way around having one last migration (Google has documentation for that). But only this one last time! And, also unfortunately, you can't do it for free. But given how central email is to our lives, I think you'll find the low cost worth it.

💡 Tip: If you don't want to deal with migration (much data cannot even be migrated!), you could still use all the tricks here and just reroute mail to your existing inbox!

Note: To make this long post more readable, I've collapsed some content into expandable sections. The little triangles that toggle them open can be easy to miss!

🤑 What'll it cost you?

Personal domain(s): You'd probably own these anyway for your portfolio/personal site. Either way, per domain they'll cost you $8 USD to... well, the sky's the limit! It all depends on your registrar, which TLD (top-level domain, like .com or .dev), whether you have to buy it from a squatter, and so on.
Google Workspace: They of course have multiple tiers here, but the lowest tier ("Starter") is $6 USD/mo per user and has all the stuff you'd be getting with a free Gmail account plus a million other things. That's all I've needed to run my business!

For example, for my personal use I have a Google Workspace subscription with one user (me) at $6 USD/mo that centralizes three domains I own (each at $12/yr through Google Domains). So it comes out to $9/mo for me to own three domains and have super-powered email.

What about non-Google alternatives?

Presumably the features I described above are also available in non-Gmail email services. I'm not specifically advocating that you use Google; it's what I know.

This article will help you understand how all these features would work for other services, so if you don't want to use Google then use this article as something to skim for an overview of what to look for in other services.

🔨 Getting and setting up Google Workspace

To use Google Workspace, you're going to need a custom domain. (Or domains, plural!) Finding ones you like is definitely the hardest part of this process. The nice thing is that you can always add another domain to your Workspace and immediately start using it as if it had been your favorite domain all along (all without having to tell people to contact you at a different address or migrate all your mail), so you won't get permanently locked into a choice your past self made.

I use Google Domains as my registrar. (Registrars are entities that sell web domains.) It's not the cheapest option, nor the most fully-featured, but for me the convenience of having everything under one roof is worth that. Plus I rarely need fancy features from my registrar: as long as I get free privacy and security features and easy DNS (Domain Name System) management I'm happy.

If your domains are registered via Google Domains (or you transfer them to it) then adding Workspace is dead simple. Just go to your domain in the Google Domains site, and then:

Domain Overview > Get a custom email with Workspace

And follow the prompts. Google will even set up your DNS records for you. Magic!

If you're using a non-Google registrar then... uh... hopefully the main Google Workspace site has good onboarding and your registrar has a nice UI.

I'm sure you'll be fine...

🔑 Log into your new Workspace Gmail account

Part of the Google Workspace setup included you creating your first account, which also results in an email address.

Let's say your domain is my-amazing-domain.tld and your name is... uh... Flux. So your email address would be flux@my-amazing-domain.tld.

Anywhere you would see a Google login page (including websites that use Google sign-in) you can plug in your new email address and password, just like you would with a plain Gmail account.

🧭 Finding and navigating the Admin console

💡 Tip: Don't use the Admin mobile app

It does have a mobile app. Technically. But the mobile app has super limited features and the general consensus seems to be: don't use it.

Use a non-mobile browser for all the stuff we talk about here.

When you create a Workspace account, you get access to a new "Admin" console for managing it. You can get to the Admin console in your browser, either directly via admin.google.com or via your Gmail inbox using that incomprehensible 3x3 dot-grid icon in the top right:

The Admin portal is where most of the power of Workspace comes in. I encourage you to poke around and explore all the things you can do.

The UI has some... I guess we'll say, "design issues." If you get confused or stuck, trust me when I say it's not your fault. If you think there should be an option somewhere but can't find it, try hovering over or clicking things. There are secrets everywhere.

Example of the console's confusing UI

At Admin > Apps > Google Workspace you may see something like this:

Once you get there you'll probably try to click the checkbox to select Gmail, only to find that this just lets you turn services on or off. Then you might notice that if you hover over the Gmail row one of those three-vertical-dot icons appears on the right! But all that lets you do is turn the service on/off, too. So then you'll think you must be in the wrong place... You're not! Turns out that the whole Gmail table-row is a button/link that takes you to the Gmail Admin settings.

👤 Aside: User Profiles

While this article is about the Gmail Admin part of Google Workspace, there's a related point of frustration you'll immediately hit: if you try to add a picture to your profile, you'll see that you don't have permission to do that.

That's a super weird default, especially for the admin user (you).

To fix it, go to Admin > Directory > Directory Settings > Profile Editing.

👥📧 Users, "recognized", and "unrecognized" addresses

You're probably used to thinking about email addresses as being synonymous with a user account. An email address is something that goes to an email inbox, right?

While an email address can point to an inbox, it doesn't have to. An email address is just like a URL: the server that controls the address/URL can decide what it means.

Any time you send an email, a server (i.e. "someone else's computer") somewhere catches that request and decides what to do with it. And it could do anything! Re-route it, modify it, block it, auto-send something back to the sender, add logs, evaluate spamminess, flag the sender as a spammer, convert the email into Pig Latin, send a copy to an archive, parse it with a robot to do cool weird stuff that has nothing to do with email, and on and on.

Literally anything!

So let's clarify and create some terms so you don't get lost:

User: An account within your Workspace, identified by a unique email address as a username, that has its own Gmail inbox, Google Docs, and so on. It's probably assigned to a single person, though it doesn't have to be. It's the same concept as any other Google account, just managed by your Workspace.
Envelope: A term used by email services that is basically synonymous with "email address".
Address: A unique email address for one of the domains attached to your Workspace. E.g. me@my-rad-domain.tld. While a User has an Address, an Address doesn't have to belong to any user. A user could have multiple addresses that all point to the same inbox. Addresses can be used for lots of things, including identifying a User, identifying a Group, and setting up routing rules.
- Recognized Address: An address that is assigned to a known resource within your workspace (e.g. a User, Group, or other kind of entity).
- Unrecognized Address: An address that doesn't point to any known resource.

💡 Tip: While you may end up with many aliases, you only have one username. Every time you sign into your account, or use Google Sign-in somewhere else, you'll need to use your account's primary address as the username.

♾ Create infinite addresses and aliases

There are three main approaches to bending email addresses to your will in Workspace:

Use "mapping" to set up one-to-one mappings of one address to another: Admin > Apps > Google Workspace > Gmail > Advanced Settings > Recipient address map
Use "routing" rules to set up more complex mappings: Admin > Apps > Google Workspace > Gmail > Advanced Settings > Routing
Use "aliased" domains so that me@professional-af.tld and me@everything-is-a-joke.tld all work synonymously with me@main-domain.tld: Admin > Account > Domains > Manage Domains

In all cases, the re-routed addresses basically work as aliases. Mail sent to them gets routed right into the inbox (or other target) you want them to go into.

⚠ Important! ⚠

Before you do any custom routing, there are two things you need to know:

You could accidentally create routing that prevents you from getting emails you expect to get. Do experiments by sending emails from other accounts to see if they appear where they're supposed to. If you mess something up, don't worry. As long as you can log into your Workspace Admin console you can fix it.
You must disable the default "Default Routing" entry or else nothing else will work!

To disable the default "Default Routing" setting, go to Admin > Apps > Google Workspace > Gmail > Default Routing.

You'll see an entry in a table, with a button that says "Disable". Disable it! You can always turn it back on later (unless you delete it).

I can't figure out what that default entry is doing. It wasn't there the first time I set up G Suite, years ago. All I know is that it prevented any other routing rules I added from working.

🗺 Recipient Address Mapping

Admin > Apps > Google Workspace > Gmail > Advanced Settings > Recipient address map

Now for the simplest option: 1-to-1 mapping. This approach is particularly useful if you want to take an allowlist approach to addresses at your domain, so that people have to know one of your explicitly-defined addresses to be able to send you anything. This is great for reducing unsolicited mail.

Sample use cases

Map sales@mydomain to your-name@mydomain to create a role-specific address that can be changed to point to a different user at any time.
Map very-important-stuff@mydomain to both your and an assistant's addresses.
Map for-robots@mydomain to a 3rd party email-based service that will do cool stuff with emails that get sent to it.
Map past-employee@mydomain to new-employee@mydomain for seamless employee onboarding.
Map old-address@secondary-domain.tld to new-address@primary-domain.tld
Map temporary-address@mydomain to permanent@mydomain, so that you can sign up for things that might end up selling your address and then later re-route messages coming into temporary-address right into spam (or cause them to bounce).

And so on and so forth. If you have any creative uses for such aliases, please share in the comments!

Google has pretty clear instructions for setting these up, and the interface isn't too complicated.

🚦 Routing

Admin > Apps > Google Workspace > Gmail > Advanced Settings > Routing

Workspace's "Recipient Address Mapping" rules (described above) are just stripped down and simplified routing rules. You can do all that stuff and more with the fancy, full-featured Routing options.

Sample use cases

Do all the stuff listed in the prior section.
Create a catch-all address, so that any Unrecognized Address will route to you. This is great if you want to use a different address for every service you sign up for, for example.
Route mail from specific senders (or groups of senders) to a different address. Maybe you want most of your mail to go to an assistant or to an automated service, but certain mail to always go right into your inbox.
For a service that requires a public email address, create a route for that address that only allows emails from that service through. That way you can use the service without getting spammed! (Looking at you, npm...)
Use a regex pattern to apply rules to a collection of addresses. This can provide a nice middle ground between the 1-to-1 mapping and catch-all approaches.
Add a prefix to messages from certain senders.
Create spam traps that dump messages right into spam before they even hit your inbox.
Prevent a range of addresses from ever going to spam.
Add custom headers, for auditing purposes or for use by robots.
Re-write the recipient address while keeping the domain or username parts constant.
Strip attachments from certain emails.

And more!

(While I've mostly been talking about routing email sent to your domain, you can do all that same stuff with email sent from your domain!)

Google's docs for routing are accurate and in-depth, though it gets a bit complicated. I encourage you to read through the details for all the options there, but the key concepts are:

Inbound vs. Outbound: You can apply a routing rule to messages coming in, going out, or both!
Envelope filter: "envelope" just means "address". You can filter (limit) which addresses are affected by a rule.
Envelope recipient: The main way you re-route email is by modifying the recipient!

Sample options for a catch-all address

✅ Inbound (applies to email being sent to you)
Headers > ✅ Add X-Gm-Original-To
Subject > Prepend Custom Subject, with something like "[rerouted]". I use this so that I can tell which emails in my inbox came from the catch-all route.
Envelope Recipient > ✅ Change envelope recipient. This is how you make it so that the fake address gets changed to a real one. You've got some options. You'll probably want to set it to recipient-address@existing-domain or fully custom recipient-address@some-other-domain. Whichever address goes to the actual inbox you want to receive this stuff.
Show Options > Account Types > ✅ Unrecognized/Catch-all (this is at the very bottom of the form) to only apply this rule to Unrecognized Addresses.

🔗 Aliased Domains

Admin > Account > Domains > Manage Domains

Finally, the simplest kind of routing in Workspace is to alias an entire domain to your primary domain, causing all emails sent to any of those domains to route to the primary one. So this-username@onedomain.tld will get routed as if it's this-username@someotherdomain.tld.

This feature is what allows you to centralize any number of domains under one Workspace, giving you tons of flexibility and address future-proofing. If you buy a new domain tomorrow and want to start using it to replace your old address, you just can! And you can route all messages sent to it right to your existing inbox. Your old addresses will keep on working, so you don't have to worry about lost connections.

Plus, you can do all that same routing stuff we talked about earlier, but can apply different rules per domain!

A big functional difference compared to using mapping/routing is that the addressee ("envelope recipient") is not rewritten. If you look at the "to" field for incoming, aliased addresses, you'll see that the originally-targeted domain is still there. Even if your inbox uses a different domain!

This is a huge benefit over the other forms of routing, since the others lose information about the original intended recipient address (you can use the Add Subject Prefix and/or Add X-Gm-Original-To header options to mitigate that).

Note that Workspace gives you two ways to centralize domains:

As Aliases (as described in this section).
As Secondary Domains, which do not work out of the box as aliases. You can still use custom routes to map addresses from one domain onto another, but they won't be true aliases (the envelope recipient will get rewritten). This option is intended for use cases where you have different sets of users for each domain.

To add your other domains you'll have to first prove ownership with a DNS Text record, and then allow email routing with a DNS MX record. This is also very easy if you're using Google Domains. In any case, you'll see help links and prompts that should get you all squared away.

💡 Tip: The "Manage Domains" Workspace UI is another confusing one: try hovering over the rows to have the things you can do pop up as buttons.

🎉 Other fun things you can do with Google Workspace

This article is all about Gmail, but I'd be remiss if I didn't point out some other cool things you can do, ever so briefly:

Replace the Google logo with a custom one: Admin > Account > Account Settings > Personalization
Get the newest Workspace features quickly: Admin > Account > Account Settings > Preferences > New Features > Rapid Release
Customize Google Apps URLs: Admin > Account > Account Settings > Custom URLs

Please share!

Are you doing anything cool with email routing? Do you have any hard-won lessons? If you're using a service besides Google Workspace, how do you like it? What about pro tips for migrating data?

Live Dev: How to set up a Node project with Typescript

Adam Coster — Fri, 14 May 2021 20:01:24 +0000

Learning how to code is a long process. Structured coursework and tutorials are a great way for most people to learn, especially at the beginning, but at some point you start to wonder... what does the actual work of development look like? How do you go from identifying a problem to a deployable project that solves that problem?

I had a (relatively) small side project I wanted to tackle for fun, and decided to do it live on video so that people could see the whole process from start to finish.

If you:

Love the Node/JavaScript/Typescript ecosystem and want to get real deep into how it all works together; or
Are having trouble using your programming knowledge to build full solutions; or
Are actively developing your fluency in programming in general or JavaScript/Node/Typescript in specific; or
Enjoy the calming background sounds of someone talking about nerd stuff

... then this will be right up your alley!

In Episode 1 I go through the process of setting up a new Node + Typescript project and present the problem being solved: converting Git logs into changelogs/patchnotes.

If you want to joint the next one live to ask questions or tell me when I'm wrong, subscribe on Twitch and/or follow me on Twitter.

How Notion's URLs change without breaking your links

Adam Coster — Wed, 05 May 2021 13:39:42 +0000

I noticed something a while ago while updating the title of a page on one of my favorite webapps (Notion): the URL changed to reflect the new title.

My first thought was, "Well that's dumb, now my links to this page will be broken!" But then my links weren't broken, so I did some experiments with Notion URLs to figure out what kind of sorcery they were doing. Turns out it was simple!

For some background: a URL is supposed to be a unique, unchanging identifier for something on the Internet. When you write, say, a blog post, you'll typically end up with a URL that uses a weird "URL-friendly" version of that title (e.g. "What I ate this morning!" becomes "example.com/post/what-i-ate-this-morning"). Once you've published, most blogging software locks your URL. Why? Because if you change it you'll break any links to that page. That's terrible for people on the Internet clicking those links and terrible for your SEO.

This leads to the annoying scenario where the title shown in the URL deviates from that in the actual post, because that first title was a draft title that got locked in.

That's not too big of a deal, since blog posts and other old-school web pages don't change that much. Even when the title does change, the content is probably largely the same. So the old title (and thus URL) are still accurate enough.

But then you have something like Notion, where content is extremely dynamic by design. In this case, the title of a page could change all the time, and the content may eventually deviate completely from the initial version.

Notion has done something clever here. They have the URL always reflect the title. If you're a Notion user you can see it happen. Live! Just start editing the title and watch your URL...

But if they change the URL when you change the title, doesn't that break all the links to that page?

Nope! Each URL ends with a unique identifier (e.g. notion.so/me/What-I-Ate-This-Morning-2345234523452345). That unique suffix stays constant while the other part changes.

What Notion is doing here is ignoring the part before the identifier when a request hits their server. It uses the never-changing identifier to find the page in question, then gets the current name of that page and computes what the URL should be with the current title of that page. If that's different from where you started, it redirects you (the redirect doesn't require re-rendering, since Notion is a Single Page App, so you'd only notice this if you watch the URL bar in your browser).

You can see this in action yourself on Notion:

Create a page
Set the title
Grab the URL (which should have your current title in it)
Delete the title part of the URL (keeping the identifier)
Paste that into a new browser tab and hit ENTER.
Watch the page and the browser's URL bar

In fact, you can put any title in front of that identifier, whether it was ever a title of your page or not, and you'll still end up in the right place.

But... SEO!

While being able to change the link at any time without breaking existing links is super neat, the immediate next question should be what happens to SEO for that page. Each distinct link will be treated as a different page by search engines, so every time the URL changes for that page you'll be diluting your SEO juice.

Fortunately, there's a mechanism you can use to keep those SEO juices flowing: set the canonical URL to the unchanging version of that URL (the one with just the ID).

The "Canonical URL" is a piece of metadata you can add to a page that tells the browser (and search engines), "Hey, no matter how you got to this page, the following URL is the real URL for this content."

You can stick this right into the head of an HTML document with a link element:

<link rel="canonical" href="https://example.com/my-full-unchanging-url"/>

Alternatively, the server can add it to the Link HTTP response header:

Link: <https://example.com/my-full-unchanging-url>; rel="canonical"

Unfortunately, Notion does not use canonical URL metadata on its pages, so this neat feature probably has a bad impact on SEO. People probably don't host a lot of public content that needs good SEO, so maybe that's no big deal.

Notion has made one mistake here, though: they don't set the Canonical URL in either the HTML head (via the <link rel='canonical'/> tag) or the HTTP response headers (via the Link header). They should have done this using the ID-only (no title) URL. Without that, search engines will think they're seeing a different page every time they follow a different URL to get there (blowing up your SEO).

For sites with less content (personal blogs and the like) you could have very short identifiers, allowing you to use this trick without hideous URLs!

I'll probably be using this in my own projects, can you think of any use cases for it?

😴 Laziness isn't real

Adam Coster — Mon, 26 Apr 2021 13:02:58 +0000

For most of my life, I was held up as a model of Puritanical work ethic through all jillion years of my schooling (K-PhD). I was the shining example of not being "lazy".

Right?

The reality was that I had enough privilege and luck to be able to design my life such that my own particular brand of "laziness" had a minimum negative impact:

I had ADHD (but didn't know it), and the hyperfocus aspect made it super easy to do things that I found interesting.
I found ways to structure my life and goals so that I could neglect the stuff I wasn't interested in.
My interests aligned with those that American society deems Important and Industrious (math, science, etc). If I wasn't interested in these things I probably wouldn't have been able to focus on them.
My forgetfulness and social detachment were easily classified by everyone else as fitting the "Absent-Minded Professor" archetype. Consequently, I was granted permission out of the gate for many things that would be seen as "laziness" in others.
I didn't have to surmount constant, systemic oppression. People believed in me by default, and I didn't have to expend enormous energy just getting to the table. I was already at the table, with all my energy, ready to work.

This list could keep going and going, but that's probably enough to make the point that "industriousness" -- and therefore its opposite, "laziness" -- are context-dependent properties that societally we treat as context-independent.

I've been thinking about this a lot recently, mostly triggered by learning that I had ADHD while also realizing I had developed a mild depression over the past couple of years.

The depression was raising all of my energy thresholds. The thing about depression is that it makes you disinterested in things. The thing about hyperfocus is that it requires your interest. So depression gave me all the downsides of ADHD without the upside.

(Note that hyperfocus is only an upside in the right context. It is often a huge downside because people with ADHD can't choose what they hyperfocus on.)

For the first time in my life, I was having trouble doing things that I wanted to do. This lead to the question... was I being lazy? I certainly thought that I was, which didn't help matters any since that added guilt on top of all the rest. It wasn't until realizing I was depressed that I stepped back to think about the context of laziness.

Fortunately, there are more knowledgeable people than me who have already been thinking about this for years and years. Go read "Laziness Does Not Exist" by Devon Price (which was recently turned into a book on the topic). I hadn't seen this until a few weeks ago, which reminded me that I wanted to do some writing on the topic.

Devon focuses that article on how mental health and privilege intersect with societal ideas of academic laziness. This is a super important aspect of the problem, which I defer to Devon and others and recommend that you go explore.

What I want to do here is take a step back to ask what we even mean when we think of ourselves or others as being "lazy".

My bet is that if you try to define it you'll end up with something like:

Laziness: Avoiding doing something you're supposed to be doing, due to a character flaw.

We could spend hours fine-tuning that definition, but I think you'll have a hard time finding one that others would find accurate that also means something substantively different.

We define laziness relative to "supposed to", and use it as a moralistic judgment about a person's fundamental being.

This whole "supposed to" thing so often goes unexplored. Why are you supposed to be doing these things in the first place? Why are you supposed to do them in that way? Why aren't you supposed to be doing these other things?

Who gets to decide?

I think that the message of the "Laziness Doesn't Exist" article is exactly right, which is that the moralistic idea of laziness is toxic, anti-human nonsense.

When the word "lazy" comes up, treat it as the discovery of a structural problem. By identifying someone (including yourself) as lazy, what you've actually done is uncovered a conflict between the context of reality (how things are) and the context of your imagination (how things "should be"). That means it's time to dig deep and find out what's really going on.

If you find yourself thinking that you are being lazy, or that someone else is, here are some questions to help you guide you to a more humanistic and empathic interpretation and hopefully reveal your next steps:

Why this task?
Why this person?
Why might this task be particularly hard for this person?

Example time!

In college I spent most of my hours glued to science textbooks and doing related coursework and lab work. I never turned something in late, and usually had it done well in advance.

For my non-science courses, on the other hand... I put in the least effort possible and delayed all tasks until the last possible moment. I was, in a word, "lazy." But... why?

Why this task? I took those courses because they were required to meet graduation requirements. I would have rather been in the lab, in another science class, or really anywhere else. Someone else decided this was important but didn't convince me of that.
Why this person? My school required that everyone take courses across a variety of disciplines. I had to because everyone did. It had nothing to do with my interests or goals.
Why is it so hard? Turns out I had undiagnosed ADHD. The hyperfocus aspect allowed me to excel at the stuff I was interested in. The other aspects made it hard to do anything else. It has always been important to me to know why, and "because!" has always been an answer that chaffed. Probably because the why is what allowed me to become interested, which in turn allowed me to excel. For a person like me, something I don't see the importance of is something I can't pay attention to.

Importantly, the fact that laziness is the outcome of a structural problem doesn't necessarily let you off the hook for not doing something. But taking out the nonsense moralistic component to get to the why allows uncovering the responsible structural issues so that they can be addressed where possible, and without blame. Where you can't address them directly, explicit awareness still lets you work on management and coping strategies.

The next time you start getting down on yourself about being "lazy", remember that there's no such thing and then dig deep.

A version of this article originally appeared in the DevChat Newsletter.