DEV Community: Maximilian

How to solve the 2 crystal balls problem in JS

Maximilian — Wed, 13 Mar 2024 21:53:05 +0000

The 2 crystal balls (or the 2 balls, or the 2 glass balls, or the 2 eggs, or the 2 fragile things) problem is a maths challenge that goes like this:

Given I have two crystal balls and a building with n number of floors, find out the maximum height from which a ball can be dropped without breaking.

In other words, find the first floor that a ball will smash if dropped from.

Let's define our assumptions and constraints

The balls' integrity does not diminish when dropped
Don't break both balls before you've found out which floor will break them
The floors will be represented by a boolean array, where false represents a floor that won't smash our ball and true represents a floor that will smash our ball. E.g. [false, false, false, false, true, true, true, true].

Our plan

If we only had 1 ball, we would only have one option: we'd have to drop the ball from each floor until it smashed. Doing it any other way to improve our running time would mean the ball would smash before we could determine the exact floor that would smash it. Doing it in this way would be linear time, or O(n).

Because we have 2 balls (cough), we can get a much better running time. Our plan is to use the first ball to walk the array in chunks and if our ball smashes, then we use our 2nd ball to walk the array from the start of the previous chunk to our known breakage point. The 'chunk' unit we'll use (I'll explain why at the end) is the sqrt(n) where n is the length of our array.

So, our plan is to walk the array with our first ball by sqrt(n), and if our ball breaks, we go back by sqrt(n) and walk the array with our 2nd ball linearly until that one breaks.

Let's explain with some pseudo code...

Some psuedo code

jumpAmount = square root of array length

i = jumpAmount

loop from i while i < array.length
    if array[i] = smash
        exit loop
    increment i by jumpAmount

decrement i by jumpAmount

loop from 0 to jumpAmount
    if (array[i]) = smash
        return i

return no smash

Real Code

// function takes in a boolean array and returns the
// index at which a ball will smash, or -1
function twoCrystalBalls(floors: boolean[]): number {

    // define our jump / chunk amount as the square root
    // of the array length
    const jumpAmount = Math.floor(Math.sqrt(floors.length));

    // loop through array starting with our jumpAmount
    // and increment by our jumpAmount
    let i = jumpAmount;
    for (; i < floors.length; i += jumpAmount) {

        // if we find a floor where the ball smashes
        // break out the loop
        if (floors[i]) break;
    }

    // decrement our iterator by our jumpAmount
    i -= jumpAmount;

    // walk the array linearly from i
    // a maximum of [jumpAmount] times
    for (let j = 0; j <= jumpAmount && i < floors.length; ++j, ++i) {
        if floors[i] {
            // if we find the floor that breaks our ball
            // return the index
            return i;
        }
    }

    // the ball didn't smash so return -1
    return -1;
}

Time complexity

The biggest question I had with this algorithm was, 'why divide the array by its square root, why not divide in half, or by 10?'. WELL... it's because, if we divided by a fraction, for example 0.25, then our time complexity would be O(1/2n), in which case, we drop our constants so our running time would be O(n) which takes us back to linear time. By dividing the array by its square root, our time complexity becomes O(sqrt(n)). Neat huh? 🤓

The Quick Sort Algorithm

Maximilian — Tue, 12 Mar 2024 22:47:39 +0000

Intro

I recently took a data structures and algorithms course as part of my pursuit to learn lower level programming and computer science concepts. As the saying goes, the best way of solidifying your knowledge in something is to teach it, and since I've already filled my quota for boring my partner with coding talk for the month (probably the year), I thought I'd write a series of posts with some of the things I've learned. This is one such post.

What is the Quick Sort Algorithm?

The Quick Sort algorithm is another example of a 'divide and conquer' algorithm that takes an un-sorted array and, you guessed it, sorts it! It works by selecting a pivot (usually the last element in the array) and then adding the remaining elements in the array to 2 sub-arrays - 1 that will contain all the elements smaller than the pivot, and the other that will contain the elements greater than, or equal to, the pivot. We then recurse this step on the sub arrays until the entire array is sorted. This algorithm has an average time complexity of O(nlogn) but its worst case is O(n^2) when the array is already sorted but in reverse order.

How does it work

To explain how it works, here's some pseudo-code:

pivot = arr.length -1
smallerItems = []
largerItems = []

for element in array to sort:
    if element < pivot
        add to smallerItems
    else
        add to largerItems

sort(smallerItems)
sort(largerItems)

How do we implement it

Here's a heavily commented implementation in Typescript:

function quickSort(arr: number[]): number[] {
    // base case
    if (arr.length <= 1) {
        return arr;
    }

    // select our pivot -- the last element in the array
    const pivot = arr[arr.length - 1];

    // create 2 sub-arrays
    const leftArr: number[] = [];
    const rightArr: number[] = [];

    // loop through the array
    for (let i = 0; i < arr.length - 1; i++) {
        // if the current item is less than the value of our pivot
        // add the item to our leftArr, otherwise, add it to our
        // rightArr
        if (arr[i] < pivot) {
            leftArr.push(arr[i]);
        } else {
            rightArr.push(arr[i]);
        }
    }

    // repeat/recurse this process for our sub arrays
    return [...quickSort(leftArr), pivot, ...quickSort(rightArr)];
}

An alternative implementation

Whilst the above implementation works, it would be pretty costly when it comes to memory as the input array scales. An alternative way of writing this function, is to split it up into 3 parts and sort our array inline without the need to create any additional arrays. Let's see what this looks like...

First, let's define our interface:

function quickSort(arr: number[]): void {}

Next, we'll create a function that gets our pivot index and handles our recursion.

function sort(arr: number[], lo: number, hi: number): void {
    if (lo >= hi) {
        return;
    }

    const pivotIndex = partition(arr, lo, hi);
    sort(arr, lo, pivotIndex - 1);
    sort(arr, pivotIndex + 1, hi);
}

function quickSort(arr: number[]): void {
    sort(arr, 0, arr.length - 1);
}

Finally, we'll create our third function which manages 3 pointers - our pivot, our index (which starts at -1), and the item we're currently checking (current). Current walks through the array, and if the item is less than or equal to our pivot, we increment the index and swap the items located at index and current. At the end of our iteration, we increment our index and swap the items at our index and pivot, then we return the index. Here's what that function looks like:

function partition(arr: number[], lo: number, hi: number): number {
    // set our pivot to the end of the array
    const pivot = arr[hi];

    // set out index to the start of the array -1
    let idx = lo - 1;

    // loop through our array
    for (let i = lo; i < hi; ++i) {
        // if the item is less than or equal to the pivot
        // increment the index, then swap it with the item
        // at the index
        if (arr[i] <= pivot) {
            idx++;
            [arr[i], arr[idx]] = [arr[idx], arr[i]];
        }
    }

    // after we've finished walking the array,
    // increment the index, then swap the item
    // at the pivot with the item at the index
    idx++;
    [arr[hi], arr[idx]] = [arr[idx], arr[hi]];

    // return the index
    return idx;
}

Ok, now let's put it all together:

function partition(arr: number[], lo: number, hi: number): number {
    const pivot = arr[hi];
    let idx = lo - 1;

    for (let i = lo; i < hi; ++i) {
        if (arr[i] <= pivot) {
            idx++;
            [arr[i], arr[idx]] = [arr[idx], arr[i]];
        }
    }
    idx++;
    [arr[hi], arr[idx]] = [arr[idx], arr[hi]];

    return idx;
}

function sort(arr: number[], lo: number, hi: number): void {
    if (lo >= hi) {
        return;
    }

    const pivotIndex = partition(arr, lo, hi);
    sort(arr, lo, pivotIndex - 1);
    sort(arr, pivotIndex + 1, hi);
}

function quickSort(arr: number[]): void {
    sort(arr, 0, arr.length - 1);
}

The Binary Search Algorithm

Maximilian — Mon, 11 Mar 2024 22:11:20 +0000

Intro

What is the Binary Search Algorithm

The binary search algorithm is an efficient way of returning the index of a given value in a sorted array. It has a worst-case time complexity of O(log n) where the element does not exist and a best-case time complexity of O(1) where the value we're searching for happens to be the first one we check!

How does it work

The idea behind the binary search algorithm is to divide and conquer. First, we check the mid point in the array. If the mid point is equal to the value we're looking for, we return the value. Otherwise, we divide the array in half and continue to look in the appropriate half and repeat the process. Remember, this only works on a sorted array.

Let's write some pseudo-code where v = the value we're looking for to further this explanation:

lo = 0
hi = arr.length
mid = (lo + hi) / 2

while lo < hi
    if mid = v
        we found it!

    if mid > v
        hi = mid

    if mid < v
        lo = mid

we didn't find it :(

Here we have 3 pointers; lo (set to the 1st element of the array), mid (set to the middle point of the array) and hi (set to the length of the array). We check the value of mid; if mid is the value we're looking for, then we're done! If mid is greater than our value, this means that the value we're looking for (if it exists) will be in the left half of our array. At this point, we move our hi pointer down to where our mid pointer is and we start the process again. If, however, mid is less than our value, then we move our lo pointer up to where our mid pointer is, and we start the process again.

How do we implement it

Here's some heavily commented Typescript:

function binarySearch(
  haystack: number[], 
  needle: number
): number {
    let lo = 0;
    let hi = haystack.length;

    do {
        const mid = Math.floor((hi + lo) / 2);
        const v = haystack[mid];

        if (v === needle) {
            return mid;
        } else if (v > needle) {
            hi = mid;
        } else {
            lo = mid + 1; // we can exclude mid as we've already checked it
        }
    } while (lo < hi);

    return -1;
}

This implementation obviously uses a do-while loop, but you can get the job done with any kind of loop, really. Alternatively, this is a good place to use recursion. Here's what a recursive version might look like:

function binarySearchRecursive(
    haystack: number[],
    needle: number,
    lo: number = 0,
    hi: number = haystack.length
): number {
    // Base case
    if (lo > hi) {
        return -1;
    }

    const mid = Math.floor((lo + hi) / 2);

    if (haystack[mid] === needle) {
        return mid; // We found it!
    }

    if (haystack[mid] < needle) {
        // recurse - search in the right half
        return binarySearchRecursive(haystack, needle, mid + 1, hi); 
    }

    if (haystack[mid] > needle) {
        // recurse - search in the left half
        return binarySearchRecursive(haystack, needle, lo, mid - 1); 
    }
}

I think we're done here

That's it for our binary search algorithm. I hope you found it useful or, at least, interesting!

The Bubble Sort Algorithm

Maximilian — Tue, 05 Mar 2024 22:46:27 +0000

Intro

What is the Bubble Sort Algorithm

The Bubble Sort algorithm is typically applied where simplicity is favoured over efficiency. It has a time complexity of O(n^2) which is less than ideal when dealing with larger lists.

How does it work

To explain how it works, here's some pseudo-code:

for pointer1 in list
    for pointer2 in list
        if list[j] > list[j + 1]
            swap list[j] and list[j + 1]
        pointer2++

    pointer1++

The idea of this algorithm is to 'bubble up' the larger items in the list to the end. We can think of this with 2 pointers, pointer1 and pointer2. We start by setting both pointers to the first item in the list. We then go through the entire list with pointer2 and swap the item at pointer2's position if the item at position pointer2 + 1 is greater in value. We keep repeating this process until we reach the end of the list. We then move pointer1 up by 1 and we start the process again with pointer2.

You may have noticed that we need a way of avoiding checking items that we have already sorted. Seeing as the aim of each iteration is to move the larger items to the end of the list, we can do this by limiting the boundary of pointer2 by the value of pointer1 + 1.

How do we implement it

Here's a heavily commented implementation in Typescript:

function bubbleSort(arr: number[]): void {
    // pointer 1
    for (let i = 0; i < arr.length; ++i) { 
       // pointer 2 + limit iterations by pointer1 + 1
        for (let j = 0; j < arr.length - 1 - i; ++j) {
            // check if current item is larger than the next item
            if (arr[j] > arr[j + 1]) {
                // temporarily store the current item so we don't lose it
                const tmp = arr[j];
                // swap the items in the array
                arr[j] = arr[j + 1];
                arr[j + 1] = tmp;
            }
        }
    }
}

Sorted

This isn't a likely go-to algorithm in most cases, but I think it's a nice and easy one to get started. I hope you found it useful or, at least, a little bit interesting!

How To Mock MongoDB's ObjectID

Maximilian — Sat, 28 Oct 2023 01:58:37 +0000

TLDR; the 2nd code block is how you do it.

I updated a service method such that it created a Mongo ID before sending it to the DB instead of relying on the DB to do it for me. There are a few reasons why you might want to do this, in my case, it was because I was linking 2 things together, so instead of running 2 mutations (the 1st to create the linked item and get back the created ID, and the 2nd to add the new ID to the item I was linking from), I created the new IDs in my method, linked the 2 together and sent them off on their merry way in one happy mutation.

I needed to write a unit test that ensured the created ID was correctly added as the ID on the linked item as well as the item it was linked from, but without mocking, we would have no way of knowing what ID would be created and so could not check that the correct ID existed in the right places.

Simplified, here's the code we're testing:

import { ObjectId } from 'mongodb';

type LinkedItem = { itemId: string };

type Item {
  id?: string; // MongoId
  newItemId: string; // temp ID used to identify new items that don't yet have an ID
  contents: Whatever;
  linkedItems?: Array<LinkedItem>;
}

const saveItemsToDb = (inputs: Array<Item>): Promise<void> {
  // Create Mongo ID if one doesn't exist
  const inputsWithIds = inputs.map(input => 
    input.id 
    ? input 
    : { ...input, id: new ObjectId().toString() }

  // Update the linked items to include the new IDs
  const inputsWithUpdatedLinks = inputsWithIds.map(input => {
    if (!input.linkedItems || !input.linkedItems.length) {
      return input;
    }

    const updatedLinkedItems: Array<LinkedItem> = [];

    input.linkedItems.forEach(linkedItemInput => {
      const linkedItem = inputsWithIds.find(
        inputWithId =>
          inputWithId.id === linkedItemInput.itemId ||
          inputWithId.newItemId === linkedItemInput.itemId
      );

      assert(linkedItem);
      assert(linkedItem.id);

      updatedLinkedItems.push({ itemId: linkedItem.id });
    })

    // Return the input with the updated links
    return {
      ...input,
      linkedItems: updatedLinkedItems,
    }
  })

  // Save the updated inputs to the DB
  await db.save(inputsWithUpdatedLinks)
}

And here's a simplified section of the unit tests with the mocks:

import { ObjectId } from 'mongodb';
import { 
  NEW_ID_1, 
  NEW_ID_2,  
  mockItemsToSave,
  mockSavedItems
} from './testData';

describe('Item service', () => {
  beforeEach(() => {
    jest
      .spyOn(ObjectId.prototype, 'toString')
      .mockReturnValueOnce(NEW_ID_1)
      .mockReturnValueOnce(NEW_ID_2);
  })

  afterEach(() => {
    jest.restoreAllMocks();
  })

  it('links new items', async () => {
    await saveItemsToDb(mockItemsToSave);
    expect(mockDb.save).toHaveBeenCalledWith(mockSavedItems);
  });
});

By using jest.spyOn() we can watch for when ObjectId().toString() gets called and override its return value so that we're able to test what happens with our data. By using a chain of .mockReturnValueOnce() methods, we're also able to dictate what the method returns when it's called multiple times. For example, in the context of our code, if we're adding multiple new items that all link to each other, we can easily check that the right item links are being set across multiple new items.

It's also important to note that we're calling jest.spyOn() from within the beforeEach callback and as such, we need to call jest.restoreAllMocks() from our afterEach callback function so that our mock return values (NEW_ID_1 and NEW_ID_2) get reset each time we run a test.

Hope that helps! I'm also leaving this here to help me when I inevitably forget how I did this in the future.

Setting up your web development machine

Maximilian — Tue, 28 Feb 2023 00:04:08 +0000

Learning how to code is easily the best thing I've done with my time as an adult, but navigating the quagmire of tutorials and courses is a big challenge. I was fortunate enough to figure out how to learn completely for free, at least well enough to land my first role, but it wasn't without its pitfalls. With the benefit of hindsight and work experience, I thought I'd collate some free resources that I've found helpful, starting with some web fundamentals and setting up your development machine.

An overview

Before we get started, I watched a video recently that I thought was great both for motivation and a reality check which I recommend watching.

Mistakes That Prevent Self-taught Developers From Landing A Job by DonTheDeveloper.

How the web works

In order to learn how to code properly, you must first understand how the internet works, what programming languages are, the different programming languages that exist and why they exist.

Throughout your coding journey, I recommend keeping the MDN documentation bookmarked and accessible at all times. This is one of the best resources for learning about all things web development and will prove extremely useful as a reference manual as you progress through your learning and career. Whilst one can learn how to code on this platform alone (the same goes for other resources I will reference later), the point of this guide is to cherry pick what I think are the best parts of all the available free tools and resources. I also recommend bookmarking W3 Schools for similar reasons!

The first resource I’d like to recommend is the free online course provided by Harvard University called CS50’s Introduction to Computer Science. If you would like to go through this entire course, it wouldn't be a waste of time, however, I only recommend watching the first lecture; Lecture 0 which passionately talks about what code actually is.

The second resource is the following MDN articles: How Does The Internet Work? and How The Web Works.

Once you’ve completed Lecture 0 and read the 2 MDN articles, you’ll hopefully be excited to get into some coding and have some solid foundational knowledge to better prepare you for your learning journey.

Some Terminology

There is no shortage of terms and acronyms in the world of software development and there are lots that you will need to learn - but don’t worry, they’re not as complicated as they sound. Here are a few definitions to hopefully make the set up process a bit easier.

Web Browser

You should already be familiar with web browsers, but they are one of the main tools for a web developer and are more capable than you may realise. Examples of web browsers include Google Chrome, Mozilla Firefox, Microsoft Edge and Apple's Safari.

Code Editor / Text Editor

Whilst we can use any text editor to write code (yes, even Notepad or TextEdit), it makes life extremely difficult. A code editor is basically a text editor with extra features like language specific syntax and error highlighting, multi-line editing (where you type and edit multiple lines at once and feel like Neo), the ability to easily click through to other areas of your codebase and more. A code editor is the place where we write and edit all of our code.

Terminal

A terminal provides a Command Line Interface (CLI) to interact with your computer. Although it may seem archaic at first when compared to a GUI (Graphical User Interface), once you get used to it, you’ll realise how quick and powerful it can be. Using the terminal, we can run our code, browse, create, edit and delete files, run programs, install and manage applications and utilities and much, much more. Typically speaking, a software developer will live between their code editor and terminal. The code editor to write and edit code, and the terminal to execute commands.

Bash

Bash is the CLI language that we use in a Unix terminal. It actually stands for ‘Bourne-Again-SHell’. You can find out more about it here.

Zsh

An alternative to Bash. This is now the default shell shipped with MacOS. You can read more about it here.

IDE

An IDE (or Integrated Development Environment) is the software that developers use to build software. At its core, it will provide a platform which integrates a code editor with a terminal, but also provides and integrates other tools to make development easier. Arguably, the most widely used IDE is Microsoft’s VSCode but another popular suite of developer tools and IDEs comes from JetBrains.

NB: There is an argument about whether or not VSCode qualifies as an IDE, but my opinion is this: at its core, VSCode is a text editor with a built-in terminal panel but through the use of extensions, we can get an IDE-like experience without the need to learn and buy different tools when we use different languages. My advice would be to spend less time worrying about whether VSCode is an IDE, and more time learning and trying different tools until you find something you like!

Package Manager

A package manager installs, updates, configures, and removes software packages. In the context of a package manager in the terminal, you can think of it like a CLI App Store where you can search for, download, install and uninstall open source apps (yes, even desktop ones with GUIs!). Popular package managers are Homebrew for the Mac and Chocolatey for Windows.

Setting up your development environment

You don’t need a powerful or new computer and you can run MacOS, Windows or Linux, but if you want to develop for iOS or the Mac, you will need a Mac. If you’re worried about the age or spec of your computer, you should only run into hardware limitations once you start working on larger projects, or need to use virtual machines and emulators. Until then, most computers are up to the task.

If you’re looking to buy a new machine and you want a hard and fast recommendation, I recommend any Apple Silicon Mac. In my experience, this will provide you with the most flexible and powerful platform for most development tasks, but do not let this be a barrier to entry as it’s by no means a requirement for learning and for working!

Now let's get set up.

MacOS

Terminal

MacOS already ships with a built-in terminal, aptly named ‘Terminal’. You can launch it by searching in Spotlight, searching in Launchpad, opening the ‘Other’ folder in Launchpad, or by going to the ‘Utilities’ folder in your Applications folder. Whilst the built-in terminal on MacOS is more than sufficient, I recommend downloading and installing iTerm 2 which will give you really useful features that will make your life easier.

Next, I recommend installing a package manager called Homebrew, arguably the most popular package manager for the Mac. You should read the documentation and play around with it to get a feel for using the terminal and a package manager, however you will learn these things as we go. If you would like to learn more about using Homebrew, I think the first ~4minutes of this video is great.

Windows

Terminal - WSL

Earlier I mentioned that we could configure Windows to use something called WSL (Windows Subsystem Linux). WSL allows you to install a Linux distribution, such as Ubuntu, Debian, or Kali, as an application in Windows. Once installed, you can open a Linux terminal and run commands, scripts, and tools as if you were running them on a native Linux system.

In order to install WSL, follow the official install guide and make sure it’s configured to WSL 2 and Ubuntu. Once installed, you will be able to open Ubuntu like an application which will open an Ubuntu terminal.

Terminal - PowerShell

As an alternative to WSL, you can use Windows’ built in terminal, PowerShell which you can search for and launch from the Start menu. I recommend installing the package manager Chocolatey, reading the docs and having a play around to get a feel for using it.

As to why you would choose the hassle of getting WSL set up; PowerShell doesn’t have Bash or Zsh, which the majority of the developer community uses due to nearly all servers running Linux and MacOS being ‘Unix-like’.

Terminal - GitBash

A great half way house (and my recommendation if you're not going to set up WSL) is GitBash. GitBash brings bash-like commands to Windows, and is what I personally use on my Windows machine, as I find it a good compromise between performance and a Bash-like experience.

MacOS & Windows

VSCode

I recommend using VSCode as your IDE as it’s currently amongst the most used, fully-featured and it's FREE! You’ll also find that a lot of online tutors will use VSCode which will make it easier to follow along. To install it, follow these instructions.

If you’re using WSL, I recommend running VSCode inside WSL which you can do by following these instructions.

Sublime Text

Alongside VSCode, I also recommend installing Sublime Text. Whilst there is an overlap in functionality between the two, I find that having a no nonsense code editor on hand can be beneficial as a scratch pad, or to quickly open files that you may want to access outside your current project.

Web Browser

I recommend Google Chrome for development, but it’s also important that you have multiple browsers installed to test web apps in browsers that your users are likely going to use, like Safari or Microsoft Edge. I personally use Firefox as my main browser and Chrome or Firefox Developer Edition for developing on as, in my opinion, they have certain superior features.

Click here to install Firefox or here to install Firefox Developer Edition (or both if you’d like!).

Node.js

Put simply, Node.js enables you to run JavaScript on your computer (or a server). It's a more advanced subject which we'll go through later, but for now, just know that you should install it! To install it, you can use your package manager, or download the installer.

Git

Git is the standard version control system (VCS) for software developers. It’s an essential tool to learn and will be one of (if not the) most used tool in your arsenal. There are 2 ways to use Git - the first is via the terminal and the second is via a GUI. Learning how to use Git via the terminal is an essential skill, so you should get used to that first. However, GUI Git clients can be useful to visualise a repository, especially when you get stuck. It can also be handy whilst you’re learning so you can see what’s going on and what your commands are doing!

To install Git, follow this guide written by Atlassian. For MacOS users, I recommend installing Git via the Homebrew package manager. For Windows, I recommend installing GitBash (if you haven’t already done so).

For Git GUI tools, I recommend the VSCode plugin, GitGraph which works conveniently within VSCode alongside the built in Git tools already available. Alternatively, 2 other popular choices are SourceTree and, my personal favourite, GitKraken.

A scratch pad

Throughout your learning (and beyond) you’ll probably want to try some things out to see how they work outside of your codebase. Rather than setting up a whole new project just to try a couple of quick things, there are some apps that allow you to do just that!

For browser based apps, check out and bookmark Code Sandbox and Codepen.

For desktop based apps, check out RunJS and the VSCode extension Quokka.js.

Customisation

Setting up VSCode

Out of the box, VSCode has a lot of features but we can extend and customise its functionality and appearance through the use of ‘extensions’ via the VSCode Marketplace. It’s a good idea to have a play around just to get a feel for it before continuing, but you will become extremely familiar with it as you learn. Check out this video by Tech With Tim to give you a comprehensive beginner’s overview of some of VSCode’s core features and concepts.

The extensions I recommend installing to get you going are:

Live Server which lets you easily run your code in a browser with automatic refresh on save, which means you don’t have to refresh the browser every time you make a change to your code.
Prettier which lets you configure rules to set how your code is formatted, and automatically formats your code on save based on your formatting rules.

My favourite extensions for aesthetics that you may want to check out:

VSCode Icons
GitHub Theme (I'm a long-term user of GitHub Dark Dimmed)
Dracula Theme

Some utility extensions I use:

Sort Lines which lets you sort lines of code in various formats
Todo Tree which lets you add ‘to do’ comments to your code and view them all in one easy to access
Spell Checker - if you only install one utility extension from this list, choose this one.

Of course there are a seemingly limitless amount of extensions available (and more are being added every day), but you will figure out which ones you need as you go. VSCode also does a good job of recommending extensions when you open up different file types. For example, if you were to open a Python script (a .py file) you’d likely get a popup asking if you’d like to install some python specific extensions if you haven't got them installed already.

Customising your terminal (WSL & MacOS)

You can completely customise the look and feel of your terminal, as well as set up your own shortcuts (called aliases) or even your own scripts to get your common tasks completed more efficiently. You’ll get more into this side of things as you go, but if you like customising and tweaking things, I'd have a look at Oh My Zsh and watch this video on how to get it set up.

Now go take over the world by writing your first app!

I switched to Windows

Maximilian — Fri, 24 Feb 2023 16:22:46 +0000

Intro

I started out my computer life, as many do, on Windows, but I switched over to Mac in 2009 and never looked back. A recent project required me to use a Windows machine for development so, as someone who would probably read about this kind of thing, I thought I'd write about this kind of thing.

A bit of background

I used Windows for everything 'back in the day' but I couldn't get it to work for me for music and video production. I switched to Mac and it solved all the problems that I had (and then some). I then learned to code on a Mac and have used Macs exclusively ever since. I hadn't had any dealings with the Windows world for 14 years, even though it felt like 4, so I was looking forward to seeing what was on the other side again especially considering a lot of the tooling I use (VSCode, TypeScript, Github etc.) was made (or bought) by Microsoft and I had heard great things about WSL and that things were 'not like they used to be'. As with everything in life, I approached this change with optimism and enthusiasm (I can hear all those who know me laughing, but I did have an open mind.... honest!).

Hardware

Let's start with the hardware. I switched to a 15" Dell Precision laptop, some £2,300 worth, with a beefy i7, 32gb RAM, 512GB SSD and an nVidia GPU with 4GB VRAM. I will be comparing this with a similarly priced 16" M1 Pro MacBook Pro with 16GB RAM, 1TB SSD, and a 13" M1 MacBook Air, at about half the price, with 16GB RAM and a 512GB SSD.

A quick few points on hardware

The weight of the Dell is really nice and sits between the MBA and MBP. The compromise is it feels a bit plasticy, but still very sturdy.
The Dell's screen has a matte finish to it which helps with glare, but isn't very sharp especially when compared to the MacBooks so coding on it can cause a bit of eye strain.
The Dell's webcam is rubbish but the little cover is nice. The MBP & MBA webcams are miles better.
The Dell has a number pad which is cool for those who want it. I do not. Because of this, the trackpad and keyboard are off centre which makes typing feel weird. The keyboard is nice though.
The ports on the Dell are great - HDMI, USB-A, USB-C, Ethernet etc. The MBP is nearly as good but with the absence of USB-A and ethernet. The MBP is better in as much as you can charge it from any USB-C port. The MBA falls miles short and I should also note that previous generations of MBPs fall miles short too!
The Dell's charger is big, but surprisingly well thought out. I just wish the plug had a charging indicator instead of always being on when connected to mains even when unplugged from the laptop - this caught me out a few times when I thought the laptop was on charge when it wasn't.
The Dell's trackpad is terrible. Not unusable, but very fiddly and awkward. I always connected a mouse to avoid tantrums.
The Dell's speakers would have been better left out. They're that offensive, especially when compared to the speakers in either MacBook.
The Dell's battery life is acceptable but about 5 times worse than either MacBook (and that's being generous under heavy load).
You can't single handedly lift the lid of the Dell laptop without the rest of the laptop coming with it. A small annoying detail.
The Dell gets hot - a problem all Intel/x86 machines suffer with, including (especially) previous generations of MacBooks. This isn't really a complaint, just a reminder of how big a leap Apple Silicon is in this department.

Aesthetics are completely subjective so I won't belabour this point. Personally, I think the Dell looks like a sturdy, no nonsense, professional machine, but I'd take either MacBook over it. To my eyes, there are plenty of PC Laptops on the market that easily rival the MacBooks aesthetically if that's what you're looking for.

Dev environment

I'm currently a full stack TypeScript and React developer. I use Docker containers with local DBs, VSCode running a load of extensions, a terminal used predominantly for Git and Chrome running the standard 2 million tabs with a bunch of dev tool add-ons.

Given that I was coming from a Mac / Unix background and that I had heard so many good things about WSL, I decided to set up my Windows machine running WSL with Ubuntu LTS, with VSCode and Docker in WSL mode too. This worked a treat and although I was still stumbling over some keyboard shortcuts (Cmd = Ctrl and it's over to the left), it was simple enough to set up and I was more than happy working on my new Windows machine.

Favourite Windows features

The built-in clipboard history that can be accessed with Win + V is insanely useful. The Mac needs this feature. The window snapping feature is also great, but you can use Rectangle or Magnet to build this capability into the Mac. I also think that the minimise, maximise and close buttons are easier and clearer to use than Apple's which now includes alt + click functionality which I personally find annoying.

Windows gripes

In my opinion, the Windows UI is garbage. I can't believe I'm still looking at windows from XP and earlier and the whole Settings / Control Panel situation is a mess. I think the whole task tray situation in the bottom right looks a state and it's fairly unclear what each thing does. Switching desktops whilst in Task View is very janky, separate displays can't have separate desktops and you can't re-order your desktops. It still seems like certain areas are designed for touch and others designed for a CRT screen. Files stored on the desktop with long filenames can stop you clicking on files that the filename obstructs, the lack of a proper preview feature (like the ability to hit the space bar on any file on MacOS) is very annoying when trying to find a file and, to top it all off, everything makes a sound. I must've heard that 3-note chime 100,000 times a day. Are you sure you want to delete this email? Da-da-ding! Do you want to empty the recycle bin? Da-da-ding! You need admin rights to modify this file. Da-da-ding! Are you sure you want to close Chrome? Da-da-ding! Unknown keyboard shortcut. Da-da-ding! You get the idea.

I also had endless issues with every bluetooth peripheral including an intermittently stuttering and delayed mouse and keyboard that could only be solved by reinstalling the bluetooth driver (except once where turning the computer off for 10 minutes seemed to fix it). Bluetooth headphones of multiple makes all struggled to stay connected throughout the day so I had to join meetings at least 5 minutes early to ensure they'd connect properly, and even then, the sound quality on calls was abysmal. I also experienced wifi drop outs frequently. Oh, and I couldn't keep other wifi devices near the laptop, or the wifi dropped out completely. I tested this. Move a MacBook closer to the laptop - no internet - move a MacBook away from the laptop - internet!

Lastly, just like my old PC from 14 years ago, a lot of the system audio (like shut down jingle etc.) was crackly.

Development problems

After a couple of hours running my dev environment each day, my whole machine would start getting 'sticky'. Code would start taking much longer to compile, apps would start taking longer to load and UI animations would start glitching. I tried stopping my task runners, rebooting WSL and stopping the Docker container, I tried closing Chrome and Slack and Outlook but it would only get marginally better for a short amount of time before it would start up again. The only solution was to reboot, which I had to do up to 5 times a day. I don't think I need to stress what an interruption to work this can be. Equally, if I ever wanted to join a video call, I couldn't if I had my dev environment running as the machine seemingly couldn't cope, and I could forget about sharing my screen. The second I shared my screen, the whole thing ground to a halt -- not very conducive to a collaborative remote engineering team! I also found that the Windows version of various cross-platform apps that I relied upon day in day out were more buggy, like MongoDB Compass and Slack.

After a few months of persevering, I decided to rebuild my dev environment without WSL as it was clear it was using a large amount of RAM and CPU. I rebuilt my dev environment and used GitBash as my terminal and I have to say, it solved a lot of my problems. GitBash, for the most part, was nice and familiar, albeit without the joys of oh-my-zsh but this wasn't a big deal. The system ran far more smoothly and consistently and I no longer had to reboot 5 times a day, but it still felt sluggish when it came to traversing the codebase, linting and autocomplete. It at least made it more possible to join calls while my environment was running, but sharing my screen was still reaching the limits of this £2,300 computer and it didn't solve my bluetooth or wifi issues.

I have to compare this, I'm afraid, to a MacBook Air costing half as much...

The project that I worked on with this Dell laptop was a React project. It's quite big, but it's still just React. The project I worked on with my MBA was using React Native running iOS and Android emulators simultaneously. I joined calls, shared my screen, connected headphones and to the internet with literally no problems. I had to reboot from time to time when an emulator got out of hand, but this was rare. The MBA also didn't even have a fan (let alone a loud one) and although it got warm, was still cooler than the Dell laptop with the fans screaming at me - something I realised is actually rather annoying to listen to all day!

I switched back to Mac

Needless to say, I went back. I'm sorry to say that the difference, especially with Apple Silicon, is stark. Running the same project on a 14" M1 Pro MBP with 32GB RAM that costs around the same as the Dell is silent, glitch free and extremely responsive, and the bugs that I was putting up with across my everyday applications have disappeared. Virtual desktops are far easier to deal with and more intuitive and is a feature I heavily rely upon. I also found that after switching back to Mac, I realised I was putting up with numerous scaling issues on my external 4k screen. These issues were fairly app-specific so I can't fully blame Microsoft here, but from a user perspective, it does make the overall experience inferior.

I really did give the switch a go with an open mind and I persevered for 9 months, but the fact remains: Windows seems to have the same problems it had for me 14 years ago. I was forever rebooting, rebuilding and tweaking instead of, well... working! In summary, whenever I've used Windows, I've used Windows. Whenever I've used a Mac, I've used my apps.

I'd be interested to hear from any of you who are running Windows to see if you've had similar experiences. Whether you think the problems I had were hardware specific, or whether you've experienced the same or similar problems. I've spoken to other developers and, unfortunately, they've all echoed very similar issues - some of them were just used to it and hadn't really thought about it. It's amazing what you get used to and tolerate over time.

A final note

One thing that genuinely baffles me (and something that I can't believe isn't talked about more) is that if you have your Windows laptop unplugged, you don't get access to the laptop's full performance.

You spend thousands of pounds for a high-performance, portable device, whose performance you can't make use of unless you're ported.

You should get a refund.

Why I always use backticks for strings

Maximilian — Fri, 26 Aug 2022 22:53:48 +0000

A quick one from me

If you're using single quotes or double quotes for your strings and not backticks (template literals), then I'm afraid your sanity may be in question.

What happens when you DON'T use backticks

If you're using single quotes (the worst offender) and your string requires an apostrophe, you're back to using double quotes. Now your code has inconsistent string syntax. The reverse is also true, albeit less common - if you're using double quotes and you need double quotes in your string, you're back to using single quotes. In both of these cases, if you ever need to pass in a variable, you're introducing template literals / backticks - a 3rd string syntax.

What happens when you use backticks

There are 2 main advantages:

Firstly, your codebase's string syntax is consistent. You can pass in apostrophes / single quotes and double quotes without thinking -- a freedom you don't appreciate until it's taken away from you -- and secondly, you are free to pass in variables whenever you like without going back and changing those damned quotation marks!

I wondered if people didn't do this because of some kind of performance advantage where template literals would be slower or more expensive, but I've found nothing that supports this theory, so it begs the question:

Why would you use any other syntax?

How to Offset a Server's Timezone to Store a UTC Datetime

Maximilian — Sun, 10 Apr 2022 10:25:06 +0000

TLDR; the Javascript Date object is nasty. This is how I stored a UTC time in a Postgres database using the Javascript Date object.

Problem

As part of building a microservice responsible for handling bookings between 2 users, each of whom could be in any timezone, and also travel into any timezone between making the booking and fulfilling the booking, I stumbled across a little issue when it came to storing the UTC time.

For this microservice, I'm using Typescript, Koa, Prisma and Postgres. It's out of the scope of this article to talk through the entire infrastructure or, indeed, how the whole microservice works, but here's what the relevant part of my Prisma schema looks like:

model Booking {
  bookingId             String        @id @default(uuid())
  user1Id               String
  user2Id               String
  lengthMins            Int
  startDateTimeUTC      DateTime
  dateTimeCreated       DateTime      @default(now())
  dateTimeUpdated       DateTime?
}

Here's how we could create a new row in our Booking table:

await db.booking.create({
  user1Id: `user_1`,
  user2Id: `user_2`,
  lengthMins: 60,
  startDateTimeUTC: new Date(`2022-04-10T14:00:00`),
});

All we're trying to do here is, using Prisma, store a new row in our Booking table that represents a booking between user_1 and user_2 at 14:00 on 10th April 2022, UTC. When running a function like this (albeit this would exist inside a try/catch block in a re-usable function), I naively expected the time to be stored as... you know.... the time I passed into the Date constructor.

Unfortunately, this didn't happen. What happens is the time gets converted into the server's local time. I was running this code locally in BST (UTC + 1), so, in this example, the time would have been stored as 15:00.

Solution

My solution for the entire system was to store all datetime information in the database in UTC, then use the user's device location (with permission, of course) to convert the stored time into the user's local time.

My solution for this specific issue (without the use of any 3rd party libraries, and without having to store extra information like the server's timezone) was to create a function that used Javascript's Date object to offset the server's timezone before we store it.

This looked like the following:

function getUTCTime(dateTimeString: string): Date {
  const dateTime = new Date(dateTimeString);
  const dateTimeNumber = dateTime.getTime();
  const dateTimeOffset = dateTime.getTimezoneOffset() * 60000;
  const dateTimeUTC = new Date();
  dateTimeUTC.setTime(dateTimeNumber - dateTimeOffset);

  return dateTimeUTC;
}

What we're doing here is:

Creating a new Date object (as we did in the first example) using a UTC datetime string that we can pass into the function.
Getting the datetime in number format (or, the number of milliseconds from midnight on January 1, 1970 to the datetime we pass in).
Using the .getTimezoneOffset() method to get the difference, in minutes, between the date as evaluated in the UTC time zone, and the same date as evaluated in the local time zone. We multiply this value by 60,000 to convert this number into milliseconds to make it more useful to use later.
Creating a new Date object (with new Date()) which will be instantiated with the current datetime.
Assigning the value of our new Date object to the datetime we passed into the function, minus our server's timezone offset (datetimeNumber - dateTimeOffset).
Returning the Date object that we've now successfully set to the dateTimeString we originally passed the function!

Now when we add a row into our table, we can do something like this:

await db.booking.create({
  user1Id: `user_1`,
  user2Id: `user_2`,
  lengthMins: 60,
  startDateTimeUTC: getUTCTime(`2022-04-10T14:00:00`),
});

Peace Out

There are libraries that cater for this kind of thing very well. The main one I use being date-fns, but this is a fairly easy manual solution which solved my problem. This will work anywhere around the world too, as it just offsets the timezone of the server you're running the code on.

As always, I hope this helped you in some way, or you at least found it an interesting read.

I would love to hear any and all timezone puzzles you may have been a part of solving. I've found that timezones are one of those annoying rabbit holes that are really easy to make mistakes with. Fingers crossed, this isn't one of them!

We trialled a 6-week project cycle and here's what happened

Maximilian — Wed, 09 Mar 2022 09:31:02 +0000

At InsideOut we've been managing our workload in a Scrum-based environment where we have 2-week long sprints and 4-sprint long iterations, but we recently switched to a project-based workflow and we've found several benefits.

The old way: Scrum-diddly-umptious

Each developer has a pre-set number of tickets that they aim to complete by the end of each sprint. Each ticket has a number of points that indicates its complexity, not time required to complete. The theory here being that if each ticket were scored by time taken, this would change per developer, whereas a complexity score is immutable. If a non-urgent issue or bug pops up, we add a ticket to the 'backlog' which will be picked up in a future sprint. Equally, tickets that contribute to a larger project (like a new app feature, or a new app entirely) will be written up and added to the backlog. These tickets may be prioritised if the project has a deadline.

Sprint iterations are a collection of 4 sprints so we can assess our longer-term productivity, but also plan and work on longer-term projects. All sprints are managed via a Kanban board.

Ok, so what's the problem?

There wasn't anything inherently wrong with this approach. It worked for us in that we were getting a lot of work done, and the methodology was agile in as much as we were only ever a maximum of 2 weeks away from working on something new. For the wider business, this meant that if a non-critical bug was spotted, or if there was a small feature request that was provably important, we could work it into our next sprint.

There were, however, 4 main problems with this approach.

1. Bigger projects took longer.
Our sprints always comprised of several types of tickets ranging from support, bug fixes and data reports, alongside tickets that contributed to larger projects. This inherently meant we were unable to focus on any single project and our time and attention was always split.

2. Requests came in thick and fast.
Because we were only ever a short amount of time away from new tickets, the wider business was more than happy to add to our workload regularly. For every request that came in, we would have to assess a solution and potentially add a ticket to our backlog, thus splitting our attention even further. Also, if you've ever written up tickets, you'll know that to do it properly requires some thought and time. Hence, we would often end up doing 'quick things' there and then which ended up being unaccounted for and distracted us further still.

3. Meetings.
On top of our daily stand-ups, we had sprint planning meetings every Friday. This involved going through our backlog of tickets and work out what needed to be prioritised in the next sprint. It also involved discussing and pointing each and every ticket. We tried to keep these meetings down to an hour, but they naturally always overran and sometimes even took up a whole morning.

4. Under appreciation 🎻😭
As is often the case, the wider business doesn't see all the work we do. This is inevitable. But with this Scrum approach, our work felt even more obfuscated as the smaller, more technical things were being done but not necessarily understood, and the larger, more visible things appeared to take far longer than they 'should'.

The new way

We have trialed, and intend to continue using an 8-week work cycle that's comprised of a 6-week project and a 2-week 'wash up' period.

Project cycle

A select few members (the best suited to the project) will be put on The Project Team. It is up to The Project Team to start and finish a project within 6 weeks. This means that by the Friday of Week 6, the project has been released to production and is in the customers' hands. The project can be anything from an internal product (the most recent being a new dashboard built in React) or a new feature for our mobile app that will be released as part of our regular app-store updates for our clients to use.

The rest of the engineering team during this period will support the wider business with support requests and bug fixes etc - i.e. smaller tasks that do not belong to a larger project but still have to get done.

Wash Up cycle

The 2 week Wash Up period is where we debrief the project and look for ways to improve the next project cycle. We can also use the time to add things that may have been missed in the project cycle, whether because they were out of scope or we ran out of time and they didn't affect the core functionality of the product. These 2 weeks are also used as Kaizen time, as well as planning the next project. It's important the projects are planned properly as the condition of any project is that it gets delivered in 6 weeks. This means that we need to properly assess ths scope of each project and ensure that the Project Team is not overstretched or, indeed, slacking ;)

How does a project get chosen?

In the 2-week Wash Up period, any engineer can advocate for a new project. They don't necessarily have to take the lead on a project they advocate for, but it's often the case that they do. It means that the developers are actively encouraged to look for areas of the business that can be improved, but also advocate for things they may want to work on. Projects also get put forward by other areas of the business where they spot areas that need improvement in their department. Ultimately, the projects get given the green light by the CTO and CEO.

Disadvantages

I'll get the downsides of this approach out of the way first, as I'm a big fan of this new approach and I'm honestly struggling to find many downsides.

1. Less agile.

It takes longer for requests and business support issues to get done as we are now dealing with an 8-week block of time, instead of 2. However, there are still engineers on hand to deal with these requests, so it's not like nothing can be done in these 8-weeks.

2. Not everyone has a project!

It is the case that developers not on a project, arguably, have less fulfilling work to complete as they're working through tickets just like before. Having said that, people might prefer grinding through tickets and might hate projects, so it's all swings and roundabouts!

3. Smaller things may get skipped... maybe.

I can foresee a situation where smaller issues do not get resolved if they're not important enough. The wider business might find this methodology more frustrating than the old way of working, however, it's actually good that developers aren't wasting their time on things that are not urgent. Smaller issues are likely to get resolved in a project cycle where an entirely new feature or product gets built that's better than any fix that could have been applied during a sprint thus, in the long run, the business is far better off even if it has to suffer minor niggles for slightly longer.

Advantages

Let's do this...

Speed. Big projects are completed and delivered quickly. Projects of the size we're able to complete within a 6-week project cycle used to take us at least 12.
Higher standards. Developers are able to work on the project intensely and properly without distractions. This has resulted in better code quality and better infrastructure design.
Planning. The wider business knows that they will have a product in 6 weeks. This makes their scheduling and planning far easier and more efficient. It also means they don't have to continuously ask the engineering team about updated delivery times.
Visibility and appreciation. The wider business can see the results of the developers' work. We deliver features and products that the wider business can see and use every 8 weeks, without fail. This has made the engineers feel more appreciated and look more accomplished.
Fewer meetings. We don't need to sprint plan every Friday. We don't need to micro-manage and change things around to suit the latest requests that come in during each sprint cycle. This means more time for development and actual work!
Fewer requests. Because the wider business knows what we're doing and how long it will take, we don't get inundated with requests. We are left to work on our project. This, of course, leads to less distraction and a better result.
Responsibility. Not everyone wants to be a manager or run a team, but if you do or you'd like to try it, being a lead on a project is a perfect way to get your feet wet. As a Project Owner, you will have the responsibility for ensuring the scope of the project is achievable and, more importantly, that the project gets delivered on time.
No micro-management. Because each project is run by a Project Owner, the project gets run in any way the Owner sees fit. This means you get exposed to different ways of working when you're not an owner, and you can trial and experiment with different ways of working when you are. On top of this, it also allows for each member of the team to work the way they like. The best way we've found so far, is for the overall 'to-do list' and timeline to be visible to the whole team, but a person's individual tasks to be managed in any way that person sees fit. This means you're not stuck working in a way that someone else's brain works, you can choose your own tooling and methodology - something that I find efficient, educational and rewarding.
Education. As a Project Owner, you could put yourself on areas you want to progress in. For example, if you're mainly a backend engineer but want to do more front end work, put yourself on the front end and elect another backend dev to work with you! So long as the project gets delivered and to a high standard, this is an excellent way of learning on the job and improving your abilities.

Summary

To sum up, with this new way of working, the engineering team are getting through bigger projects more quickly, more efficiently and to a higher standard. Less time is being spent on the minutiae of day-to-day business and the wider team has a better understanding of what the developers are up to, and a better appreciation for what the developers do.

This is a way of working that we certainly intend to continue with. We and the business are really happy with the results and, as a developer, it's more rewarding and more fun!

Kaizen: Little and Often

Maximilian — Tue, 08 Mar 2022 08:55:35 +0000

Introduction

I'm going to explain what Kaizen sessions are and how and why engineering teams have them. I'm then going to explain how and why the InsideOut Engineering Team incorporates Kaizen sessions and the Kaizen philosophy in a more unconventional manner.

What are Kaizen Sessions?

Kaizen Sessions are an opportunity for members of the engineering team to use a portion of their working day for self improvement. You may have heard of these sessions also be referred to as 'self-learning', 'self-study', 'self-improvement', 'innovation-time' etc. but the reason we at InsideOut use the term 'Kaizen', is because of the philosophy behind it that we try to adopt.

The term 'Kaizen' is a Japanese business term meaning 'change for the better', or 'continuous improvement'. There are a few different implementations of Kaizen, but for the sake of this blog post, I'd like to touch on the 5 core elements of the Kaizen approach:

Teamwork
Personal Discipline
Improved Morale
Quality Circles (groups specifically brought together to identify potential improvements)
Suggestions for Improvement

By keeping these 5 principals at the heart of an engineering team, one creates a culture not only of self-development, but co-operative development. That is to say, everyone feels supported and motivated to better themselves and each other.

Why have Kaizen sessions?

We all know how quickly technology moves and having the time to keep up with the latest tech is a big challenge. As such, it's common practice for tech companies to afford a given amount of time to members of their engineering team for 'self-learning'. This is both beneficial for the employee as it keeps their skills sharp and their knowledge up to date, ultimately, making them more valuable in their own careers, but also to the employer who, in turn, will have sharper engineers at their disposal, as well as engineers who feel valued by the company.

InsideOut vs. The World

The amount of time given to engineers for self-learning varies from company to company, but the most common schedules I've come across are one afternoon a month, one day a month, one day a quarter and one afternoon a quarter.

At InsideOut, however, we've upped this to one afternoon a week and it makes everyone happier, more collaborative (somewhat more competitive) and, importantly, more productive.

Let me explain...

We tried once a quarter and we tried once a month and they were both fine. Everyone looked forward to tackling their own projects and felt grateful for having the time to learn.

But...

Sessions spread out this far apart presented some glaringly obvious problems which, I would argue, made them next to pointless! In no particular order:

Frustration: We'd be left wanting to have learned more in the allotted time, and had to wait another month (or 3!) for our next learning opportunity.
Fatigue: We tried to cram as much as possible into a relatively short amount of time, meaning energy and productivity levels were noticeably down for the following day (and sometimes for the rest of the week).
Retention: It was difficult for us to retain what we'd learned due to the gap between learning sessions. It's far better to learn 'little and often'.
A lack of desire to learn something directly associated with current company-work. After-all, we all have things we want to learn and projects we want to work on that aren't necessarily associated with work!
An unfair 'urgent' work distribution. So, if something urgent came up that required an engineer, it's pretty unfair for that engineer to lose (potentially) a quarter of their yearly learning time to fix something.
A lack of desire to teach someone else something. Even though we wanted to, we were conscious that knowledge-sharing took up a large chunk of our self-learning time.

At InsideOut we feel like we've solved all of these problems and in doing so, found further benefits. In summary, we participate in weekly Kaizen sessions that start at 14:00 and span the rest of the day. At 14:00, the engineering team joins a video call and our aim is to spend about 60 minutes knowledge sharing where our target is for every person to bring something to the table - even if it's a question that they know another engineer has the expertise to answer and teach everyone. Afterwards, everyone goes away and progresses with their self-learning for the rest of the day.

The benefits we've noticed to this approach, amongst others, are:

Creates a culture of continuous learning and development (self-improvement).
Creates a culture of teaching and progressing the people around you.
Creates a culture of business improvements.
Gives each engineer the time to pursue a proper educational path, with better knowledge retention due to a 'little and often' learning approach.
Engineers are happy to jump on bits of work during Kaizen, should they need to.
Engineers are happy to prioritise a deadline they may have over their self-learning.
Engineers are much more likely to further their knowledge in areas directly beneficial to their work, with some choosing to alternate week-by-week between personal projects and work-related learning.
Engineers are happier in general and feel more appreciated and cared for.
Having one afternoon that's almost guaranteed to be slower-paced makes the team more productive and feel more rested for the rest of the week.

In other words:

The InsideOut engineering team are happier, healthier, more productive and more collaborative!

Conclusion

By having regular sessions like this, a culture of self-improvement, self-discipline and improving and helping others is almost guaranteed. We trialled it with a view to monitoring our output and seeing if our work got affected, and if so, by how much. The results were that our output measurably improved week-on-week, and moreover, everyone felt encouraged and supported to get better which made everyone happier and more achieved in their personal growth.

I'd highly recommend this approach for all engineering teams, not least because I consider it a win for both parties in a professional capacity, but because it's a huge win in a personal one. On paper, of course, the engineering team gets 1 afternoon 'off' a week, but we all know that what's on paper can often be a far cry away from reality.

Maximilian Coghlan
Full Stack Software Engineer at InsideOut

Mocking Node-Fetch with Jest, Sinon and Typescript

Maximilian — Tue, 09 Nov 2021 23:50:10 +0000

TLDR

If you don't care about the context of these tests and want to go straight to the node-fetch mocks, skip to the here's my solution section.

Introduction

I wrote a middleware library to be used by micro-services that decodes and verifies JWTs and works with Express and Koa. The requirement was for the library to make an API request to an external service in order to refresh tokens if the provided token had expired. I'm not writing this post to discuss the library itself, but to talk about how I wrote the unit tests for it as I found it a little tricky to implement a solution that catered for sending and receiving dynamic data to and from the external service, whilst keeping the tests isolated. Hopefully this will be helpful to someone trying to do a similar thing.

The middleware

The controller function looks a little something like this:

async function checkToken(
    reqHeaders: IncomingHttpHeaders
): Promise<ITokenData> {
    // Get access token from auth header
    const accessToken = reqHeaders.authorization?.split(/\s+/)[1];

    // Decode token
    const decodedToken = await verifyAndDecodeToken(accessToken, SECRET);
    // Token is valid, return the decoded token
    if (decodedToken.exp > Date.now() / 1000) return decodedToken.tokenData;

    // Store the data from the decoded token in a variable
    const tokenData: ITokenData = decodeToken.tokenData;

    // Call the external API using the data decoded from the access token
    const newAccessToken = await refreshTokens(tokenData);
    // Decode token returned from external API
    const decodedNewToken = await verifyAndDecodeToken(newAccessToken, SECRET);

    // Return the decoded new token
    return checkNewToken.tokenData;
}

The refreshTokens() function looks something like this:

async function refreshTokens(
    tokenData: ITokenData
): Promise<string | undefined> {
    const res = await fetch(`https://refreshmytokensyouslag.com`, {
        method: `post`,
        body: JSON.stringify({ tokenData }),
        headers: {
            "content-type": `application/json`,
        },
    });

    const resJson = await res.json();
    return resJson?.data.newAccessToken;
}

And, just for the sake of context, the wrapper functions (or 'factories') for Koa and Express look something like this:

/**
 * Middleware factory for Express
 */
function checkTokenExpress() {
    return async function checkTokenMiddleware(
        req: express.Request,
        res: express.Response,
        next: express.NextFunction
    ): Promise<void> {
        const decodedToken = await checkToken(req.headers);
        req.userData = decodedToken;
        return void next();
    };
}

/**
 * Middleware factory for Koa
 */
function checkTokenKoa() {
    return async function checkTokenMiddleware(
        ctx: Koa.Context,
        next: Koa.Next
    ): Promise<void> {
        const decodedToken = await checkToken(ctx.headers);
        ctx.userData = decodedToken;
        await next();
    };
}

Code explanation

We have our 2 'middleware factories'; one for Express and one for Koa. Both are exported, ready to be used in any other Express or Koa services as middleware. Both factories call the checkToken() function, append a decoded token to the req or ctx objects respectively, then call next().

Our controller function, checkToken(), verifies and decodes access tokens. If the token is valid and hasn't expired, it returns the decoded token object. If the token is invalid, it will throw an error, and if the token is valid but has expired, it calls the refreshTokens() function.

The refreshTokens() function makes a request to an external API which handles the issuing of new access tokens if certain conditions are met. Our checkToken() function will then decode and return this new token.

The tests

Testing for the token being valid was pretty simple as the code is already isolated. Here's what the code looks like for both the Koa and Express implementations:

// Express
test(`middleware calls next if access token is valid`, async () => {
    // Create a token to test
    const testAccessToken = jwt.sign({ foo: `bar` }, SECRET, {
        expiresIn: `1h`,
    });

    // Mock the request object
    const mockReq = {
        headers: { authorization: `Bearer ${testAccessToken}` },
    };
    // Mock the response object
    const mockRes = {};

    const req = mockReq as unknown as ExpressRequest;
    const res = mockRes as unknown as ExpressResponse;

    // Mock the next() function
    const next = Sinon.stub();

    // Invoke Express
    const middleware = express(SECRET);

    void (await middleware(req, res, next));

    // Our test expectation
    expect(next.callCount).toBe(1);
});

// Koa
test(`middleware calls next if access token is valid`, async () => {
    // Create a token to test
    const testAccessToken = jwt.sign({ foo: `bar` }, SECRET, {
        expiresIn: `1h`,
    });

    // Mock the ctx object
    const mockCtx = {
        headers: { authorization: `Bearer ${testAccessToken}` },
    };
    const ctx = mockCtx as unknown as KoaContext;

    // Mock the next() function
    const next = Sinon.stub();

    // Invoke Koa
    const middleware = koa(SECRET);

    void (await middleware(ctx, next));

    // Our test expectation
    expect(next.callCount).toBe(1);
});

Code explanation

The tests for Express and Koa are nearly identical, we just have to cater for Express' request object and Koa's ctx object.

In both tests, we're creating a valid token testAccessToken and mocking the next() functions with Sinon. We're then mocking the request and response objects for Express, and the ctx object for Koa. After that, we're invoking the middleware and telling Jest that we expect the next() function to be called once, i.e. we're expecting the token to be valid and the middleware to allow us to progress to the next step in our application.

What does a test for a failure look like?

From this point onwards, I'll only give code examples in Koa as there's slightly less code to read through, but you should have no problem adapting it for Express using the examples above.

test(`middleware throws error if access token is invalid`, async () => {
    const testAccessToken = `abcd1234`;

    const mockCtx = {
        headers: { authorization: `Bearer ${testAccessToken}` },
    };
    const ctx = mockCtx as unknown as KoaContext;
    const next = Sinon.stub();

    const middleware = koa(SECRET, API_URI);

    await expect(middleware(ctx, next)).rejects.toThrowError(
        /access token invalid/i
    );
});

Code explanation

Here, we're creating a testAccessToken that is just a random string, and giving it to our middleware. In this case, we're expecting the middleware to throw an error that matches the regular expression, access token invalid. The rest of the logic in this test is the same as the last one, in that we're just mocking our ctx object and next function.

The tricky bit: testing dynamic calls to an external API

We always need tests to run in isolation. There are several reasons for this, but the main one is that we're not interested in testing anything that's not a part of our code, and therefore outside of our control.

So the question is, how can we dynamically test for different responses from an external API or service?

First, we mock the node-fetch library, which means that any code in the function we test that uses node-fetch is mocked. Next, in order to make the responses dynamic, we create a variable that we can assign different values to depending on what we're testing. We then get our mocked node-fetch function to return a function, which mocks the response object provided by Express and Koa.

That's a bit of a mouth full. So let's look at some code...

Here's my solution

At the top of my .spec file, we have the following (in JS to make it easier to read):

// The variable we can change for different tests
let mockTokenFromAPI;

// Mocking the 'node-fetch' library
jest.mock(`node-fetch`, () => {
    // The function we want 'node-fetch' to return
    const generateResponse = () => {
        // Mocking the response object
        return { json: () => ({ data: { newAccessToken: mockTokenFromAPI } }) };
    };

    // Put it all together, Jest!
    return jest.fn().mockResolvedValue(generateResponse());
});

We first get Jest to mock the node-fetch library by returning a function. We then get the mocked library to return another function called generateResponse(). The purpose of generateResponse is to mock the response objects in Express and Koa, so it returns an object with the json key. The value of json is a function, thus mocking the .json() method, which finally returns the data structure we're expecting from the API, using our mockTokenFromApi variable. So now in order to make the whole thing dynamic, all we have to do in our tests is change the value of this variable!

Let's Typescript this up...

interface IJsonResponse {
    data: {
        newAccessToken: string | undefined;
    };
}
interface IResponse {
    json: () => IJsonResponse;
}

let mockTokenFromAPI: string | undefined;

jest.mock(`node-fetch`, () => {
    const generateResponse = (): IResponse => {
        return {
            json: (): IJsonResponse => ({
                data: { newAccessToken: mockTokenFromAPI },
            }),
        };
    };

    return jest.fn().mockResolvedValue(generateResponse());
});

And now here's how we can test our middleware with dynamic responses from an external API using the node-fetch library:

test(`Middleware throws error if refresh token errors`, async () => {
    // Create an expired but valid access token to send
    const testAccessToken = jwt.sign({ tokenData: { authId: `1234` } }, SECRET, {
        expiresIn: `0`,
    });

    // DYNAMICALLY SET WHAT WE WANT THE EXTERNAL API / SERVICE TO RETURN
    // In this case, an invalid token
    mockTokenFromAPI = `abc123`;

    const mockCtx = {
        headers: { authorization: `Bearer ${testAccessToken}` },
    };
    const ctx = mockCtx as unknown as KoaContext;
    const next = Sinon.stub();

    const middleware = koa(SECRET, API_URI);

    await expect(middleware(ctx, next)).rejects.toThrowError(
        /refresh token error/i
    );
});

test(`Middleware calls next if refresh token exists and is valid`, async () => {
    // Create an expired but valid access token to send
    const testAccessToken = jwt.sign({ tokenData: { authId: `1234` } }, SECRET, {
        expiresIn: `0`,
    });

    // DYNAMICALLY SET WHAT WE WANT THE EXTERNAL API / SERVICE TO RETURN
    // In this case, a valid token
    mockTokenFromAPI = jwt.sign({ tokenData: { authId: `1234` } }, SECRET, {
        expiresIn: `1h`,
    });

    const mockCtx = {
        headers: { authorization: `Bearer ${testAccessToken}` },
    };
    const ctx = mockCtx as unknown as KoaContext;
    const next = Sinon.stub();

    const middleware = koa(SECRET, API_URI);

    void (await middleware(ctx, next));
    expect(next.callCount).toBe(1);
});

Conclusion

We now have the ability to get 100% isolated test coverage on our middleware, even though it relies on an external API.

I hope this helped you in some way, and if it didn't, I hope you learned something or at least found it interesting!