The Tinkering Hobbit (excerpt from The hidden language within the language)

The night falls over The Shire and gray smoke billows from the chimneys as every house gets ready for a hot supper and some stories before bed.

Every house except one.

In the distance, off the beaten path, in a strange house (strange by hobbit standards) something else is going on. You could tell it was strange by the plumes of green and purple smoke coming out of its roof, or by the clinks and clanks coming from inside.

This is not an ordinary hobbit house, in here lives...

The Tinkering Hobbit

Tinky the Tinkering Hobbit is the one responsible for all the cool hobbit contraptions you see out there. Adventuring hobbits have no time to learn how stuff actually works.

Back in high-school, some hobbit bullies called Tinky "non-canon" because in the 9 hours of chasing a ring all over the land (extended director's cut), plus some other 9 hours of stealing treasure from a mean dragon, there was absolutely no mention of Tinky.

The Tinkering Hobbit never paid any attention to those comments. Understanding the inner workings of things, that was the ultimate goal.

But Tinky knew understanding was not all, sharing the knowledge was equally important, and so, a community was born, almost like a "fellowship", although they preferred to be called "The Tinkerhood".

For months Tinky has been researching, tinkering, trying, failing, and trying again.

One day after knocking down half a sink with a headbutt and having flux capacitor flash forwards, Tinky saw something in the edge of the metaphorical vision. Something so clear and yet so elusive, something that was there all along, hidden in plain sight, something that could change everything.

But Tinky knew well that the key to truly understanding stuff was sharing the knowledge so a book was born. A book we partially reproduce here for all of you honorary members of The Tinkerhood. A book called...

The hidden language within the language

(excerpt from Tinky's book)

Hi there, my name is Tinky, the Tinkering Hobbit. You know me, I've been tinkering since forever. Today I'd like to tell you about my recent explorations into the world of types, more specifically of Typescript.

Some of the rabbit holes in this field are deeper than the deepest pits of Moria, so we'll tread carefully and take it slowly.

Let's start with...

Unions, really?

Typescript has something called "Union Types". You write union types using a | like this:

type Monster = Orc | GiantSpider;

You read this as "a Monster is either an Orc or a GiantSpider".

I'm sure there's an occult field of magic where calling this a "union" makes sense, but if you are a simple tinkerer like myself, one perhaps familiar with set-union or the bitwise operator |, then this will be at least a little confusing.

If you inspect that Monster type, you'll see it has readily available only the parts that both an Orc AND a GiantSpider have in common. Now if you think about sets, a set-union has the stuff that EITHER or BOTH of its components have.

I guess what I'm trying to say is: don't pay much attention to the name. These are called "union types" (even though they work like intersections or something) but it doesn't matter they could've been called something else.

Do know that in this book when we refer to "union types" we are talking about these things and not what you might intuitively think of as "unions".

An objective view

Have you ever been into object-oriented programming? In there you used to have classes and such. Some classes were abstract representations of other classes, like:

class Monster { /*... */ }

class Orc extends Monster { /* ... */ }

A Monster is an abstract representation of an Orc. Or an Orc is a more concrete type of Monster.

Now, in OOP, a recurring problem (and cause of great anxiety) is called "downcasting".

Downcasting is the act of taking a generic thing, like a Monster that you know (how?) to be a specific thing, like an Orc and asserting that this thing is an Orc, like this:

function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?
}

Note that the reverse problem, know as "upcasting" or treating something specific as something generic, just works:

const orc = new Orc();
const monster: Monster = orc;

Why are we talking about this, you ask?

Well...

Narrow and wide

A very similar problem occurs with union types. Let's say we have:

type Monster = Orc | GiantSpider;

And we have a monster variable of type Monster that we know (?) it's an Orc and we need to assert that.

When dealing with union types, what we used to call "downcasting" is now called "narrowing" as in "you take a wider, more generic type and you narrow it into a more specific one".

That being said, narrowing and downcasting are pretty much the same and we could use a similar hack as we did with classes:

function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?

But there are a couple of better options. We could use a discriminating property like this:

interface Orc {
  type: 'orc';
}
interface GiantSpider {
  type: 'giant-spider';
}
type Monster = Orc | GiantSpider;

So that we can later use monster.type to check for 'orc' and get type-checked narrowing:

function f(monster: Monster) {
  if (monster.type === 'orc') {
    // in here the type of `monster` is `Orc`
  }
}

En garde!

Another, more advanced, narrowing alternative is to use something called "type guards" (also known as type predicates). These are just functions that take monsters and return a fancy boolean stating that you just checked and yeap, this monster is an Orc.

const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// note the `m is Orc` part which is the "fancy boolean"

Type guards are the answer to those "how did you know that?" questions above.

To be fair, technically all these strategies for type-checked narrowing are called "type guards" (e.g. type predicates, discriminating properties, the typeof operator, etc.), but I grew up calling type predicates "type guards" and old habits die hard.

With all that warm up out of the way, it's time for a crucial choice.

Red berry vs blue berry

I once read of a powerful wizard called Morpheus that once offered his apprentice N00b a choice between two berries: eat the red berry and your eyes will be open, but there will be no turning back..., or take the blue berry and enjoy blissful ignorance.

Dear reader, I'm giving you the same choice right now. Keep reading at your own peril or choose to walk away now oblivious to the power (and responsibility) ahead.

Hidden in plain sight

Let's say that we have a function:

const someFn = (t: number): number =>
  /* do something with `t` */

This function takes a single argument t of type number and returns another number.

Now look at this:

type SomeFn<T> =
  /* do something with `T` */

Now squint your eyes and look at both together. Do you see it? Both look like functions, right?

someFn is a run-time function while SomeFn is a compile-time function!

We could say that SomeFn takes a type argument T and returns some other type value.

In other words the type alias SomeFn acts as a function over types instead of a function over values.

Let's see another example:

type MakeArray<T> = T[];

This MakeArray function takes a type argument T and returns a new type T[].

So we already have variables like T above (capable of holding types) and we have functions (like MakeArray and SomeFn) capable of producing new types from existing types...

Do you see where I'm going with this?

The big reveal

We should be able to write pretty complex programs just by combining the elements above (i.e. variables and functions). These are not regular programs, they are programs that operate on the type-system of our "real" (or should I say "other") programs.

We could craft our type-system programs to provide better domain-specific compilation (think DSLs, encode business rules in the type-system, etc.).

But for us to be able to do really cool stuff in this hidden language we need a couple more elements.

Who types the types?

Did you spot it back in the SomeFn example? Here's again all together:

const someFn = (t: number): number =>
  /* do something with `t` */

type SomeFn<T> =
  /* do something with `T` */

See how we know t is a number but we have no clue what T is?

We went to all the trouble of using Typescript because we believe types are important and now that we are coding at the type-system level we have no types?

Outrageous!

...actually, there is a way for us to specify types in our type-system programs:

type SomeFn<T extends number> =
  /* do something with `T` */

So what you do with : in run-time land you can do with extends in compile-time land.

But what about...

Narrowing^2

What if we have something like:

type F<T extends Monster> =
  // do something if T is an Orc

How can we narrow T from Monster to Orc? Well, the answer is something called conditional types but I like to think of these as good old ternary operators:

type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc

If you don't want to allow one of the branches of your conditional types you could use the handy type never (I shudder to think of a run-time analogy for this):

type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;

OK, that's more like it, we have functions, variables and conditional expressions. What else do we need?

Unions as lists

We talked a lot about union types, truth is, for our type-system programs a union can function pretty well as a list. You need to return a bunch of stuff, you could return a union type of that.

Let's see some examples. You're probably familiar with good old Object.keys:

const a = { a: 1, b: true };

const keysOfA = Object.keys(a); // ['a', 'b']

Object.keys takes an object and gives you back an array of its keys (strings, numbers or symbols).

Ready for another squinting eyes moment? Here it goes:

interface A { a: number; b: boolean }

type KeysOfA = keyof A; // 'a' | 'b'

By now you are an expert at this. keyof T is the type-system equivalent to Object.keys(t), but instead of returning an array, it returns a union type.

Let's see the other obvious example, I'll throw it all together this time:

const a = { a: 1, b: true };

const valuesOfA = Object.values(a); // [1, true]

// -----

interface A { a: number; b: boolean }

type ValuesOfA = A[keyof A]; // number | boolean

Clearly there's a pattern here.

Unions and the Occult

One of my favorite run-time operations over arrays is without a doubt .map. Here's a refresher:

const a = [1, 2];

const b = a.map(value => value % 2 === 0) // [false, true]

.map takes a function and an array (via this), it then applies that function to every element of the array and returns a new array with these mapped values.

When dealing with this type of operations, some people with a magical proclivity like to use a spell called "The Functor". We tinkerers are not usually into magicks so we'll just stick to thinking of map as a function that can change every element in an array to something else, preserving order, cardinality, etc.

We said some time ago that union types could double as type-system arrays.

Wouldn't it be great to be able to map over the types wrapped in a union type? Ha, you knew this was coming didn't you?

Type-system map

The type-system version of map (i.e. map over types rather than run-time values) requires a bit more work but it's perfectly doable:

type Monster = Orc | GiantSpider;

type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;

type GoodMonster = MakePet<Monster>;
  //             =
  //               | WashCleanAndDress<Orc>
  //               | WashCleanAndDress<GiantSpider>

Let's go over all of this nonsense step by step!

We start with a Monster type that is a union of Orc and GiantSpider (remember that we think of unions as arrays for now).

Then we create a mapping function that is called MakePet. MakePet will unpack each monster type from the union, call WashCleanAndDress for each type and then neatly pack them back into a union type.

MakePet works like map because it does the unpacking / packing by exploiting something called distributive conditional types.

This essentially means that whenever you do T extends U ? SomeFn<U> : never this will not only pattern match T and U but also, when T is a union type, "distribute" SomeFn over every type in T that extends U.

If you combine this knowledge that extends distributes with the fact that every type extends unknown you get the raw mapping pattern:

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown

If you've been paying attention you might have noticed that the type-system map requires us to bind two type functions:

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown

type MappingResult = ApplySomeMapping<A | B>;

This is necessary because if we tried to inline A | B in ApplySomeMapping we'd break distribution. Here's why, step by step:

// step 0 (this one works)
type X = A | B;

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never;

type Result = ApplySomeMapping<X>;

// -----

// step 1 (inline X in ApplySomeMapping, it's broken)
type X = A | B;

type Result = X extends unknown
  ? SomeMapping<X>
  : never;

// -----

// step 2 (inline X in Result to make it obvious it's broken)
type Result = (A | B) extends unknown
  ? SomeMapping<A | B>
  : never;

If we tried to merge Result with ApplySomeMapping we would be pattern matching (and distributing) X, but then we'd end up applying SomeMapping over the whole X.

If this was run-time, it'd be the same as doing:

x.map(_ => someMapping(x));

When what you wanted probably was:

x.map(value => someMapping(value));

Wow, that was harder on the brain than a honey mead hangover.

Maybe it's time for a recap...

Cheatsheet of Typescript type-system programming

Let's summarize all this craziness.

Functions

// run-time
const someFn = (t: number) => /* use `t` */

// compile-time
type SomeFn<T extends number> = /* use `T` */

Narrowing

type Monster = Orc | GiantSpider;

// run-time
if (monster.type === 'orc') {
  // in here the type of `monster` is `Orc`
}
// or
const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// compile-time
type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc
// or
type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;

Unions as lists

// run-time
const a = { a: 1, b: true };
const keysOfA = Object.keys(a); // ['a', 'b']
const valuesOfA = Object.values(a); // [1, true]

// compile-time
interface A { a: number; b: boolean }
type KeysOfA = keyof A; // 'a' | 'b'
type ValuesOfA = A[keyof A]; // number | boolean

Mapping over unions

type Monster = Orc | GiantSpider;

// run-time
const pets = monsters.map(washCleanAndDress);

// compile-time
type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;
type Pets = MakePet<Monster>;
//        =
//          | WashCleanAndDress<Orc>
//          | WashCleanAndDress<GiantSpider>

Only returns void

Cool so now that we have that cheetsheet, it's time for an example and then a challenge.

Let's say I have an object type, and I really don't care about the keys but I do want every value to be a function that returns void.

You might be tempted to say:

type OnlyReturnsVoid = Record<string, () => void>;

But what I meant was this:

interface OnlyReturnsVoid {
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}

See how functions in OnlyReturnsVoid take different numbers and types of arguments? We want to preserve that (i.e. if I call onlyReturnsVoid.log(1) it should complain that 1 is not a string).

How can we assert that future changes to our OnlyReturnsVoid don't break the rule? In other words how can we make this change break the build?

interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}

We can approach this problem by writing a type function to validate that a function returns void:

type IsVoidFn<T> = T extends (...args: any[]) => infer U
  ? U extends void
  ? T // This is OK (`T` is a fn returning void)
  : never  // `T` is a function that returns stuff
  : never // `T` is not a function

Note that here we are using infer. You can read more about it here.

Now that we have that validation function we'll use another advanced type-system construct called mapped types to map over the keys of OnlyReturnsVoid and apply IsVoidFn to every value:

type Validator = {
  [K in keyof OnlyReturnsVoid]: IsVoidFn<OnlyReturnsVoid[K]>
};
//             = {
//                 getLogLevel: never;
//                 log: (text: string) => void;
//                 queue: (message: Message) => void;
//                 start: () => void;
//             };

See how our Validator type now has never for the invalid function? The last trick is to create a new interface that extends from both the OnlyReturnsVoid and our Validator:

interface OnlyReturnsVoidGuard extends
  OnlyReturnsVoid,
  Validator {}

You can play around with this here.

Now the challenge: can you change this example so that OnlyReturnsVoid allows embedded objects that also return void, like this one?

interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  service: {
    getCount: () => number; // compilation error!
    start: () => void;
    stop: () => void;
  }
}

You can do it here. Can you change the validator so that it checks nested objects of arbitrary depth?

(the excerpt of the book ends here)

Final thoughts (and a shameless cliffhanger)

I don't know about you, dear reader, but I really love Tinky's explorations, to the point that I almost consider myself part of "The Tinkerhood".

About Tinky's current adventures little is known but it is said that a powerful wizard atop an evil tower once pointed an eye-shaped telescope in Tinky's direction and got a glimpse of a new book in the making.

The working title for that book appears to be: Tunneling into the unknown: portaling between run-time and compile-time programs in Typescript...

I can hardly wait!