DEV Community: Philippe Poulard

Mutually exclusive types done right

Philippe Poulard — Wed, 13 Apr 2022 10:24:43 +0000

The difference between types and interfaces in Typescript is not necessarily clear since they look very similar. However, there are some subtle differences and in this article I will focus on a feature that can be achieved only with types : describing mutually exclusive types.

Let's start with an exemple :

type Resource<Type = any> = {
    progress: 'pending' | 'success' | 'error'
    error?: Error
    data?: Type
}

Such structure would be used as follow :

set progress to "pending" when accessing the resource
then set progress to "success" and the data to data in case of success
or set progress to "error" and the error to error in case of failure

However, this is not enforced in any way ; let's try to write some hardcoded values :

// this one is a possible expected value :
const res: Resource = {
    progress: 'pending'
}

// but this one is unexpected, however I can write it !
const res: Resource = {
    progress: 'pending',
    data: 42             // 😵 oh no ! this is allowed
}

// this one doesn't make sense either !
const res: Resource = {
    progress: 'pending',
    data: 42,            // 😵
    error: new Error()   // 😵
}

Union type to the rescue

One strength of types in Typescript is to have the possibility to combine types together. Let's rewrite the type as the union of 3 mutual exclusive types :

type Resource<Type = any> = {
    progress: 'pending'
} | {
    progress: 'success'
    data: Type
} | {
    progress: 'error'
    error: Error
}

And it works fine ! Now if I write :

// I can't write it anymore ! There is an error !
const res: Resource = {
    progress: 'pending',
    data: 42             // 👈 yes ! there is an error
}

// the fix :
const res: Resource = {
    progress: 'success', // 👌
    data: 42
}

So far so good, Typescript now prevent mistakes the way expected. And it also prevent writing the following code, because data might not be a field of res :

const res: Resource = getSomeResource();
doSomething(res.data); // 👈 error !

To access the data, we must use type guards which allow Typescript to refine the real type in an alternative branch of code :

const res: Resource = getSomeResource();
if (res.progress === 'success') {
    doSomething(res.data); // 👌
}

Type guards are certainly one of the most valuable features in Typescript, but unfortunately, they prevent also writing this :

const res: Resource = getSomeResource();
const data = res.data ?? 42; // 😖 oh no ! error !

Why ? Let's say it again: data might not be a field of res !

Mutually exclusive type done right

So let's fix our type in order to have the data field defined in any case :

type Resource<Type = any> = {
    progress: 'pending'
    data?: never
    error?: never
} | {
    progress: 'success'
    data: Type
    error?: never
} | {
    progress: 'error'
    data?: never
    error: Error
}

That way, the data field is an unconditionally part of the result union type ; what changes is that sometimes we can't set it :

// I still can't write that !
const res: Resource = {
    progress: 'pending',
    data: 42             // 👈 yes ! there is an error !
}

...but all the times we can access it :

// but now, I can write that :
const res: Resource = getSomeResource();
const data = res.data ?? 42; // 👌 got it !

...and of course type guards are still working the expected way !

As a result, the last type definition is a bit verbose, but the code will be less ! Therefore it will be worth to apply this technique only in certain circumstances.

Thank you for reading, Typescript Padawan !

Class patterns for static interfaces

Philippe Poulard — Wed, 06 Apr 2022 15:29:48 +0000

When working with classes, there are two types to consider: the type of the static side and the type of the instance side.

The most common is of course the instance side, and things are working like expected :

interface HasName {
    name: string
}

class Person implements HasName {
    name!: string
    birthDate!: Date
}

In the example above, all the fields are describing an instance of a Person, that is defined partially with the interface HasName.

Then, we might add some convenient static methods :

class Person implements HasName {
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<Person> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

// try it :
const bob = await Person.load('Bob')
bob.birthDate.getFullYear();

At this point of the design of such outstanding application, we guess that some other classes will follow the same pattern for loading and saving data, therefore we create an interface for that purpose :

interface Storable<Type, Key> {
    load(key: Key): Promise<Type>
    save(item: Type): Promise<void>
}

Since the static keyword can't be used on interfaces, I just removed it, but how to enforce my Person class to implement those methods as static ?

Language evolution proposal

There has been some discussions on that topic, but my preferred solution would be :

// unfortunately, this is not allowed by Typescript :
class Person implements HasName, static Storable<Person, string> {
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<Person> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

Unlike other proposals, it has the advantage to not restrict the interface to classes and keep the intend of Typescript of being a structural type system.

But that syntax doesn't exist : we must use the possibilities of the language.

Understanding class type duality

Let's recall that 2 types exist around each class :

the type of the static side,
the type of the instance side.

Before showing acceptable patterns, we must understand what is the type of the static side of a class. For that purpose, let's examine how they are managed on predefined classes, say let's have a look at String :

interface String {
    charAt(pos: number): string;
    charCodeAt(index: number): number;
    // etc, all instance methods
}

interface StringConstructor {
    new(value?: any): String;
    (value?: any): string;
    readonly prototype: String;
    fromCharCode(...codes: number[]): string;
}

declare var String: StringConstructor;

So, String instances are fully defined by the interface definition, and at the end, the same String stuff has a type, the StringConstructor interface.

Strictly speaking, the name endorsed is not the best choice, because the string constructor is just the function new(value?: any): String; : all other definitions of that interface are static definitions, therefore the name StringStatic would have certainly been a better choice ; but let's refrain from rewriting history...

In fact, the last line declare var String: StringConstructor is used to enforce the built-in variable String to be of the type StringConstructor. But String as a variable doesn't refer to an instance but to the class.

Therefore, we might apply the same recipe for our own classes : what is needed is just a typed variable to which one can assign the class, it will enforce the class to honour that type.

Pattern 1 : class expression

Let's try it on our type Person ; first we can make some observations if we assign that class to some variable (actually, a constant !) :

In our preferred IDE, we see that person has the type typeof Person : this is the static side of the class Person. Unfortunately, unlike with the built-in String, we can't call the constant Person because that symbol already exist in the value space for the class itself, therefore we call it person.

Hopefully, we can use class expressions:

interface PersonInterface extends HasName {
    birthDate: Date
}
interface PersonStatic extends Storable<PersonInterface, string> {
    new(): PersonInterface
}
const Person: PersonStatic = class Person implements HasName {
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<PersonInterface> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

Ouch ! There are too much disadvantages with this pattern :

we can't apply a decorator on the class
generics are left out (if we had Person<Foo> ?)
lots of boilerplate code are necessary

Let's take a closer look at the last issue. Why do we have this new interface PersonInterface ? This is because we lost the type typeof Person of the class : if we use Storable<typeof Person>, it would refer the type of the constant Person which is not what we want. On the other hand, Storable<Person> is not usable since Person is a constant. Do you follow ? Well... we need that new interface PersonInterface.

In fact, we have to fix the things lost when using the class as an expression.

Pattern 2 : static field of itself

One better solution, is to have a standalone typed variable that refers the class. But wait, where is the preferred place to define such a variable ? In the class itself as a static variable of course ! But wait, could we define an interface for such classes ? Sure we can :

interface HasStatic<Type> {
    class: HasStatic<Type>
}

Then, we can define a type for the static side of our class :

type PersonStatic = Storable<Person, string> & HasStatic<Person>;

class Person implements HasName {
    static class: PersonStatic = Person; // 👈 
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<Person> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

The thing that enforce our static methods is due to the line static class: PersonStatic = Person : it has the type PersonStatic and has the value Person, which is the class itself.

I personally like that solution because it is in the spirit of the language evolution that I suggest before.

Pattern 3 : static block with field of itself

The following pattern is a variant of the previous one :

class Person implements HasName {
    static {
        const clazz: Storable<Person, string> = Person;
    }
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<Person> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

It has the advantage of being more straightforward and to drop the clazz constant after being set. Notice that class couldn't be used as the constant name; hence clazz.

Pattern 4 : class decorator

Another solution is to call a function that takes as argument the static definition of the class. The best way to apply that function on our class is by using the semantic of the decorator.

Here is the decorator factory, that does nothing and returns nothing, which means that the decorated class is not altered ; its role is to enforce its argument (the static side of our class) to be of the type given :

function HasStatic<Type>() {
    return (target: Type) => {};
}

And this is how it has to be apply to our class :

@HasStatic<Storable<Person, string>>()
class Person implements HasName {
    name!: string
    birthDate!: Date
    static async load(name: string): Promise<Person> {
        // get data and create a new Person someway
    }
    static async save(person: Person) {
        // save the Person someway
    }
}

Notice that we used a decorator factory instead of a decorator, because we wouldn't be able to use the latter like this : @HasStatic<Storable<Person, string>>.

Thank you for reading, Typescript Padawan !

How to stringify and parse a graph

Philippe Poulard — Mon, 21 Feb 2022 18:27:56 +0000

JSON.stringify() (and JSON.parse()) is working well for tree structures; in fact, it doesn't work as-is for graphs.

Let's see it in action (with relevant outputs from the console in the comments) :

const a = {}
const b = {}
b.a = a // same as b={a}
a.b = b
// <ref *1> { a: { b: [Circular *1] } }
const json = JSON.stringify(a);
// Uncaught TypeError: Converting circular structure to JSON
//    --> starting at object with constructor 'Object'
//    |     property 'b' -> object with constructor 'Object'
//    --- property 'a' closes the circle
//    at JSON.stringify (<anonymous>)

We clearly set a.b = b and b.a = a which leads to an infinite loop when traversing the graph. Fortunately, this is detected by JSON.stringify() !

Of course, there are existing tools to inspect a graph of objects, but the purpose of JSON is to exchange a tree of objects, not a graph. Typically, you create some data structure server-side, you stringify it, then send the JSON to the client that can parse it.

Let's go on with some realistic data ; we are using Typescript to have clean data types, but it will work identically with Javascript :

class Person {
    hobbies: Hobby[] = []
    constructor(
        public firstName: string,
        public birthDate: Date
   ) {}
}

class Hobby {
    constructor(
        public name: string,
        public person: Person
    ) {
        person.hobbies.push(this);
    }
}

const bob = new Person('Bob', new Date('1998-12-20'));
new Hobby('cooking', bob);
new Hobby('programming', bob);

const personJson = JSON.stringify(bob);
// TypeError: Converting circular structure to JSON...

There are two things to fix : not only we expect to get a clean JSON string, but we also expect to get back that graph of instances after using JSON.parse().

Basically, we need one recipe to stringify, and another recipe to revive, one being the opposite of the other.

`JSON.stringify()`

If we want to turn our graph to a tree, we must get rid of circular references, that implies that we must decide which data is hosting the other. In our case, it is clear that a person has hobbies: Person is left as-is.

Then, we have to fix things in the subordinate class Hobby, which can be made in various ways :

Customize .toJSON()
Auto-discard the unwanted field

Customize `.toJSON()`

Just return the fields that you want to have in the result JSON :

class Hobby {
    constructor(
        public name: string,
        public person: Person
    ) {
        person.hobbies.push(this);
    }
    toJSON() {
        return { name: this.name }
    }
}

With that update, the stringified result will be :

{
    "firstName": "Bob",
    "birthDate": "1998-12-20T00:00:00.000Z",
    "hobbies": [
        { "name": "cooking" },
        { "name": "programming" }
    ]
}

A variant could be toJSON() { return this.name }, that would give "hobbies": [ "cooking", "programming" ]

Auto-discard the unwanted field

We can either make the field non enumerable, or use a Symbol, like shown below :

const PERSON: unique symbol = Symbol();
class Hobby {
    [PERSON]: Person
    constructor(
        public name: string,
        person: Person
    ) {
        this[PERSON] = person;
        person.hobbies.push(this);
    }
}

Of course, the stringified result will be the same.

`JSON.parse()`

Getting back a tree or a graph of classes instances is not as obvious as you may think, since the reviver argument of JSON.parse(data, reviver) is a function that is not aware of the hierarchy each time it is invoked, and there are many corner cases to take care of.

Fortunately, I wrote a library that does the job simply ; let's use it :

npm install @badcafe/jsonizer

import { Reviver } from '@badcafe/jsonizer';

In a nutshell, @badcafe/jsonizer let you define revivers contextually. For a given structure, you describe in a plain Javascript object the expected mappings, plus the recipe that allow to create new instances (this latter is bound to the 'self' familiar key '.'). Then that object may be bound to a class thanks to a decorator, or applied as a normal function to a class.

You are lost ? Let's see some code with a reviver defined as a decorator :

@Reviver<Hobby>({
    // '.' is the 'self' entry,
    //      that tells how to create new Hobby instance
    '.': ({name, person}) => new Hobby(name, person) // 💥
})
class Hobby {
    // same code as shown previously
}

Then a reviver defined as a normal function

Reviver<Person>({
    // '.' is the 'self' entry,
    //      that tells how to create new Person instance    
    '.': ({firstName, birthDate}) => new Person(firstName, birthDate),
    // then, the fields that require a mapping
    birthDate: Date, // bound to a Date
    hobbies: { // bound to a submapping
        // '*' is the familiar 'any' key for any array item
        '*': Hobby // bound to a Hobby
    }
})(Person) // bound the reviver to the class

If you have some difficulties to understand everything so far, you may read another post that explains more progressively how all that is working.

So far so good... in fact, not really :

If we examine again how our classes are defined, we understand that a Hobby can be created after having created a host Person.
Unfortunately, the reviver function is applied by JSON.parse() bottom-up, that is to say every Hobby instance is supposed to be revived before its host Person instance !

There is clearly some chicken 🐔 and egg 🥚 issue here...

Worse 💥, you also may have noticed that the builder function of the hobby, that is to say : '.': ({name, person}) => new Hobby(name, person) was wrong, because the JSON string of a hobby is made just of a name without a person, like this : { "name": "cooking" }, therefore, it is normal that it doesn't work...

The fix

To fix this issue, we understand that we don't have on that builder a person instance, therefore we will supply it later.

So, instead of building an instance of Hobby, we will build a factory. In order to be compliant with the JSON source structure, we create a source type that describe it :

// describing the JSON structure 
// will prevent a wrong usage of the person field
type HobbyDTO = { name: string }

// the type arguments of Reviver() are <Target,Source>
//                              (when omitted, Source=Target)
@Reviver<Hobby, HobbyDTO>({
    // return a factory that takes a person argument
    '.': ({name}) => (person: Person) => new Hobby(name, person)
})
class Hobby {
    // same code as shown previously
}

As a consequence, we have somewhat inserted an intermediate structure in the flow ; let's define a type for it :

type PersonDTO = {
    firstName: string,
    birthDate: Date,
    // an array of Hobby factories
    hobbies: {(person: Person): Hobby}[]
}

If there were too much fields to copy, just focus on the replacement expected, like this :
type PersonDTO = Omit<Person, 'hobbies'> & {
    hobbies: {(person: Person): Hobby}[]
}

Then fix the reviver of the Person class accordingly :

Reviver<Person, PersonDTO>({
    '.': ({firstName, birthDate, hobbies}) => {
        const person = new Person(firstName, birthDate);
        // then apply the person to the factories
        hobbies.forEach(hobby => hobby(person))
        return person;
    },
    birthDate: Date,
    hobbies: {
        '*': Hobby
    }
})(Person)

Job done ! You just need to parse the JSON to revive your graph of object instances :

const personJson = await read('person.json');
const personReviver = Reviver.get(Person);
const person = JSON.parse(personJson, personReviver);

As a bonus, with Typescript the person const result of the parsing is a typed data (its type is Person).

See also :

Reviving JSON classes made easy

How to fix a bad JSON structure

RPC done right for browser APIs and server APIs

Philippe Poulard — Tue, 15 Feb 2022 15:36:40 +0000

In the browser, many APIs are used to exchange data :

fetch (the today preferred API used for the good old HTTP protocol)
web socket (an upgrade of HTTP)
broadcast API, to make browser tabs and windows discuss together
web worker (with their variant shared workers ; service worker is not an exchange API), to make running expansive tasks in another thread
RTC, to make peer to peer data exchanges

For those APIs that are talking to a web server (the 2 formers), there are the counterpart server APIs : HTTP and web socket. It's worth to mention that even server side, you may find similar exchange APIs within a NodeJS environment:

cluster API, to run your app in several threads
IPC, to scale your app in several servers

The latest is also used in desktop environments such as electron.

The problems

If we examine all those APIs (in the browser and server side), they are all doing more or less the same thing, but with really different APIs, which let us coding the wiring of our applications in a specific way according to the channel considered, whereas we just need to send some data and (sometimes) receive some response.

Let's start with some data to exchange ; we describe them with Typescript :

type Person = {
    firstName?: string,
    lastName: string,
    birthDate?: Date
}
const bob: Person = {
    firstName: 'Bob',
    lastName: 'Marley',
    birthDate: new Date('1945-02-06')
}

In a Chat app, for example, this is how you would like to talk to other users :

say(bob, 'Hello World !');

And this is how you would send and receive a message client side with web sockets :

const socket = new WebSocket('wss://example.com/ws');
function say(user: Person, message: string) {
    const data = JSON.stringify({ user, message });
    socket.send(data);
}
type ChatMessage = { user: Person, message: string }
socket.onmessage = function(e) {
   const {user, message} = JSON.parse(e.data) as ChatMessage;
   user.birthDate = new Date(user.birthDate);
   // render in HTML
   renderChatMessage(user, message);
}

Now, this is how you would send and receive a message with the broadcast API, provided that it would have some sense for a chat application :

const bc = new BroadcastChannel('Chat');
function say(user: Person, message: string) {
    bc.postMessage({ user, message });
}
type ChatMessage = { user: Person, message: string }
bc.onmessage = function(ev: MessageEvent<ChatMessage>) {
   const {user, message} = ev.data;
   user.birthDate = new Date(user.birthDate);
   // render in HTML
   renderChatMessage(user, message);
}

Let's point some difference : the former use send() to send messages, the latter use postMessage() ; the former deals with text, the latter with plain Javascript objects.

This is a really minimal exchange: just a simple one-way message with no response. But things can quickly go more and more complicated :

how to handle several different type of messages ? well... you'd have to send an additional information such as { type: 'Chat.say' } and when receiving messages, you will have to dispatch them with a big switch.
how to handle return values ? when you receive some data, they are unrelated to a message that you would have been sent that expect some response (except with HTTP fetch), how do you follow the data flow ?
how to handle errors ?
did you notice that class instances such as birthDate have to be revived ?

And if you think about what happens server side, there are more things to take care. So far, we considered that the client was the sender and the server was the receiver, but :

how would you reverse the polarity, that is to say make the server send a message and expecting the client sending back a response ?
what are becoming all the previous considerations in that case ?
how to broadcast a message to all users ?
when receiving a message sent by the client, how to multicast that message to others except the sender ? or to arbitrary recipients ?
how to sometimes send back an acknowledgement (and some times don't) ?
how to deal with errors ?
how to manage timeouts ?
how the client would manage all that stuff (return values, errors, timeouts, etc) when the server is the sender of the message ?

If you consider a single protocol, you'll certainly be able to tackle those problems one by one, but if you switch to another protocol, you'll have to rewrite everything.

Let's go on with more general purpose considerations :

how to manage the security efficiently when you use both the HTTP channel and the web socket channel in your app ?
how to ensure that all kind of messages sent in one side are processed properly with the right type on the other side ?
how to avoid writing boilerplate code when you have to deal with a so common API such as CRUD ?
how to have a nice mapping to REST with less efforts ?

The solution

RPC to the rescue: be abstract

There is a way to think of all that with a good abstraction : firstly, let's evacuate all the wiring considerations, that is to say how the data are exchanged through any channel. What is remaining is just our intents (the what, not the how), actually the function say() with its signature : here we are, in a nutshell, we just want the client app to send messages like say(bob, 'Hello World !'); with optionally a response, and just supply the code that process that message server side: this is called a remote procedure call, or RPC. The fact that we are using HTTP or web socket, or even that we are not on a client-server exchange but peer to peer, or on anything else is not taken into account.

So we will separate our intents from the underlying concrete channel ; then as an expectation, the channel would send the data at its charge (no code to write) ; on the other side, we would just supply the implementation of our function ; this is how RPC works.

Let's describe it with Typescript ; we could use an interface, but you will see soon that a class is better, although at this stage it has to be considered as an interface, because we just need the signature of our RPC function :

abstract class Chat {
    say(user: Person, message: text) {}
}

Asynchronizer to the rescue

We go on with Asynchronizer, a nice library that does all the required stuff...

Then the code client side would be :

// the wiring :
const th = // TODO: get a transfer handler some way (see below)
// generate a proxy Data Transfer Handler for the Chat class :
const chatDTH = th.bindSender(Chat);
// send a message with ACK :
await chatDTH.say(bob, 'Hello World !);

Not only the types of the arguments passed to the function are constrained by Typescript, but the function was turned to a Promise, which let us ensure to get back an acknowledgement.

Moreover, what is nice is that the transfert layer can be anything ; let's try a web socket :

import { SocketClient } from '@badcafe/asynchronizer/SocketClient';

const socket = new WebSocket('wss://example.com/ws');
const channel = SocketClient.Channel.get(socket);
const th = channel.transferHandler();

...or HTTP :

import { HttpClient } from '@badcafe/asynchronizer/HttpClient';

const channel = HttpClient.Channel.get('/api');
const th = channel.transferHandler();

...what differs is getting and configuring some channel, but in both cases the chatDTH proxy will be able to invoke the functions defined in the Chat class regardless the concrete channel used. It's worth to mention again that the say() function was defined in that Chat class as a normal function, but was turned to a promise in the chatDTH proxy instance generated. What is nice is that at design time we focus on the payload, not the fact that it will be asynchronous. Why ? Because in certain case you expect it to return a Promise (this is an acknowledgement) and in other case you expect it to return nothing (void) ; more about that later...

Explanations about the vocabulary

Before having a look on how it is received server side, few words about the vocabulary used here :

the channel is the concrete component that emit data with send(), postMessage() or fetch() or whatever. This is the low-level protocol. Asynchronizer supplies most channels invoked at the start of this article, but you can implement your own.
the transfer handler (set in the th const) is an intermediate layer that makes the abstraction between the channel and the RPC functions to which one can bind a sender at one side or a receiver on the other side. This is the high-level of the protocol exchange.
the data transfer handler holds the definition of the remote service ; it is not called just "a service" because the concept is wider since a service is related to something that runs within a server, whereas we are dealing with RPC APIs that are also considering exchanges within the browser, or within a NodeJS server. Hence the concept of Data Transfer Handler. DTH is almost like DAO that would access to some store, but a DAO deals with persistence, DTH with data transfer.
the sender is the side that initiate the data exchange (a request) and is a proxy generated from the transfer handler, and the receiver is the side that implements what to do with the data (at the charge of the developer), and wether there are some results to send back to the sender.

The isomorphic architecture (1/2)

Let's go on with our Chat app. Server side, we have to write the code that implements the say() function, that is to say what are we supposed to do when such messages are received.

Here is the skeleton :

const th = // TODO: get a transfer handler from a socket
           //       server channel or an HTTP server channel
th.bindReceiver(Chat, {
    // the arguments types are inferred to the right types
    async say(user, message) {
        // TODO (see below)
    }
});

According to the channel used client side, you will get the counterpart channel server side (SocketServer.Channel or HttpServer.Channel) that requires some configuration that have to be done once (code not shown for simplicity), then get its transfer handler.

Did you notice that the same DTH definition (the Chat class) were used server side and client side ? What is nice in such isomorphic architecture is that the same code can be used in both sides ; thanks to Typescript, we are sure that the function signature used to send data in one side will match the one used to receive data on the other side. And if you define several functions in the DTH class, the type system will warn you if you omit to write the implementation of one function, or if the arguments types don't match. Of course, you may define as many DTH class as you want with as many functions as you want.

The last but not the least, our data are revived as necessary. The birthDate field of the person is a Date instance when we enter the function.

Broadcast, Multicast, Unicast

So far, we learned how to send a message from the client, and how to receive it in the server ; a good chat application should be able to send messages from the server and to receive it in the client. Let's see how to reverse the polarity.

Server side, apply the following modifications :

// the server can also be a sender
const chatDTH = th.bindSender(Chat);
th.bindReceiver(Chat, {
    async say(user, message) {
        // when a message is received, send it to all users
        chatDTH.say(user, message);
    }
});

Client side, just append to the previous code :

th.bindReceiver(Chat, {
    async say(user, message) {
       // display HTML
       renderChatMessage(user, message);
    }
});

In fact, things are not so different when the client is the sender or when the server is the sender. However, there is a subtle difference :

Client side :

await chatDTH.say(bob, 'Hello World !);

Server side :

chatDTH.say(user, message);

Well, the senders are not the same : the former returns a Promise and the latter returns nothing ; if you omit await in the former, your IDE will tell you so thanks to Typescript.

This is because the server will broadcast the message to all clients, and broadcasting doesn't require an acknowledgement. In fact, each channel kind has a default behaviour for sending messages according to its capabilities. The HTTP server channel and the web socket server channels have broadcast capabilities, whereas the HTTP client channel and the web socket client channels don't : it's unicast.

Your IDE will show you the difference if you hover the variable (below, in the client code we have the DTH is a Sender.Unicast, then in the server code the DTH is a Sender.Broadcast) :

But wait, since the server is broadcasting every message it receives, all clients will receive it, including the user that write that message ? Can't we change the behaviour ? Sure we can, for that purpose, the DTH supply additional functions that are accessible thanks to Symbols (like [MULTICAST]()) to avoid naming conflicts with existing methods of the Chat class :

Server side, apply the following modifications :

import { MULTICAST, OTHERS } from '@badcafe/asynchronizer';

const chatDTH = th.bindSender(Chat);
th.bindReceiver(Chat, {
    async say(user, message) {
        chatDTH[MULTICAST](this[OTHERS]) // other clients
            .say(user, message);
    }
});

Note that in the body of the receiver function :

this is bound to the endpoint port of the channel, that may be in our case the web socket (SocketServer.Endpoint) or a wrapper object around the HTTP request (HttpServer.Endpoint)
this[OTHERS] contains conveniently an array of all clients that are not this client (the one that sends the message).
In fact, chatDTH[MULTICAST](this[OTHERS]) gives another chatDTH object but that will send messages to a different scope, this is why we can call directly our .say() function on it.

That way, not only we can send a response to other clients, but we could also send a response to the sender only (note that sending a response in another message is not the same as returning a value, our say() function doesn't return a value) :

Server side, apply the following modifications :

import { UNICAST, MULTICAST, OTHERS } from '@badcafe/asynchronizer';

const chatDTH = th.bindSender(Chat);
th.bindReceiver(Chat, {
    async say(user, message) {
        if (message === 'ping') {
            await chatDTH[UNICAST](this) // only this client
                .say({lastName: 'Tennis table'}, 'pong');
        } else {
            chatDTH[MULTICAST](this[OTHERS]) // other clients
                .say(user, message);
        }
    }
});

chatDTH[UNICAST]() turns the DTH to a Sender.Unicast which returns a Promise that you have to await ; you can drop the acknowledgement by chaining the [NOACK] property if you'd like to. It's Typescript, your IDE will show you it and will import the Symbol if you choose it :

The isomorphic architecture (2/2)

So far, we have an almost full featured chat application, and we focused just on what our app is supposed to do, not how the messages are sent when we are using HTTP or web sockets. It works for both.

For that purpose, our application is an isomorphic application with at least 3 parts :

[CHAT_ROOT]
    ┣━chat-shared      contains DTH classes definition
    ┃
    ┣━chat-client      is the client app
    ┃
    ┗━chat-server      is the server app

In Asynchronizer, an isomorphic application is an application that uses Javascript/Typescript both on the client and the server, and that contains an amount of code common to both sides. From that base, you might consider additional parts for client workers or other server layers.

Don't forget the REST

But I want more :

some nice REST mappings
check whether the current user has the right role in the app
avoid writing boilerplate code for common CRUD entity management

Almost all that takes place in the shared part of the app ; let's update our DTH class :

@DataTransferHandler()
abstract class Chat {
    @POST
    say(@BodyParam('user')    user: Person,
        @BodyParam('message') message: string) {}
}

...will be map to :

POST /api/Chat/say
(and the HTTP body will contain the 2 parameters)

That way, we are mapping our RPC functions to some REST API ; the decorators are applied with the HTTP channel and ignored with the web socket channel or other channels. We have several other decorators : @GET, @PUT, @DELETE, @PathParam, @HeaderParam, etc. You can also rename path steps, or insert wherever additional path steps.

Finally, let's manage the persons entity in our application :

@DataTransferHandler()
@AccessControl.checkRole('admin')
abstract class PersonDTH extends Crud {
    // @POST create()
    // @GET read()
    // @PUT update()
    // @DELETE delete()
    // @GET list()
    // are inherited from Crud, you don't need to write'em

    @GET
    listPersonBornAfter(@PathParam date: Date): Person[]
        { throw '' } // dummy code
}
// this example was simplified for readability

Extending Crud will create automatically the expected methods with their REST annotations, so that you can use directly in the client :

try {
    await personDTH.update(someone);
} catch (err) {
    // TODO: Ooops ! Unable to update someone
}

...and it's async/await, you handle errors regardless the channel invoked just as usual.

@AccessControl.checkRole('admin') is a decorator that we must develop ourselves in this application ; there is not a generic mechanism because access controls exist in very various flavors so it's up to the developer to define and implement its own access policy. But it's not so difficult to bind it server-side to the socket endpoint or the HTTP endpoint since they are easily customizable, and to make it working in both cases, because Asynchronizer supply some convenient hooks. Here, we might apply that decorator to the class, but also to each functions of the class.
{ throw '' } // dummy code is the only boilerplate code that we are compelled to write : this is a limitation due to Typescript that doesn't let us writing @decorators to abstract functions. We expect to be able to use @decorators in abstract functions in a future version of Typescript.

Asynchronizer

This is the library I work on since 3 years.

It is used since the beginning in a critical application in my company and is about to be released soon with lots of outstanding features. It requires some final polish and documentation before being available.

Asynchronizer was designed to make things simple :

you write some RPC definitions in so-called DTH classes ; you just have to focus on the functions payload : their name, arguments, return values ; they might be decorated for nice REST mapping (available in the library), security checking (at the charge of the developer), etc
you bind senders and receivers with the function implementation the same way everywhere : web worker, node clusters, web server, etc ; thanks to Typescript, your IDE will help you to write the right code. Broadcast, multicast, and unicast capabilities are included
you choose a concrete transport, if you change it later you don't need to touch your code: it just works

Server-side, Asynchronizer fits well in Express or Koa, or can be used as a standalone server.

I also intend to generate the full OpenAPI (Swagger) definitions from the REST @decorations, but it will not be ready for the first public realease.

Final thoughts :

You might think that with GraphQL you also have some kind of RPC API ? Not at all, afaik GraphQL client is just for HTTP fetch and don't have nice REST mappings facilities. How would you push messages from the server to the client with GraphQL ? or from the browser window to the browser worker ? You can still use GraphQL with Asynchronizer server-side for the data access layer ; after all, the entry point of all your GraphQL query is also a function, is it ?
So, you still manage routes by hand in the server ? You think you have the same with your preferred REST library ? Not at all, when you use a REST library you define the mappings server side and you write the right HTTP query client side with no help : no type checking, no automatic class reviving, etc. Conversely, Asynchronizer let define our REST mapping in the shared code, but client and server side you just use the same RPC function !

I hope you find like me that those kind of features looks very interesting, because Asynchronizer rocks !

Asynchronizer will be available here :

https://badcafe.github.io/asynchronizer/

How to fix a bad JSON structure

Philippe Poulard — Mon, 07 Feb 2022 12:50:19 +0000

Sometimes, you receive external data that are not well designed, like shown in the following JSON file with comments :

{
  "firstName": "Bob",
  "numberOfHobbies": "3",         // 👈  should be a number
  "birthDate": "21/10/1998",      // 👈  formatted Date
  "hobbies": "cooking,skiing,programming" // 👈  not JSON-friendly
}

But you prefer a clean target structure ; let's use Typescript to describe it :

interface Person { // 👈  Target with clean types
    firstName: string
    numberOfHobbies: number
    birthDate: Date
    hobbies: string[]
}

How to parse with JSON.parse() the incoming data to the expected target ?

Just use Jsonizer's revivers to fix anything :

npm install @badcafe/jsonizer

import { Jsonizer } from '@badcafe/jsonizer';

Describe the source shape as it is :

interface PersonDTO { // 👈  Source with bad types
    firstName: string
    numberOfHobbies: string
    birthDate: string
    hobbies: string
}

DTO stands for Data Transfer Object

Then define the mappings for every field to fix ; in Jsonizer, a mapping is just a plain object that contains an entry for each field to map :

                                   //  Target  Source
                                   //    👇       👇
const personReviver = Jsonizer.reviver<Person, PersonDTO>({
    numberOfHobbies: {
        //  👇 fix the type
        '.': n => parseInt(n)
    },
    birthDate: Date,
    hobbies: {
        //  👇 split CSV to array
        '.': csv => csv.split(',')
    }
})

Each entry is bound to its reviver that can be a class such as Date, or a nested mapping for hierarchic structures, or nothing to left the field as-is, such as for the firstName.

The special mapping entry '.' stands for the familiar 'self' reference ; it is bound to a function that returns the expected data. Jsonizer also supply the '*' mapping that stands for the familiar 'any' item (object field or array item) and it is also possible to use Regexp matchers and range matchers for arrays.

However, there is a mapping that doesn't work ; let's try it with the incoming data :

new Date('21/10/1998')
// Invalid Date

Since the birthDay input field is such a formatted date, we have to rewrite the mapping for it :

                                   //  Target  Source
                                   //    👇       👇
const personReviver = Jsonizer.reviver<Person, PersonDTO>({
    numberOfHobbies: {
        //  👇 fix the type
        '.': n => parseInt(n)
    },
    birthDate: {
        //  👇 fix the Date
        '.': date => {
            const [day, month, year] = date.split('/')
                .map(part => parseInt(part));
            return new Date(Date.UTC(year, month - 1, day));
        }
    },
    hobbies: {
        //  👇 split CSV to array
        '.': csv => csv.split(',')
    }
})

Notes:

don't use new Date(year, month - 1, day) because it may shift due to the local time zone.

Javascript use month indexes, so we use month - 1.

Finally, parse the data :

const personJson = await read('person.json');
const person = JSON.parse(personJson, personReviver);

Since this exemple is somewhat simple with a flat structure, you might be tempted to write your own reviver function, but for nested structures it will become harder than you think.

With Jsonizer, you'll be able to define mappings for classes, plain objects, nested structures... and more.

See also :

Reviving JSON classes made easy

Pulling Strings with Typescript

Philippe Poulard — Mon, 31 Jan 2022 14:10:01 +0000

In the Typescript zoo, there are union types that are very helpful, specifically to narrow down strings :

type size = 'small' | 'medium' | 'large';

This works well for known list of items, but for other types, one often use the common type string.

Could we do better ? Why should we do better ?

Let's start with some data :

{
    const madrid: string = 'Madrid';
    const spain: string = 'Spain';
    const localisation = spain;
}

It's not clear whether the localisation has to be a city or a country, so let's try to define some type :

{
    type City = string
    const madrid: City = 'Madrid';
    type Country = string;
    const spain: Country = 'Spain';
    const localisation: Country = spain;
    const localisation2: Country = madrid; // 😵 oh no !!!
                                           // it passes...
    // well, Madrid is not a Country !
}

Above, defining a type City doesn't help ; Typescript just takes this type as an alias to string, therefore it won't enforce anything, except our intention (that makes the answer to the question : why should we do better ?)

Could we do better ? Yes : even if you have an open list of values, you may define a type by examples :

{
    type City = 'London' | 'Paris' | '[any city]'
    const madrid: City = 'Madrid' as City;
    type Country = 'United-Kingdom' | 'France' | '[any country]';
    const spain: Country = 'Spain' as Country;
    const localisation: Country = spain;
    const localisation2: Country = madrid; // ❌ oh yes, it fails !!!
    // Typescript will say that a City
    // is not assignable to a Country
}

What is nice, is that your code can either take any data source with valid types as well as hard-coded data like in the example above : writing 'Spain' as Country makes sense, 'Spain' as City wouldn't.

This is a technique that I use and abuse because it prevents putting a string in another string when they are semantically unrelated. And believe me, it saves me from lots of runtime errors !

And it works for numbers too :

/** Any HTTP port (other values are accepted
 * as long they are related to HTTP)
 */
type HttpPort = 80 | 8080 | 443 | 8443;
const port1: HttpPort = 8080; // ok, a known value
const port2: HttpPort = 8888 as HttpPort; // because I'm sure

Reviving JSON classes made easy

Philippe Poulard — Fri, 28 Jan 2022 17:24:10 +0000

JSON.parse() is the standard Javascript function that one use to parse some JSON text to a plain Javascript object or array.

But what you get is only a mix of plain objects, arrays, strings, numbers, booleans or nulls value. If the data source were made of instance of classes, or just dates objects, you are done. You can't recover them efficiently.

However, JSON.parse() accept as a second argument a revive function that is invoked during parsing for each key-value pair encountered, but what you have is only the key name regardless of any context : where are you in the hierarchy of objects ? For arrays it is worse : if the key is "0", which array of your structure are we talking about if you have many ? Really, you are done...

What we need is to design a reviver that takes care of the hierarchy of the target data structure. We want it simple to use. This is the job of @badcafe/jsonizer, a Typescript library that helps you revive any JSON data to an arbitrary hierarchy of plain objects, arrays, and class instances.

Let's start with an example :

{
    "name": "Bob",
    "birthDate": "1998-10-21T00:00:00.000Z",
    "hobbies": [
        {
            "hobby": "programming",
            "startDate": "2021-01-01T00:00:00.000Z"
        },
        {
            "hobby": "cooking",
            "startDate": "2020-12-31T00:00:00.000Z"
        }
    ]
}

This is a typical JSON file that contains Date instances that were stringified. JSON.parse() will give you... strings only. But Jsonizer let you express the expected mapping to revive date instances :

const personReviver = Jsonizer.reviver({
    birthDate: Date,
    hobbies: {
        '*': {
            startDate: Date
        }
    }
});
// personJson contains the JSON string displayed previously
const personFromJson = JSON.parse(personJson, personReviver);

It's easy to understand that Jsonizer's reviver function takes as an argument a plain object that contains a mapping for fields that have to be transformed (we should say 'augmented' to typed data) ; there is also an entry that matches any array item '*' inside the hobbies array ! ('*' looks familiar isn't it ?)

This is where Jsonizer shines : it is able to take care of the hierarchy making any single mapping contextual, and the mapper is really simple to express.

Let's go on. Say that a hobby is a class ; how Jsonizer let us define a reviver ? For classes, we will use the decorator @Reviver :

@Reviver<Hobby>({ // 👈  bind the reviver to the class
    '.': ({hobby, startDate}) => new Hobby(hobby, startDate),
    // 👆 instance builder
    startDate: Date
})
class Hobby {
    constructor(
        public hobby: string,
        public startDate: Date
    ) {}
}

we switched to Typescript, but the example also works in Javascript: just remove type infos in the code and write the fields assignment in the constructor... or consider it's time to move to Typescript !
the decorator function @Reviver let us decorate our class with once again a mapper, but this mapper has a special entry '.' the (looks familiar) "self" entry that indicates how to create an instance of Hobby
oh ! another advantage of using Typescript is that the mapper is constrained to fields that exist in the source data structure, so it helps you to define correct mappings easily.

Now, we can refer it in our first mapping :

const personReviver = Jsonizer.reviver({
    birthDate: Date,
    hobbies: {
        '*': Hobby // 👈  we can refer a class
                   //     decorated with @Reviver
    }
});

After parsing, hobbies will contain an array of Hobby instances !

Fine... but what about Date ? Well, it is so common that Jsonizer ships its reviver ; so you can just use it.

Finally, we could also define a Person class and bound it to a reviver :

@Reviver<Person>({
    '.': ({name, birthDate, hobbies}) => new Person(name, birthDate, hobbies), 
    birthDate: Date,
    hobbies: {
        '*': Hobby
    }
})
class Person {
    constructor(
        public name: string,
        public birthDate: Date,
        public hobbies: Hobby[]
    ) {}
}

Using it is also simple :

const personReviver = Reviver.get(Person); // 👈  extract the reviver from the class
const personFromJson = JSON.parse(personJson, personReviver);
// this is 👆 an instance of Person

Thanks for reading ! I invite you to have a look at @badcafe/jsonizer: you will find more outstanding features !

DEV Community: Philippe Poulard

Mutually exclusive types done right

Union type to the rescue

Mutually exclusive type done right

Class patterns for static interfaces

Language evolution proposal

Understanding class type duality

Pattern 1 : class expression

Pattern 2 : static field of itself

Pattern 3 : static block with field of itself

Pattern 4 : class decorator

How to stringify and parse a graph

JSON.stringify()

Customize .toJSON()

Auto-discard the unwanted field

JSON.parse()

The fix

RPC done right for browser APIs and server APIs

The problems

The solution

RPC to the rescue: be abstract

Asynchronizer to the rescue

Explanations about the vocabulary

The isomorphic architecture (1/2)

Broadcast, Multicast, Unicast

The isomorphic architecture (2/2)

Don't forget the REST

Asynchronizer

How to fix a bad JSON structure

Pulling Strings with Typescript

Reviving JSON classes made easy

`JSON.stringify()`

Customize `.toJSON()`

`JSON.parse()`