DEV Community: ClementVidal

Use Typescript mapped types to "Promisify" interfaces

ClementVidal — Tue, 13 Apr 2021 12:07:30 +0000

Intro

Imagine you're specifying an interface in Typescript with a bunch of methods (sync and async) and attributes, randomly, this one:

// This is the "Original interface" we'll refer to during this article.
interface DataService {
   cache: Map<string, Data>;
   invalidateCache();
   processData( data: Data ): Promise<Result>;
   compressData( data: Data ): Result;
}

Now imagine again that you need to derive it to another interface that will only expose it's methods:

interface DataService {
   invalidateCache(): void;
   processData( data: Data ): Promise<Result>;
   compressData( data: Data ): Result;
}

Or to another one that will convert all sync method to async:

interface DataService {
   invalidateCache(): Promise<void>;
   processData( data: Data ): Promise<Result>;
   compressData( data: Data ): Promise<Result>;
}

Let's see how we can do that using Typescript Mapped types

Filter out attributes from an interface:

First thing we want to do is to convert the "DataService" interface to this one:

interface DataServiceWithoutAttributes {
   invalidateCache();
   processData( data: Data ): Promise<Result>;
   compressData( data: Data ): Result;
}

As you can see the only thing that changes between the original version and the "remapped one" is that the cache attribute is gone.

Here is the Mapped type used to do that:

type FilterOutAttributes<Base> = {
  [Key in keyof Base]: Base[Key] extends (...any) => any ? Base[Key] : never;
}

And here is how to use it:

type DataServiceWithoutAttributes = FilterOutAttributes<DataService>;

Let's break this down:

type FilterOutAttributes<Base>

This declares a mapped type named FilterOutAttributes that takes one generic parameter Base.
We'll use it to provide the "original" interface that we want to remap.

{
  [Key in keyof Base]: ...
}

The keyof operator will return a list of all the keys available in Base and the in operator will iterate over that list and store each value in Key
This is like writing the following loop:

const Keys = ["cache", "invalidateCache", "processData", "compressData"];
for( let i=0; i<Keys.length; i++ ) {
  const Key = Keys[i];
}

Now, for each key, we need to determine if it will be present in the mapped type or not. (or in our case, if the key matches a function in the Base interface)

To do so we need a way to express condition:

if( Base[Key] is a function ) {
  It's a function
} else {
  It's not a function
}

Or using ternary condition:

Base[Key] is a function ? It's a function : It's not a function;

Well, turns out we can express that condition using [Conditional types] in typescript:(https://www.typescriptlang.org/docs/handbook/2/conditional-types.html)

Base[Key] extends (...any) => any ? ... : ...

See! This is a ternary expression and the left side is the test expression:
If Base[Key] extends any kind of function ( (...any) => any ) then the condition is true.

Thinks of extends as a mechanism to constraint the parameter type to match what we want, you can mentally replace it by "can be safely converted into". So in our case, we could read that as: If Base[Key] can be safely converted into (...any) => any then expression is true

So if the condition is true, we return Base[Key] from the ternary expression, which corresponds to the type of the attribute Key in the Baseobject.

And if the condition is false, we return the keyword never which will discard the current Key from the mapped type.

Convert sync methods to async methods:

What we want to do is to convert the "DataServiceWithoutAttributes" interface to this one:

interface DataServicePromisified {
   invalidateCache(): Promise<void>;
   processData( data: Data ): Promise<Result>;
   compressData( data: Data ): Promise<Result>;
}

Now, each method returns a "Promisified" version of the original:
The return value is kept but wrapped inside a promise.

Before going into the detail of this mapped type, let's introduce two builtin mapped type provided by Typescript:

ReturnType<Base> If Base is a function, this will return the type of the return value of that function
Parameters<Base> If Base is a function, this will return the type of every parameter of that function as a tuple

Using those 2 primitives, we'll build a third one: PromisifyFunction<Base>.
This mapped type will convert a sync function into an async one by wrapping its return value inside a Promise.

type PromisifyFunction<Function extends (...any) => any> =
  (...args: Parameters<Function>) => Promise<ReturnType<Function>>;

So:

If the Function parameter is a function: extends (...any) => any
Return a function that:
- take the same set of parameters: (...args: Parameters<Function>)
- Wrap the return type inside a promise: Promise<ReturnType<Function>>

Ok, so now that have our 3 primitives let's have a look at the implementation of the PromisifyObject<Base> mapped type:

type PromisifyObject<Base extends { [key: string]: (...any) => any }> = {
  [Key in keyof Base]: ReturnType<Base[Key]> extends Promise<any> ?
  Base[Key] :
  PromisifyFunction<Base[Key]>;
}

Now, this should make sense for you as we are following the same pattern as in the previous chapter:

We remap each key of the input parameter: [Key in keyof Base]
For each key we use a ternary operator to decide what to do: ReturnType<Base[Key]> extends Promise<any> ? ... : ...
If the return value of the current key in the input parameter is a Promise: ReturnType<Base[Key]> extends Promise<any>
Then we return that the same type
Otherwise use the PromisifyFunction<Base> defined earlier to convert this sync function into an async one.

And here we are!
Now we can combine our two custom mapped type to "Promisify" any kind of interface:

type DataServicePromisified = PromisifyObject<FilterOutAttributes<DataService>>;

The syntax is not the easiest to read, but once you get the things with the extends mechanism to apply constraint, and how to use it within a ternary operator, understanding such code is much easier :)

Now you may ask:
Okay, that's funny, but at which point in my day-to-day life is this thing useful?

Weel, check out my previous article for an example of where that could help you!

I hope you enjoy reading this article and that it was useful for you ;)

Happy (typed) coding!

Use Javascript Proxy for isolated context intercommunication

ClementVidal — Mon, 05 Apr 2021 21:34:34 +0000

What are "isolated context intercommunication"

When writing a web app we spend our time invoking functions, that's what applications are made of:
Functions that call other functions.

While calling function is a trivial operation in most environments, it can become more complicated when dealing with isolated Javascript contexts.

Isolated Javascript contexts are independent Javascript execution context that lives aside each other.
Most of the time they are sandboxed, meaning you can't access objects, variables, or functions created in one context from the other one.

The only way to do "inter-context communication" is to use a dedicated API (provided by the underlying system) that allows to send messages from one side to the other.

There are more and more API that use that approach:

Web Worker API with Worker.postMessage()
Web Extension API with runtime.sendMessage()
Electron API with ipcRenderer.send()

Once a message is sent from one side, you have to set up a message handler on the other side to do the effective processing and optionally return a value back to the sender.

The downside with that approach is that you are not "calling" a regular method anymore.

Instead of doing:

processData(inputData);

You have to send a message using one of the previous API in one context and install a handler in the other context to handle that message:

// In context A
sendMessage({name: "proccessData", payload: inputData});

// In context B
onMessage( msg => {
  switch (msg.name) {
     case "proccessData":
       processData( msg.payload );
  }
})

Wouldn't it be nice if we could just call processData(inputData) from context A, get the implementation executed on context B, and have all the messaging logic hidden behind implementation details?

Well, that's what this article is about:
Implementing a remote procedure call (RPC) that will abstract the messaging layer.

How Es6 proxy can help us

If you don't know what Javascript proxy is you can have a look at this article

In short, proxy allows us to put custom logic that will get executed when accessing an object's attribute.

For example:

// Our exemple service
const service = { processData: (inputData) => { } };

const handler = {
  // This function get called each time an attribute of the proxy will be accessed
  get: function(target, prop, receiver) {
    console.log( `Accessing ${prop}` );
    return target[prop];
  }
};

// Create a new proxy that will "proxy" access to the service object
// using the handler "trap"
const proxyService = new Proxy( service, handler );

const inputData = [];
// This will log "Accessing processData"
proxyService.processData(inputData);

Ok, now what's happen if we try to access an attribute that does not exist on the original object ?

// This will also log "Accessing analyzeData"
proxyService.analyzeData(inputData);

Even if the attribute does not exist, the handler is still called.
Obviously, the function call will fail as return target[prop] will return undefined

We can take the benefit of that behavior to implement a generic remote procedure call mechanism.

Let's see how.

In the upcoming sections I'll refer to "context A" and "context B" as being 2 isolated Javascript contexts.
With an electron app, "context A" could be the render thread and "context B" the main thread.
With a WebExtension, "context A" could be a content script and "context B" the background script.

Implementing the remote procedure call system

The code presented below is only for "explanation" purposes, do not copy/paste it.
Check out the end of this article, I've provided a github repo with a fully working project.

The "send request part"

At the end of this section, you'll be able to use our remote procedure call API on the "sender side" this way:

// In context A

const dummyData = [1, 4, 5];
const proxyService = createProxy("DataService");
const processedData = await proxyService.processData(dummyData);

Let's build that step by step:

First let's implement a createProxy() method:

// In context A

function createProxy(hostName) {
  // "proxied" object
  const proxyedObject = {
    hostName: hostName
  };

  // Create the proxy object
  return new Proxy(
    // "proxied" object
    proxyedObject,
    // Handlers
    proxyHandlers
  );
}

Here the interesting thing is that the proxied object only has one attribute: hostName.
This hostName will be used in the handlers.

Now let's implement the handlers (or trap in es6 proxy terminology):

// In context A

const proxyHandlers = {
  get: (obj, methodName) => {

    // Chrome runtime could try to call those method if the proxy object
    // is passed in a resolve or reject Promise function
    if (methodName === "then" || methodName === "catch")
      return undefined;

    // If accessed field effectivly exist on proxied object,
    // act as a noop
    if (obj[methodName]) {
      return obj[methodName];
    }

    // Otherwise create an anonymous function on the fly 
    return (...args) => {
      // Notice here that we pass the hostName defined
      // in the proxied object
      return sendRequest(methodName, args, obj.hostName);
    };
  }
}

The tricky part resides in the last few lines:
Any time we try to access a function that does not exist on the proxied object an anonymous function will be returned.

This anonymous function will pass 3 pieces of information to the sendRequest function:

The invoked method name
The parameters passed to that invoked method
The hostName

Here is the sendRequest() function:

// In context A

// This is a global map of ongoing remote function call
const  pendingRequest = new Set();
let nextMessageId = 0;

function sendRequest(methodName, args, hostName) {
  return new Promise((resolve, reject) => {

    const message = {
      id: nextMessageId++,
      type: "request",
      request: {
        hostName: hostName,
        methodName: methodName,
        args: args
      }
    };

    pendingRequest.set(message.id, {
        resolve: resolve,
        reject: reject,
        id: message.id,
        methodName: methodName,
        args: args
    });

    // This call will vary depending on which API you are using
    yourAPI.sendMessageToContextB(message);
  });
}

As you can see the promise returned by sendRequest() is neither resolved nor rejected here.
That's why we keep references to its reject and resolve function inside the pendingRequest map as we'll use them later on.

The "process request part"

At the end of this section, you'll be able to register a host into the remote procedure system.
Once registered all methods available on the host will be callable from the other context using what we build in the previous section.

// In context B

const service = { processData: (inputData) => { } };
registerHost( "DataService", service );

Ok, let's go back to the implementation:
Now that the function call is translated into a message flowing from one context to the other, we need to catch it in the other context, process it, and return the return value:

// In context B

function handleRequestMessage(message) {
  if (message.type === "request") {
    const request = message.request;
    // This is where the real implementation is called
    executeHostMethod(request.hostName, request.methodName, request.args)
      // Build and send the response
      .then((returnValue) => {
        const rpcMessage = {
          id: message.id,
          type: "response",
          response: {
            returnValue: returnValue
          }
        };

        // This call will vary depending on which API you are using
        yourAPI.sendMessageToContextA(rpcMessage);
      })
      // Or send error if host method throw an exception
      .catch((err) => {
        const rpcMessage = {
          id: message.id,
          type: "response",
          response: {
            returnValue: null,
            err: err.toString()
          }
        }

        // This call will vary depending on which API you are using
        yourAPI.sendMessageToContextA(rpcMessage);
      });
    return true;
  }
}

// This call will vary depending on which API you are using
yourAPI.onMessageFromContextA( handleRequestMessage );

Here we register a message handler that will call the executeHostMethod() function and forward the result or any errors back to the other context.

Here is the implementation of the executeHostMethod():

// In context B

// We'll come back to it in a moment...
const hosts = new Map();

function registerHost( hostName, host ) {
   hosts.set( hostName, host );
}

function executeHostMethod(hostName, methodName, args) {

  // Access the method
  const host = hosts.get(hostName);
  if (!host) {
    return Promise.reject(`Invalid host name "${hostName}"`);
  }
  let method = host[methodName];

  // If requested method does not exist, reject.
  if (typeof method !== "function") {
    return Promise.reject(`Invalid method name "${methodName}" on host "${hostName}"`);
  }

  try {
    // Call the implementation 
    let returnValue = method.apply(host, args);

    // If response is a promise, return it as it, otherwise
    // convert it to a promise.
    if (!returnValue) {
      return Promise.resolve();
    }
    if (typeof returnValue.then !== "function") {
      return Promise.resolve(returnValue);
    }
    return returnValue;
  }
  catch (err) {
    return Promise.reject(err);
  }
}

This is where the hostName value is useful.
It's just a key that we use to access the "real" javascript instance of the object which holds the function to call.

We call that particular object the host and you can add such host using the registerHost() function.

The "process response part"

So now, the only thing left is to handle the response and resolve the promise on the "caller" side.

Here is the implementation:

// In context A

function handleResponseMessage(message) {
  if (message.type === "response") {
    // Get the pending request matching this response
    const pendingRequest = pendingRequest.get(message.id);

    // Make sure we are handling response matching a pending request
    if (!pendingRequest) {
      return;
    }

    // Delete it from the pending request list
    pendingRequest.delete(message.id);

    // Resolve or reject the original promise returned from the rpc call
    const response = message.response;
    // If an error was detected while sending the message,
    // reject the promise;
    if (response.err !== null) {
      // If the remote method failed to execute, reject the promise
      pendingRequest.reject(response.err);
    }
    else {
      // Otherwise resolve it with payload value.
      pendingRequest.resolve(response.returnValue);
    }
  }
}

// This call will vary depending on which API you are using
yourAPI.onMessageFromContextB( handleResponseMessage );

Once we receive the response, we use the message id attribute that was copied between the request and the response to get the pending request object containing our reject() and resolve() method from the Promise created earlier.

So let's recap:

In context A:
- We have created a proxy object on host "DataService".
- We have called a method processData() on that proxy.
- The call was translated into a message sent to the other context.
- When the response from context B is received the Promise returned by processData() is resolved (or rejected).
In the context B:
- We have registered a host called "DataService".
- We have received the message in our handler.
- The real implementation was called on the host.
- The result value was ended back to the other context.

Final words

I've assembled all the code sample provided in this article in the following github repo:

ClementVidal / rpc-web-worker

Companion project for the Use Javascript Proxy for isolated context intercommunication article on Dev.to

Install and build:

Install this project using npm i and start it with npm start

See the result:

Open http://127.0.0.1:8080 in your browser

If that doesn't work, checkout your terminal output to see on which port the file are served.

Once done, checkout the console output.

The

Enjoy !

View on GitHub

It provides a full implementation of the remote procedure call system and demonstrates how it can be used with Web Workers.

Well...
That's it friends, I hope you enjoy reading that article.

I'll soon provide another one that will cover how to correctly handle Typescript typings with this system ;)

Happy coding !