A trick to improve speed when you are interfacing with a slow API

Christos Panagiotakopoulos — Mon, 05 Oct 2020 10:59:42 +0000

TLDR; I created a small npm package that acts as a wrapper around node-fetch, and returns the same promise for the same request, until it resolves. You can visit the repo of this package here. Below, I explain my motivation, and how I tackled the issue.

So here's the scenario:

You have a system that interfaces with a really slow third-party API. User Bob, needs some data, so your system performs a request to the third-party API, and waits for a response. In the meantime, user Alice needs the same date and the system performs the same request to the API on behalf of her. Both users are now waiting for two requests that the only difference they have, is the execution time.

If a request to this API has an average response time of 1 second, both users will wait 1 second. Also, you would need to occupy resources in your system and the third-party API for more than 1 second, and for 2 seconds at most!

The solution

What if you could have both users, Bob and Alice, wait for the same request? Then, although Bob will still wait for the request for 1 second, Alice will use Bob's request, and wait less time for the response.

To achieve that, we'll need a promise-cache subsystem. This subsystem will consist of a data structure to store our requests' promises and of a way to retrieve them/delete them when they are not needed.

The data structure

We need a data structure to store our promises inside. This data structure needs to be able to store and retrieve a new promise in one operation (O(1)). So, the best choice would be a key/value store. Javascript, offers two such structures, the basic object and the Map() instance. The most preferrable data structure for our use-case among the two is the Map().

So, let's create it:

const promiseCache: Map<string, Promise<Response>> = new Map();

The retrieval/storage

Now, let's create a function that wraps around the request function and retrieves the same promise for the same request, if it exists. If it doesn't, it performs a new request and stores it in the cache.

function memoizedRequest(url: string) {
    const key = url;
    if (promiseCache.has(key)) {
        return promiseCache.get(key);
    }

    const promise = request(url);
    promiseCache.set(key, promise);

    return promise;
}

With this, we have achieved the basic function of our promise-cache subsystem. When our system performs a request using the memoizedRequest function, and the request has already happened, it returns the same promise.

But, we haven't yet implemented the mechanism for the deletion of the promise from the cache when the promise resolves (when the request returns results)

The deletion - cache invalidation

For this, we'll create a function that awaits for the promise to resolve and then delete the promise from the cache.

async function promiseInvalidator(key: string, promise: Promise<any>) {
    await promise;
    promiseCache.delete(key);

    return promise;
}

And then we'll modify our memoizedRequest function to include this invalidation function:

function memoizedRequest(url: string) {
    const key = url;
    if (promiseCache.has(key)) {
        return promiseCache.get(key);
    }

    const promise = promiseInvalidator(key, request(url));
    promiseCache.set(key, promise);

    return promise;
}

But what happens with more complicated requests?

Not all requests can be differentiated by just the url they are performed on. There are many other parameters that make a request different (eg: headers, body etc).

For that, we'll need to refine our promise-cache's key and add an options object on our function:

function memoizedRequest(url: string, options: RequestOptions) {
    const key = url + JSON.stringify(options);
    if (promiseCache.has(key)) {
        return promiseCache.get(key);
    }

    const promise = promiseInvalidator(key, request(url));
    promiseCache.set(key, promise);

    return promise;
}

Now, only the requests that use exactly the same options will return the same promise until they resolve.

With this, we implemented all the basic functionality of our package. But we haven't taken into account the possibility of a request failure. Let's add this on our code, by making the promiseInvalidator function to always remove the promise from the cache either when it resolves, or when it rejects.

async function promiseInvalidator(key: string, promise: Promise<any>) {
    try {
        await promise;
    } finally {
        promiseCache.delete(key);
    }

    return promise;
}

More improvements

This implementation has a small drawback, that can prove serious on a production system. All the requests' data, are stored within the key of our data store, highly increasing the memory requirements of our application, especially when our requests contain a lot of data. The solution to this is to use a hash function on our key, to assign a unique value to each different request, without needing to include all the actual of the request.

const key = hasher(url + JSON.stringify(options));

Caveats

This solution, isn't applicable to any situation. To use this solution, you need to ensure that the API you are interfacing with, is not providing different responses for two different requests in the amount of time it will take for those requests to resolve.

The package

If you don't want to code this for yourself, I created a simple npm package that does all of the above, as a wrapper to node-fetch (or any other fetch-like function you choose).

import memoizedNodeFetch from 'memoized-node-fetch';

const fetch = memoizedNodeFetch();

(async () => {
    const fetch1 = fetch('https://jsonplaceholder.typicode.com/todos/1');
    const fetch2 = fetch('https://jsonplaceholder.typicode.com/todos/1');

    // This should return true because both requests return the same promise.
    console.log(fetch1 === fetch2);

    const res1 = await fetch1;
    const res2 = await fetch2;

    console.log(await res1.json());
    console.log(await res2.json());
})();

You can see all of the above work, on its Github repository here:

https://github.com/chrispanag/memoized-node-fetch

PS. 1: Although this can be used in the front-end, I can't find a very useful use-case for it, especially when you have other packages such as react-query/swr, that although they perform a different function than the above, can sometimes remove the need for it.

PS. 2: Special thanks to the other two contributors of this repository (ferrybig and Bonjur for their invaluable input and suggestions!

Why would you want an API to print a PDF?

Christos Panagiotakopoulos — Tue, 15 Sep 2020 12:14:01 +0000

Why would somebody need an API to just print a PDF file? Well... Hear my short story:

At the company where I work, we developed a small "kiosk" React app that is running on a full-screen browser and uses speech recognition to gather some details and then automatically print a receipt.

The issue is when you issue the command at the browser to print the page, it opens the system's print dialog. As a result, the experience becomes mediocre for the user. The app is running on localhost (and will never run on the web) so this allowed me to create a simple solution:

Run a small nodejs backend on localhost that directly interfaces with the printer and issues the command to print without opening any system dialog.

Because I some of you might benefit from it, I published it on Github here, to check it out. It's fairly basic but it does its job (and why develop something when it is already working?)

It could also be used for IoT applications, I have some usecases in mind for my small RasPi.

You can check it out here:

chrispanag / printer-api

Upload a PDF and automatically print it 🖨️

Printer API

Ever wanted to upload a PDF from your web app and just print it? Here's the solution ;)

Getting Started

Clone the repo
yarn
Edit the .env file. (See below for details)
yarn run build
yarn start

How to edit the .env file

PRINTER_NAME=<THE_NAME_OF_YOUR_PRINTER> (or delete the key completely to use your 'default' printer)
PORT=<PORT> (Default is 3000, change it as you wish)

How to perform the request

curl --location --request POST 'http://localhost:3000/print' \
--form 'pdf=@<path_to_file>'

Troubleshooting

I've only tested it on a Linux system (Ubuntu), and most probably it won't work on Windows. Also, it won't run (at least out of the box) on Docker.

Acknowledgments

Thanks to @thiagoelg for maintaining the npm module to interface with the printer.

View on GitHub

Ebony: An open-source multi-channel chatbot framework

Christos Panagiotakopoulos — Tue, 01 Oct 2019 19:45:41 +0000

I've been working with chatbots for almost 4 years. The result of all this work is the Ebony Framework, a multi-channel, easy to use, module-based chatbot framework.

It already powers a few production-grade high-traffic bots.

The thing is that up until now, the only person that ever used this framework was me. As a result, I need people to take a look at it and share feedback so I can refine it even more and make it a decent open-source project.

You can find the Ebony Framework on GitHub. Feel free to share any comments, feedback and questions with me! :)

DEV Community: Christos Panagiotakopoulos