Supercharge your API with Compression

Mike Eason — Wed, 14 Apr 2021 10:12:09 +0000

Performance is critical to any API, taking the time to reduce API response times to be as low as possible is absolutely worth the effort.

Take a look at this example API request in Solaris:

The API response is 44.6kB with a time of 584ms. This isn't terrible but could be smaller and faster with compression.

Express Compression

If you're familiar with Node.js you have probably used Express.js at some time or another.

Express is extensible and has a large number of middleware libraries that can be bolted on. One such library is compression.

To install compression:

npm install compression

Then its a case of simply calling the Express use function to register the middleware like this:

const express = require('express');
const compression = require('compression');
const app = express();

...

// Compress all responses
app.use(compression({
    threshold: 0 // Byte threshold (0 means compress everything)
}));

...

Easy, right? Now calling the same API endpoint we get this:

The response is now 8.1kB and a time of 101ms, that's over 5x faster than before!

Compressing Specific Responses

With the above code, we'll be compressing all responses, if for some reason you'd like to not compress a response from the API then we can override the filter function like this:

app.use(compression({
    threshold: 0,
    filter: (req, res) => {
        if (req.headers['x-no-compression']) {
            // don't compress responses if this request header is present
            return false;
        }

        // fallback to standard compression
        return compression.filter(req, res);
    }
}));

Any API request with the x-no-compression header will be ignored.

And that's it, your API will now serve compressed responses and should now be performing even better than before!

In my spare time I develop an open source strategy game called **Solaris, check it out.

How I migrated my development environment to a Raspberry Pi

Mike Eason — Mon, 12 Apr 2021 14:39:19 +0000

In my spare time I develop an open source strategy game called Solaris, check it out.

One of the problems with software development is that you need to set up a development environment on every device that you code on, also there can be differences between dev and production which often lead to unforeseen issues.

An ideal solution would be to be able to develop on any device without having to go through a long setup process other than installing VS Code (even this is optional as you will see later) on my development machine and have all of the code and apps hosted on a remote server. In this case, a Raspberry Pi.

Introducing the Raspberry Pi

The Raspberry Pi is an awesome little device capable of a lot of different tasks, like most tinkerers I have a few lying around doing nothing.

Solaris runs on Ubuntu Server, the Pi is the perfect candidate to mimic production as it can run the same software in an almost identical environment.

The Plan

The end result of this exercise will be set up the development environment to be as close as possible to what runs in production:

Install everything required to run Solaris.
Run the client and server applications in dev mode.
Use Visual Studio Code with ssh to edit files and debug the applications.

Pi Installation and Setup

The setup for Ubuntu running on the Pi is super simple. Using the Pi imager software, I just chose Ubuntu Server and using the advanced options, set up the hostname, WiFi and SSH there and then. There was no need to even plug in a keyboard and monitor once this installation was complete, just insert the SD card, power the Pi and SSH in from another machine for the rest of the setup process.

Setting Up Node and MongoDB

Solaris requires Node.js and MongoDB to run, it was straightforward to get these running as they are very well supported and documented.

The next step was to git clone the repository down and walk through the steps in the README:

git clone https://github.com/mike-eason/solaris.git

Setting Up Apps and Services

The Node applications will be managed on the server by PM2. After installing pm2 the setup is easy:

API - pm2 start ~/solaris/server/api/index.js --name solaris-api --watch
Server jobs - pm2 start ~/solaris/server/jobs/index.js --name solaris-jobs --watch
Vue.js client - pm2 start ~/solaris/client/node_modules/@vue/cli-service/bin/vue-cli-service.js --name solaris-client --node-args="serve"

The above is where the magic happens. pm2 will manage the 3 processes, automatically watch for changes and restart the apps when necessary. The app is now accessible via the local IP address 192.168.1.xxx:8080, amazing!

Setting Up Visual Studio Code

Now the apps are running and watching for changes, I needed a way to actually edit files. I chose to use VS Code for this as it has an awesome extension for remote development via ssh.

Debugging

The only caveat with debugging is that it is slightly inconvenient to stop the pm2 process and then launch a VS Code debugging session.

The best way to get around this is to enable the --inspect flag when launching the pm2 process:

pm2 start solaris-jobs --node-args="--inspect=9230" --watch

Then using a VS Code configuration to attach to the process.

{
    "type": "node",
    "request": "attach",
    "name": "Attach to Jobs",
    "port": 9230
}

For more info on the --inspect flag, see here.

Final Thoughts

With all of that setup I can now simply open VS Code on my development machine and get straight into coding without really having to think about getting set up first. The environment is remote so any changes are persistent, I can code on my desktop and then switch over to my laptop and continue what I was working on.

I could take this a step further and use something like code-server which will actually host VS Code itself on the Pi. Then all I need is a device with a browser and I'm good to go. Finally, I'll be able to code on my Samsung Smart Fridge!

Let me know what you think in the comments or if you have any suggestions. Thanks for reading.

DEV Community: Mike Eason