Building a News API with Express and Redis to integrate AppSignal for performance tracking.

Badmus Kola Sulaiman — Fri, 08 Apr 2022 10:11:03 +0000

Introduction

Among the popular caching tool that is useful for speeding up an API response time is Redis. Heavy applications with a higher number of requests daily demand a higher response time.

In this article, you will not only learn how to build an API proxy with Redis but also learn how to integrate with AppSignal for measuring performance metrics.

AppSignal is a remote monitoring SAAS that helps developers understand how their application performs.

How do AppSignal works?

In modern tech teams, the need for productive team collaboration via remote monitoring cannot be overemphasised.

AppSignal provides API tools for tracking errors, monitoring app performances, and host metrics. This involves a notification message to the team messaging app of their choice when an AppSignal deployment or incident occurs, including exception and performance incidents. Integrating your app with AppSignal is very simple. In the next section, I will guide you through the steps involved. Sign up for a free AppSignal account here to follow the tutorial.

Introducing Redis

Redis has an interesting set of data structures, and it is sometimes called the data structure server.

Caching enables a web page to load faster by serving a web stage from mini storage from the browser.

Installing Redis.

Linux:

https://redis.io/topics/quickstart \ sudo service redis start

macOS:

Install brew here if you don't have it on your machine.
brew install redis
brew services start redis

Windows:

Install it like on Linux.

To check that Redis is working.

Given that you have started Redis, communicate with it via the redis-cli. Type **redis-cli ping **and Redis should answer with PONG. That means Redis is working.

To learn more about Redis, check this resource

Building our App

This section will kick start the guides to building the app, so let's get started.

Requirements

Node.js installed or by downloading it here
Redis installed or referred to the previous section
Any IDE of your choice
Sign up for the News-API key here
Sign up for a free AppSignal account here

Project set up

Go to your terminal, create ‘newsapp’ directory and cd into it.

Provided you have node.js installed, initialise a node project by typing npm init in your terminal to generate a package.json file.
npm init

Install project modules.

We need to install modules these modules; dotenv, newsapi, express, Redis, and AppSignal. In the later section, we shall learn more about why we used these modules.

npm install --save-dev dotenv express newsapi redis @appsignal/nodejs @appsignal/express

Source files breakdown

There are five files, namely app.js, appsignal.js, cacheClient.js, newsClient.js and .env file.

The app.js file

This is the entry point to our application. We must import the AppSignal module first, then other modules afterwards. Later in this section, we go deeper into how to configure our app with AppSignal.

//app.js
import { appsignal } from './appsignal.js';
import 'dotenv/config';
import express from 'express';
import { expressMiddleware } from "@appsignal/express";
import { getRedisClient } from './cacheClient.js';
import { fetchNews } from './newsClient.js';

const app = express();
const router = express.Router();
router.get('/news', async (req, res) => {
    try {
        if(!req.query.page) return res.status(500).send({ error: 'Invalid params '});
        const cacheClient = await getRedisClient();
        const key = `news${req.query.page}`;
        const newsInCache = await cacheClient.get(key);
        if (newsInCache) return res.send({ success: true, data: JSON.parse(newsInCache) });
        const newsFromApi = await fetchNews(req.query.page);

        // Add data from direct api to cache set time to live to 30minutes
        await cacheClient.set(key, JSON.stringify(newsFromApi), 'EX', 1800);
        return res.send({ success: true, data: newsFromApi })
    } catch(error) {
        res.status(400).send(error);
    }
});
app.use('/', router);
app.use(expressMiddleware(appsignal));
app.listen(process.env.PORT, () => {
    console.log('Proxy server running, on port ' + process.env.PORT);
});

Fetching data from NewsApi

The JSON fed to the Redis cache is from the NewsApi. We allow AppSignal to manage errors. Follow this link to learn more about this API.

//newsClient.js
import NewsAPI from "newsapi";
const newsapi = new NewsAPI(`${process.env.NEWS_API_KEY}`);
export const fetchNews = async (page) => {
  try {
    const result = await newsapi.v2.everything({
      q: "bitcoin",
      sources: "bbc-news,the-verge",
      domains: "bbc.co.uk, techcrunch.com",
      language: "en",
      sortBy: "relevancy",
      page,
    });
    console.log(result);
    return result;
  } catch (error) {
      console.log(error);
    throw new Error(error);
  }
};

Configuring Redis

In this section, we are configuring Redis as TLS. We are using Redis here to cache the response for 30 minutes.
Use the Redis URI generated on Heroku at deployment as your URL in the createClient function. If on development mode, use the default URL (redis://127.0.0.1:6379) for redis given that you have Redis installed on your machine. Below we used the one generated on Heroku.

//cacheClient.js
import { createClient } from "redis";
let globalClient = null;
export const getRedisClient = async () => {
  const client = createClient({url: process.env.REDIS_URL,
  socket: {
    tls: true,
    rejectUnauthorized: false
  }
  });
  client.on("error", (err) => console.log("Redis Client Error", err));
  await client.connect();
  return client;
};

Environment variables

These are the values in our environment variables.

NEWS_API_KEY=9eafe26143454c6abac986b2b4e6a788
PORT= 3000
APPSIGNAL_PUSH_API_KEY=1758908e-6cd0-49c3-af24-06e6d0ca72f6
APPSIGNAL_APP_NAME= newsapp
REDIS_URL = redis://:p8282f4c43f55111d669c29b9df088eee953f0dcab8d315496d94897b5bc32a32@ec2-3-226-160-187.compute-1.amazonaws.com:12900

How to install and configure our app with AppSignal.

These guides walk you through sending data from your local development environment to AppSignal for measuring performance metrics in our app.
The major part of our appsignal.js is used here.

Create a free AppSignal account
Install your app
Configure your app
Fire up your main app.js to send data to AppSignal
Explore your data

Installation Guide

AppSignal is very easy to integrate.

First, create an account.
Given that you have logged in click on the language you would like to use for the integration.
In our case, choose node.js and click next.

Now go back to your IDE or terminal to configure AppSignal.

Configuration Guide

This involves two steps;

Install modules
Import and construct AppSignal object

Let's create a separate file for AppSignal; so we have appsignal.js.

Install AppSignal modules.

Ignore if you have already installed them in the earlier sections.

npm install --save @appsignal/nodejs @appsignal/express

Import and construct AppSignal object

Import the AppSignal object and supply its constructor with the required info.

AppSignal constructor requires the following;

Set active to true to actively track errors.
Name your app; in our case, we say news-app.
pushApiKey; add the API key generated at the installation stage

To use this function as a predefined module, we use the keyword “export” in line 3.

//appsignal.js
import { Appsignal } from "@appsignal/nodejs";
export const appsignal = new Appsignal({
  active: true,
  name: 'news-app',
  pushApiKey: '1758908e-6cd0-49c3-af24-06e6d0ca72f6'
});

Given that we are app.js file, let's import the above function from appsignal.js.

import { appsignal } from './appsignal.js';

Note that the above module is predefined and must be imported before other modules.

The final step is to use the middleware.

Use the middleware with AppSignal as an argument at the end of your routes.

Importing the middleware

import { expressMiddleware } from "@appsignal/express";

The usage

Note // ADD THIS AFTER ANY OTHER EXPRESS MIDDLEWARE, AND AFTER ANY ROUTES!

app.use(expressMiddleware(appsignal));

And boom, our app is connected to the AppSignal. To see your data, make sure your server is running, make some calls to your endpoints, log in to your AppSignal account, go to your dashboard and choose your app.

Conclusion

At the end of this article, we learned how to use Redis and AppSignal for Node and Express applications. We also built a NEWS API using News-API.
Find the link to the deployed app here.
Happy Coding!!!

DEV Community: Badmus Kola Sulaiman