DEV Community: Ilya Pirozhenko

✨ The new way of testing REST API's

Ilya Pirozhenko — Thu, 06 Jun 2019 16:08:52 +0000

I’m a software engineer in a small company. In short, this means I do everything related to the software used in the business, from dev-ops and CI setup, to quality assurance. Most of my time is spent developing REST API’s for our company’s clients. The only problem I have is this: I don’t feel confident in the quality of my REST API’s.

I guess some people would recommend using TDD, make a test-coverage of 100% and you’re sorted. I’ve done all of that, but it will not rescue you in production environment.

In my experience, unit tests and test-driven development work pretty well for small encapsulated segments of code. It works if you’re developing some library or part of code that performs math, for example. Generally speaking, the code that doesn’t depends on some external services. If you’re developing REST API’s, you’ll mainly rely on database or queue or cache services, e.g. an external service.

Maybe smoke tests? Yes, they’ll alert you if a known problem occurs, but they won’t solve the problem completely.

What I actually need is some type of integration test. This is the only one you can trust as it works in the production environment and covers the whole process, from the client request to the client response.

What is the best way to carry out the integration test?

The standard way is to run a script that will imitate the client’s behaviour and call the API’s methods in a specific order. This script runs on the “stage” or “production” environment under a special test account. After each run, you’ll also need to delete all the records created by the test account or recreate the whole database. Sounds difficult. You’ll also need to come up with test scenarios and describe them in code, as well as supporting these scenarios after each small change in the API’s interface. I think this approach is good for an enterprise level company with a dedicated QA team, but not for an indie developer. Indie developers need a more simple and efficient approach.

Maybe monitoring?

After looking around, my first idea was to use some monitoring service to solve my problem. Actually, my teammate wrote a whole article about Monitoring in 2019.

For a time, we tried APMs (like NewRelic and ELK stack) and “pinging” services (like PingDom, Assertible or Checkly).

The APM solutions have an agent that captures all the required data, but lacks the ability to quickly and easily define assertions. As proof, try to set up alerting in ELK stack for some unobvious case. I also found that NewRelic’s pricing policy is a bit too high for adding a batch of REST APIs.

Alternatively, there are several “pinging” services that have absolutely wonderful ways of defining assertions for each and every method of REST API, but lack access to the client’s requests and responses.

That’s why I came up with an idea:

What if I could capture each and every client request and corresponding response to my API with the help of an agent library (like an APM service)? But then let the developer define a set of assertions on each of the REST API methods (like a “pinging” service).

I call it the “indie hacker way” of doing an integration test for the whole API in just two or three clicks.

What are the pros?

You don’t need to create test scenarios and then describe them in code, as the actions of your clients are recorded for you. And it would always be up to date because the client is always right :) I mean if a client can’t call your API in the proper order, then it probably means that your documentation sucks.

Secondly, your test coverage will be around 100% because every request will go through assertions. You could argue that there would be methods that clients do not call and these would not be tested. Ok, that’s true, but do you need such methods?

Another feature of "the indie hacker way" is the possibility of rate testing. For example, you can define an assertion for the minimum or maximum number of requests per second for each and every method of your API. If something goes over the threshold, you’ll be notified.

Last but not least, you can assert response times for the customer and be confident that all requests are running under 100ms, for example.

What are the cons?

Yes, it’s not perfect. As I said above, if you have dedicated QA team then it’s better to go with integration tests in the usual sense, or configure a proper APM solution.

First of all, you’re adding some computational load to your app as a result of capturing and sending each and every request. It’s not a big deal, but for some apps it may be a problem.

Secondly, you can’t define any custom logic or dependency between methods in the test.

Implementation

Inspired by this idea, I quickly developed a web service to define assertions and receive notifications. It’s called SLAO. I use it on a daily basis and have added it to all my REST APIs.

How does it work?

Install agent to capture requests (now only node.js/express is supported)
Define assertions in simple web interface
Receive notification if something goes wrong

If you like the idea, try it now on 📊 SLAO and provide your feedback via chat or via email hello@slao.io

Originally posted: https://slao.io/blog/posts/new-way-of-testing/

💯 Counting requests in your Node.js + Express application

Ilya Pirozhenko — Thu, 18 Apr 2019 10:59:37 +0000

Foreword

What is the purpose of counting requests to your web application?

As I wrote in previous post, knowing the number of requests helps you answer next important business questions:

Is anyone using my API? (if requests count is zero then it’s probably nobody)
Is my API working? (if requests count is zero than it’s probably broken)
Is my API under a DDoS attack ? (if requests count during the last hour is much higher than average than probably it is)

In my case it was a business need - every request with status code “200” to a specific method of my REST API was adding a few cents to our company’s bank account. That’s why we decided to go into the matter.

First of all, we explored all existing paid and free monitoring tools to make a long story short, none of them was a perfect fit.

Secondly, I googled for npm libraries that count requests. I found that in 90% of cases developers count requests for requests rate limiting purposes. Rate limiting is another subject not related to my task in this case.

Roughly speaking, my task was to count all the requests grouped by methods and status codes.

Writing a middleware

My web app is a REST API written on Node.js + Express. To simplify things here is the boilerplate:

const app = require('express')()

app.get('/api/', (req, res) => {
    res.sendStatus(200)
})

app.listen(3000, () => {
    console.log('Server started')
})

The only one legit method to capture all the requests in Express framework is to implement a middleware function and load it before any other handlers.

Quote from the official Express.js docs:

Middleware functions are functions that have access to the request object (req), the response object (res), and the next function in the application’s request-response cycle. The next function is a function in the Express router which, when invoked, executes the middleware succeeding the current middleware. Middleware functions can perform the following tasks: Execute any code. Make changes to the request and the response objects. End the request-response cycle. Call the next middleware in the stack. If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function. Otherwise, the request will be left hanging.

Just to understand what was happening in my app, I wrote this middleware function (see below) and made several requests.

app.use((req, res, next) => {
   console.log(`${req.method} ${req.originalUrl}`) 
   next()
})

The results are

> curl http://localhost:3000/api
GET /api

> curl http://localhost:3000/api/
GET /api/

> curl http://localhost:3000/api?q=test
GET /api?q=test

Ok, it’s working. Let’s add an ability to capture the response status code. Node.js has a default event that is fired when the response has been sent. More specifically, this event is emitted when the last segment of the response headers and body have been handed off to the operating system for transmission over the network. This hook is res.on("finish").

I should notice that not every request comes to the “finish” state, in real life client can close the connection before the response is sent. In this case Node.js emits only res.on("close") event. To keep this post as simple as it can be I decided to ignore these types of requests.

I modified my middleware to add the info about the response status code

app.use((req, res, next) => {
   res.on("finish", () => {
       console.log(`${req.method} ${req.originalUrl} ${res.statusCode}`) 
   })
   next()
})

The results are

> curl http://localhost:3000/api
GET /api 200

> curl http://localhost:3000/api/
GET /api/ 200

> curl http://localhost:3000/api/?q=test
GET /api?q=test 200

We captured the http verb, the status code and the original url. As you can see the originalUrl is different for each request but the handler path is always the same, it’s api.get("/api"). Let’s capture the handler path instead of the originalUrl. It’s a bit tricky.

Express stores the data about the handler path in req.route object. The object is filled with data only after the handler processed the request. As mentioned above the hook res.on("finish") is called after all the handlers have been executed and the response has been sent. So we should inject a capturing code right in res.on("finish"). Also we should keep in mind that there may be requests without a handler and we also should process them somehow.

I wrote a small helper function to get the correct handler path

getRoute(req) {
   const route = req.route ? req.route.path : '' // check if the handler exist
   const baseUrl = req.baseUrl ? req.baseUrl : '' // adding the base url if the handler is a child of another handler

   return route ? `${baseUrl === '/' ? '' : baseUrl}${route}` : 'unknown route'
 }

And modified the middleware

app.use((req, res, next) => {
   res.on(‘finish’, () => {
       console.log(`${req.method} ${getRoute(req)} ${res.statusCode}`) 
   })
   next()
})

Now the results are consistent

> curl http://localhost:3000/api
GET /api 200

> curl http://localhost:3000/api/
GET /api 200

> curl http://localhost:3000/api?q=test
GET /api 200

> curl http://localhost:3000/
GET unknown route 404

> curl -X POST http://localhost:3000/
POST unknown route 404

Data persistance

The last but not least step is storing the captured data. I decided to store the data in next format:

{
    "GET /stats/ 200": 11, // "route name": "number of requests"
    "GET /api/ 200": 7,
    "GET unknown route 404": 2,
    "POST unknown route 404": 1
}

For demo purposes we will store the statistics in a JSON file. Let’s add two helper methods to read and dump the data.

const fs = require('fs')
const FILE_PATH = 'stats.json'

// read json object from file
const readStats = () => {
    let result = {}
    try {
        result = JSON.parse(fs.readFileSync(FILE_PATH))
    } catch (err) {
        console.error(err)
    }
    return result
}

// dump json object to file
const dumpStats = (stats) => {
    try {
        fs.writeFileSync(FILE_PATH, JSON.stringify(stats), { flag: 'w+' })
    } catch (err) {
        console.error(err)
    }
}

Also, I modified the middleware to add persistance to the statistics

app.use((req, res, next) => {
    res.on('finish', () => {
        const stats = readStats()
        const event = `${req.method} ${getRoute(req)} ${res.statusCode}`
        stats[event] = stats[event] ? stats[event] + 1 : 1
        dumpStats(stats)
    })
    next()
})

And created the /stats method which returns the statistics.

app.get('/stats/', (req, res) => {
    res.json(readStats())
})

We’re done, let’s make a few requests and check the stats.

> curl -X GET  http://localhost:3000/api/
> curl -X POST http://localhost:3000/api/
> curl -X PUT http://localhost:3000/api/
> curl http://localhost:3000/stats/
{
    "GET /api/ 200": 1,
    "POST unknown route 404": 1,
    "PUT unknown route 404": 1
}

As you can see, we have number of request for every route in our app. The whole code of this sample app can be found on GitHub.

Conclusion and next steps

In this post I described the basics of request counting. Keeping all your data in a file might not work properly in production, you should persist it in somewhere less volatile, such as a database. It could be Redis, InfluxDB, ElasticSearch, MongoDB, etc. Personally, our Node.js + Express monitoring service SLAO uses an InfluxDB cluster.

Also, in the real world you’d like to have more detailed stats like requests per day/minute/hour along with an ability to view your data in a more convenient way than a JSON returned by your API. What’s more, a plain dashboard with statistics is useless unless you have an alerting attached to it. We’ll cover all these topics later on.

I’m building 📊SLAO: Node.js + Express monitoring. Sign up for a free trial!
Not sure yet? Just press 🧡 for this post.

Originally posted here

📍 Four key metrics of REST API every developer should worry about

Ilya Pirozhenko — Wed, 17 Apr 2019 17:42:39 +0000

Hi, software engineer, did you think what happens with your app when it goes into the wild? Have you ever faced the situation when requests from your clients differ a lot from your test ones? Do you recall all your complaining clients who were sending requests with wrong “Content-Type”?

First time I connected an Application Performance Monitoring (APM) to my REST API was like: “Oooh what should I do with this volume of new data?”. What is bad and what is good? Should I worry about CPU load? Or maybe requests count?

To answer these questions you should think about your REST API from the client’s side. Your client doesn’t care about the CPU load when your API runs smoothly, but he will definitely panic if your API responds slowly, this can happen even when the CPU usage is low. So let’s agree on that not all metrics are valuable. After my 10+ years of experience in developing REST API’s I came up with a rule: Every metric should be an answer to a simple question related to your business and your clients’ experience.

Good metrics are related to your business and affect clients, for example: “What amount of time a client should wait till he gets the response?” (e.g. request duration), “How many clients is my app serving right now?” (e.g. count of requests by clients), “Has a client received the response?” (e.g. connection state).

On the other hand, here is the list of metrics that are pretty interesting for a developer, but don’t have a direct impact on your client: “What amount of memory does my application use?” (e.g. memory consumption), “What number of i/o ops does my app performs?” (e.g. i/o ops per minute). All these metrics are not strictly connected to your business, because who cares If your app consumes all the memory but still responds in 10 ms.

Now you known the philosophy behind my key metrics, based on it I made a compilation of critical REST API metrics you should worry about as a developer:

1. Requests count

Requests count is a simple but super useful metric. With this metric you can answer next important business questions:

Is anyone using my API? (if requests count is zero then it’s probably nobody)
Is my API working? (if requests count is zero than it’s probably broken)
Is my API under a DDoS attack ? (if requests count during the last hour is much higher than average than probably it is)

2. Response codes

When I was a beginner software developer I was wondering why we just can’t send code 200 for each request? Response status code is the easiest way to understand what happened with the request without reading and decoding the response body. So it’s important always to respond with a proper code in your REST API. With response status code you can answer next questions:

Are clients properly calling my REST API methods? (e.g. no 4xx codes)
Has my API ever crashed? (e.g. 5xx codes)
Is my API running properly? (e.g. only 2xx codes)

3. Requests duration

Gold standard for request duration is 200 ms, but let’s be honest it’s not true for the real world. In my experience some methods’ calls could take up to 60 seconds and it was still OK for the client. So you should negotiate the upper threshold of requests’ duration with your client, and fix its value in the SLA (Service Level Agreement). Armed with this metric you can answer next valuable business question:

Is my API responding not slower than it should? (e.g. all responses are sent faster than the threshold time)

4. Is request finished

This metric describes the connection state. It’s highly related to the requests duration metric. Having unfinished requests means that your client had closed the connection before the response arrived. Next question comes to my mind:

Do my clients always receive the response? (If there are a lot of unfinished requests, then it’s definitely a reason to check this out)

Sum up

Requests count, response time, status codes and connection states are the four simple metrics you should keep an eye on. I personally have been using these metrics for the last 3 years and have no plans to stop doing it.

I’m building 📊SLAO: Node.js + Express monitoring. Sign up for a free trial!
Not sure yet? Just press 🧡 for this post.

Originally posted here

📊 SLAO: Node.js + express monitoring tool

Ilya Pirozhenko — Tue, 26 Mar 2019 14:08:45 +0000

I've created a tool to monitor the performance and correctness of your REST API from a single dashboard. Unlike other overloaded APMs, SLAO is about the essentials.

Give it a try here 👉 https://slao.io