DEV Community: Anudeep

Why Choose TestBeats Over Direct Slack Webhooks for Test Notifications

Anudeep — Tue, 10 Feb 2026 18:25:14 +0000

When teams automate tests, getting meaningful results into Slack is essential. Many projects start by sending simple alerts using Slack incoming webhooks — a direct way to push messages to a channel when tests run or fail. But while webhooks are straightforward, they’re just a delivery mechanism — they don’t add structure, analysis, context, or insights to your test results.

TestBeats, by contrast, is a dedicated test reporting platform that takes your raw automation results and turns them into actionable information before they hit Slack. Rather than only sending a basic pass/fail message via a webhook, TestBeats processes, analyzes, and enriches your test data first.

Better Context & Insight

With TestBeats, Slack messages aren’t just notifications — they’re summaries with meaning. The platform can package test failures with AI-powered summaries, categorize errors (like new failures, flaky tests, or recurring issues), and highlight what matters most. This means your team spends less time deciphering logs in Slack and more time fixing real issues.

Unified & Structured Reporting

Slack webhooks simply send whatever payload you give them — but you’re responsible for formatting, parsing, and contextualizing that data. TestBeats collects results from all major test frameworks (JUnit, Mocha, Playwright, Cypress, etc.), normalizes them, and provides structured reports. Teams get consistent, easy-to-read summaries rather than fragmented raw notifications.

Seamless Toolchain Integration

Slack webhooks are a low-level notification channel; integrating them well typically requires custom scripts or tooling to wrap your test runners and format messages. TestBeats handles this orchestration for you — you publish your test results once, and it sends smart, configurable notifications to Slack (and other platforms).

Team Collaboration & Workflow

Because TestBeats understands test results across runs and time, it helps teams track trends, historical failures, and patterns — things that simple Slack messages can’t deliver on their own. This broader visibility improves collaboration and speeds decision-making.

In summary

: if your goal is simple alert delivery, Slack webhooks can do that. But if you want context-rich, structured, and actionable test reporting that helps your whole team understand and act on test outcomes more efficiently, TestBeats offers a smarter, more powerful solution — integrating Slack not just as a channel, but as part of a full test reporting workflow.

Slack Test Reporting

Anudeep — Wed, 12 Nov 2025 05:44:24 +0000

In the world of continuous integration and delivery (CI/CD), fast feedback is crucial. Waiting for someone to manually check a dashboard for test results can slow down your entire team. This is where TestBeats and its powerful Slack integration step in, bringing test visibility directly to your team's communication hub.

TestBeats is an AI-Powered Test Analysis Platform that unifies test results from various frameworks (like Cypress, Selenium, Playwright, and PyTest) and delivers enhanced reports instantly.

Why Report Tests to Slack?

The core benefit of using TestBeats with Slack is simple: instant, actionable collaboration.

Get Results Where You Work: Instead of forcing developers and QAs to check a separate reporting tool, TestBeats pushes a summary of every test run directly into a designated Slack channel.
AI-Powered Insights: TestBeats doesn't just send a pass/fail count; it uses AI to summarize test failures, highlight the most common errors, and categorize issues (e.g., newly failed, recurring, or flaky tests). This immediate context accelerates troubleshooting.
Actionable Alerts & Context: The Slack notification includes a clean summary along with crucial links and metadata. You get one-click access to detailed logs, artifacts, and the full report.
Smart Tagging: TestBeats can even tag the right on-call engineers directly in the Slack alert, ensuring the correct person is notified immediately to address new failures.

The Simple Setup

Getting automated test results into Slack with TestBeats is designed to be straightforward:

Centralize Results: Run your tests using any popular framework.
Configure Slack: Set up a webhook URL for your target Slack channel.
Publish: Use the TestBeats Command Line Interface (CLI) to send your test results (e.g., JUnit, TestNG, or Cucumber files) to the TestBeats platform, which then automatically formats and pushes the smart report to your configured Slack webhook.

npx testbeats publish --slack <slack-url> --junit <file-path>

By integrating TestBeats with Slack, you transform passive test reports into active, collaborative alerts, ensuring that everyone on the team has the real-time quality status needed to ship better software faster.

References

Github

From Requests to Reports: Clean Logging in API Testing

Anudeep — Sun, 04 May 2025 09:19:19 +0000

While many testers are familiar with the basics of API testing—validating status codes, parsing response bodies, and testing CRUD operations—expert-level API testing goes far beyond that. To build a resilient and maintainable test suite, testers must adopt advanced practices like structured logging, traceability, and clear reporting.

In this article, we explore logging best practices that are largely tool-agnostic, but we'll demonstrate them using PactumJS, a powerful and extensible API testing tool, along with Mocha, a popular JavaScript test framework. For logging, we’ll use Pino, one of the fastest and most reliable structured loggers for Node.js.

Here is the sample repo https://github.com/pactumjs/pactum-mocha-boilerplate to see the code in live action.

🎯 Why Logging Matters in API Testing

Running thousands of API tests is meaningless if, when something fails, you can’t answer the following:

❓ What failed?

❓ Why did it fail?

❓ When did it start failing?

❓ Is it a consistent or intermittent failure?

Without structured and persistent logging, diagnosing failures becomes time-consuming and unreliable. Reporting+Logging helps testers see the full picture, improves debugging speed, and strengthens collaboration across the team.

🤓 Concept: See the Full Picture with Perfect Logging

In real-world API automation suites—especially those integrated into CI/CD pipelines—you need more than just a test failure message. You need context:

📤 Request payload

📥 Response body and headers

🔄 Dynamic variables passed between steps

Without these, troubleshooting is reduced to guesswork.

🧰 Solution: Pino + PactumJS Event Logging

To implement clean and contextual logging, we’ll use Pino to persist logs and hook into PactumJS’s event system to automatically log request and response details after every API interaction.

Step 1: Set Up a Custom Logger

Create a logger using pino, and write all logs to a file:

// logger.js
const { pino } = require('pino');

const logger = pino({
  level: 'info',
  transport: {
    target: 'pino-pretty',
    options: {
      destination: `logs/${Date.now()}.log`,
      colorize: false,
      translateTime: 'UTC:yyyy-mm-dd HH:MM:ss.l o',
    },
  },
});

module.exports = { logger };

Step 2: Hook Into Pactum Events

Use Pactum’s built-in AFTER_RESPONSE event to capture and log each request/response:

// event.logger.js
const { events } = require('pactum');
const { logger } = require('./logger');

const { pactumEvents, EVENT_TYPES } = events;

pactumEvents.on(EVENT_TYPES.AFTER_RESPONSE, ({ request, response }) => {
  logger.info({
    request: {
      method: request.method,
      url: request.url,
      headers: request.headers,
      body: request.body,
    },
    response: {
      statusCode: response.statusCode,
      responseTime: response.responseTime,
      headers: response.headers,
      body: response.body,
    }
  }, `Request: ${request.method} ${request.path}`);
});

With this setup, every test execution will generate rich logs that capture all HTTP activity in a structured format.

✅ Pro Tip: Always stream logs to a file rather than printing to the console. Log files are easier to archive, reference later, or integrate with monitoring/reporting tools.

Sample Logs

[2025-04-13 07:26:13.741 +0000] INFO (51775): Setting up base test
[2025-04-13 07:26:14.522 +0000] INFO (51775): Request: POST /challenger
    request: {
      "method": "POST",
      "url": "https://apichallenges.eviltester.com/challenger"
    }
    response: {
      "statusCode": 201,
      "responseTime": 780,
      "headers": {
        "x-challenger": "3f9331e7-8f26-45bf-af46-0ffd9582a64d",
      },
      "body": ""
    }
[2025-04-13 07:26:15.345 +0000] INFO (51775): Request: POST /todos
    request: {
      "method": "POST",
      "url": "https://apichallenges.eviltester.com/todos",
      "headers": {
        "X-CHALLENGER": "3f9331e7-8f26-45bf-af46-0ffd9582a64d"
      },
      "body": {
        "title": "process payroll",
        "doneStatus": false,
        "description": ""
      }
    }
    response: {
      "statusCode": 201,
      "responseTime": 822,
      "headers": {
        "content-type": "application/json",
        "x-challenger": "3f9331e7-8f26-45bf-af46-0ffd9582a64d",
      },
      "body": {
        "id": 11,
        "title": "process payroll",
        "doneStatus": false,
        "description": ""
      }
    }

📝 Reporting: Make Logs Actionable

Logging is only half the story — reporting turns those logs into insights. Once your logs capture rich request and response data, the next step is to organize and deliver that information in a way that your team can actually use. Actionable reporting ensures test results don’t go unnoticed and helps drive faster decisions.

🤓 Concept: Automated Reporting with Context

Once tests are run, logs and results need to be visible and shareable. Reporting tools help teams:

Detect regressions early
Share real-time feedback in Slack/Teams
Build test history and trends

🧪 Tool: TestBeats for Result Publishing

We’ll use TestBeats to push test reports (e.g., JUnit XML) directly to Slack or dashboards.

npx testbeats@latest publish \
  --slack ${{ secrets.SLACK_WEBHOOK_URL }} \
  --junit reports/junit.xml

This integration ensures your team doesn’t need to hunt through CI logs—they’ll be notified of test outcomes instantly, along with rich failure context.

App Dashboard

Logs

Slack Alert

Conclusion

Logging is not just about debugging—it’s about observability, accountability, and collaboration. Whether you're building a simple smoke suite or a complex regression pipeline, structured logs help you move from confusion to clarity.

By combining PactumJS, Mocha, and Pino, and by integrating tools like TestBeats, you can build a robust E2E testing workflow that gives you complete visibility from requests to reports.

Test Reporting Tools - Exploring the Best

Anudeep — Mon, 07 Oct 2024 09:40:02 +0000

In the ever-evolving landscape of software development, effective testing is paramount. As organizations strive for higher quality and faster releases, the need for robust test reporting tools becomes increasingly important. This article explores three notable tools: ReportPortal, Test Observability, and TestBeats, highlighting their features and benefits.

Report Portal

ReportPortal is an open-source test automation dashboard designed to provide real-time reporting and analysis for automated tests. It consolidates results from various testing frameworks and provides a interface for teams to monitor their testing efforts.

Pros

Open Source: Open-source tools often have vibrant communities that contribute to the project, providing support.
Real-Time Analytics: Offers live reporting, enabling quick identification of issues.
Integration with Multiple Frameworks: Supports various testing frameworks like JUnit, TestNG, and Cucumber, enhancing versatility.
Historical Data Analysis: Facilitates trend identification and proactive issue resolution.

Cons

Setup Complexity: Initial setup and configuration can be time-consuming.
Infrastructure Maintenance: Maintaining within an organization’s infrastructure can lead to challenges with uptime and reliability.
Cost of Infrastructure: Organizations may incur additional costs associated with hosting, scaling, and maintaining the infrastructure needed to run ReportPortal effectively.

TestBeats

TestBeats is an innovative open source reporting tool designed to enhance the way teams publish and analyze test results from various automation testing frameworks. By integrating with communication platforms like Slack, Microsoft Teams, Google Chat and more, TestBeats ensures that teams stay informed and can collaborate more efficiently. It brings AI-powered reporting to deliver smart analysis and rapid insights on test failures.

test-results-reporter / testbeats

Publishes test results to Microsoft Teams, Google Chat & Slack

This npm package has been renamed from test-results-reporter to testbeats. test-results-reporter will soon be phased out, and users are encouraged to transition to testbeats.

Publish test results to Microsoft Teams, Google Chat, Slack and many more.

TestBeats is a tool designed to streamline the process of publishing test results from various automation testing frameworks to communication platforms like slack, teams and more for easy access and collaboration. It unifies your test reporting to build quality insights and make faster decisions.

It supports all major automation testing frameworks and tools.

Read more about the project at https://testbeats.com

Sample Reports

Alerts in Slack

Results in Portal

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions. Head over to it and feel free to start a discussion. We are always happy to help 😊.

Support Us

Like this project! Star it on Github ⭐. Your support…

View on GitHub

Pros

Open Source: Open-source tools often have vibrant communities that contribute to the project, providing support.
Integration with Multiple Platforms: TestBeats can publish test results to popular communication platforms like Slack, Teams, and Google Chat, fostering better team collaboration and quick access to insights.
Wide Tool Support: TestBeats supports various automation testing tools such as Selenium, Cypress, Playwright, RestAssured, and PactumJS, making it highly versatile. Handles numerous test result formats, including TestNG, JUnit, Cucumber, Mocha, XUnit, NUnit, and MSTest, ensuring flexibility.
AI-Powered Reporting: TestBeats AI-driven insights help in identifying test failures quickly, providing actionable intelligence that helps in rapid decision-making.
Easy Setup: TestBeats offers a straightforward installation process that enables teams to quickly get started with minimal configuration, allowing for rapid adoption in testing workflows.

Cons

Steady Growth: The tool is experiencing consistent growth, driven by a dedicated community that continues to contribute and enhance its capabilities.

Test Observability

Test Observability by BrowserStack is a tool aimed at providing comprehensive test reporting, debugging, and analytics for automation tests. It promises real-time test reporting, AI-driven test failure analysis, and flake detection to improve automation.

Pros

Deep Insights: Offers granular views of test execution for quick diagnosis of issues.
Enhanced Debugging: Provides contextual information that reduces debugging time when you run your tests in BrowserStack.
Wide Tool Support: It supports various automation testing tools.

Cons

Setup Complexity: Initial setup and configuration can be time-consuming.
Cost: While BrowserStack offers a free tier, the advanced features and full capabilities of Test Observability are expensive
Dependency on BrowserStack: The tool is optimized for use with BrowserStack's infrastructure, which means that teams using other testing environments might not get the full benefit of its features.

Conclusion

As the demand for high-quality software grows, so does the importance of effective test reporting tools. ReportPortal, Test Observability, and TestBeats each offer unique advantages and drawbacks tailored to different organizational needs. By carefully evaluating the pros and cons, teams can choose the right tool to enhance their testing strategies, improve collaboration, and ultimately deliver better software products. Whether your focus is on real-time reporting, deep insights, or metric-driven analysis, these tools can help drive your testing efforts to new heights.

Supercharge Your Test Results with AI-Driven Insights from TestBeats

Anudeep — Thu, 08 Aug 2024 17:14:20 +0000

Automation testing is the backbone of any robust CI/CD pipeline, but the flood of test results can quickly become overwhelming. Identifying patterns, understanding recurring issues, and keeping track of test failures can consume valuable time and resources.

TestBeats — a tool designed to transform the way we handle automation test results by integrating AI-powered insights directly into your communication channels.

test-results-reporter / testbeats

Publishes test results to Microsoft Teams, Google Chat & Slack

This npm package has been renamed from test-results-reporter to testbeats. test-results-reporter will soon be phased out, and users are encouraged to transition to testbeats.

Publish test results to Microsoft Teams, Google Chat, Slack and many more.

It supports all major automation testing frameworks and tools.

Read more about the project at https://testbeats.com

Sample Reports

Alerts in Slack

Results in Portal

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions. Head over to it and feel free to start a discussion. We are always happy to help 😊.

Support Us

Like this project! Star it on Github ⭐. Your support…

View on GitHub

AI-Powered Failure Summary

TestBeats enhances your workflow by providing an AI-Powered Failure Summary—a short, precise summary that captures all the key points from your test failures. Instead of wading through pages of detailed test results, you get a clear snapshot of what’s happening with your tests.

Smart Analysis of Test Failures

TestBeats goes beyond simple test result aggregation by offering a Smart Analysis feature that categorizes and summarizes your test failures in an intuitive way. Here's what you get:

⭕ Newly Failed Tests: Easily spot tests that have started failing in the latest run.
🔴 Always Failing Tests: Identify tests that have been consistently failing, signaling the need for deeper investigation.
🟡 Flaky Tests: Detect tests that fail intermittently, helping to distinguish between genuine issues and environmental or timing-related problems.
🟢 Recovered Tests: Celebrate small victories by tracking tests that were failing but have now passed.
🟥 Error Clusters: Group similar test failures together, making it easier to understand systemic issues or common problem areas.
🟠 Recurring Errors: Highlight tests that are failing due to the same error across multiple runs, pointing to unresolved issues.

Integrated Notifications for Your Workflow

TestBeats doesn’t just stop at summarizing your test failures — it also integrates seamlessly with your team’s communication tools, including Microsoft Teams, Slack, and Google Chat.

Integrating TestBeats is quick and straightforward, allowing you to get up and running in just a few minutes.

Here is an example of Cypress tests reported to Slack.

Read more about it at docs.testbeats.com

On-Call for QA Engineers

Anudeep — Sat, 10 Sep 2022 05:28:43 +0000

QA Engineers spend a considerable amount of time and effort in developing automation frameworks and even more on adding tests and maintaining them.

Given the flaky nature of these end-to-end automation tests, most of the teams design alerting mechanisms for test failures.

Communication platforms like slack or microsoft teams are becoming the new graveyard for these alerts as most engineers end-up ignoring them over a period of time. The primary reasons for ignoring these alerts are

Excess Alerts
Lack of Visibility
Poor Accountability

In the following sections, sample config files of Test Results Reporter cli tool are shared to follow the best practices while sending your test results to slack, microsoft teams or any other communication platforms.

test-results-reporter / testbeats

Publishes test results to Microsoft Teams, Google Chat & Slack

This npm package has been renamed from test-results-reporter to testbeats. test-results-reporter will soon be phased out, and users are encouraged to transition to testbeats.

Publish test results to Microsoft Teams, Google Chat, Slack and many more.

Get Started

TestBeats is a tool designed to streamline the process of publishing test results from various automation testing frameworks to communication platforms like slack, teams and more for easy access and collaboration. It unifies your test reporting to build quality insights and make faster decisions.

Read more about the project at https://testbeats.com

Sample Reports

Alerts in Slack

Results in Portal

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions. Head over to it and feel free to start a discussion. We are always happy to help 😊.

Support Us

Like this project! Star it on Github ⭐. Your support means a lot to us.

View on GitHub

Excess Alerts

During the initial phases, looking at these alerts in your slack channels or any other place is going to be a fancy experience. After some days, we get used to ignoring them if there are more alerts than we can handle. Due to this, the real problems are lost in the noise.

One way to solve this is by sending the failure alerts to a different channel by keeping your primary channel clean and out of noise.

Primary Channel

Failure Channel

Config

[
  {
    "name": "slack",
    "inputs": {
      "url": "<primary-channel-webhook-url>",
      "publish": "test-summary-slim"
    }
  },
  {
    "name": "slack",
    "condition": "fail",
    "inputs": {
      "url": "<failure-channel-webhook-url>",
      "publish": "failure-details"
    }
  }
]

Lack of Visibility

Test Failure Analysis is where value is realised and quality is improved. It is highly critical for the velocity of your releases. Failures happen for a wide variety of reasons and keeping a track of these failure reasons is a tedious task.

ReportPortal is an AI powered test results portal. It uses the collected history of executions and fail patterns to achieve hands-free analysis of your latest results. With auto analysing power, it helps to distinguish between real failures and noise (known failures or flaky tests).

It provides the following failure categories

Product Bug (PB)
Automation Bug (AB)
System Issue (SI)
No Defect (ND)
To Investigate (TI)

Report Portal Analysis

Config

{
  "name": "report-portal-analysis",
  "inputs": {
    "url": "<report-portal-base-url>",
    "api_key": "<api-key>",
    "project": "<project-id>",
    "launch_id": "<launch-id>"
  }
}

Poor Accountability

Investigating and fixing the test failures is the most important part of maintaining automation test suites. It makes the automation healthy and trustworthy.

One way we often see to infuse accountability is by asking the test owners (who has written or adopted the tests) to look at the failures. In this approach, the knowledge is restricted and creates a dependency on others.

Another way is to make the entire team responsible for all test failures. It would be a disaster, if the entire team tries to investigate the same failure at the same time or no one takes a look at them assuming others are investigating. It would be worth creating a roster for the person to look at the failures.

Auto tagging the person who is on-call to investigate the test failures would definitely help in maintaining the automation suites healthy.

Mentions

Config

{
  "name": "mentions",
  "inputs": {
    "schedule": {
      "layers": [
        {
          "rotation": {
            "every": "week",
            "users": [
              {
                "name": "Jon",
                "slack_uid": "ULA15K66N"
              },
              {
                "name": "Anudeep",
                "slack_uid": "ULA15K66M"
              }
            ]
          }
        }
      ]
    }
  }
}

Report Automation Test Results to Google Chat

Anudeep — Mon, 18 Jul 2022 14:27:06 +0000

In the previous post we have seen the integration of test results to microsoft teams and slack.

In this article, we will check on reporting your test results to Google Chat.

test-results-reporter / testbeats

Publishes test results to Microsoft Teams, Google Chat & Slack

This npm package has been renamed from test-results-reporter to testbeats. test-results-reporter will soon be phased out, and users are encouraged to transition to testbeats.

Publish test results to Microsoft Teams, Google Chat, Slack and many more.

It supports all major automation testing frameworks and tools.

Read more about the project at https://testbeats.com

Sample Reports

Alerts in Slack

Results in Portal

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions. Head over to it and feel free to start a discussion. We are always happy to help 😊.

Support Us

Like this project! Star it on Github ⭐. Your support…

View on GitHub

TestBeats is a command line tool that can publish test results to numerous communication platforms.

Steps

Create an incoming webhook url for a chat space - docs
Create a config file either in json or js format.

    {
      "targets": [
        {
          "name": "chat",
          "inputs": {
            "url": "<chat-incoming-webhook-url>"
          }
        }
      ],
      "results": [
        {
          "type": "junit",
          "files": ["path/to/junit-results.xml"]
        }
      ]
    }

Install Node.js in your system.
Run this command to report the results.

npx test-results-reporter publish -c path/to/config.json

Report test results to slack or microsoft teams

Anudeep — Tue, 26 Apr 2022 10:27:36 +0000

Sharing the test results to the right places at the right time is very important when you are working in a high velocity organisations.

And also having the right platform where teams can discuss transparently on the test failures is also equally important.

These test results come in different forms like JUnit, TestNG, xUnit and many more from testing tools like Selenium, Cypress or RestAssured. Similarly organisations use different communication platforms like Slack, Microsoft Teams., etc for sending different kinds of alerts.

TestBeats

TestBeats is a command line tool that solves this problem elegantly. It allows us to publish test results to different entities. Even it allows us to customise the report to include hyperlinks, reportportal analysis, mentioning users based on a on-call schedule and more.

test-results-reporter / testbeats

Publishes test results to Microsoft Teams, Google Chat & Slack

This npm package has been renamed from test-results-reporter to testbeats. test-results-reporter will soon be phased out, and users are encouraged to transition to testbeats.

Publish test results to Microsoft Teams, Google Chat, Slack and many more.

It supports all major automation testing frameworks and tools.

Read more about the project at https://testbeats.com

Sample Reports

Alerts in Slack

Results in Portal

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions. Head over to it and feel free to start a discussion. We are always happy to help 😊.

Support Us

Like this project! Star it on Github ⭐. Your support…

View on GitHub

Sample Reports

Slack

Teams

Top clickbait titles

Anudeep — Fri, 25 Mar 2022 12:11:31 +0000

🙂

API Integration Testing Made Easy

Anudeep — Fri, 22 Oct 2021 14:37:51 +0000

Writing integration tests for API's is challenging in a micro-services world as it involves multiple API's from multiple components.

In this article we will be focusing on the two major challenges of writing API Integration Testing. It also talks about how to overcome them by using modern testing tools and techniques. We will be using PactumJS to write automated API integration test cases.

pactumjs / pactum

REST API Testing Tool for all levels in a Test Pyramid

PactumJS

REST API Testing Tool for all levels in a Test Pyramid

PactumJS is a REST API Testing Tool used to automate e2e, integration, contract & component (or service level) tests.

⚡ Swift
🎈 Lightweight
🚀 Simple & Powerful
🛠️ Compelling Mock Server
💎 Elegant Data Management
🔧 Extendable & Customizable
📚 Clear & Comprehensive Testing Style
🔗 Component, Contract & E2E testing of APIs

Documentation

This readme offers an basic introduction to the library. Head over to the full documentation at https://pactumjs.github.io

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions.

Installation

# install pactum as a dev dependency
npm install --save-dev pactum

# install a test runner to run pactum tests
# mocha / jest / cucumber
npm install --save-dev mocha

or you can simply use

npx pactum-init

Usage

…

View on GitHub

Challenges

These are the two things of many that I personally felt challenging while writing API Integration Tests.

Passing data across tests.
Retry on failed expectations.

Example

It's always better to have an example to understand the core concepts of a topic under discussion.

Let's take a simple example of an e-commerce application with the following API endpoints for processing an order.

POST /api/orders (for placing an order)
POST /api/payments (for making a payment)
GET /api/payments/{payment_id} (for fetching the status of payment)

Workflow

To make things clear, the requests and responses shown below are overly simplified.

Step 1 - Place Order

A user comes in and makes a POST request to /api/orders with the following payload to place an order.

Request Payload



{
  "product": "PlayStation 5"
}

At the time of writing this article, it's highly impossible to buy the above product. At-least in some places. 🙂

Now the server responds with the following response body which contains the order id.

Response



{
  "id": "1f4c99e9-12df-45d4-b455-98418f4e3b1e"
}

This order id is dynamically generated by the API server. We need to grab it and pass it to the other endpoints like payments to complete the order.

Using any testing library, we can save the response in a variable and use them later. It works but not efficient. Because when we write integration tests for large scale applications, it forces us to pass significant amount of data between tests and API calls. Declaring intermediary variables will damage the readability of the code.

To overcome this challenge PactumJS comes with a concept of Data Store to pass data between API calls across tests.

Let's look at the test first.



await pactum.spec()
    .post('/api/orders')
    .withJson({
      "product": "PlayStation 5"
    })
    .expectStatus(200)
    .stores('OrderID', 'id');

The above test will make a POST request to /api/orders with given json payload and once the response is received it expects the status should be 200 and stores the value of id into a special variable called OrderID which is internal to PactumJS.

Step 2 - Make Payment

The next step is to make the payment. Now the user makes a POST request to /api/payments with the following payload.

Request Payload



{
  "order_id": "1f4c99e9-12df-45d4-b455-98418f4e3b1e",
  "card_info": {
    "number": "1111-1111-1111-1111",
    "expiry": "11/11",
    "cvv": "111"
  }
}

Now the API responds with the following response body which contains payment id.

Response



{
  "id": "a32fce50-d4e8-4d95-b16f-57fd13fbb7df"
}

Now let's talk about the test case.

As you observed, the order id from the previous request is included in the request payload.

To get the value of special internal variable, PactumJS uses a special pattern - $S{<variable-name>} to access it.

Let's look at the test.



await pactum.spec()
    .post('/api/payments')
    .withJson({
      "order_id": "$S{OrderID}",
      "card_info": {
        "number": "1111-1111-1111-1111",
        "expiry": "11/11",
        "cvv": "111"
      }
    })
    .expectStatus(200)
    .stores('PaymentID', 'id');

PactumJS will internally replace $S{OrderID} with 1f4c99e9-12df-45d4-b455-98418f4e3b1e before making the request.

In the above test case we are also saving the payment id into the special variable PaymentId using the stores method. Using the payment id we can track the status of the payment. So this brings us to the final step of our integration test.

Step 3 - Wait for Payment to be completed.

To get the status of the payment, user makes a GET request to the /api/payments/{payment_id} endpoint.

The API responds with the following response body.

Response



{
  "status": "in-progress"
}

As you see, the status is still in-progress. We need to wait for few seconds for the payment to be completed.

Including hard waits is a bad practice in testing. PactumJS comes with a concept of retry mechanism which retries on failed expectations. It is similar to fluent wait in selenium.

Let's look at the test case.



await pactum.spec()
    .get('/api/payments/{id}')
    .withPathParams('id', '$S{PaymentID}')
    .expectStatus(200)
    .expectJson({
      "status": "completed"
    })
    .retry();

By default it retries 3 times with a delay of 1000ms between each retry.

Complete Test

Now let's take a look at the entire test using PactumJS and mocha.



const pactum = require('pactum');

it('order PS5 and make payment', async () => {

  await pactum.spec()

    .post('/api/orders')

    .withJson({

      "product": "PlayStation 5"

    })

    .expectStatus(200)

    .stores('OrderID', 'id');

await pactum.spec()

    .post('/api/payments')

    .withJson({

      "order_id": "$S{OrderID}",

      "card_info": {

        "number": "1111-1111-1111-1111",

        "expiry": "11/11",

        "cvv": "111"

      }

    })

    .expectStatus(200)

    .stores('PaymentID', 'id');

await pactum.spec()

    .get('/api/payments/{id}')

    .withPathParams('id', '$S{PaymentID}')

    .expectStatus(200)

    .expectJson({

      "status": "completed"

    })

    .retry();

});

Conclusion

Writing readable and maintainable tests is very important to make API testing productive and enjoyable experience.

PactumJS abstracts the challenging parts to write tests in an easy and fun way and ultimately making API Integration testing super easy.

API Testing Tools in JavaScript

Anudeep — Sun, 06 Jun 2021 12:03:00 +0000

API testing is a type of software testing that focuses on determining if APIs meet expectations. It is critical for automating testing because APIs now serve as the primary interface to application logic.

Tools

Here is the list of popular tools available in JavaScript for API Automation Testing in alphabetical order.

Find other options at the end.

Chakram

dareid / chakram

REST API test framework. BDD and exploits promises

Chakram

Chakram is no longer actively maintained, PRs are welcomed

Chakram is an API testing framework designed to perform end to end tests on JSON REST endpoints.

The library offers a BDD testing style and fully exploits javascript promises - the resulting tests are simple, clear and expressive. Chakram is built on node.js, mocha, chai and request.

This readme offers an introduction to the library. For more information, visit Chakram's documentation and tests which demonstrate all of Chakram's capabilities. In addition, example tests of publicly accessible APIs are available in the examples directory. If required, assistance can be found in the project's gitter chat room.

Features

HTTP specific assertions. Allows testing of
- Status codes
- Cookie presence and value
- Header presence and value
- JSON values
- JSON structure (using the JSON schema specification)
- Compression
- Response times
BDD formatting and hooks (e.g. beforeEach, afterEach)
Promise based
Plugin…

View on GitHub

Chakram is a REST API testing framework offering a BDD testing style and fully exploiting promises.

Chakram extends Chai.js, adding HTTP specific assertions. It allows simple verification of returned status codes, the compression used, cookies, headers, returned JSON objects and the schema of the JSON response.

describe("HTTP assertions", function () {
  it("should make HTTP assertions easy", function () {
    var response = chakram.get("http://httpbin.org/get");
    expect(response).to.have.status(200);
    expect(response).to.have.header("content-type", "application/json");
    expect(response).not.to.be.encoded.with.gzip;
    return chakram.wait();
  });
});

Pros

Easy to use
Simple Testing Style
Extendable & Customizable

Cons

API Chaining is inelegant

Frisby.js

vlucas / frisby

Frisby is a REST API testing framework built on Jest that makes testing API endpoints easy, fast, and fun.

Frisby

Introduction

Frisby.js an API testing tool built on top of Jest that makes testing API endpoints easy fast and fun.

Installation

Install Frisby v2.x from NPM into your project:

npm install --save-dev frisby joi

Creating Tests

Simple Example

The minimum setup to run a single test expectation.

const frisby = require('frisby');

it('should be a teapot', function () {
  // Return the Frisby.js Spec in the 'it()' (just like a promise)
  return frisby.get('http://httpbin.org/status/418')
    .expect('status', 418);
});

Nested Dependent HTTP Calls

A more complex example with nested dependent Frisby tests with Frisby's Promise-style then method.

const frisby = require('frisby');
const Joi = require('joi');
describe('Posts', function () {
  it('should return all posts and first post should have

…

View on GitHub

Frisby makes REST API testing easy, fast, and fun. Frisby.js comes loaded with many built-in tools for the most common things you need to test for to ensure your REST API is working as it should, and returning the correct properties, values, and types.

const frisby = require('frisby');

it ('POST should return a status of 201 Created', function () {
  return frisby
    .post('http://api.example.com/posts', {
      title: 'My New Blog Post',
      content: '<p>A cool blog post!</p>'
    })
    .expect('status', 201);
});

Pros

Easy to use & setup
Good Community Support
Extendable & Customizable

Cons

Only CRUD
API Chaining is inelegant

PactumJS

pactumjs / pactum

REST API Testing Tool for all levels in a Test Pyramid

PactumJS

REST API Testing Tool for all levels in a Test Pyramid

PactumJS is a REST API Testing Tool used to automate e2e, integration, contract & component (or service level) tests.

⚡ Swift
🎈 Lightweight
🚀 Simple & Powerful
🛠️ Compelling Mock Server
💎 Elegant Data Management
🔧 Extendable & Customizable
📚 Clear & Comprehensive Testing Style
🔗 Component, Contract & E2E testing of APIs

Documentation

This readme offers an basic introduction to the library. Head over to the full documentation at https://pactumjs.github.io

Need Help

We use Github Discussions to receive feedback, discuss ideas & answer questions.

Installation

# install pactum as a dev dependency
npm install --save-dev pactum

# install a test runner to run pactum tests
# mocha / jest / cucumber
npm install --save-dev mocha

or you can simply use

npx pactum-init

Usage

…

View on GitHub

PactumJS is a REST API Testing Tool for all levels in a Test Pyramid and used to automate e2e, integration, contract & component (or service level) tests.

const { spec } = require('pactum');

it('should save a new user', async () => {
  await spec()
    .post('https://jsonplaceholder.typicode.com/users')
    .withHeaders('Authorization', 'Basic xxxx')
    .withJson({
      name: 'bolt',
      email: 'bolt@swift.run'
    })
    .expectStatus(200);
});

Pros

Lightweight
Active & Growing
Simple & Powerful
Compelling Mock Server
Elegant Data Management
Advanced Retry Mechanism
Extendable & Customizable
Clear & Comprehensive Testing Style
Component, Contract & E2E testing of APIs

Cons

Limited Community Support

SuperTest

ladjs / supertest

🕷 Super-agent driven library for testing node.js HTTP servers using a fluent API. Maintained for @forwardemail, @ladjs, @spamscanner, @breejs, @cabinjs, and @lassjs.

SuperTest

HTTP assertions made easy via superagent. Maintained for Forward Email and Lad.

About

The motivation with this module is to provide a high-level abstraction for testing HTTP, while still allowing you to drop down to the lower-level API provided by superagent.

Getting Started

Install SuperTest as an npm module and save it to your package.json file as a development dependency:

npm install supertest --save-dev

Once installed it can now be referenced by simply calling require('supertest');

Example

You may pass an http.Server, or a Function to request() - if the server is not already listening for connections then it is bound to an ephemeral port for you so there is no need to keep track of ports.

SuperTest works with any test framework, here is an example without using any test framework at all:

const request = require('supertest');
const express = require(

…

View on GitHub

SuperTest is built on a HTTP client called SuperAgent. The motivation with this module is to provide a high-level abstraction for testing HTTP, while still allowing you to drop down to the lower-level API provided by superagent.

const request = require('supertest');
const express = require('express');

const app = express();

app.get('/user', function(req, res) {
  res.status(200).json({ name: 'john' });
});

describe('GET /user', function() {
  it('responds with json', function(done) {
    request(app)
      .get('/user')
      .set('Accept', 'application/json')
      .expect('Content-Type', /json/)
      .expect(200, done);
  });
});

Pros

Simple & Powerful
Good Community Support

Comparison

#	Chakram	Frisby.js	PactumJS	SuperTest
CRUD Operations	✔️	✔️	✔️	✔️
Non CRUD Operations	❌	❌	✔️	✔️
Global Request Setup	✔️	✔️	✔️	✔️
Custom Assertions	✔️	✔️	✔️	✔️
Exception Handling	❌	❌	✔️	❌
BDD Style	✔️	❌	✔️	❌
Cucumber Support	❌	❌	✔️	❌
Inspectors	❌	✔️	✔️	❌
Retry Mechanism	❌	❌	✔️	✔️
Data Management	❌	❌	✔️	❌
Custom Reporting	❌	❌	✔️	❌
Mock Server	❌	❌	✔️	❌
Contract Testing	❌	❌	✔️	❌

Others

Boost your code coverage with API Tests

Anudeep — Fri, 07 May 2021 17:40:26 +0000

Code coverage is an important quality metric that determines the number of lines of source code that is tested & covered by automated tests. Usually, developers achieve code coverage close to 80% by writing unit tests (most popular tests to generate code coverage).

Targeted Code

In general unit tests targets happy paths, core business logic, and rarely sad paths. Most likely they can give us close to 80% of code coverage. The remaining 20% of source code might be responsible for handling external interfaces and exceptions/errors. Unit tests generally omit testing external interface logic that interacts with outside applications & databases.

Testing the external interface logic at the early phases of SDLC is very critical for delivering a quality software product.

API Testing

API testing is critical for automating testing because APIs now serve as the primary interface to application logic. API tests could greatly improve the code coverage of applications and the overall confidence in the product.

Let's see how to get code coverage from API tests.

Example

In this example, we will be looking at a basic Node.js web application.

Source Code

Look at the sample web app server written in express. (Express is a minimal and flexible Node.js web application framework)

// index.js
const express = require('express');
const app = express();
const port = 3333;

app.get('/hello', (req, res) => {
  res.send('Hello World!');
});

app.listen(port, () => {
  console.log(`App listening at http://localhost:${port}`);
});

To run the application, execute the below command in terminal

node index.js

API Tests

Look at the sample API tests written using PactumJS and mocha.

pactumjs / pactum

REST API Testing Tool for all levels in a Test Pyramid

// app.test.js
const pactum = require('pactum');

describe('App', () => {

  it('GET /hello', async () => {
    await pactum.spec()
      .get('http://localhost:3333/hello')
      .expectStatus(200)
      .expectBody('Hello World!');
  });

});

To run the tests, execute the below command in terminal

mocha app.test.js

Code Coverage

We have seen how to run the application & execute tests against it. To generate code coverage from API tests, we will be using an npm package called nyc.

Install the package globally

npm i nyc -g

Now run your application with this magical tool nyc.

nyc node index.js

Output

Run Tests

mocha app.test.js

Output

Stop the application by pressing CTRL + c. Now the nyc tool will generate and display the code coverage in the terminal.

Output

For real and complex web applications, the code-coverage setup might not be straightforward. It might require additional steps and advanced configurations.

Conclusion

Importance of API testing is growing day by day. Not only using these tests for validating applications but also for generating code coverage metrics is an added advantage.

In most scenarios, a simple API test could cover a big chunk of source code. It means with fewer API Tests we can get more code coverage and confidence in the application.