DEV Community: MykolaGolubyev

Simple and powerful HTTP API tests

MykolaGolubyev — Tue, 03 Jan 2023 14:33:40 +0000

I want to show you how to write and run concise, readable and powerful HTTP API tests using WebTau tool.

First Test

We are going to test a Todo API. JSON Placeholder website is going to help us with this.

GET https://jsonplaceholder.typicode.com/todos/1

{
  "userId": 1,
  "id": 1,
  "title": --> "delectus aut autem", <--
  "completed": false
}

Our first test will send GET request to https://jsonplaceholder.typicode.com/todos/1 and assert on the title response field

scenarios/apitest.groovy

scenario("check todo item title") {
    http.get("https://jsonplaceholder.typicode.com/todos/1") {//1
        title.should == "delectus aut autem"//2
    }
}

GET call to a provided URL (full URL is not required)
automatic mapping of the JSON response

To run the test, we will use WebTau command line tool. One way to install it is to use brew.
Other options available in documentation Installation section.

$ brew install testingisdocumenting/brew/webtau
$ webtau scenarios/apitest.groovy

Let's take a look at the output produced by the test run

Few things to note:

WebTau produces detailed log of what happens
Automatic assertion on statusCode when no explicit assertion provided
Highlighting asserted fields in the response

Base URL

In the example above we use the full URL to the TODO item resource. To be able to run the test against different environments (e.g. localhost)
we should extract base URL.

There are multiple ways to specify base URL, most commons are:

Command line parameter
Config file

scenario("check todo item title") {
    http.get("/todos/1") {
        title.should == "delectus aut autem"
    }
}

To set base URL via command line we need to pass --url parameter to the CLI call

$ webtau scenarios/apitest.groovy --url https://jsonplaceholder.typicode.com

To set base URL using a config file use

webtau.cfg.groovy

url = "https://jsonplaceholder.typicode.com"

Extracting Data

Next, let me show you how to use data from one response to initiate the next one.
We will create a TODO item and then extract its id for further usage.

POST https://jsonplaceholder.typicode.com/posts

{
  "userId": 1,
  "id":  --> 1 <--,
  "title": "delectus aut autem",
  "completed": false
}

scenario("create and update todo item") {
    //1
    def itemId = http.post("/posts", [title: "my todo item", body: "write test", userId: 1]) {
        return id //2
    }

    http.get("/todos/${itemId}") { //3
        id.should == itemId
    }
}

create new item
extract auto parsed response field
use extracted id as part of URL

Reporting

I already show you console output WebTau produces. Let me finish this post with a screenshot of an HTML report that WebTau generates.

Generated report is a self-contained single HTML file that you can Slack around. Report captures a lot of details and provides permalink behavior.
E.g. open a specific test and specific HTML call and then share the link.

Outro

I demoed basic functionality of testing HTTP based API. Should be enough for you to start writing your tests.

There are so many other things you can do with WebTau and HTTP API, Browser, CLI, DataBase and Business Logic.

Web UI tests reinforcement with webtau framework (Selenium based)

MykolaGolubyev — Tue, 13 Aug 2019 14:22:40 +0000

One of the biggest problem with Web UI tests is their brittleness. Moving a page element around or replacing an input box with a drop down can cause a ripple effect that breaks dozens of your tests.

In this article I will show you how to encapsulate your UI implementation details so your tests can focus on testing features.

I will be using the webtau framework that has primitives designed to help with tests brittleness

First test attempt

Let’s test a search feature of an imaginary app.

scenario('search by specific query') {
    browser.open('/search')

    $('#search-box').setValue('search this')
    $('#search-box').sendKeys("\n")

    $('#results .result').count.shouldBe > 1
}

Test:

open a search page
set a value to an input box that can be located by search-box id
simulate enter key and check that there are some results

I claim that there should be only one reason to change this test and that is if the actual functionality of your search experience changes (Single Responsibility Principle anyone?).

However there are four non functional reasons that can force our test to change:

search page url changes
the way we initiate the search changes: enter key vs mouse click
ids/classes/attributes associated with elements are changed
server response time becomes slower

Let’s address them one by one.

Extract elements definition

In webtau $(css) creates an instance of PageElement that you use to simulate user actions and query values.

Created instances are lazy and can be defined before the browser is even opened.

def searchBox = $('#search-box')
def numberOfResults = searchBox.count

scenario('search by specific query') {
    browser.open('/search')

    searchBox.setValue('search this')
    searchBox.sendKeys("\n")

    numberOfResults.shouldBe > 1
}

This is a step in a right direction, but we still have exposed page url and hardcoded Enter key press.

Extract user actions

Let’s create a submit method to encapsulate the way users are supposed to initiate the search.

def searchBox = $('#search-box')
def numberOfResults = searchBox.count

scenario('search by specific query') {
    submit('search this')
    numberOfResults.shouldBe > 1
}

def submit(query) {
    browser.open("/search")

    searchBox.setValue(query)
    searchBox.sendKeys("\n")
}

I think the changes we made make the test easier to reason about. But I still want to give it a final touch: move definitions out of a test file so they can be potentially used by other test scenarios.

Encapsulation

Let’s move the action and page elements definitions to a separate class. You may have heard of the PageObject concept before.

In webtau page objects are simple class instances. It is the lazy nature of PageElement that makes this simplicity possible.

package pages

import static com.twosigma.webtau.WebTauDsl.*

class SearchPage {
    def searchBox = $('#search-box')
    def numberOfResults = $('#results .result').count

    def submit(query) {
        browser.open("/search")

        searchBox.setValue(query)
        searchBox.sendKeys("\n")
    }
}

It doesn’t matter when we create the instance of the class since searchBox and numberOfResults will be lazily initiated.

Let’s create the Pages class with static page object instances to make it convenient to access them from different scenarios.

package pages

class Pages {
    static final def search = new SearchPage()
    static final def calculation = new CalculationPage()
    static final def form = new FormPage()
}

Our page object is one import away.

import static pages.Pages.*

scenario("search by specific query") {
    search.submit("search this")
    search.numberOfResults.shouldBe > 1
}

Dealing with asynchronous brittleness

Our test is now reinforced and can sustain UI non-functional changes. But there is one unprotected area left uncovered.

During local development our server is blazingly fast and search results are produced in under 5ms. Weeks later we will run the tests in QA environment and most likely we will be dealing with the failed assertion.

Let’s do a final test reinforcement.

import static pages.Pages.*

scenario("search by specific query") {
    search.submit("search this")
    search.numberOfResults.waitToBe > 1
}

We replaced shouldBe with waitToBe. As a result, instead of failing the assertion right away waitTo will re-query numberOfResults multiple times until it times out (driven by configuration).

Groovy, Java and beyond

I have been using Groovy for testing for the last decade and I think it is perfect for the job. But if you are not a Groovy fan, you can use Java or other JVM languages.

If you are interested in Java or other JVM examples, please hit me in the comments and I will be happy to help.

Links

GitHub

Introduction to REST API testing with webtau

Introduction to concise and expressive REST API testing framework — WebTau

MykolaGolubyev — Tue, 28 May 2019 20:57:00 +0000

Introduction

Webtau (short for web test automation) is a tool and API to write expressive and concise REST API tests.

Let’s start with a simple get-weather example. You have a server that returns a current temperature every time you hit /weather end point.

{
  "temperature": 88
}

I want to write a simple test to validate that a temperature I receive is lower than 100 Fahrenheit. Here is a webtau test script.

scenario("simple get") {
    http.get("/weather") {
        temperature.shouldBe < 100
    }
}

temperature.should expression gets automatically mapped to the server response and triggers validation against temperature value.

To run the test, execute webtau webtau-simple-get.groovy command line.

If a command line and standalone scripts is not your cup of tea, you can use JUnit and similar runners.

package com.example.tests.junit4

import com.twosigma.webtau.junit4.WebTauRunner
import org.junit.Test
import org.junit.runner.RunWith

import static com.twosigma.webtau.WebTauGroovyDsl.*

@RunWith(WebTauRunner.class) // required for html report generation only
class WeatherGroovyTest {
    @Test
    void checkWeather() {
        http.get("/weather") {
            temperature.shouldBe < 100
        }
    }
}

I am using Groovy for testing for the last decade and I think it is perfect for the job. But if you are not a Groovy fan, you can use Java or other JVM languages.

package com.example.tests.junit4;

import com.twosigma.webtau.junit4.WebTauRunner;
import org.junit.Test;
import org.junit.runner.RunWith;

import static com.twosigma.webtau.WebTauGroovyDsl.*;

@RunWith(WebTauRunner.class)
public class WeatherJavaTest {
    @Test
    public void checkWeather() {
        http.get("/weather", (header, body) -> {
            body.get("temperature").shouldBe(lessThan(100));
        });
    }
}

CRUD example

Let’s complicate the example and test the full Create, Read, Update, Delete cycle.

scenario("CRUD operations for customer") {
    def customerPayload = [firstName: "FN", lastName: "LN"]

    def id = http.post("/customers", customerPayload) {
        return id // return id value from response body
    }

    http.get("/customers/${id}") {
        body.should == customerPayload // only specified properties will be asserted against
    }

    def changedLastName = "NLN"
    http.put("/customers/${id}", [*:customerPayload, lastName: changedLastName]) {
        lastName.should == changedLastName // specifying body is optional
    }

    http.get("/customers/${id}") {
        lastName.should == changedLastName
    }

    http.delete("/customers/${id}") {
        statusCode.should == 204
    }

    http.get("/customers/${id}") {
        statusCode.should == 404
    }
}

Note how test is very data focused and tries to minimize the data management boilerplate by re-using the initial payload as response expectation.

It the test above all CRUD operations are part of the same test. Below is the example of the separating CRUD operations into their own tests.

def customerPayload = [firstName: "FN", lastName: "LN"]

def customer = createLazyResource("customer") { // lazy resource to be created on the first access
    def id = http.post("/customers", customerPayload) {
        return id
    }

    return new Customer(id: id, url: "/customers/${id}") // definition is below
}

scenario("customer create") {
    customer.id.should != null // accessing resource for the first time will trigger POST (in this example)
}

scenario("customer read") {
    http.get(customer.url) { // convenient re-use of url defined above
        body.should == customerPayload
    }
}

scenario("customer update") {
    def changedLastName = "NLN"
    http.put(customer.url, [*:customerPayload, lastName: changedLastName]) {
        lastName.should == changedLastName
    }

    http.get(customer.url) {
        lastName.should == changedLastName
    }
}

scenario("customer delete") {
    http.delete(customer.url) {
        statusCode.should == 204
    }

    http.get(customer.url) {
        statusCode.should == 404
    }
}

class Customer {
    Number id
    String url // store url of the created entity
}

Example above is using a lazyResource concept to make each test to be self contained. I.e. if I just want to run delete test, a resource will still be created. On the other hand if I run all CRUD tests, there will be only one post.

Reporting

One of the time consuming tasks maintaining tests is figuring out what when wrong when a test fails. Webtau provides comprehensive console output as well as a rich, self contained html report.

Note that asserted values are highlighted inside the console output. Failures are highlighted in a similar manner

At the end of the tests run you will be given a location of html report

Open API, UI Testing, matchers and more

There are a lot more features I want to cover. There is Open API integration for tracking coverage, there is web UI testing parts of webtau that let you create comprehensive tests of your app. There are powerful matchers and data management techniques. Integration with JUnit5.

While I am tempted to cover them now I think dedicated posts may help to focus on one feature at a time.

If you have questions or suggestions, please leave a comment here or create an issue on github.

Getting Started

To get started follow the official docs.