Consuming and Testing third party API's using Spring Webclient

Introduction

More often than not developer's build and call API's. Therefore it's guaranteed that you are going to have to build or consume an API sometime in your day to day development. In this article I will show you how you can build a springboot service that calls a third-party api and most importantly we are going to write test to guarantee our consuming service works as expected.

Getting Started

Start a new Spring Boot Project on Spring Initializr here or just create from the command line using a http request to Spring Initializr as follows:

curl --location 'https://start.spring.io/starter.zip?type=maven-project&language=java&bootVersion=3.2.2&baseDir=ms-xcoffee&groupId=com.xcoffee&artifactId=ms-xcoffee&name=ms-xcoffee&description=Demo%20project%20for%20Spring%20Boot&packageName=com.xcoffee.ms-xcoffee&packaging=jar&javaVersion=21&dependencies=webflux%2Clombok%2Cvalidation' | tar -xzvf -

Add Spring Webflux, Validation and Lombok as the starting dependencies. (Above is your first first API interaction).

Building

We are going to use https://sampleapis.com/api-list/coffee as our third party API. They have provided us a very nice API endpoint that accepts a GET request and responds with a list of JSON representation of the coffee data.

GET /coffee/iced HTTP/1.1
Host: api.sampleapis.com

Curl Request:

curl --location 'https://api.sampleapis.com/coffee/iced'

Response:

[
    {
        "title": "Iced Coffee",
        "description": "A coffee with ice, typically served with a dash of milk, cream or sweetener—iced coffee is really as simple as that.",
        "ingredients": [
            "Coffee",
            "Ice",
            "Sugar*",
            "Cream*"
        ],
        "image": "https://upload.wikimedia.org/wikipedia/commons/d/d8/Blue_Bottle%2C_Kyoto_Style_Ice_Coffee_%285909775445%29.jpg",
        "id": 1
    }, ...
]

Next we will build a POJO(plain old java object) representation of the json response.

package com.xcoffee.msxcoffee.schema.pojos;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.util.List;


@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public class CoffeeResponse {
    @JsonProperty("title")
    private String title;
    @JsonProperty("description")
    private String description;
    @JsonProperty("ingredients")
    private List<String> ingredients;
    @JsonProperty("image")
    private String image;
    @JsonProperty("id")
    private int id;
}

The above class maps the json data to a java object we can work with. We use Lombok to generate constructors, getters and setters for our code and the Jackson Project to handle serialization and deserialization of json to pojo . We know the response is an array of objects representing the coffee and so above data structure is fit for this.

Webclient Action

We will build an interface in which we will base our api consumer implementation. The interface supports Polymorphism and allows us to use different implementations for different purposes e.g. testing.

package com.xcoffee.msxcoffee.services;

import com.xcoffee.msxcoffee.schema.pojos.CoffeeResponse;
import reactor.core.publisher.Mono;

import java.util.List;

public interface CoffeeService {
    Mono<List<CoffeeResponse>> getHotCoffees();
    Mono<List<CoffeeResponse>> getIcedCoffees();
}

Next we dive into the actual implementation.
We will create a class called CoffeeServiceImpl.java that implements our interface methods.

package com.xcoffee.msxcoffee.services.impl;

import com.xcoffee.msxcoffee.schema.pojos.CoffeeResponse;
import com.xcoffee.msxcoffee.services.CoffeeService;
import reactor.core.publisher.Mono;

public class CoffeeServiceImpl implements CoffeeService {
    @Override
    public Mono<List<CoffeeResponse>> getHotCoffees() {
        return null;
    }

    @Override
    public Mono<List<CoffeeResponse>> getIcedCoffees() {
        return null;
    }
}

Then we build our api consumption logic.

But wait, we need to configure WebClient , here's the implementation.

package com.xcoffee.msxcoffee.config;

import io.netty.channel.ChannelOption;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import io.netty.handler.timeout.ReadTimeoutHandler;
import io.netty.handler.timeout.WriteTimeoutHandler;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.reactive.ClientHttpConnector;
import org.springframework.http.client.reactive.ReactorClientHttpConnector;
import org.springframework.web.reactive.function.client.ExchangeStrategies;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.netty.http.client.HttpClient;

import javax.net.ssl.SSLException;
import java.time.Duration;
import java.util.concurrent.TimeUnit;

@Configuration
public class WebClientConfig {

    @Bean
    public WebClient getWebClient() throws SSLException {

        ExchangeStrategies exchangeStrategies = ExchangeStrategies.builder()
                .codecs(configurer -> configurer.defaultCodecs()
                        .maxInMemorySize(1048576)) // Set buffer size to 1 MB
                .build();

        // Disable ssl verification
        SslContext context = SslContextBuilder.forClient()
                .trustManager(InsecureTrustManagerFactory.INSTANCE)
                .build();

        HttpClient httpClient = HttpClient.create()
                .secure(t -> t.sslContext(context))
                // .proxyWithSystemProperties() // Use JVM level System proxy
                .responseTimeout(Duration.ofSeconds(30))
                .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30 * 1000)
                .doOnConnected(conn -> conn
                        .addHandlerLast(new ReadTimeoutHandler(30,
                                TimeUnit.SECONDS))
                        .addHandlerLast(new WriteTimeoutHandler(30)));

        ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);

        return WebClient
                .builder()
                .exchangeStrategies(exchangeStrategies)
                .clientConnector(connector)
                .build();
    }
}

Then we can use WebClient in our Service Impl class:

Here's the code:

package com.xcoffee.msxcoffee.services.impl;

import com.xcoffee.msxcoffee.config.AppConfig;
import com.xcoffee.msxcoffee.schema.pojos.CoffeeResponse;
import com.xcoffee.msxcoffee.services.CoffeeService;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatusCode;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;

import java.util.List;

@Service
@RequiredArgsConstructor
@Slf4j
public class CoffeeServiceImpl implements CoffeeService {

    private final WebClient webClient;
    private final AppConfig cfg;

    @Override
    public Mono<List<CoffeeResponse>> getIcedCoffees() {
        return webClient.method(HttpMethod.GET)
                .uri(cfg.getCoffeeURL() + "/iced")
                .contentType(MediaType.APPLICATION_JSON)
                .retrieve()
                .onStatus(HttpStatusCode::isError, response -> {
                    log.error("An Error Occurred Getting Iced Coffees");
                    return Mono.just(new Exception("An Error Occurred Getting Iced Coffees"));
                })
                .bodyToFlux(CoffeeResponse.class)
                .collectList();
    }

    @Override
    public Mono<List<CoffeeResponse>> getHotCoffees() {
        return webClient.method(HttpMethod.GET)
                .uri(cfg.getCoffeeURL() + "/hot")
                .contentType(MediaType.APPLICATION_JSON)
                .retrieve()
                .onStatus(HttpStatusCode::isError, response -> {
                    log.error("An Error Occurred Getting Hot Coffees");
                    return Mono.just(new Exception("An Error Occurred Getting Hot Coffees"));
                })
                .bodyToFlux(CoffeeResponse.class)
                .collectList();
    }
}

The code above uses org.springframework.web.reactive.function.client.WebClient to perform GET request to the respective coffee endpoints and then map the responses to the List<CoffeeResponse object which is a List of Coffees.

@RequiredArgsConstructor creates a webclient and config constructor for the service class.

We also created a config class to externalize configurations such as API URLS :

package com.xcoffee.msxcoffee.config;


import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@Getter
@Setter
@Configuration
@ConfigurationProperties(prefix = "app")
public class AppConfig {
    private String coffeeURL;
}

Unit Testing and Mocking

Suppose the HTTP calls to the coffee endpoints perform a data update or a paid transaction, then we cannot perform test by calling the real endpoint. So here comes Mocking .

We will use Square’s Mock Webserver to spin up a mock server which we can use to simulate real api's request to the get coffee endpoint.

Add the dependency as follows:

        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>mockwebserver</artifactId>
            <version>5.0.0-alpha.12</version>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>5.0.0-alpha.12</version>
        </dependency>

Checkout the latest version here on maven central https://central.sonatype.com/artifact/com.squareup.okhttp3/mockwebserver/5.0.0-alpha.12

Then we can write our test as follows:

package com.xcoffee.msxcoffee.services.impl;

import com.xcoffee.msxcoffee.config.AppConfig;
import com.xcoffee.msxcoffee.schema.pojos.CoffeeResponse;
import com.xcoffee.msxcoffee.services.CoffeeService;
import jakarta.validation.constraints.NotNull;
import okhttp3.mockwebserver.MockResponse;
import okhttp3.mockwebserver.MockWebServer;
import org.junit.jupiter.api.*;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
import reactor.test.StepVerifier;

import java.io.IOException;
import java.util.List;
import java.util.Objects;

class CoffeeServiceImplTest {


    private static MockWebServer mockWebServer;
    private static CoffeeService coffeeService;

    private static final String COFFEE_API_URL = "dummy";

    @BeforeAll
    static void setUp() {
        AppConfig cfg = new AppConfig();
        mockWebServer = new MockWebServer();
        WebClient mockedWebClient = WebClient.builder()
                .baseUrl(mockWebServer.url(COFFEE_API_URL).toString())
                .build();

        coffeeService = new CoffeeServiceImpl(mockedWebClient,cfg);

    }

    @AfterAll
    static void tearDown() throws IOException {
        mockWebServer.close();
    }

    @Test
    @DisplayName("Unit | Get Hot Coffees Success")
    void getHotCoffeeSuccess() {
        // Enqueue a successful response
        mockWebServer.enqueue(
                new MockResponse().setResponseCode(HttpStatus.OK.value())
                        .setHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                        .setBody(getCoffeeSuccess()));

        // Test the success case
        Mono<List<CoffeeResponse>> responseMono = coffeeService.getHotCoffees();

        StepVerifier.create(responseMono)
                .assertNext(response -> {
                    Assertions.assertInstanceOf(List.class, response);
                })
                .verifyComplete();

    }

    @Test
    @DisplayName("Unit | Get Hot Coffees Failure")
    void getHotCoffeeFailure() {
        // Enqueue a failed response
        mockWebServer.enqueue(new MockResponse()
                .setResponseCode(HttpStatus.INTERNAL_SERVER_ERROR.value())
                .setBody("{}")
        );

        // Test the failure case
        Mono<List<CoffeeResponse>> hotCoffees = coffeeService.getHotCoffees();

        StepVerifier.create(hotCoffees)
                .expectErrorMatches(throwable -> throwable instanceof Exception
                        && Objects.equals(throwable.getMessage(),
                        "An Error Occurred Getting Hot Coffees"))
                .verify();

    }

    @NotNull
    private String getCoffeeSuccess() {
        return """
                [
                     {
                         "title": "Iced Coffee",
                         "description": "A coffee with ice, typically served with a dash of milk, cream or sweetener—iced coffee is really as simple as that.",
                         "ingredients": [
                             "Coffee",
                             "Ice",
                             "Sugar*",
                             "Cream*"
                         ],
                         "image": "https://upload.wikimedia.org/wikipedia/commons/d/d8/Blue_Bottle%2C_Kyoto_Style_Ice_Coffee_%285909775445%29.jpg",
                         "id": 1
                     }
                 ]
                """;
    }
}

We initialize the components to be mocked via setUp() method annotated with @BeforeAll, here we also configure the mock server and mock webclient and inject to our coffeeService , remember the interface we created? That helps us easily modify the implementation of coffee service and this is what achieves mocking.

We Enqueue success and failure responses respectively inside the test methods and finally use project reactor Step Verifier to test the events that happen upon subscription.

Finally run the test with IntelliJ, VS Code, Maven or tool of your choice they run as follows:

1. IntelliJ

   

2. Maven

   mvn test

Conclusion

In this article we learnt how we can consume API's using non blocking WebClient and mock API calls using MockServer.