DEV Community: Nikolay Stanchev

Integrating a Java REST API With a Database

Nikolay Stanchev — Sun, 31 Jul 2022 15:35:25 +0000

Introduction

This article is a follow-up from my last tutorial on building a fully functional Java REST API for managing TODO tasks. For the purpose of simplicity, last time we used an in-memory database as an implementation of the storage interface defined by our business logic. It is clear that this solution is not feasible for a production-ready API mainly because:

we will lose the stored tasks every time we stop the application
different application instances will not be able to share memory - in other words if we vertically scale up our service by using two servers - A and B, then tasks created through server A will not be seen by tasks created through server B.

This is illustrated in the diagram below:

In this tutorial I want to demonstrate how we can integrate the API we built as part of the previous article with a very famous open-source non-relational database, namely MongoDB. This means changing the diagram above to the following:

For the purpose of simplicity, I chose MongoDB, but technically we could have also chosen a relational database such as PostgreSQL. If we were to choose between the two DBs , we would need to know a bit more about the software requirements of our service - things like:

scalability - how much is the application expected to grow in the future
expected traffic pattern - how many clients are going to use the service, what request rate they will use, read vs write ratio, etc.
access patterns - do we have a well-defined access pattern requested by our business stakeholders or do we need an approach that allows for general search queries

The API specification is not enough on its own to make this decision, but for the purpose of this tutorial, we can go ahead and use MongoDB.

Current State

A quick reminder of what we built before is given below:

a repository interface defining the functionality our business logic requires for storing and retrieving tasks

public interface TaskManagementRepository {

    void save(Task task);

    List<Task> getAll();

    Optional<Task> get(String taskID);

    void delete(String taskID);
}

an in-memory DB implementation of the repository interface that uses a hash map to store and retrieve tasks

public class InMemoryTaskManagementRepository implements TaskManagementRepository {

    private Map<String, Task> tasks = new HashMap<>();

    @Override
    public void save(Task task) {
        tasks.put(task.getIdentifier(), task);
    }

    @Override
    public List<Task> getAll() {
        return tasks.values().stream()
                .collect(Collectors.toUnmodifiableList());
    }

    @Override
    public Optional<Task> get(String taskID) {
        return Optional.ofNullable(tasks.get(taskID));
    }

    @Override
    public void delete(String taskID) {
        tasks.remove(taskID);
    }

}

a Guice binding that ensures every piece of business logic that needs to use a TaskManagementRepository instance will use the same InMemoryTaskManagementRepository instance

public class ApplicationModule extends AbstractModule {

    @Override
    public void configure() {
        bind(TaskManagementRepository.class).to(InMemoryTaskManagementRepository.class).in(Singleton.class);
    }

}

Target State

The goal of this tutorial is to implement a new class called MongoDBTaskManagementRepository which will be an implementation of the repository interface that uses MongoDB for persistent storage.

We are going to have the DB running as a separate Docker container. Therefore, we will now have an application made of two containers - the service container and the DB container. We will use Docker Compose to manage the multi-container application as a single entity. Keep in mind that this is not a Docker-focused tutorial, so the focus won't be on explaining best practices for setting up a database with Docker or how to use Docker and Docker Compose in general. Feel free to leave a comment if you would like to see a separate article for this.

Database Integration

MongoDB Container Setup

We start by pulling a MongoDB Docker image and setting up our Docker Compose file to start two containers - one for the service itself and one for a MongoDB instance:

docker pull mongo:5.0.9

The Docker Compose file is given below. Notice that we mount a persistent volume to the MongoDB container since we don't want the data to be wiped out whenever the container crashes or is rebuilt). What this means is that any data stored in /data/db on the MongoDB container will be stored under data/mongo (relative to the project root folder) on your laptop. Keep in mind that /data/db is where the MongoDB container stores data by default.

version: "3.9"
services:
  mongo:
    image: "mongo:5.0.9"
    volumes:
      - .data/mongo:/data/db
  webapp:
    build: .
    depends_on:
      - "mongo"
    ports:
      - "8080:8080"

To start the application, we run docker-compose up --build -d which effectively does three things:

creates a container running the MongoDB image - using the default MongoDB port 27017 for connections
re-builds the image for our REST API and creates a container running the service - the container will have port 8080 exposed so that you can connect from your laptop to the API
creates a virtual network connecting the two containers

To stop the application and cleanup the resources (containers and network), run docker-compose down.

The full commit for this step can be found here.

Repository Interface Implementation

Now that we have a MongoDB instance running, we need to create a new class - MongoDBTaskManagementRepository - that will implement the repository interface.

public class MongoDBTaskManagementRepository implements TaskManagementRepository {

    private final MongoCollection<MongoDBTask> tasksCollection;

    @Inject
    public MongoDBTaskManagementRepository(MongoCollection<MongoDBTask> tasksCollection) {
        this.tasksCollection = tasksCollection;
    }

    @Override
    public void save(Task task) {
        MongoDBTask mongoDBTask = toMongoDBTask(task);
        ReplaceOptions replaceOptions = new ReplaceOptions()
                .upsert(true);
        tasksCollection.replaceOne(eq("_id", task.getIdentifier()), mongoDBTask, replaceOptions);
    }

    @Override
    public List<Task> getAll() {
        FindIterable<MongoDBTask> mongoDBTasks = tasksCollection.find();

        List<Task> tasks = new ArrayList<>();
        for (MongoDBTask mongoDBTask: mongoDBTasks) {
            tasks.add(fromMongoDBTask(mongoDBTask));
        }

        return tasks;
    }

    @Override
    public Optional<Task> get(String taskID) {
        Optional<MongoDBTask> mongoDBTask = Optional.ofNullable(
                tasksCollection.find(eq("_id", taskID)).first());

        return mongoDBTask.map(this::fromMongoDBTask);
    }

    @Override
    public void delete(String taskID) {
        Document taskIDFilter = new Document("_id", taskID);
        tasksCollection.deleteOne(taskIDFilter);
    }

    private Task fromMongoDBTask(MongoDBTask mongoDBTask) {
        return Task.builder(mongoDBTask.getTitle(), mongoDBTask.getDescription())
                .withIdentifier(mongoDBTask.getIdentifier())
                .withCompleted(mongoDBTask.isCompleted())
                .withCreatedAt(Instant.ofEpochMilli(mongoDBTask.getCreatedAt()))
                .build();
    }

    private MongoDBTask toMongoDBTask(Task task) {
        MongoDBTask mongoDBTask = new MongoDBTask();
        mongoDBTask.setIdentifier(task.getIdentifier());
        mongoDBTask.setTitle(task.getTitle());
        mongoDBTask.setDescription(task.getDescription());
        mongoDBTask.setCreatedAt(task.getCreatedAt().toEpochMilli());
        mongoDBTask.setCompleted(task.isCompleted());

        return mongoDBTask;
    }

    private static class MongoDBTask {
        @BsonProperty("_id")
        private String identifier;

        private String title;

        private String description;

        @BsonProperty("created_at")
        private long createdAt;

        private boolean completed;
        ...
    }
...
}

Given that this is not a MongoDB-specific tutorial, I skipped a few implementation details around the Mongo DB Java driver, but in essence, we are using a very basic implementation that maps the internal MongoDBTask POJO to a Mongo DB document. For more details, see the official tutorial.

The full commit for this step can be found here.

Binding Everything Together

You might have noticed that the MongoDB repository implementation relies on a MongoCollection<MongoDBTask> instance to be injected. This class is an abstraction for interacting with an actual MongoDB collection. In this section, we will write the code for connecting to the database and binding an instance of this class. Since we are using Guice for dependency injection, we will encapsulate this logic into its own Guice module and then use the new module in the main application module.

public class MongoDBModule extends AbstractModule {

    @Provides
    private MongoCollection<MongoDBTaskManagementRepository.MongoDBTask> provideMongoCollection() {
        ConnectionString connectionString = new ConnectionString(System.getenv("MongoDB_URI"));

        CodecRegistry pojoCodecRegistry = fromProviders(PojoCodecProvider.builder().automatic(true).build());
        CodecRegistry codecRegistry = fromRegistries(MongoClientSettings.getDefaultCodecRegistry(), pojoCodecRegistry);

        MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
                .applyConnectionString(connectionString)
                .codecRegistry(codecRegistry)
                .build();

        MongoClient mongoClient = MongoClients.create(mongoClientSettings);
        MongoDatabase mongoDatabase = mongoClient.getDatabase("tasks_management_db");
        return mongoDatabase.getCollection("tasks", MongoDBTaskManagementRepository.MongoDBTask.class);
    }
}

To use this new module, we need to install it in our main application module and change the binding to use the MongoDB repository implementation:

public class ApplicationModule extends AbstractModule {

    @Override
    public void configure() {
        bind(TaskManagementRepository.class).to(MongoDBTaskManagementRepository.class).in(Singleton.class);

        install(new MongoDBModule());
    }

}

You can see in the MongoDB module that we are using a database with name tasks_management_db and a collection with name tasks but we never actually created these. The reason is that MongoDB will automatically create both the database and the collection as soon as we insert the first document (when we create the first task through our API).

The final piece for binding everything together is to configure the MongoDB_URI environment variable used for connecting to the database in the task management service container. This can be done through the Docker Compose file:

version: "3.9"
services:
  webapp:
    ...
    environment:
      - MongoDB_URI=mongodb://taskmanagementservice_mongo_1:27017
    ...

The URI is built based on the default port the MongoDB container uses and the default hostname convention Docker Compose uses when building containers.

The full commit for this step can be found here.

Testing The Service

Before we start testing, let's re-build the full stack from scratch:

docker-compose down
docker-compose up --build -d

Now, both the service and MongoDB should be up and running. We will once again use curl to test the CRUD API (ideally these manual tests should be automated as integration tests but we will leave that for another article where we focus on testing):

creating a few tasks

curl -i -X POST -H "Content-Type:application/json" -d "{\"title\": \"test-title\", \"description\":\"description\"}" "http://localhost:8080/api/tasks" 

HTTP/1.1 201 
Location: http://localhost:8080/api/tasks/cb06d6a1-960b-47eb-b44b-de0b01303020
Content-Length: 0
Date: Fri, 29 Jul 2022 09:20:25 GMT

curl -i -X POST -H "Content-Type:application/json" -d "{\"title\": \"test-title\", \"description\":\"description\"}" "http://localhost:8080/api/tasks" 

HTTP/1.1 201 
Location: http://localhost:8080/api/tasks/3c4c7a7d-b680-4be5-9dd4-51d02225a700
Content-Length: 0
Date: Fri, 29 Jul 2022 09:20:32 GMT

retrieving a task

curl -i -X GET "http://localhost:8080/api/tasks/cb06d6a1-960b-47eb-b44b-de0b01303020"

HTTP/1.1 200 
Content-Type: application/json
Content-Length: 159
Date: Fri, 29 Jul 2022 09:21:04 GMT

{"identifier":"cb06d6a1-960b-47eb-b44b-de0b01303020","title":"test-title","description":"description","createdAt":"2022-07-29T09:20:25.410Z","completed":false}

retrieving a non-existing task

curl -i -X GET "http://localhost:8080/api/tasks/random-task-id-123"

HTTP/1.1 404 
Content-Type: application/json
Content-Length: 81
Date: Fri, 29 Jul 2022 09:21:19 GMT

{"message":"Task with the given identifier cannot be found - random-task-id-123"}

retrieving all tasks

curl -i -X GET "http://localhost:8080/api/tasks" 

HTTP/1.1 200 
Content-Type: application/json
Content-Length: 321
Date: Fri, 29 Jul 2022 09:21:30 GMT

[{"identifier":"cb06d6a1-960b-47eb-b44b-de0b01303020","title":"test-title","description":"description","createdAt":"2022-07-29T09:20:25.410Z","completed":false},{"identifier":"3c4c7a7d-b680-4be5-9dd4-51d02225a700","title":"test-title","description":"description","createdAt":"2022-07-29T09:20:32.851Z","completed":false}]

deleting a task

curl -i -X DELETE "http://localhost:8080/api/tasks/3c4c7a7d-b680-4be5-9dd4-51d02225a700"                             

HTTP/1.1 204 
Date: Fri, 29 Jul 2022 09:22:11 GMT

patching a task

curl -i -X PATCH -H "Content-Type:application/json" -d "{\"completed\": true, \"title\": \"new-title\", \"description\":\"new-description\"}" "http://localhost:8080/api/tasks/cb06d6a1-960b-47eb-b44b-de0b01303020"

HTTP/1.1 200 
Content-Length: 0
Date: Fri, 29 Jul 2022 09:22:34 GMT

So far, all the tests we executed look pretty much the same as the ones executed in the previous article. However, we know that previously, restarting the service implied losing the data. Let's try this now by rebuilding the full stack again:

docker-compose down
docker-compose up --build -d


curl -i -X GET "http://localhost:8080/api/tasks" 

HTTP/1.1 200 
Content-Type: application/json
Content-Length: 163
Date: Fri, 29 Jul 2022 09:27:25 GMT

[{"identifier":"cb06d6a1-960b-47eb-b44b-de0b01303020","title":"new-title","description":"new-description","createdAt":"2022-07-29T09:20:25.410Z","completed":true}]

As we can see, the data is now persistently stored - we still have the original task we initially created and then updated as part of testing.

Summary

In conclusion, what we've done as part of this tutorial is:

setup a MongoDB container
use Docker Compose to manage a multi-container application
integrate a Java REST API with a MongoDB database

The key point I was trying to show in this article was the fact that all we had to do to switch from an in-memory DB implementation to a proper DB technology was to create a new implementation of the repository interface, configure the connection to the DB inside a new guice module and then change one line in the already existing code so that our business logic will now use the new repository implementation.

We changed the guice binding from:

bind(TaskManagementRepository.class).to(InMemoryTaskManagementRepository.class).in(Singleton.class);

to:

bind(TaskManagementRepository.class).to(MongoDBTaskManagementRepository.class).in(Singleton.class);

We never touched any of the core business logic for our API and this is the beauty of clean architecture - we don't couple our code with the fact that we chose to persist our data in a database rather than in-memory. This is an implementation detail and changing our storage mechanism should not affect the rest of our code.

Step-By-Step Tutorial for Building a REST API in Java

Nikolay Stanchev — Tue, 28 Jun 2022 17:27:37 +0000

Motivation

Having seen many tutorials on how to build REST APIs in Java using various combinations of frameworks and libraries, I decided to build my own API using the software suite that I have the most experience with. In particular, I wanted to use:

Maven as the build and dependency management tool
Jersey as the framework that provides implementation of the JAX-RS specification
Tomcat as the application server
- in particular, I wanted to run Tomcat in embedded mode so that I would end up with a simple executable jar file
Guice as the dependency injection framework

The problem I faced was that I couldn't find any tutorials combining the software choices above, so I had to go through the process of combining the pieces myself. This didn't turn out to be a particularly straightforward task, which is why I decided to document the process on my blog and share it with others who might be facing similar problems.

Project Summary

For the purpose of this tutorial, we are going to build the standard API for managing TODO items - i.e. a CRUD API that supports the functionalities of C reating, R etrieving, U pdating and D eleting tasks.

The API specification is given below:

The full specification can be viewed in the Appendix.

To implement this API, we will use:

Java 11 (OpenJDK)
Apache Maven v3.8.6
Ecplipse Jersey v2.35
Apache Tomcat v9.0.62
Guice v4.2.3

For the purpose of simplicity, I will avoid the use of any databases as part of this tutorial and instead use a pseudo in-memory DB. However, we will see how easy it is to switch from an in-memory testing DB to an actual database when following a clean architecture.

The goal is to end up with an executable jar file generated by Maven that will include the Tomcat application server and our API implementation. We will then dockerize the entire process of generating the file and executing it, and finally run the service as a Docker container.

The following coding steps will only outline the most relevant pieces of code for the purpose of this tutorial, but you can find the full code in the GitHub repository. For most steps, we will add unit tests that won't be referenced here but included in the code change itself. To run the tests at any given point in time, you can use mvn clean test.

Coding Steps

Step 1 - Project Setup

As with every Maven project, we need a POM file (the file representing the P roject o bject M odel). We start with a very basic POM which describes the project information and sets the JDK and JRE target versions to 11. This means that the project can use Java 11 language features (but no features from later versions) and will require a JRE version 11 or later to be executed. To avoid registering a domain name for this example project, I am using a group ID that corresponds to my GitHub username where this project will be hosted - com.github.nikist97.

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <!-- Project Information -->
    <groupId>com.github.nikist97</groupId>
    <artifactId>TaskManagementService</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>TaskManagementService</name>

    <properties>
        <!-- Maven-related properties used during the build process -->
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>
        <!-- This is where we will declare libraries our project depends on -->
    </dependencies>

    <build>
        <plugins>
            <!-- This is where we will declare plugins our project needs for the build process -->
        </plugins>
    </build>
</project>

The full commit for this step can be found here.

Step 2 - Implementing the Business Logic

We start with the most critical piece of software in general, which is our business logic. Ideally, this layer should be agnostic to the notion of any DB technologies or API protocols. Whether we implement an HTTP API using MongoDB on the backend or we use PostgreSQL and implement a command-line tool for interacting with our code, it should not affect the code for our business logic. In other words, the business logic should not depend on the persistence layer (the code interacting with the database) and the API layer (the code that will define the HTTP API endpoints).

The first thing to implement is our main entity class - Task. This class follows the builder pattern and provides argument validation. The required attributes are the task's title and description. The rest of the attributes we can default to sensible values when not explicitly provided:

identifier is set to a random UUID
createdAt is set to the current date time
completed is set to False

public class Task {

    private final String identifier;
    private final String title;
    private final String description;
    private final Instant createdAt;
    private final boolean completed;

    ...

    public static class TaskBuilder {

        ...

        private TaskBuilder(String title, String description) {
            validateArgNotNullOrBlank(title, "title");
            validateArgNotNullOrBlank(description, "description");

            this.title = title;
            this.description = description;
            this.identifier = UUID.randomUUID().toString();
            this.createdAt = Instant.now();
            this.completed = false;
        }

        ...

    }
}

Then, we define the interface we need for interacting with a persistence layer (i.e. a database or another storage mechanism). Notice that this interface belongs to the business layer because, ultimately, it is the business logic that decides what storage functionality we will need. The actual implementation of this interface, though (a MongoDB implementation or an in-memory DB or something else) will belong to the persistence layer, which we will implement in a subsequent step.

public interface TaskManagementRepository {

    void save(Task task);

    List<Task> getAll();

    Optional<Task> get(String taskID);

    void delete(String taskID);
}

Finally, we implement the service class, which has the CRUD logic. The critical piece here is that this class doesn't rely on a concrete implementation of the repository interface - it is agnostic to what DB technology we decide to use later.

public class TaskManagementService {

    private final TaskManagementRepository repository;

    ...

    public Task create(String title, String description) {
        Task task = Task.builder(title, description).build();

        repository.save(task);

        return task;
    }

    public Task update(String taskID, TaskUpdateRequest taskUpdateRequest) {
        Task oldTask = retrieve(taskID);

        Task newTask = oldTask.update(taskUpdateRequest);
        repository.save(newTask);

        return newTask;
    }

    public List<Task> retrieveAll() {
        return repository.getAll();
    }

    public Task retrieve(String taskID) {
        return repository.get(taskID).orElseThrow(() ->
                new TaskNotFoundException("Task with the given identifier cannot be found - " + taskID));
    }

    public void delete(String taskID) {
        repository.delete(taskID);
    }
}

The way this code was written allows us to easily unit test our business logic in isolation by mocking the behavior of the repository interface. To achieve this, we will need to add two dependencies in the POM file:

        ...
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>3.5.13</version>
            <scope>test</scope>
        </dependency>
        ...

The full commit for this step can be found here.

Step 3 - Creating Stub API Endpoints

The next step is to implement the API layer. For this project, we are implementing an HTTP REST API using Jersey. Therefore, we start by adding the dependency in the POM file.

        ...
        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-servlet</artifactId>
            <version>2.35</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.inject</groupId>
            <artifactId>jersey-hk2</artifactId>
            <version>2.35</version>
        </dependency>
        ...

The second dependency is needed after Jersey 2.26 - https://eclipse-ee4j.github.io/jersey.github.io/release-notes/2.26.html - following this version users need to explicitly declare the dependency injection framework for Jersey to use - in this case we go with HK2 which is what was used in previous releases.

Then we implement the resource class, which at this point only has stub methods that all return a status code 200 HTTP response with no response body.

@Path("/tasks")
public class TaskManagementResource {

    @POST
    public Response createTask() {
        return Response.ok().build();
    }

    @GET
    public Response getTasks() {
        return Response.ok().build();
    }

    @PATCH
    @Path("/{taskID}")
    public Response updateTask(@PathParam("taskID") String taskID) {
        return Response.ok().build();
    }

    @GET
    @Path("/{taskID}")
    public Response getTask(@PathParam("taskID") String taskID) {
        return Response.ok().build();
    }

    @DELETE
    @Path("/{taskID}")
    public Response deleteTask(@PathParam("taskID") String taskID) {

        return Response.ok().build();
    }
}

We will also need an application config class to define the base URI for our API and to inform the framework about the task management resource class:

@ApplicationPath("/api")
public class ApplicationConfig extends ResourceConfig {

    public ApplicationConfig() {
        register(TaskManagementResource.class);
    }

}

The full commit for this step can be found here.

Step 4 - Implementing the API Layer

For this project, we will use JSON as the serialization data format for HTTP requests and repsonses.

In order to produce and consume JSON in our API, we need to add a library that's going to be responsible for the JSON serialization and deserialization of POJOs. We are going to use Jackson. The library we need in order to integrate Jersy with Jackson is given below:

        ...
        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
            <version>2.35</version>
        </dependency>
        ...

Then we need to customize the behavior of the JSON object mapper that will be used for serializing and deserializing the request and response POJOs. In this case, we disable ALLOW_COERCION_OF_SCALARS - this means that the service won't attempt to parse strings into numbers or booleans (e.g. {"boolean_field":"true"} will be rejected)

@Provider
public class JsonObjectMapperProvider implements ContextResolver<ObjectMapper> {

    private final ObjectMapper jsonObjectMapper;

    /**
     * Create a custom JSON object mapper provider.
     */
    public JsonObjectMapperProvider() {
        jsonObjectMapper = new ObjectMapper();
        jsonObjectMapper.disable(ALLOW_COERCION_OF_SCALARS);
    }

    @Override
    public ObjectMapper getContext(Class<?> type) {
        return jsonObjectMapper;
    }
}

Once again, we need to make Jersey aware of this provider class:

@ApplicationPath("/api")
public class ApplicationConfig extends ResourceConfig {

    public ApplicationConfig() {
        register(TaskManagementResource.class);
        register(JsonObjectMapperProvider.class);
    }

}

Then we define the request and response POJOs. I will skip the code for these classes, but in summary, we need:

TaskCreateRequest - represents the JSON request body sent to the service when creating a new task
TaskUpdateRequest - represents the JSON request body sent to the service when updating an existing task
TaskResponse - represents the JSON response body sent to the client when retrieving task(s)

The last part of this step is to replace the stub logic in the resource class with the actual API implementation that relies on the business logic encapsulated in the service class from step 2.

@Path("/tasks")
public class TaskManagementResource {

    private final TaskManagementService service;

    public TaskManagementResource(TaskManagementService service) {
        this.service = service;
    }

    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    public Response createTask(TaskCreateRequest taskCreateRequest) {
        validateArgNotNull(taskCreateRequest, "task-create-request-body");

        Task task = service.create(taskCreateRequest.getTitle(), taskCreateRequest.getDescription());

        String taskID = task.getIdentifier();

        URI taskRelativeURI = URI.create("tasks/" + taskID);
        return Response.created(taskRelativeURI).build();
    }

    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public List<TaskResponse> getTasks() {
        return service.retrieveAll().stream()
                .map(TaskResponse::new)
                .collect(Collectors.toUnmodifiableList());
    }

    @PATCH
    @Path("/{taskID}")
    @Produces(MediaType.APPLICATION_JSON)
    public Response updateTask(@PathParam("taskID") String taskID, TaskUpdateRequest taskUpdateRequest) {
        validateArgNotNull(taskUpdateRequest, "task-update-request-body");

        TaskUpdate update = new TaskUpdate(taskUpdateRequest.getTitle(), taskUpdateRequest.getDescription(),
                taskUpdateRequest.isCompleted());

        service.update(taskID, update);

        return Response.ok().build();
    }

    @GET
    @Path("/{taskID}")
    @Produces(MediaType.APPLICATION_JSON)
    public TaskResponse getTask(@PathParam("taskID") String taskID) {
        Task task = service.retrieve(taskID);
        return new TaskResponse(task);
    }

    @DELETE
    @Path("/{taskID}")
    public Response deleteTask(@PathParam("taskID") String taskID) {
        service.delete(taskID);
        return Response.noContent().build();
    }

The full commit for this step can be found here.

Step 5 - Implementing the Storage Mechanism

For simplicity, we are going to implement an in-memory storage implementation of the repository interface rather than relying on a database technology. The implementation will store all tasks inside a map - the key is the task identifier and the value is the task itself. This is just enough for simple CRUD functionality.

public class InMemoryTaskManagementRepository implements TaskManagementRepository {

    Map<String, Task> tasks = new HashMap<>();

    @Override
    public void save(Task task) {
        tasks.put(task.getIdentifier(), task);
    }

    @Override
    public List<Task> getAll() {
        return tasks.values().stream()
                .collect(Collectors.toUnmodifiableList());
    }

    @Override
    public Optional<Task> get(String taskID) {
        return Optional.ofNullable(tasks.get(taskID));
    }

    @Override
    public void delete(String taskID) {
        tasks.remove(taskID);
    }

}

The full commit for this step can be found here.

Step 6 - Binding Everything Together

Now that we have all the layers implemented, we need to bind them together with a dependency injection framework - in this case, we will use Guice to achieve that.

We start by adding Guice as a dependency in the POM file:

        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
            <version>4.2.3</version>
        </dependency>

Then we create a simple guice module to bind the in-memory DB implementation to the repository interface. This basically means that for all classes that depend on the repository interface, Guice will inject the in-memory DB class. We use the Singleton scope because we want all classes that depend on the repository to re-use the same in-memory DB instance.

public class ApplicationModule extends AbstractModule {

    @Override
    public void configure() {
        bind(TaskManagementRepository.class).to(InMemoryTaskManagementRepository.class).in(Singleton.class);
    }

}

Note that if we decide to use an actual database, the code change is as simple as:

implementing the wrapper class for the DB we choose - e.g. MongoDBTaskManagementRepository
changing the binding above to point to the new implementation of the repository interface

Now that we have the module implemented, we can add Inject annotation to all classes where the constructor has a dependency which needs to be injected by Guice. These would be the TaskManagementResource and the TaskManagementService classes. The magic of guice (and dependency injection in general) is that the module above is enough to build the entire tree of dependencies in our code.

TaskManagementResource depends on TaskManagementService which depends on TaskManagementRepository. Guice knows how to get an instance of the TaskManagementRepository interface so following this chain it also knows how to get an instance of the TaskManagementService and TaskManagementResource classes.

The final piece of work is to make Jersey aware of the Guice injector - remember Jersey uses HK2 as its dependency injection framework, so Jersey will rely on HK2 to be able to build a TaskManagementResource class. In order for HK2 to build a TaskManagementResource it needs to know about Guice's dependency injector container. To connect Guice and HK2, we are going to use something called the Guice/HK2 Bridge. It is basically a process of bridging the Guice container (the Injector class) into the HK2 container (the ServiceLocator class).

So we declare a dependency on the Guice/HK2 bridge library:

        ...
        <dependency>
            <groupId>org.glassfish.hk2</groupId>
            <artifactId>guice-bridge</artifactId>
            <version>2.6.1</version>
        </dependency>
        ...

Then we change the ApplicationConfig class to create the bridge between Guice and HK2. Notice that since the ApplicationConfig class is used by Jersey (and thus managed by HK2) we can easily inject the ServiceLocator instance (the HK2 container itself) into it.

        @Inject
        public ApplicationConfig(ServiceLocator serviceLocator) {
            register(TaskManagementResource.class);
            register(JsonObjectMapperProvider.class);

            // bridge the Guice container (Injector) into the HK2 container (ServiceLocator)
            Injector injector = Guice.createInjector(new ApplicationModule());
            GuiceBridge.getGuiceBridge().initializeGuiceBridge(serviceLocator);
            GuiceIntoHK2Bridge guiceBridge = serviceLocator.getService(GuiceIntoHK2Bridge.class);
            guiceBridge.bridgeGuiceInjector(injector);
        }

The full commit for this step can be found here.

Step 7 - Creating the Application Launcher

The final critical step is configuring and starting the application server through a launcher class, which will serve as our main class for the executable jar file we are targeting.

We start with the code for starting an embedded Tomcat server. The dependency we need is:

    ...
    <dependency>
        <groupId>org.apache.tomcat.embed</groupId>
        <artifactId>tomcat-embed-core</artifactId>
        <version>9.0.62</version>
    </dependency>
    ...

Then we need a launcher class. This class is responsible for starting the embedded Tomcat server and registering a servlet container for the resource config we defined earlier (when we registered the resource class).

public class Launcher {

    public static void main(String[] args) throws Exception {
        Tomcat tomcat = new Tomcat();

        // configure server port number
        tomcat.setPort(8080);

        // remove defaulted JSP configs
        tomcat.setAddDefaultWebXmlToWebapp(false);

        // add the web app
        StandardContext ctx = (StandardContext) tomcat.addWebapp("/", new File(".").getAbsolutePath());
        ResourceConfig resourceConfig = new ResourceConfig(ApplicationConfig.class);
        Tomcat.addServlet(ctx, "jersey-container-servlet", new ServletContainer(resourceConfig));
        ctx.addServletMappingDecoded("/*", "jersey-container-servlet");

        // start the server
        tomcat.start();
        System.out.println("Server listening on " + tomcat.getHost().getName() + ":" + tomcat.getConnector().getPort());
        tomcat.getServer().await();
    }
}

If using InteliJ to code this project, then you should ideally be able to run the main method of the Launcher class. There is one caveat here - with the release of JDK 9 and after (and hence the introduction of the Java Platform Module System), reflective access is only allowed to publicly exported packages. This means that Guice will fail at runtime because it uses reflection to access JDK modules. See this StackOverflow post for more information.

The only workaround I found so far for this was to add the following as a JVM option --add-opens java.base/java.lang=ALL-UNNAMED to the run configuration of the main method as suggested in the StackOverflow post I linked. This basically allows Guice to continue doing its reflection as in the pre-JDK 9 releases.

After we use the workaround above and test our launcher, we get to the part of generating an executable JAR file which can be used to start the service. To achieve this, we need the appassembler plugin. Note that we still need to add the --add-opens java.base/java.lang=ALL-UNNAMED JVM argument in order for the executable jar file to work.

         ...
         <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>appassembler-maven-plugin</artifactId>
                <version>2.0.0</version>
                <configuration>
                    <assembleDirectory>target</assembleDirectory>
                    <extraJvmArguments>--add-opens java.base/java.lang=ALL-UNNAMED</extraJvmArguments>
                    <programs>
                        <program>
                            <mainClass>taskmanagement.Launcher</mainClass>
                            <name>taskmanagement_webapp</name>
                        </program>
                    </programs>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>assemble</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
        ...

With this plugin, we can finally generate an executable file and then use it to start the service:

mvn clean package
./target/bin/taskmanagement_webapp

The full commit for this step can be found here.

Step 8 - Adding Exception Mappers

You might have noticed that so far we have defined two custom exceptions that are thrown when the service receives input data it cannot handle:

TaskNotFoundException
InvalidTaskDataException

If these exceptions aren't handled properly when encountered, then the embedded tomcat server will wrap them inside an internal server error (status code 500) which is not very user friendly. As per the API specification we defined in the beginning (see Appendix), we want clients to receive a 404 status code if, for example, they use a task ID that doesn't exist.

To achieve this, we use exception mappers. When we register those mappers, Jersey will use them to transform instances of these exceptions to proper HTTP Response objects.

public class TaskNotFoundExceptionMapper implements ExceptionMapper<TaskNotFoundException> {

    @Override
    public Response toResponse(TaskNotFoundException exception) {
        return Response
                .status(Response.Status.NOT_FOUND)
                .entity(new ExceptionMessage(exception.getMessage()))
                .type(MediaType.APPLICATION_JSON)
                .build();
    }

}


public class InvalidTaskDataExceptionMapper implements ExceptionMapper<InvalidTaskDataException> {

    @Override
    public Response toResponse(InvalidTaskDataException exception) {
        return Response
                .status(Response.Status.BAD_REQUEST)
                .entity(new ExceptionMessage(exception.getMessage()))
                .type(MediaType.APPLICATION_JSON)
                .build();
    }

}


    @Inject
    public ApplicationConfig(ServiceLocator serviceLocator) {
        ...
        register(InvalidTaskDataExceptionMapper.class);
        register(TaskNotFoundExceptionMapper.class);
        ...
    }

Notice the use of a new POJO - ExceptionMessage - which is used to convey the exception message as a JSON response. Now, whenever the business logic throws any of these exceptions, we will get a proper JSON response with the appropriate status code.

The full commit for this step can be found here.

Dockerizing the Application

There are lots of benefits of using Docker but given that this article is not about containers, I won't spend time talking about them. I will only mention that I always prefer to run applications in a Docker container because it makes the build process much more efficient (think application portability, well-defined build behavior, improved deployment process, etc.)

The Dockerfile for our service is relatively simple and based on the maven OpenJDK image. It automates what we did in step 7 - packaging the application and running the executable jar file.

FROM maven:3.8.5-openjdk-11-slim
WORKDIR /application

COPY . .

RUN mvn clean package

CMD ["./target/bin/taskmanagement_webapp"]

With this, we can build the container image and start our service as a Docker container. The commands below assume you have the Docker daemon running on your local machine.

docker build --tag task-management-service .
docker run -d -p 127.0.0.1:8080:8080 --name test-task-management-service task-management-service

Now the service should be running in the background and be accessible from your local machine on port 8080. For starting/stopping it, use this command:

docker start/stop test-task-management-service

Testing the Service

Now that we have the service running, we can use Curl to send some test requests.

creating a few tasks

curl -i -X POST -H "Content-Type:application/json" -d "{\"title\": \"test-title\", \"description\":\"description\"}" "http://localhost:8080/api/tasks" 

HTTP/1.1 201 
Location: http://localhost:8080/api/tasks/d2c4ed20-2538-44e5-bf19-150db9f6d83f
Content-Length: 0
Date: Tue, 28 Jun 2022 07:52:46 GMT

curl -i -X POST -H "Content-Type:application/json" -d "{\"title\": \"test-title\", \"description\":\"description\"}" "http://localhost:8080/api/tasks"

HTTP/1.1 201 
Location: http://localhost:8080/api/tasks/64d85db4-905b-4c62-ba10-13fcb19a2546
Content-Length: 0
Date: Tue, 28 Jun 2022 07:52:47 GMT

retrieving a task

curl -i -X GET "http://localhost:8080/api/tasks/64d85db4-905b-4c62-ba10-13fcb19a2546"

HTTP/1.1 200 
Content-Type: application/json
Content-Length: 162
Date: Tue, 28 Jun 2022 07:54:21 GMT

{"identifier":"64d85db4-905b-4c62-ba10-13fcb19a2546","title":"test-title","description":"description","createdAt":"2022-06-28T07:52:47.872859Z","completed":false}

retrieving a non-existing task

curl -i -X GET "http://localhost:8080/api/tasks/random-task-id-123"                                                       

HTTP/1.1 404 
Content-Type: application/json
Content-Length: 81
Date: Tue, 28 Jun 2022 09:44:53 GMT

{"message":"Task with the given identifier cannot be found - random-task-id-123"}

retrieving all tasks

curl -i -X GET "http://localhost:8080/api/tasks"     

HTTP/1.1 200 
Content-Type: application/json
Content-Length: 490
Date: Tue, 28 Jun 2022 07:55:08 GMT

[{"identifier":"64d85db4-905b-4c62-ba10-13fcb19a2546","title":"test-title","description":"description","createdAt":"2022-06-28T07:52:47.872859Z","completed":false},{"identifier":"d2c4ed20-2538-44e5-bf19-150db9f6d83f","title":"test-title","description":"description","createdAt":"2022-06-28T07:52:46.444179Z","completed":false}]

deleting a task

curl -i -X DELETE "http://localhost:8080/api/tasks/64d85db4-905b-4c62-ba10-13fcb19a2546"

HTTP/1.1 204 
Date: Tue, 28 Jun 2022 07:56:55 GMT

patching a task

curl -i -X PATCH -H "Content-Type:application/json" -d "{\"completed\": true, \"title\": \"new-title\", \"description\":\"new-description\"}" "http://localhost:8080/api/tasks/d2c4ed20-2538-44e5-bf19-150db9f6d83f"

HTTP/1.1 200 
Content-Length: 0
Date: Tue, 28 Jun 2022 08:00:37 GMT

curl -i -X GET "http://localhost:8080/api/tasks/d2c4ed20-2538-44e5-bf19-150db9f6d83f"   
HTTP/1.1 200 
Content-Type: application/json
Content-Length: 164
Date: Tue, 28 Jun 2022 08:01:07 GMT

{"identifier":"d2c4ed20-2538-44e5-bf19-150db9f6d83f","title":"new-title","description":"new-description","createdAt":"2022-06-28T07:52:46.444179Z","completed":true}

patching a task with empty title

curl -i -X PATCH -H "Content-Type:application/json" -d "{\"title\": \"\"}" "http://localhost:8080/api/tasks/d2c4ed20-2538-44e5-bf19-150db9f6d83f"

HTTP/1.1 400 
Content-Type: application/json
Content-Length: 43
Date: Tue, 28 Jun 2022 09:47:09 GMT
Connection: close

{"message":"title cannot be null or blank"}

Future Improvements

What we have built so far is obviously not a production-ready API, but it demonstrates how to get started with the software suite I mentioned in the beginning of this article when building a REST API. Here are some future improvements that can be made:

using a database for persistent storage
adding user authentication and authorization - tasks should be scoped per user rather than being available globally
adding logging
adding KPI (Key Performance Indicators) metrics - things like the count of total requests, latency, failures count, etc.
adding a mapper for unexpected exceptions - we don't want to expose a stack trace if the service encounters an unexpected null pointer exception, instead we want a JSON response with status code 500
adding automated integration tests
adding a more verbose response to the patch endpoint - e.g. indicating whether the request resulted in a change or not
scanning packages and automatically registering provider and resource classes instead of manually registering them one-by-one
adding CORS (Cross-Origin-Resource-Sharing) support if we intend to call the API from a browser application hosted under a different domain
adding SSL support
adding rate limiting

If you found this article helpful and would like to see a follow-up on the topics above, please comment or message me with a preference choice of what you would like to learn about the most.

Appendix

The full API specification using the Open API description format can be found below. You can use the Swagger Editor to display the API specification in a more friendly manner.

swagger: '2.0'

info:
  description: This is a RESTful task management API specification.
  version: 1.0.0
  title: Task Management API
  license:
    name: Apache 2.0
    url: 'http://www.apache.org/licenses/LICENSE-2.0.html'

host: 'localhost:8080'
basePath: /api

schemes:
  - http

paths:

  /tasks:
    post:
      summary: Create a new task
      operationId: createTask
      consumes:
        - application/json
      parameters:
        - in: body
          name: taskCreateRequest
          description: new task object that needs to be added to the list of tasks
          required: true
          schema:
            $ref: '#/definitions/TaskCreateRequest'
      responses:
        '201':
          description: successfully created new task
        '400':
          description: task create request failed validation
    get:
      summary: Retrieve all existing tasks
      operationId: retrieveTasks
      produces:
        - application/json
      responses:
        '200':
          description: successfully retrieved all tasks
          schema:
            type: array
            items:
              $ref: '#/definitions/TaskResponse'

  '/tasks/{taskID}':
    get:
      summary: Retrieve task
      operationId: retrieveTask
      produces:
        - application/json
      parameters:
        - name: taskID
          in: path
          description: task identifier
          required: true
          type: string
      responses:
        '200':
          description: successfully retrieved task
          schema:
            $ref: '#/definitions/TaskResponse'
        '404':
          description: task not found
    patch:
      summary: Update task
      operationId: updateTask
      consumes:
        - application/json
      parameters:
        - name: taskID
          in: path
          description: task identifier
          required: true
          type: string
        - name: taskUpdateRequest
          in: body
          description: task update request
          required: true
          schema:
            $ref: '#/definitions/TaskUpdateRequest'
      responses:
        '200':
          description: successfully updated task
        '400':
          description: task update request failed validation
        '404':
          description: task not found
    delete:
      summary: Delete task
      operationId: deleteTask
      parameters:
        - name: taskID
          in: path
          description: task identifier
          required: true
          type: string
      responses:
        '204':
          description: >-
            successfully deleted task or task with the given identifier did not
            exist

definitions:
  TaskCreateRequest:
    type: object
    required:
      - title
      - description
    properties:
      title:
        type: string
      description:
        type: string
  TaskUpdateRequest:
    type: object
    properties:
      title:
        type: string
      description:
        type: string
      completed:
        type: boolean
  TaskResponse:
    type: object
    required:
      - identifier
      - title
      - description
      - completed
      - createdAt
    properties:
      identifier:
        type: string
      title:
        type: string
      description:
        type: string
      createdAt:
        type: string
        format: date-time
      completed:
        type: boolean

A Guide for Common SSH Use Cases

Nikolay Stanchev — Mon, 13 Jun 2022 10:30:33 +0000

What is SSH

SSH is a network protocol allowing two machines (physical or virtual) to establish an encrypted connection and comminucate with each other over an unsecure network without compromising the confidentiality and integrity of the exchanged messages. The protocol uses public-key (a.k.a. asymmetric) cryptography to authenticate an SSH client that wants to connect to an SSH server.

OpenSSH is an implementation of the SSH protocol. All the usage examples given in the next sections are based on it.

SSH Basic Usage

A very common usage of SSH is to login to a remote server. Assuming you have a private/public key pair on your local machine and the public key is also distributed on the server you are trying to login to, then connecting is as simple as running:

ssh <server domain name>
------------------------------
ssh myserver.com

This assumes two things:

you have an SSH agent running with the appropriate private key added or the path to the private key follows a standard naming convention (e.g. ~/.ssh/id_rsa) - you might need to run ssh-add <path to private key> if this is not the case
- ~/.ssh/id_rsa is one of the standard naming conventions that the SSH client automatically attempts to use when authenticating with a server
you want to authenticate with the username on your local machine
- this assumption will not hold if on my laptop I use something like nikolay as the username but on a server running Ubuntu only the ubuntu user exists

The following command can be used to change the username to authenticate with:

ssh <username>@<server domain name>
---------------------------------------------
ssh ubuntu@myserver.com

Advanced SSH Usage - Tunneling

A more advanced usage of SSH is the so called SSH tunneling (a.k.a. SSH port forwarding). SSH tunneling is a process of establishing communication between a client (in this case, your local machine) and a server (a remote machine) through a jump server (the machine running an SSH server). This is achieved by mapping ports between the client and the SSH server (hence the name "port forwarding").

This can be visualized with the following diagram:

Keep in mind that the remote machine might be the jump server itself.

There are 3 types of port forwarding, the behavior of which will be illustrated in the following sections. All the example commands assume that the client's public key is already distributed to the SSH server and that the client can successfully login to the SSH server using the command given in the previous section.

N.B. remember that if using a different username on your local machine and on the SSH server, then you need to include <SSH server username>@ in front of all the "jump server" references for the examples below to work - for example, instead of mysshserver.com you might have to use something like user@mysshserver.com

Local Port Forwarding

Local port forwarding is the process of tunneling connections initiated by the client machine through a jump server and forwarding these connections to a remote server or the jump server itself.

To initiate this, run the command below on your local machine:

ssh -N -f -L <local port>:<target server domain name>:<remote port> <jump server domain name>
------------------------------------------------------------------------------------------------------
ssh -N -f -L 5001:mytargetapp.com:5002 mysshserver.com

With the example above, all connections to port 5001 on your local machine will be tunneled (in an encrypted manner) to the SSH jump server (mysshserver.com) and then forwarded to mytargetapp.com at port 5002.

The CLI flags are explained below:

-L - this is the key flag that instructs the SSH client to execute local port forwarding
-N - by default the SSH client will open a session even when executing local port forwarding; this flag instructs the client to not execute a remote command (i.e. only forward ports)
-f - this instructs the ssh client to run in the background (so that you don't need to keep your terminal open while port forwarding is running)

Another example where we are not forwarding to a remote machine but to the jump server itself:

ssh -N -f -L 5001:localhost:5002 mysshserver.com

With this, any connections to port 5001 on your local machine are tunneled to port 5002 on the jump server itself (mysshserver.com).

Remote Port Forwarding

Remote port forwarding is the reverse process of local port forwarding - forwarding connections initiated by a remote machine (or the jump server itself) through the jump server and tunneling these connections back to your local machine on the specified target port.

To initiate this, run the command below on your local machine:

ssh -N -f -R <remote port>:localhost:<local port> <jump server domain name>
------------------------------------------------------------------------------------------------------
ssh -N -f -R 5001:localhost:5002 mysshserver.com

The example above shows a scenario where port 5001 will be open for connections on the jump server (mysshserver.com), any connection to it will be tunneled back to your local machine (in an encrypted manner), and then forwarded to the application listening on the target port - 5002.

The only new CLI flag in this case is -R which instructs the SSH client to execute remote port forwarding. Remote port forwarding is a very common solution if you want to temporarily expose an application running on your local machine to the public (assuming the SSH server is publicly available on the internet) or to someone that has network access to the SSH server in general (for example, through a VPN).

Dynamic Port Forwarding

Dynamic port forwarding is similar to local port forwarding in the sense that local connections are tunneled (in an encrypted manner) to the SSH jump server. The difference is that with dynamic port forwarding you don't need to specify a target host or port - all connections are forwarded to the original target host and target port through the SSH server. In technical terms, dynamic port forwarding is the process of using the SSH server as a SOCKS5 proxy server.

To initiate this run:

ssh -N -f -D <local proxy port> <jump server domain name>
------------------------------------------------------------------------------------------------------
ssh -N -f -D 1080 mysshserver.com

The new CLI flag in this case is -D which instructs the SSH client to execute dynamic port forwarding. The recommendation is to use port 1080 as it is the standard SOCKS protocol port number.

Then you can configure your browser (say Firefox - see this tutorial) or any other application that supports SOCKS/SOCKS5 proxies to use this tunnel and securely forward your network traffic through the SSH jump server (mysshserver.com). In essense, this gives you a very basic VPN functionality by hiding data about your local machine - websites you visit will see traffic that seems to be coming from the SSH server.

The diagram above illustrates an example where the client is accessing somewebsite.com through dynamic port forwarding. The browser uses the proxy to connect to the website. The communciation between the local machine and the SSH server (mysshserver.com) is encrypted. The SSH server then forwards the request to the actual website. From the perspective of somewebsite.com the conenction was initiated from the SSH server rather than the local machine.

6 Things You Can Easily Do With Nginx

Nikolay Stanchev — Fri, 10 Jun 2022 15:22:19 +0000

Introduction

Nginx is by far one of the most famous and most commonly deployed web servers out in the world - check the web servers' usage statistics provided by W3Techs. Nginx can also act as a proxy server, load balancer, caching server, etc. - feel free to read the official docs for more information on what the product can offer you.

My first experience with Nginx was when trying to use it as an API gateway (more like a single entry point for a system) proxying requests to multiple back-end services. I fell in love with the simplicity of a product with so many capabilities. Since then, I've used it for multiple projects and would still prefer it in comparison to alternative web servers.

I decided to write this post to illustrate some of the features I found super useful during my experience with Nginx which in my opinion turned out to be incredibly simple to configure. In no particular order, the list is given in the next section.

If you don't understand the http/server/location directives in the configuration examples below or you haven't seen any Nginx configuration before, please refer to this guide for more details.

How-tos

Limit request body size

Very often when developing an API we let users upload arbitrary files - be they image files, video files, archives, etc. This would commonly imply sending a large amount of bytes as part of the HTTP request body. When left with no validation this functionality can lead to unexpected storage costs or even memory exhaustion if the backend server loads the whole file into memory. Nginx can easily provide protection for this by enforcing a limit on the size of the request body. Here is an example configuration.

http {
    ...
    client_max_body_size 500M;
    ... 
}

Yes, it is that simple. This line is enough to ensure the body of all HTTP requests sent to this host does not exceed 500 megabytes. Any requests that exceed the limit will receive a client error response with status code 413 (Request Entity Too Large).

Another example is given below - enforcing the limit for a particular URI pattern rather than for every single request:

http {
    ...
    server {
        ...
        listen 80;

        location /api/pictures {
            ...
            client_max_body_size 500M;
            ...
        }
        ...
    }
    ...
}

The config above defines an HTTP server listening on port 80 and enforces all requests sent to URIs starting with /api/pictures to have a maximum body size of 500 megabytes. For more details on this, please see the official docs

N.B. the default threshold is 1 megabyte

Add HTTP basic authentication

A common use case website developers have is restricting access to certain sub-pages of their website. Personally, what I also commonly do during manual testing of a new website/API is restrict access to the entire application to a particular set of test users until the release is official. Nginx's auth basic module can easily allow us to do just that. For example:

http {
    ...
    auth_basic "Please authenticate";
    auth_basic_user_file /etc/nginx/.htpasswd;
    ... 
}

This config restricts the entire Nginx server using HTTP Basic Authentication. Keep in mind the /etc/nginx/.htpasswd file must contain the list of <username>:<password> pairs and is expected to follow a certain format. Please see this tutorial for how to generate the necessary password file.

The same configuration can be used to only restrict a particular URI pattern:

http {
    ...
    server {
        ...
        listen 80;

        location /admin {
            ...
            auth_basic "Please authenticate";
            auth_basic_user_file /etc/nginx/.htpasswd;
            ...
        }
        ...
    }
    ...
}

Limit API access to read-only requests

Sometimes when exposing a local back-end service to the public through Nginx (acting as a proxy) I only want to allow read-only (e.g. GET and HEAD) requests that the front-end needs for retrieving content - for example, I might not want to expose publicly any ops-related DELETE API endpoints. Nginx supports this through the limit_except directive:

http {
    ...
    server {
        ...
        listen 80;

        location /api {
            ...
            limit_except GET {
                deny all;
            }
            ...
        }
        ...
    }
    ...
}

N.B. the HEAD method is automatically included when allowing the GET method

Add SSL support

Who doesn't love the message from our browsers indicating that the connection with a particular website is secure? To enable this for an Nginx web server we can use the Nginx SSL module:

http {
    ...
    server {
        ...
        listen 443 ssl;

        ssl_certificate <path to certificate file in PEM format>;
        ssl_certificate_key <path to private/secret key file in PEM format>;
        ...
    }
    ...
}

This config assumes you know how to generate an SSL certificate. One of the simplest ways to do this is to use Let's encrypt - follow this tutorial. Ultimately, you should end up with fullchain.pem and privkey.pem ; these are the files you need for the ssl_certiciate and ssl_certificate_key directives respectively.

Rate limiting clients

Rate limiting is the process of ensuring that an individual client cannot exceed a given threshold of requests for a given unit of time - e.g. ensuring that each client can send a maximum of 5 requests per second. The module for configuring this is the Nginx HTTP Limit Request Module.

http {
    ...
    limit_req_zone $binary_remote_addr zone=books:10m rate=5r/s;
    ...
    server {
        ...
        listen 80;

        location /api/books {
            ...
            limit_req zone=books;
            ...
        }
        ...
    }
    ...
}

Here we need one directive ( limit_req_zone ) for defining the rate limiting "zone" (a re-usable rate limiting configuration) and another directive ( limit_req ) for indicating that we want to enable the rate limiting zone for a particular context (e.g. a URI location pattern). The limit_req directive can also be used inside a server or http context - in other words, rate limiting can be applied for all virtual servers, for an individual virtual server or for a particular URI location pattern.

By default, clients that exceed the rate limiting threshold will receive a status code 503 Service Unavailable. This is also configurable with the limit_req_status directive:

http {
    ...
    limit_req_zone $binary_remote_addr zone=books:10m rate=5r/s;
    ...
    server {
        ...
        listen 80;

        location /api/books {
            ...
            limit_req zone=books;
            limit_req_status 429;
            ...
        }
        ...
    }
    ...
}

Redirect HTTP traffic to HTTPs

A very very very common scenario for web servers is to automatically redirect all clients who connect to the non-encrypted version of a website (HTTP traffic - i.e. server listening on port 80) to the safer encrypted version of the website (HTTPs traffic - i.e. server listening on port 443). Redirection is another easy thing to configure in Nginx:

http {
    ...
    server {
        listen 80;
        return 301 https://$host$request_uri;
    }
    server {
        ...
        listen 443;
        ...
    }
    ...
}

What this config means is that Nginx will return a status code 301 (Moved Permanently) to all clients that connect to port 80. The redirection target will be a URL that uses the same host name and request URI but with https rather than http as the URL scheme. Browsers will automatically follow the redirection and move clients to the secure version of our website.