DEV Community: Akinwale Folorunsho Habibullah

How to improve test isolation in integration testing using Jest and testcontainers

Akinwale Folorunsho Habibullah — Sun, 10 Dec 2023 08:02:11 +0000

Integration testing is a software testing technique that verifies the interactions and data exchange between different components or modules of a software application. It aims to identify any problems or bugs that arise when different components are combined and interact with each other. Integration tests provide better guarantee than unit tests but come at a cost, which is speed.

Integration tests are slower than unit tests because they interact with software dependencies and possibly wait for long running asynchronous or I/O intensive processes to complete, such as database queries. Integration tests however provide better guarantees than unit tests, run less often and have a broader scope including many units and dependencies.

In a bid to make our integration tests faster, we can run them in parallel. Jest runs tests in parallel by default, using the number of available CPU cores. Running tests in parallel does save time, but we must ensure our tests are well isolated from each other and have very little to no shared resources.

If tests are not properly isolated or have too many shared resources, you can have a test that invalidates the state of another test. For example, a test can write in a MongoDB database collection, while another test deletes all the documents in the same MongoDB database collection as part of its setup, possibly in the beforeEach hook.

The first thing we need to do regarding database integrations in our test is to use a separate database instance.

This gives us a good starting point because we don't risk polluting or deleting our development data or in the worst case scenario, production data.

Using dotenv, we can easily use a separate test specific database instance by having an environment specific dotenv file such as test.env, dev.env etc. In our application, we can then read the relevant env file like this:


require('dotenv').config({ path: `${process.env.NODE_ENV}.env` })
......

const MONGODB_URI = process.env.MONGODB_URI;
mongoose.connect(MONGODB_URI);

When running your application in test mode with Jest (identified by the NODE_ENV variable being set to test), the application will automatically load the test-specific environment file. This file can be used to configure your application for testing, including connecting to a dedicated test database.

In simpler terms, Jest automatically switches to the test environment settings when running your tests. This allows you to use a separate database for testing purposes, ensuring your tests are isolated and don't affect your development or production data.

Tests in separate files could however still invalidate each other's state if they both access the same database collection. Given that tests run in parallel, we cannot be sure of the order the tests will run, and it it's generally a bad practice to rely on the order of execution in our tests, because we can never be sure.

A solution is to run tests sequentially. To run tests sequentially, use the --runInBand option when running Jest. When using the option with npm script, you need to prefix the option with -- before using any option, like this

npm run test -- --runInBand

when running Jest

jest --runInBand

JEST_WORKER_ID is set to 1 for all tests when runInBand is set to true, and rightly so because we wont have more than one worker process. Only one test will run at a time before execution proceeds to the next test.

Running tests sequentially solves our problem, but at the cost of speed. Meaning out tests will run slower.

Another option is to keep running our tests in parallel mode but first check if the application is running in test environment, and then add change the MongoDB connection string to make sure the database we are connecting to is unique for each test.

let connection
let { MONGODB_URI, NODE_ENV, JEST_WORKER_ID } = process.env
if (NODE_ENV === 'test') {
  MONGODB_URI = `${MONGODB_URI}_${JEST_WORKER_ID}`
}
mongoose.connect(databaseUrl);

...
module.exports = {
  connection,
  server
};

Besides setting the environment variable to test, Jest also sets the JEST_WORKER_ID environment variable. Each worker process is assigned a unique id (index-based that starts with 1). Using JEST_WORKER_ID, we can ensure we are using a unique database when making connections to the database.

With MongoDB's creation on first use, we don't need to do anything besides connecting to the unique database. If you use a relational database in your application, where the database has to exist first before connecting to it, then you will need to connect to the database using a default database and superadmin or root user, execute create database DDL query first, and then switch connection to the newly created unique database.

It is worthy to note that a testable application is designed with testing in mind, so checking and then manipulating the connection string if its a test context in our application is not entirely bad, but I will say inconvenient.

We can get rid of having to provision a database for testing purposes, so that its a self contained application especially in a test context. This makes integration tests easier especially in CI/CD. This is where testcontainers shines. It is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container.

To get started, add testcontainers as a dev dependency.


npm install testcontainers --save-dev

Then remove the the MONGODB_URI and its value in your test.env configuration file.

We then need to create a MongoDB container and get the connection details to the container. We also need to inject the MongoDB connection string into the app, so the application can connect to MongoDB database as it would when running in any other environment apart from test.

First, let us create a MongoDB container for our tests.

const { GenericContainer } = require('testcontainers')

....

let container;
let app
let connection

beforeAll(async () => {
  container = await new GenericContainer('mongo')
    .withExposedPorts(27017)
    .start()

const connectionUrl = `mongodb://${container.getHost()}:${container.getMappedPort(27017)}/kittytest`
process.env.MONGODB_URI = connectionUrl
app = require('./app').server
connection = require('./app').connection
});

In the beforeAll hook of our test suite, we created a MongoDB container using a GenericContainer from testcontainers. All we needed to do create the container was tell GenericContainer to create a container of type mongo. You will agree with me that this declarative approach to creating a container is terser than using a Dockerfile. We then read the host using the getHost method, and the exposed port using the getMappedPort method.

To inject the MongoDB container's connection string into our application, we create the MongoDB string using the details we read from the container and formatted it to match a MongoDB connection string format. We then added this connection string to process.env.MONGODB_URI, which the app reads from the process.env values, after dotenv must have loaded some environment variables from dotenv files.

Lastly, we imported our app and mongoose connection and saved them into app and connection global variables, so we can close both after all tests finish running.

We delayed importing the application till this point, after we have added the MongoDB connection string to the process.env. If we imported the app before this, the value of MONGODB_URI will be null, and the application will not be able to connect to a database, causing our tests to fail.

Now we can run our integration tests, and our application will run smoothly against a MongoDB container provided by testcontainer.

To ensure we don't leave any resources open or processes running which will leave Jest hanging, we can use the afterAll Jest hook.


afterAll(async () => {
  app.close()
  connection.close()
  await container.stop()
});

Besides closing both our express application, and mongoose connection, we also need to stop our container using container.stop(). The stop method is an asynchronous method, so we use await to wait for the container to shut down before execution proceeds.

In this article, we saw how to roperly isolate tests when doing integration testing and also create a MongoDB container using testcontainers. We also learnt how to guarantee that each test file will run against a unique MongoDB database. We also used testcontainers to replace our test specific MongoDB database.

I hope you find this article informative and useful. Happy testing.

Spring Integration with Mail support full stack app.

Akinwale Folorunsho Habibullah — Tue, 11 Jul 2023 18:11:34 +0000

In this tutorial, you will learn to use Spring Integration to receive emails from Gmail, create an API endpoint using Spring MVC and use React to fetch data from the backend API. In this tutorial, we will keep track of payments into a Bank account, store the information in a MongoDB database and show a list of transactions in a React app.

Scenario

Assuming a small business owner needs his or her account department to keep track of payments made by customers or clients for goods or services without relying on any platform to do that. For privacy reasons, the business owner also cannot grant the accounts department access to the business's email.

We will create a full-stack React and Spring boot app. We will also create a Spring integration workflow that listens for new transaction notification emails and persists the transaction data in a MongoDB database. A React app will consume the transaction data and show a list of transactions to the account department staff.

Introduction to Spring Integration.

Spring Integration is a simple model for building enterprise integration solutions. Spring Integration extends the Spring programming model to support Enterprise Integration Patterns. Spring Integration provides lightweight messaging within Spring-based applications and supports external systems via declarative adapters.

We will use Spring Boot and other Spring projects like Spring MVC, Spring Data JPA, and Spring Integration.

Environment setup

I will create the Spring application using Spring initializer at https://start.spring.io. You can also start a new project using Spring Tool Suite. Spring Tool Suite is a Spring development environment and provides extensions for Eclipse IDE and Visual Studio Code.

Download your new Spring project by clicking the Generate button on the Spring initializer web page. Next is to unzip the folder and open the project in your favourite IDE of Text Editor. I currently use VSCode with Spring Tool Suite.

Here's what our pom.xml list of dependencies should look like;



<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-integration</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.integration</groupId>
    <artifactId>spring-integration-http</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <scope>runtime</scope>
    <optional>true</optional>
  </dependency>
  <dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <optional>true</optional>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.springframework.integration</groupId>
    <artifactId>spring-integration-test</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

We need to add other dependencies which do not have a Spring Boot starter project yet. We should also remove spring-integration-http because we will not need it. spring-integration-http is an endpoint adapter that enables Spring Integration to integrate with external systems using the HTTP protocol. We will replace spring-integration with spring-integration-mail dependency.



<dependencies>
  <dependency>
    <groupId>org.eclipse.angus</groupId>
    <artifactId>angus-mail</artifactId>
  </dependency>
  <dependency>
    <groupId>com.sun.mail</groupId>
    <artifactId>jakarta.mail</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.integration</groupId>
    <artifactId>spring-integration-mail</artifactId>
    <version>6.1.0</version>
  </dependency>
  .... list of dependencies
</dependencies>

"angus-mail" dependency is an SMTP protocol provider for Jakarta Mail, while the "jakarta-mail" dependency sends and receives emails via SMTP, POP3 and IMAP protocols.

Create Integration Flow

Let us create a basic integration flow which connects to Gmail and logs a message to the standard output saying we've received an email.
There are different ways to create integration flows in Spring Integration

via XML configuration
via Java configuration
via Java configuration with a DSL (Domain Specific Language)

I will use Java configuration with a DSL option because it is terser and easier to read.

I will create a new file, "MailIntegrationConfig.java", in com. akinwalehabib.transactiontracker package. This file will contain our basic integration flow configuration.



package com.akinwalehabib.transactiontracker;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.Pollers;
import org.springframework.integration.mail.dsl.Mail;

@Configuration
public class MailIntegrationConfig {

  @Bean
  public IntegrationFlow mainIntegration(
    EmailProperties props
  ) {
    return IntegrationFlow
      .from(
        Mail.imapInboundAdapter(props.getImapUrl())
          .shouldDeleteMessages(false)
          .simpleContent(true)
          .autoCloseFolder(false),
        e -> e.poller(
          Pollers.fixedDelay(props.getPollRate())
        )
      )
      .handle(message -> {
        System.out.println("New message received: " + message);
      })
      .get();
  }
}

We created a class and annotated it @configuration, which indicates to Spring that it is a configuration class that will provide beans to the Spring application context.

In the mailIntegration method inside MailIntegrationConfig class, we injected EmailProperties class, which we need to create next. We will use the EmailProperties class to inject configuration properties from the application.yml file.



package com.akinwalehabib.transactiontracker;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

import lombok.Data;

@Data
@ConfigurationProperties(prefix = "email")
@Component
public class EmailProperties {

  private String username;
  private String password;
  private String host;
  private String port;
  private String mailbox;
  private long pollRate = 30000;

  public String getImapUrl() {
    return String.format("imaps://%s:%s@%s:%s/%s", this.username, this.password, this.host, this.port, this.mailbox);
  }
}

We annotated EmailProperties with @data. @data is from the Lombok dependency, which generates our setter, getter, toString and hashCode, thereby saving us some keystrokes. @Component indicates to Spring that this is a component, making it a candidate for component scanning or auto-detection. @ConfigurationProperties annotation injects configuration properties from the Spring environment.

The attributes username, password, host, port, mailbox and pollRate are the details we will inject from our Spring environment, which we will add to the application.yml file.

There are different ways to provide configuration properties to Spring, but we will use the application.yml file in the "src/main/resources" folder. Go ahead and rename the "application.properties" file to "application.yml".



email:
  username: akinwalehabib
  password: somerandomstring
  host: imap.gmail.com
  port: 993
  mailbox: INBOX
  pollRate: 30000

Replace "username" with your email id without the @domainname, then add your app password. To work with Gmail, you need to create an app password and use that in place of your actual password. See instructions here: https://support.google.com/mail/answer/185833?hl=en.

Go ahead and run your application, and then send an email to the configured email address. The spring application will log a message to the terminal when an email arrives.

💡 New message received: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@249c8b65, headers={closeableResource=org.springframework.integration.mail.AbstractMailReceiver$$Lambda$916/0x0000000801155be0@3b6d02de, id=64b0e7d7-70f6-9432-260d-a1fc0941874b, timestamp=1688931330198}]

Because we are only interested in keeping track of payments, we only care about transaction notification emails from the bank. In Spring Integration, we can place a Filter in an integration flow to allow or disallow messages from proceeding to the next step in the flow.
We will add a filter that checks the subject of the email. If it contains the keyword "Transaction Alert", we will pass the message to the next step in our integration flow. We can also add other checks in our filter method, like checking if the sender is our bank's notification email address etc.



....
import jakarta.mail.Message;
import jakarta.mail.MessagingException;

@Configuration
public class MailIntegrationConfig {

  private final String SUBJECT_KEYWORDS = "TRANSACTION ALERT";

  @Bean
  public IntegrationFlow mainIntegration(
    EmailProperties props
  ) {
    return IntegrationFlow
      .from(
        Mail.imapInboundAdapter(props.getImapUrl())
          .shouldDeleteMessages(false)
          .simpleContent(true)
          .autoCloseFolder(false),
        e -> e.poller(
          Pollers.fixedDelay(props.getPollRate())
        )
      )
      .<Message>filter((Message) -> {
        boolean containsKeyword = false;
        try {
          containsKeyword = Message.getSubject().toUpperCase().contains(SUBJECT_KEYWORDS);
        } catch (MessagingException e1) {
          e1.printStackTrace();
        }

        return containsKeyword;
      })
      .handle(message -> {
        System.out.println("New message received: " + message);
      })
      .get();
  }
}

The filter method uses an anonymous function which returns a boolean value. The integration flow will only pass the message to the next step in the integration flow if the returned value from the anonymous function is True.

If you send an email whose subject doesn't contain the keyword "Transaction alert", you should see the following message in your terminal.

💡 The message [GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@20625346, headers={closeableResource=org.springframework.integration.mail.AbstractMailReceiver$$Lambda$953/0x000000080118bc10@9bc0049, id=bdb31273-a3fd-3d6e-ef2e-4a9f98942254, timestamp=1688935506023}]] has been rejected in filter: bean 'mainIntegration.filter#0' for component 'mainIntegration.org.springframework.integration.config.ConsumerEndpointFactoryBean#0'; defined in: 'class path resource [com/akinwalehabib/transactiontracker/MailIntegrationConfig.class]'; from source: 'bean method mainIntegration’

The next step is to transform the received message into an Object. We will create a domain model and then persist it in the database using Spring Data MongoDB.

Let's add Spring data MongoDB dependency to our project and then add mongoDB configuration.



<dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>

Then update our application.yml file.



....
spring:
  data:
    mongodb:
      host: localhost
      port: 27017
      database: emailintegration

Create our domain model.




package com.akinwalehabib.transactiontracker;

import java.time.LocalDateTime;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;

import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@NoArgsConstructor
@ToString
@Document(collection = "emails")
public class Email {

  @Id
  private String id;
  private String email;
  private String subject;
  private Double amount;
  private String remarks;
  private LocalDateTime receiptDate;
  private String content;

  public Email (String email, String subject, String content) {
    this.email = email;
    this.subject = subject;
    this.content = content.toUpperCase();

    this.amount = retrieveAmount(content);
    this.receiptDate = retrieveReceiptDate(content);
    this.remarks = retrieveRemarks(content);
  }

  public double retrieveAmount(String content) {
    Pattern pattern = Pattern.compile("Amount : NGN [1-9]\\d?(?:,\\d{3})*(?:\\.\\d{2})");
    Matcher matcher = pattern.matcher(content);
    String amountString = "";

    if (matcher.find()) {
      String substring = matcher.group();
      String[] substringParts = substring
        .split(":");                                        
      String[] substringAmountPart = substringParts[1]
        .trim()
        .split(" ");                                        
      amountString = substringAmountPart[1].replace(",", "");
      double value = Double.parseDouble(amountString);
      return value;
    }

    return 0.0;
  }

  public String retrieveRemarks(String content) {
    Pattern pattern = Pattern.compile("Remarks :([\\s]+\\w+\\s+[\\w]+)+");
    Matcher matcher = pattern.matcher(content);
    String remarks = "";

    if (matcher.find()) {
      String substring = matcher.group();
      remarks = substring.split(":")[1]
        .trim();
    }

    return remarks;
  }

  public LocalDateTime retrieveReceiptDate(String content) {
    Pattern pattern = Pattern.compile("Time of Transaction : (\\d+-\\d+-\\d+ \\d+\\d+:\\d+)");
    Matcher matcher = pattern.matcher(content);
    LocalDateTime receiptDate = LocalDateTime.now();

    if (matcher.find()) {
      String substring = matcher.group();                                 
      String[] DateTimeParts = substring.split(":");                
      String[] DateParts = DateTimeParts[1].trim().split("-"); 

      int day = Integer.parseInt(DateParts[0]);
      int month = Integer.parseInt(DateParts[1]);
      int year = Integer.parseInt(DateParts[2].substring(0, DateParts[2].length() - 3));

      int hour = Integer.parseInt(DateParts[2]
        .substring(DateParts[2].length() - 2)
        .trim());
      int minute = Integer.parseInt(DateTimeParts[2]);
      receiptDate = LocalDateTime.of(year, month, day, hour, minute);
    }

    return receiptDate;
  }
}

We annotated the Email domain model with @data, @noArgsConstructor, and @ToString from Lombok dependency. We also added @Document(collection = "emails") annotation from "spring-data-mongodb", which specifies the database collection we will store our document into.

In the email domain model, we want to store the subject, the amount received, the transaction remarks and the receipt date. We will retrieve the amount, the transaction remarks and the receipt date from the email content, and that's why we have the retrieveAmount, retrieveRemarks, and retrieveReceiptDate methods.

We also need to create an interface that extends MongoRepository so our application can save our Email domain to the database and also retrieve information from the database.



package com.akinwalehabib.transactiontracker;

import org.springframework.data.mongodb.repository.MongoRepository;

public interface EmailRepository extends MongoRepository<Email, String>{}

We should create a Class to transform our incoming transaction notification email into an Email domain object. We'll then create a transformer in our integration flow, which uses the newly created class.



package com.akinwalehabib.transactiontracker;

import java.io.IOException;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.integration.mail.transformer.AbstractMailMessageTransformer;
import org.springframework.integration.support.AbstractIntegrationMessageBuilder;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.stereotype.Component;

import com.akinwalehabib.transactiontracker.Email;

import jakarta.mail.BodyPart;
import jakarta.mail.Message;
import jakarta.mail.MessagingException;
import jakarta.mail.internet.ContentType;
import jakarta.mail.internet.InternetAddress;
import jakarta.mail.internet.MimeMultipart;

@Component
public class EmailTransformer extends AbstractMailMessageTransformer<Email> {

  private static Logger log = LoggerFactory.getLogger(EmailTransformer.class);
  // private boolean textIsHtml = false;

  @Override
  protected AbstractIntegrationMessageBuilder<Email> doTransform(Message mailMessage) {
    Email email = processPayload(mailMessage);
    return MessageBuilder.withPayload(email);
  }

  private Email processPayload(Message mailMessage) {
    try {
      String subject = mailMessage.getSubject();
      String email = ((InternetAddress) mailMessage.getFrom()[0]).getAddress();
      String content = getTextFromMessage(mailMessage);

      return parseEmail(email, subject, content);
    } catch (MessagingException e) {
      log.error("MessagingException: {}", e);
    } catch (Exception e) {
      log.error("IOException: {}", e);
    }

    return null;
  }

  private String getTextFromMessage(Message message) throws IOException, MessagingException {
    String result = "";
    if (message.isMimeType("text/plain")) {
      result = message.getContent().toString();
    } else if (message.isMimeType("multipart/*")) {
      MimeMultipart mimeMultipart = (MimeMultipart) message.getContent();
      result = getTextFromMimeMultipart(mimeMultipart);
    }
    return result;
  }

  private String getTextFromMimeMultipart(MimeMultipart mimeMultipart) throws IOException, MessagingException {
    int count = mimeMultipart.getCount();
    if (count == 0) throw new MessagingException("Multipart with no body parts not supported.");

    boolean multipartAlt = new ContentType(mimeMultipart.getContentType()).match("multipart/alternative");
    if (multipartAlt) {
      return getTextFromBodyPart(mimeMultipart.getBodyPart(count - 1));
    }

    String result = "";
    for (int i = 0; i < count; i++) {
        BodyPart bodyPart = mimeMultipart.getBodyPart(i);
        result += getTextFromBodyPart(bodyPart);
    }
    return result;
  }

  private String getTextFromBodyPart(BodyPart bodyPart) throws IOException, MessagingException {
    String result = "";
    if (bodyPart.isMimeType("text/plain")) {
      result = (String) bodyPart.getContent();
    } else if (bodyPart.isMimeType("text/html")) {
      String html = (String) bodyPart.getContent();
      result = org.jsoup.Jsoup.parse(html).text();
    } else if (bodyPart.getContent() instanceof MimeMultipart){
      result = getTextFromMimeMultipart((MimeMultipart)bodyPart.getContent());
    }

    return result;
  }

  private Email parseEmail(String senderEmailAddress, String subject, String content) {
    Email email = new Email(senderEmailAddress, subject, content);
    return email;
  }
}

The EmailTransformer is lengthy. It extends AbstractMailMessageTransformer and then overrides the AbstractIntegrationMessageBuilder method. It retrieves the subject and content from an email message and then creates an Object from the Email domain model class. The Email domain class is responsible for retrieving the amount, remarks, and receipt date from the email content passed to it.

We need to add another dependency to our application which we use to parse HTML and extract the text content in the Email Transformer class.



<dependency>
        <groupId>org.jsoup</groupId>
        <artifactId>jsoup</artifactId>
        <version>1.16.1</version>
</dependency>

Then add the transformer to our integration flow. Note how we inject the EmailTransformer class into the method, and then use in our integration flow. We also injected EmailRepository and used it in our handle’s callback function to persist the transaction details in the database.



@Bean
  public IntegrationFlow mainIntegration(
    EmailProperties props,
    EmailTransformer emailTransformer,
    EmailRepository emailRepository
  ) {
    return IntegrationFlow
      .from(
        Mail.imapInboundAdapter(props.getImapUrl())
          .shouldDeleteMessages(false)
          .simpleContent(true)
          .autoCloseFolder(false),
        e -> e.poller(
          Pollers.fixedDelay(props.getPollRate())
        )
      )
      .<Message>filter((Message) -> {
        boolean containsKeyword = false;
        try {
          containsKeyword = Message.getSubject().toUpperCase().contains(SUBJECT_KEYWORDS);
        } catch (MessagingException e1) {
          e1.printStackTrace();
        }

        return containsKeyword;
      })
      .transform(emailTransformer)
      .handle(message -> {
        Email email = (Email) message.getPayload();
        emailRepository.save(email);
      })
      .get();
  }

Here is a sample transaction notification email from my bank. The email consists of different MIMEBodyParts, and the main content is HTML.

Go ahead and run your application, then send an email to your configured email address. Ensure the email subject contains "Transaction alert". Also, ensure the email's content has the "remarks", "amount", and the "time of transaction" as shown above.

After running the application, check the database, and you should have the transaction details stored in the database.

Create API endpoint

We will create an API endpoint to expose our list of transactions stored in the MongoDB database. We already added the "spring-boot-starter-web" dependency through the Spring initializer.
Create a new file titled "EmailIntegrationAPI.java". This new file will contain our API endpoint.



package com.akinwalehabib.transactiontracker;

import java.util.List;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping(path = "/api/payments",
                produces = "application/json")
public class EmailIntegrationAPI {

  private EmailRepository emailRepository;

  public EmailIntegrationAPI(EmailRepository emailRepository) {
    this.emailRepository = emailRepository;
  }

  @GetMapping
  public List<Email> getPaymenets() {
    List<Email> payments = emailRepository.findAll();
    return payments;
  }
}

Run the application and visit http://localhost:8080/api/payments using your favourite browser. You should have a JSON response containing the received payment notification emails.

Create React client

Let us create a web client to interact with our newly created API. We will use "create-react-app" to quickly create a React application inside the src/frontend folder. Ensure your Spring application is running before proceeding.



npx create-react-app src/frontend && cd src/frontend

Add a "proxy" key in package.json, which helps avoid CORS issues and redirects unknown requests to configured host or port during development.



"proxy": "http://localhost:8080"

Let us install MUI component library. MUI provides a DataGrid component that is ideal for our use case.



npm install @mui/material @mui/styled-engine-sc styled-components @mui/x-data-gridnpm install @fontsource/roboto @mui/x-data-grid                                                                                                                  ─╯



npm install @fontsource/roboto @mui/x-data-grid

I will create a new component titled Payments. The Payments component will be in the src/folder/src folder. This component will fetch all payment data from our API and then populate a DataGrid in the DOM.




import React, { useEffect, useState } from 'react'
import {
  Box,
  Skeleton
} from '@mui/material';
import { DataGrid } from '@mui/x-data-grid';

const columns = [
  { field: 'id', headerName: 'ID', width: 70 },
  { field: 'amount', headerName: 'Amount', width: 130, type: 'number' },
  { field: 'receiptDate', headerName: 'Receipt Date', width: 130 },
  { field: 'remarks', headerName: 'Remarks', width: 700 }
];

function PaymentsSkeleton() {
  return (
    <>
      <Skeleton variant="rectangular" animation="wave" width={"100%"} height={"80vh"} />
    </>
  )
}

function Payments() {
  const [payments, setPayments] = useState([]);
  const [loading, setLoading] = useState(true);

  useEffect(() => {
    async function getTransactions() {
      let response = await fetch("/api/payments");
      return await response.json();
    }

    getTransactions()
      .then(payments => {
        setPayments(payments);
        setLoading(false);
      })
      .catch(err => console.error('Err: ' + err));
  }, [])

  if (loading) {
    return <PaymentsSkeleton />
  }

  return (
    <Box>
      <DataGrid
        columns={columns}
        rows={payments}
        initialState={{
          pagination: {
            paginationModel: { page: 0, pageSize: 20 },
          },
        }}
        pageSizeOptions={[ 5, 10, 15, 20, 50, 100 ]}
        checkboxSelection
      />
    </Box>
  );
}

export default Payments

Edit the App.js file in the src/frontend/src folder to display the Payment component.



import * as React from 'react'
import './App.css';
import Payments from './Payments';
import { Container, CssBaseline, Typography } from '@mui/material';

function App() {
  return (
    <>
      <CssBaseline />
      <Container>
        <Typography variant="h2" gutterBottom>
          Spring Integration - Mail
        </Typography>

        <Typography variant="subtitle1" gutterBottom>
          Payments received
        </Typography>

        <hr />

        <Payments />
      </Container>
    </>
  ) 
  ;
}

export default App;

Go to http://localhost:3000 using your browser, your webpage should look like this.

Conclusion

We have learned how to create a Spring integration flow using the Spring integration mail endpoint. We used Spring data MongoDB to persist our domain models. We also created an API with a GET endpoint. In conclusion, we developed a React and MUI-based web client to interact with our backend API.

Spring Integration is great for developing enterprise integration solutions using Enterprise Integration Patterns. Learn more at https://docs.spring.io/spring-integration/docs/current/reference/html/index.html.

I beleive that with Open Banking API, applications should be able to integrate with banking services and listen for events such as debits into the account.

Please feel free to share your thoughts, particularly challenges you’ve faced using Spring Integration.