DEV Community: François Farge

Easy Integration Tests for Event-Driven AWS Architectures with EventScout 📨🔭

François Farge — Thu, 16 Feb 2023 11:26:37 +0000

When building event-driven Serverless applications on AWS, EventBridge is a must-have. It's simple to use, scalable and inexpensive.

However, a challenge I often faced on event-driven projects was testing. I could not find an easy way to validate that the events sent by my application were matching my expectations. The critical challenge was to list events sent through an event bus, which is not natively possible with EventBridge.

TL;DR

I made a cool EventBridge integration testing library for Typescript, check it out!

What could go wrong in my event-driven architecture?
Events? What events
Searching for and EventBridge testing tool
Building a scalable EventBridge testing infrastructure
- Simple to use
- Scalable
- Cheap
- Secure
Use EventScout in your infrastructure
- Deploy resources with EventScout
- Use the EventScout client in your tests

What could go wrong in my event-driven architecture?

Since its initial release in 2019, EventBridge has been widely used to build event-driven Architecture on AWS, progressively replacing older tools such as SQS and SNS (although these still have valid specific use cases).

I have always been a fan of EventBridge. It’s simple to use, powerful and inexpensive. However, during my first projects with EventBridge, I found it very difficult to test my asynchronous applications.

For example, let’s imagine a very simple application with EventBridge:

In this architecture, we have two Lambdas. The first one is synchronously triggered by ApiGateway. It then puts an ORDER_CREATED event into EventBridge. That event triggers the execution of a second Lambda. Pretty simple, right?

However, many things could go wrong in our system:

our application code can behave unexpectedly and fail to send the event

there can be configuration issues:
- invalid IAM permissions
- missing environment variables
- onOrderCreated could listen to the wrong event

To alleviate these risks, we can test our system at 3 different levels:

unit tests: they can address the application failure cause. They validate that the code behaves as expected, and mock its external interactions. Efficient unit testing for Lambda is a complex topic in itself, that I will address in another article about hexagonal architecture for Serverless
integration tests: they assert that several components in our system behave as expected together. There are multiple types of integration tests, and in particular cloud-native integration tests. Fortunately for us, these tests can prevent our configuration failure cause
end-to-end tests: they consider our whole system as a black box and only interact with its interfaces. They are beyond the scope of this article

During the rest of this article, we will focus on integration tests and EventBridge. In our system, two integration tests would seem relevant.

The simplest one tests the interactions between EventBridge and onOrderCreated.

In order to validate that EventBridge and onOrderCreated are correctly configured, we can simply put a valid event in EventBridge at the beginning of our test, then wait and check that our Lambda has been invoked, by checking CloudWatch logs for example.

The second integration test is more complex and aims to check the interactions between the resources at the start of our process.

Here, the goal of our test is to call the endpoint provided by API Gateway and assert that the createOrder Lambda has sent an event matching our expectations in EventBridge. And this is where it gets truly complicated.

Events? What events?

Testing event-driven architectures is a whole topic in itself, but in particular, EventBridge doesn’t make it easier.

But what makes EventBridge-powered applications so difficult to test? EventBridge provides no way of listing events it has received, or check that an event has been put to it. While these features would make little sense in a production environment, they would have been much helpful to check that our code has produced some events.

In order to assert that events have transited through a bus, we need to project them somehow in an observable place.

Searching for an EventBridge testing tool

In order to bypass EventBridge's limitations on the testing topic, I searched for an existing tool.

But what would I want from this tool?

I’d want it to be simple to set up. Developers need to encounter the least possible friction to write good integration tests
I’d want it to be scalable: I need my integration tests to be able to run in parallel without interference
I’d want it to remain cheap: my whole architecture uses the serverless pay-as-you-go pricing model, and my tests must remain inexpensive enough
Obviously, it needs to be secure (do I need to explain why?)

In found several solutions online:

Setting up a Step Function for each test
Creating an EventBridge target to dump events to SQS. This approach was proposed in several articles (here and here) and implemented in the sls-test-tools library
Using CloudWatch to dump and debug events, introduced by David Boyne. Although it was not directly aimed at testing, maybe this solution could be used for integration tests

I wanted to evaluate the perks and drawbacks of each solution, so I scored them against my constraints and put them in a comparative table:

	Simple setup	Scalable	Cheap	Secure
Step functions	❌ Manual setup required	⚠️ Probably one infrastructure per test	✅ Pay-as-you-go	✅ IAM
SQS	✅ NPM package ✅ Automated resources creation	❌ No parallelism: SQS can only have one consumer ⚠️ SQS creation limits	✅ Pay-as-you-go ⚠️ No automatic disabling of resources	✅ IAM
CloudWatch	❌ Manual setup required	✅ Unlimited parallel reads	❌ CloudWatch logs are expensive! ❌ No automatic rule disabling	✅ IAM ⚠️ Sensible event data may persist in the logs

Although all of these solutions inspired me and made me realize that testing EventBridge was possible, none of them completely fulfilled my requirements.

Building a scalable EventBridge testing infrastructure

I therefore decided to build and open-source an EventBridge integration tests library. This is why I am very proud to introduce EventScout!

It is designed following the requirements I had applied to existing online solutions.

Simple to use

EventScout is composed of two highly reusable parts:

A CDK construct to deploy the necessary resources
A lightweight client to use in the tests, for example with Jest of Vitest

If you wish to learn how to use EventScout on you project, head to the documentation.

Scalable

Making our testing infrastructure scalable requires that our tests can be run in parallel. For this:

We must not be constrained by AWS quotas (e.g. after an SQS queue is deleted, another with the same name cannot be created for 60 seconds). Therefore EventScout ensures only the minimal resources are created during a test suite, and reuse resources between tests
We must be able to receive events independently for each test suite:
- Each test suite declares a pattern of events to watch (a trail of events)
- Each trail is then completely independent from all other trails to allow parallel query. Each trail can be queried as many times as necessary

Therefore, a test suite’s sequence diagram looks like the following:

From an architectural point of view, let’s dive in this sequence:

During its setup phase, the test creates an EventBridge rule with the desired pattern, linking to EventScout’s ingestion lambda (storeEvents)
The test produces some events that match the pattern
The newly created rule is called
The rule triggers the ingestion lambda
Using metadata information from the rule, storeEvents saves the event in the correct trail.
The test can query the trail and perform assertions
A the end of the test, the only thing left is to delete the EventBridge rule

Two features are key here to make EventScout scalable:

Using one EventBridge rule per test suite:
- it makes each test fully independent from all others
- it reuses infrastructure and only needs to create and delete the rule, which is quite fast
Using the rule metadata to store events in a trail:
- there is no need for additional request or computation to know which test suites are interested in the received event
- if an event matching two rules is sent, it will be stored in two trails, which enables parallel tests

Cheap

EventScout only uses serverless resources (Lambda, DynamoDB, API Gateway) to take full advantage of the pay-as-you-go model. Moreover, EventScout ensures that no unnecessary resources are used through automatic trail cleanup. Leveraging DynamoDB’s time-to-live capabilities, it automatically:

cleans recorded trail events after 15 minutes
stops the event rules after 15 minutes to handle the case when they haven’t been manually stopped

This makes EventScout the safest way to run integration tests without worrying about their cost.

Secure

All interactions between the EventScout client and the construct are secured by IAM
Events are automatically cleaned after 15 minutes, removing any privacy issue

Use EventScout on your infrastructure

Leveraging EventScout capabilities on your infrastructure is pretty straightforward.

Deploy resources with EventScout construct

Start by installing the EventScout construct:

npm install @event-scout/construct

Then, instantiate the CDK construct in your CDK app:

import { EventScout } from '@event-scout/construct';
import { CfnOutput } from 'aws-cdk-lib';
import { EventBus } from 'aws-cdk-lib/aws-events';

// create the necessary resources
const { restEndpoint } = new EventScout(this, 'EventScout', {
  eventBus: EventBus.fromEventBusName(this, 'EventBus', eventBusName),
});

// export the endpoint value for easier use in tests
new CfnOutput(this, 'EventScoutEndpoint', {
  value: restEndpoint,
  description: 'EventScout endpoint',
  exportName: '<your export name>',
});

The export here is not required, but will be useful to retrieve the EventScout endpoint for your tests.

Use the EventScout client in your tests

npm install --save-dev @event-scout/client

Then you can head to the documentation for details on how to instantiate and use the client in your tests.

Congratulations, you have unlocked the power of EventScout! Use it wisely...

Introducing Swarmion 🐝, a Type-safe Serverless Microservices Framework

François Farge — Thu, 19 May 2022 12:23:00 +0000

The serverless computing paradigm introduced a lot of opportunities to create more scalable and cost-efficient software, but it also came with its own challenges, some of which we harshly experienced.

After more than two years spent building multiple real-world applications, ranging from small startups to large groups, we decided to share our learnings and help other teams build awesome products.

This is why we would like to introduce Swarmion, an open-source framework, dedicated to Serverless microservices. A great emphasis has been done on developer experience, since working with a Typescript monorepo is a challenge.

What is Swarmion? 🐝

Swarmion is an open-source framework for building type-safe Serverless microservices at scale. It takes full advantage of the Serverless Framework to handle provisioning and deployment of resources, while adding support for microservices and end-to-end type-safety.

It is composed of two parts:

a template for starting state-of-the-art projects
a set of tools and plugins, to ease the communication between microservices and secure their deployment

Today, we are going to focus on the first part, stay tuned for more articles...

TL:DR: try it out! Head to the end of this article for a quick tutorial!

Our core beliefs

Your codebase should adapt to your team organization

Changes in the way you organize your teams should not have an impact on the speed at which you can develop and deploy new features. Therefore, Swarmion uses a flexible microservices approach in a monorepo.

DRY (Don't Repeat Yourself)

Having several teams working in the same environment requires efficient collaboration. Swarmion allows to clearly separate the shared logic and interfaces from the service-specific logic for better decoupling.

Developer experience is key for code quality

As your codebase grows, testing and deployment times are likely to skyrocket. Swarmion uses optimized low-level software (esbuild, vitejs) to reduce testing and building times and a smart monorepo management tool (Nx) to provide a top-level developer experience and reduce CI/CD delays.

✨ Template features

The template comes with state of the art tooling for a Typescript monorepo. The main philosophy is to allow easy customization of the different tools used in the packages, without having to write too many code; every tool can be configured at the root level and extended through composition at the package level.

Yarn 3 with workspaces

Yarn 3 and workspaces allows to create a monorepo with multiple packages, each with their own dependencies. Every package in the monorepo is considered as an npm package inside the monorepo. Every modification to its source code is instantly available to the other packages.

Nx

A powerful and extensible build system, Nx, is used to manage dependencies between packages, provide visualization of the monorepo dependencies through the yarn nx graph command and gain access to powerful custom generators. Our nx plugin already has two generators, for libraries and serverless services (more are coming).

Nx also avoids unnecessary computation when running commands inside the monorepo, using local caching and changes detection for an smooth developer experience and an optimized CI/CD.

Typescript

Since every package inside the monorepo is using Typescript, every line of code can be statically checked. A strict and strong Typescript configuration is located at the root level, extended throughout the packages using composition.

ESLint

A comprehensive set of formatting (through eslint-plugin-prettier) and linting rules, generated with Clinter. Once again, each package can easily extend the root configuration.

Shared Typescript libraries

One of the main challenges of a Typescript monorepo is to build shared Typescript libraries. Packages are transpiled in cjs, esm (thanks to Babel) and .d.ts Typescript declaration file, to enable any usage across the monorepo.

Jest

Every package has testing configured through jest, with a default configuration that can easily be extended, thanks to the jest.config.ts file at the package level.

VS Code support

Each package in the monorepo is defined as a workspace in the multi-root workspaces. It enables easier browsing and searching features, bringing the same navigation than in a standard codebase.

Using multi-root workspaces makes it possible to use the great vs-code extension, which makes it possible to run and debug tests directly inside VS Code.

🛠 Generate a project with Swarmion

To create a Swarmion project, head to https://github.com/swarmion/template.

Click the "use this template" button

And you're all set! Follow the install docs.

Quick overview

The generated repo will have the following structure.

├── backend/
|   ├── core/                       # core service
|   ├── users/                      # users service
|   └── ...                         # other deployed services
|
├── frontend/
|   ├── app/
|   ├── cloudfront/
|   └── ...                         # other deployed services
|
├── commonConfiguration/            # configuration files such as jest, babel...
|   ├── babel.config.js
|   └── lintstaged-base-config.js
|
├── contracts/                      # JSONSchema-based binding contracts
|   ├── core-contracts/
|   ├── users-contracts/
|   └── ...                         # other contracts, used between deployed services
|
├── packages/
|   ├── configuration/              # common constants used in all services
|   ├── serverless-configuration/   # common constants used in all serverless deployed services
|   ├── serverless-helpers/         # a set of shared helpers
|   └── ...                         # other internal shared packages
|
├── package.json                   # shared dependencies and global scripts
└── yarn.lock                      # unique lock file, using yarn workspaces

To learn more about this structure, head to the documentation!

🎁 Wrapping up

Be sure to check out https://www.swarmion.dev/ for docs about Swarmion, the template repo and the tools repo. Any feedback is welcome!

🎄 Upcoming features

Swarmion is being maintained by a core team of several people, we are striving to deliver more and more features:

a create-swarmion-app script to create an app even more easily
more contracts in our library to define type-safe and runtime interactions between services
frontend generators for apps and libraries

🤝 Acknowledgments

Swarmion was created by Adrien Cacciaguerra and François Farge.

Thanks to the swarmion team: Guillaume Duboc, Maxime Vivier, Axel Fournier and Stanislas Hannebelle.

Thanks to all the contributors!

Thanks to Theodo for sponsoring this project!