DEV Community: mrasu

Observable API Testing with YAML using Echoed

mrasu — Thu, 22 Feb 2024 15:14:55 +0000

Introduction

I'm building a package that makes integration testing easy by visualizing OpenTelemetry's traces and logs.

GitHub is 👉 👉 https://github.com/mrasu/echoed

What is Echoed

Echoed enhances Integration testing, aka API testing by visualizing OpenTelemetry's traces and logs.
Because it is built on top of Jest, it holds Jest's features like parallel execution or debuggability.
In addition to those features, Echoed offers unique capabilities:

Coverage Analysis: Gain insights into the coverage of your API endpoints based on OpenAPI or Protocol Buffers specifications.
Detect Propagation Leaks: Uncover spans that don't propagate OpenTelemetry's context to their children.
etc.

This is an overview of Echoed

Echoed now suppors YAML!

Because Echoed is working with Jest, JavaScript and TypeScript are originally supported.
And now, you can also write tests with YAML!
With this, you can create tests with less effort and it requires little knowledge about JavaScript or TypeScript.

YAML is like below:

variable:
  productId: OLJCESPC7Z
scenarios:
  - name: Get product detail
    steps:
      - description: fetch /products/{id}
        act:
          runner: fetch
          argument:
            endpoint: http://localhost:8080/products/${productId}
        assert:
          - expect(_.response.status).toBe(200)
          - expect(_.jsonBody.id).toBe(productId)

In the above YAML.
The test does:

Create a request to http://localhost:8080/products/${productId}
Check status code of the response is 200
Check body of the response is JSON and value of id in the JSON is OLJCESPC7Z

After writing the YAML, Echoed compiles it to TypeScript using npx echoed compile.
Then, you can run test by npx jest without knowing TypeScript for it.

Additionally, Echoed generates an HTML report for the tests.
It contains OpenTelemetry's data and coverage for each service.

Here are the screenshots of these features

Complex example

When you want to tests more complicated flow like retry or set up database, Echoed supports them.
Below YAML does more things:

retry: 3 # <- retry failed tests.
variable:
  productId: OLJCESPC7Z # <- define variables.
  db: ${connectDB()}
runners:
  - runner: fetch
    option:
      baseEndpoint: ${_env.BASE_ENDPOINT}/api
      headers:
        content-type: application/json
hook:
  beforeEach: # <- set up DB before each test
    - db.insertProduct(productId)
  afterEach:
    - db.cleanup() # <- clear DB
scenarios:
  - name: Adding item to cart
    variable:
      userId: 1234
    steps:
      - description: Add a product to cart
        arrange:
          - db.createUser(userId) # <- establish preconditions
        act:
          runner: fetch # <- executes HTTP request
          argument:
            endpoint: cart?currencyCode=USD
            method: POST
            jsonBody:
              item:
                productId: ${productId}
                quantity: 1
              userId: ${userId}
        assert:  # <- test anything
          - expect(_.response.status).toBe(200)
          - |
            expect(_.jsonBody).toEqual({
              items: [{
                productId: productId,
                quantity: 1
              }],
              userId: userId,
            })

It does,

Create required data in beforeEach
Establish preconditions in arrange
Create an HTTP request in act
Check the response is what you want in assert
Then delete data in afterEach

Echoed's YAML respects the AAA(Act-Arrange-Assert) pattern. So you can separate code for preparing data and code that actually you want to test.

You can see more examples at GitHub

Bonus. Using Echoed without OpenTelemetry

While the primary feature of Echoed is troubleshooting or analyzing tests by visualizing OpenTelemetry data, it can also be used to write Jest tests in YAML.
Because conversion from YAML to Jest test (TypeScript) doesn't use OpenTelemetry, even when you haven't had time to install OpenTelemetry to you project, you can write tests with YAML using Echoed.

That's all!
Echoed has more features making test easy.
Visit GitHub! 👉👉 https://github.com/mrasu/echoed 👈👈

I'd appreciate any feedback – whether you liked it or not!

How OpenTelemetry Organizes Distributed Tracing

mrasu — Thu, 01 Feb 2024 13:44:14 +0000

OpenTelemetry is a framework for sending traces, metrics, and logs. It consists of various components, such as protocols and SDKs tailored for many programming languages.

This article explores how OpenTelemetry accomplishes distributed tracing and delves into the underlying mechanisms that make it work.

What is Distributed Tracing?

Distributed tracing is a vital mechanism for monitoring and tracking traces across multiple servers, commonly employed in microservices architectures.

OpenTelemetry's distributed tracing primarily consists of two elements: Trace and Span.
A Span contains execution details such as timestamps or SQL query, while a Trace is a tree structure with Span as its nodes.

In the visual representation below, the entire figure represents a Trace, and each bar represents a Span.

It's important to note that a Trace is not a tangible entity; programs generate only Span.
The backend organizes a Trace from received Spans.

Generating Traces from Spans

Distributed tracing can be achieved when the backend constructs a Trace based on Spans.

To create a Trace based on Spans, you need to consider three key elements within the Span:

TraceId
SpanId
ParentSpanId

You can find them in the OpenTelemetry Proto file, specifically in the trace.proto.

TraceID: This is the unique identifier for the entire trace. If, for instance, the TraceID is 9023c11c... in hexadecimal, any span sharing this TraceID (9023c11c...) is part of the same trace.
SpanId: Each Span has a unique identifier within the same Trace
ParentSpanId: This is the identifier of the parent span. When the SpanId of one Span matches the ParentSpanId of another, the Span is considered as the parent of another one.

By combining these elements, we can construct a Trace tree.
Let's illustrate this with an example:

In this example, observe that the ParentSpanId of Spans matches the SpanId of their parent, while different TraceId give rise to different trees.

Propagation in Distributed Tracing

As established in the previous chapter, we learned that Traces can be constructed from elements in Spans.

Now, the question arises: How does a program determine its own TraceId and ParentSpanId?

While sharing TraceId within the same process can be achieved through memory, distributed tracing in a multi-machine environment necessitates a more sophisticated approach.

This is where Propagation becomes essential.

The idea is simple: "Pass your TraceId and SpanId to other machines somehow."
For instance, when making an HTTP request, you can include these identifiers in the headers.

There are some methods for transmitting this information in HTTP headers, due to historical reasons, but I'll focus on the approach standardized by the W3C, which has gained adoption in the industry.

W3C Trace Context

The format of the W3C Trace Context is specified by the W3C, as indicated by its name.

When utilizing W3C Trace Context, the traceparent field is employed in the HTTP request header.

The header follows the format ${version}-${trace-id}-${parent-id}-${trace-flags}.
When using curl, it looks like the example below:

curl \
  -H "traceparent:00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01" \
  localhost

This example demonstrates how the traceparent field is included in an HTTP request header, conveying essential trace information to the destination:

Version: 00 (currently, there is no version other than 00)
TraceId: 4bf92f3577b34da6a3ce929d0e0e4736
ParentSpanId: 00f067aa0ba902b7
Flag: 01 at the end indicates whether the request is being sampled or not.

This header enables the sharing of Trace information with another service.

Conclusion

In conclusion, despite the formidable name of distributed tracing, it essentially boils down to passing data (TraceId, SpanId, ParentSpanId).

That's the way OpenTelemetry facilitates distributed tracing.

Echoed: Observable Integration Testing using OpenTelemetry on top of Jest

mrasu — Thu, 18 Jan 2024 15:00:00 +0000

Troubleshooting integration test failures can be challenging, especially when you run multiple servers.
If you're running multiple requests in one spec, the complexity increases further.

To tackle this issue, I've created a package called Echoed, integrating OpenTelemetry into Jest.
https://github.com/mrasu/echoed

I'd like to get feedback for it.

What is Echoed?

Echoed simplifies integration testing, aka API testing, by analyzing requests executed during testing and presenting the results in HTML format.

Key Features:

Efficient Debugging: Easily debug failed tests by aggregating OpenTelemetry traces and logs and outputting HTML.
Coverage Measurement: Measure the coverage of each service's endpoint in a microservices architecture.
Propagation Leak Detection: Identify the lack of OpenTelemetry's context propagation.
Span Verification: Verify the content of spans, such as duration, SQL queries, and more.
Compatibility: No changes are required for existing tests or packages.
CI-Friendly: No external services are needed, making it easy to execute in CI.
Jest features: Since Echoed is built on top of Jest, it supports debugging and parallel execution.

Visualizing Traces

Echoed creates HTML to visualize the traces of executed requests.

The following image illustrates the trace graph generated by Echoed.

In addition to trace visualization, Echoed extends its functionality to offer coverage measurements for each service's endpoints, even when accessed internally within a microservices architecture.

Installation

Installing Echoed is a straightforward process, offering two options:

New Setup:
- Run the command npx echoed@latest in your terminal. This will generate a codebase with samples, streamlining your initial setup process.
Integration into Existing Test Suite:
- Adjust Jest's configuration by modifying jest.config.js.
- Add the Echoed configuration file (.echoed.yml).

For more guidance, refer to the README included with GitHub's repository.

Code

To demonstrate the power of Echoed, let's explore a couple of practical examples.
The following code snippets showcase how Echoed can be seamlessly integrated into your test suites to visualize traces and ensure the correctness of your services.

Visualizing Traces

Visualizing traces is a breeze with Echoed. Check out the examples below:

describe("Awesome test", () => {
  it("should pass", async () => {
    const response = await fetch(`http://localhost:8080/api/cart`);
    expect(response.status).toBe(200);

    const body = await response.json();
    expect(body.items.length).toBe(10);
  });
});

Because Echoed inspect fetch automatically, you can visualize a trace for the http://localhost:8080/api/cart without any special code.

Verifying Executed SQL

To verify executed SQL, waitForSpan function is:

describe("Awesome test", () => {
  it("should create an OpenTelemetry span", async () => {
    const response = await fetch(`http://localhost:8080/api/products`, {
      method: "POST",
      body: JSON.stringify({
          name: "Awesome Product",
          price: 100,
      }),
    });
    expect(response.status).toBe(200);

    const span = await waitForSpan(response, { // <--This line!
      name: "oteldemo.ProductCatalogService/CreateProducts",
      resource: {
        attributes: {
          "service.name": "productcatalogservice",
        },
      },
      attributes: {
        "db.system": "postgresql",
        "db.statement": /INSERT INTO products .+/,
      }
    });

    const query = span.attributes.find(attr => attr.key === "db.statement");
    expect(query?.value?.stringValue).toBe("INSERT INTO products (name, price) VALUES ('Awesome Product', 100)");
  });
});

This test checks for the SQL statement, INSERT INTO....
Like this code you can verify the behavior of your services at a detailed level.

Conclusion

Echoed improves the experience of integration testing.
Visualize traces, measure coverage, and verify the content of spans effortlessly.
Check out the GitHub repository, and give it a try.

mrasu / echoed

Observable Integration Testing using OpenTelemetry on top of Jest.

Echoed

Observable Integration Testing using OpenTelemetry on top of Jest.

Features

Echoed enhances Integration testing, aka API testing with the following features:

Effortless Test Troubleshooting: Quickly identify issues in failed tests by visualizing OpenTelemetry's traces and logs.
YAML Supported: Write tests effortlessly using YAML and easily expand functionality through your plugins.
Coverage Analysis: Gain insights into the coverage of your API endpoints based on OpenAPI or Protocol Buffers specifications.
Detect Propagation Leaks: Uncover spans that don't propagate OpenTelemetry's context to their children.
Validate Spans: Validate span's fields, such as SQL or requests going outside.
CI-Friendly: Integrates with CI without relying on external services.
IDE Debugging: Debug your tests in your preferred IDE, leveraging Jest's built-in debugging capabilities.
Code Compatibility: No need to modify your existing…

View on GitHub

I'd appreciate any feedback – whether you liked it or not!

DEV Community: mrasu

Observable API Testing with YAML using Echoed

Introduction

What is Echoed

Echoed now suppors YAML!

Complex example

Bonus. Using Echoed without OpenTelemetry

How OpenTelemetry Organizes Distributed Tracing

What is Distributed Tracing?

Generating Traces from Spans

Propagation in Distributed Tracing

W3C Trace Context

Conclusion

Echoed: Observable Integration Testing using OpenTelemetry on top of Jest

What is Echoed?

Visualizing Traces

Installation

Code

Visualizing Traces

Verifying Executed SQL

Conclusion

mrasu / echoed

Observable Integration Testing using OpenTelemetry on top of Jest.

Echoed

Observable Integration Testing using OpenTelemetry on top of Jest.

Table of contents

Features