DEV Community: Josh Gibson

Introducing the @nxpansion/opentelemetry-tasks-runner

Josh Gibson — Fri, 26 Nov 2021 22:29:21 +0000

As your applications grow in complexity, so do the tools required to build those applications. The Nx Build Framework and its accompanying product [Nx Cloud])(https://nx.app/) have made it easier than ever for JavaScript (and even non Javascript) developers to manage their build pipelines. Through its executors and tasks runners, Nx provides developers with an extensible monorepo to help developers manage dependencies between applications, create build scripts, cache results, and much more. When paired with Nx Cloud, these build artifacts can be cached in the cloud and distributed across build servers to speed up your CI Pipeline.

With this complexity, however, developers need a way understand how their build process is performing. CI platforms have come a long way help deliver these insights about your pipelines and workflows, but if you're using Nx, you're still probably left wondering "What is going on when I run nx affected build?" Nx Cloud is a great first step to see build performance and statistics, but that data still lives in the Nx Cloud and isn't queryable or correlatable. This is where @nxpansion/opentelemetry-tasks-runner comes in. This plugin instruments every command ran by the Nx CLI using OpenTelemetry, allowing you to generate traces and send them to the observability platform of your choice. Below are example traces sent to Honeycomb from an example application. Whether you're using stock Nx build executors, using incremental builds and Nx Cloud build cache, or you're using the Nx Cloud distributed build agents, the OpenTelemetry Tasks Runner can instrument you build commands.

Setup

The @nxpansion/opentelemetry-tasks-runner works by wrapping your existing tasks runner. For most users this will be either the @nrwl/workspace/tasks-runners/default or @nrwl/nx-cloud runners. In your nx.json file, simply replace the tasks runner with @nxpansion/opentelemetry-tasks-runner and move your previous settings into the wrappedTasksRunner and wrappedTasksRunnerOptions properties.

Install the plugin

yarn add --dev @nxpansion/opentelemetry-tasks-runner

Default Tasks Runner Example

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nxpansion/opentelemetry-tasks-runner",
      "options": {
        "wrappedTasksRunner": "@nrwl/workspace/tasks-runners/default",
        "wrappedTasksRunnerOptions": {
          "cacheableOperations": ["build", "lint", "test", "e2e"]
        }
      }
    }
  },
}

Nx Cloud Example

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nxpansion/opentelemetry-tasks-runner",
      "options": {
        "wrappedTasksRunner": "@nrwl/nx-cloud",
        "wrappedTasksRunnerOptions": {
          "cacheableOperations": ["build", "lint", "test", "e2e"],
          "accessToken": "SECRET_VALUE"
        },
        "accessToken": "SECRET_VALUE"
      }
    }
  }
}

Note that the access token is left at the top level and also provided in the wrappedTaksRunnerOptions. The nx-cloud cli expects that accessToken value to live at in the options object of the default tasks runner while the wrapped tasks runner needs the token to be passed into it when it is wrapped by the OpenTelemetry Tasks Runner. Putting the token in both places allows for compatibility with both cases.

Exporting Traces

Out of the box, the @nxpansion/opentelemetry-tasks-runner supports sending traces to an OpenTelemetry Collector via OTLP (via gRPC) or printing them out to the console. Those configurations can be set in the tasksRunnerOptions.

OTLP

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nxpansion/opentelemetry-tasks-runner",
      "options": {
        "wrappedTasksRunner": "@nrwl/workspace/tasks-runners/default",
        "wrappedTasksRunnerOptions": {
          "cacheableOperations": ["build", "lint", "test", "e2e"]
        },
        "exporter": "otlp",
        "otlpOptions": {
          "url": "grpc://localhost:4317"
        }
      }
    }
  },
}

Console

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nxpansion/opentelemetry-tasks-runner",
      "options": {
        "wrappedTasksRunner": "@nrwl/workspace/tasks-runners/default",
        "wrappedTasksRunnerOptions": {
          "cacheableOperations": ["build", "lint", "test", "e2e"]
        },
        "exporter": "console"
      }
    }
  },
}

Custom OpenTelemetry Setup

In some cases you want a little more control of the OpenTelemetry SDK being used to collect the traces. For example, if you want to send your traces directly to a platform like Honeycomb rather than first sending them through an OpenTelemetry Collector, you will need to set up a custom NodeSdk with these configurations. To do so, you can provide a custom setup file that exports a function that returns a custom NodeSdk with your desired configurations.

Honeycomb Example

nx.json

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "@nxpansion/opentelemetry-tasks-runner",
      "options": {
        "wrappedTasksRunner": "@nrwl/workspace/tasks-runners/default",
        "wrappedTasksRunnerOptions": {
          "cacheableOperations": ["build", "lint", "test", "e2e"]
        },
        "setupFile": "setup-honeycomb.js"
      }
    }
  },
}

setup-honeycomb.js

const { credentials, Metadata } = require('@grpc/grpc-js');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-otlp-grpc');
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-base');

const getOtelNodeSdk = (defaultConfiguration) => {
  const metadata = new Metadata();
  metadata.set('x-honeycomb-team', process.env.HONEYCOMB_WRITEKEY);
  metadata.set('x-honeycomb-dataset', process.env.HONEYCOMB_DATASET);
  const traceExporter = new OTLPTraceExporter({
    url: 'grpc://api.honeycomb.io:443/',
    credentials: credentials.createSsl(),
    metadata,
  });
  const spanProcessor = new BatchSpanProcessor(traceExporter);

  defaultConfiguration.traceExporter = traceExporter;
  defaultConfiguration.spanProcessor = spanProcessor;

  const sdk = new NodeSDK(defaultConfiguration);

  return { sdk };
};

module.exports = getOtelNodeSdk;

Example Traces

Example Repo

In the below examples, we are building a simple Nx repo that has two apps example-app-1 and example-app-2. Both apps share the libraries lib-2 and lib-3 while lib-1 is used exclusively by example-app-1 and lib-4 is used exclusively by example-app-2. common-lib-5 and common-lib-6 are both libs are are used internally by libs 1-4. Below we will use a variety of strategies to build these two apps and see how these various traces look with the @nxpansion/opentelemetry-tasks-runner when viewing the traces in Honeycomb.

Standard Build (Non-incremental)

This image shows a trace with two applications being built by the @nrwl/node:build executor. One span is created for the command to build these two apps and one child span is created for each application. Since this build is not using buildable libs, both applications have to rebuild all the libraries shared between them, thus each about takes about 7 seconds to compile all of the libs into a single webpacked bundle.

Incremental Build

For the above trace, we have enabled incremental builds for the repo using the @nrwl/node:package executor. The trace shows a span for each library indicating that each lib was built individually. As a result, the libraries were only built once between both apps, however, there is significant overhead to build each library. Instead of taking 14 seconds to build both apps, it took about 3.5 seconds per lib, 1.5 second per app, and 25 seconds in total. This use case clearly isn’t as helpful in a full repository rebuild scenario, but it is promising if we can cache the results of each library. If only one or two libraries change, we should be able to build both apps in well under the initial 14 seconds.

Partial Incremental Build

This trace shows an example of a partial build. We have the same number of spans as the previous incremental build, but for all libraries other than lib-3, their build time is only a few milliseconds (the time it takes to determine the result was already cached). As a result, the only time spent building is the 3.5 seconds for lib-3 and the 1.5 seconds per app, bringing us in at under 7 seconds.

Distributed Build

In this trace, we have taken the repository and built it in a distributed GitHub actions workflow using Nx Cloud’s distributed execution agents. The spans in this trace were generated across 5 different virtual machines. Using W3C Trace Propagation and the otel-cli, we were able to collect traces from all of these different machines into one trace. The @nxpansion/opentelemetry-tasks-runner can create spans under a parent span via the TRACEPARENT environment variable. Thus it is possible to create a root span for your workflow, pass that context to your agents executing the builds and create a single trace representing builds split across multiple machines.

Help Contribute!

Hopefully you can see all of the the possibilities of the OpenTelemetry Tasks. Check us out at GitHub and consider giving the project a star and contributing if you have suggestions!

Triggering GitHub Actions Using Repository Dispatches

Josh Gibson — Sun, 15 Mar 2020 20:30:04 +0000

TL;DR: Using HTTP requests, you can send repository dispatches to trigger GitHub Actions workflows

GitHub Actions is a quickly growing and quickly developing CI/CD platform that is free to all GitHub users. While the platform is still relatively young, features are being released constantly by the Actions team and the community of developers utilizing the platform and developing open source actions for it continues to expand.

One feature that has been at the front of many current and potential users minds is the flexibility of triggering actions. Unsurprisingly, triggering actions based on commits, releases, and branches is simple to configure. GitHub has also made it easy to schedule tasks using cron notation. What becomes less obvious, though, is how users could either manually trigger an action or have a third party service trigger an action. Other platforms like Jenkins make this simple with the ability to build jobs from the click of a button in their web console, or by installing plugins that enable remote services to act as triggers.

While GitHub actions does not yet have a similar feature in their web console, this functionality does exist in the form of repository dispatches. A repository dispatch is essentially just an HTTP request to your repository asking GitHub to kick off any action or webhook. Using this feature, you can both manually trigger GitHub actions using repository dispatches, or you can set up an application to trigger the action by sending a request.

Setting up your action

In your workflow yml file, you need to add a trigger for the on key that says that the workflow will accept repository dispatches.

The simplest configuration is to have the workflow trigger on any repository dispatch:



on:
    repository_dispatch:

If you have multiple workflows that might be triggered by separate dispatches, however, you can specify specific event types that will trigger the workflow.



on:
    repository_dispatch:
        types: [start-example-workflow]

In the above example, only dispatches with the event type "start-example-workflow" will trigger the workflow.

Custom Parameters

Throughout your action, you can also use information provided by the "client-payload" of the dispatch. Using the expression



${{ github.event.client_payload.example-key }}

will give you access to all of the data provided in the client payload. This will allow you to use the data as environment variables for bash commands or as inputs to other actions.

Constructing the request

The payload of a repository dispatch is quite simple:



{
    "event_type": "my_event_type",
    "client_payload": {
        "example-key": "example-value"
    }
}

This request also needs the following set of headers:



Accept:  application/vnd.github.everest-preview+json
Content-Type: application/json
Authorization: Bearer {{personal_access_token}}

At this point, I should clarify that in order to trigger the workflow, the request must be authenticated with a personal access token for a user that is authorized to access the repository.

To create an access token, follow these steps from GitHub and be sure that you have given the token access to the repo scope.

With that payload, you can POST the request to https://api.github.com/repos/:owner/:repo/dispatches to trigger workflows that are set up to accept the dispatch.

Using Postman

If your goal is to simply trigger these workflows with a push of a button, Postman might be one of the easiest ways to set up these POST requests and send them GitHub.

The first step is to create a new environment in Postman to store your personal access token. To do this, click the gear in the upper right hand corner to manage environments and create a new environment called GitHub.

From there, you should add a new environment with your personal access token as the variable token.

Once this environment is set up, you can now create a new collection of requests that will trigger your GitHub Actions workflows. To use the token that is a part of your environment, make sure your environment is selected in the upper right hand corner of your application, and then add the token to the Bearer Authorization of your request.

Next set the appropriate headers.

You can now set the appropriate payload for your dispatch and send the request. If everything is appropriately set up, you should get a 204 No Content Response.

Finally, you can save the request and send the dispatch whenever you want to manually trigger the workflow.

What's next?

Given that these triggers are simple HTTP Requests, you can obviously take advantage of this feature to integrate your actions with third party services. In the next article, we'll investigate strategies to trigger actions with integrations from services such as Slack and Sanity.io, a configurable CMS.

Creating a Transaction Interceptor Using Nest.js

Josh Gibson — Mon, 10 Feb 2020 15:22:13 +0000

TL;DR - Nest.js interceptors allow you to create transactions, attach them to each request, and use those in your controllers. See the final implementation here.

Nest.js is one of the fastest growing backend libraries in the Node.js world. Its clear structure, adjacency to Angular, and its similarities to Spring have all led to the framework to bloom in adoption. Additionally, abstracting away repetitive behavior that might have been tedious or verbose in an express app is a breeze with Nest. Between the built-in decorators, pipes, and the ability to extend all of this behavior, serious abstraction is highly encouraged by the framework. If you haven't used Nest.js, we at TeamHive, highly recommend it. Streamlining API development has never been easier than with Nest and it has simplified our development rapidly.

One such repetitive behavior is the process of setting up and appropriately tearing down transactions. If you use a transactional database with your Nest app, one of the greatest benefits is that you can ensure data integrity. If you wrap your requests in a transaction, and the request fails at any point, you can always rollback the transaction and all modifications made up to that point will be reverted. This process, however, is quite tedious. At the start of each request, you must create a transaction, run all of your queries within that transaction, determine if the request succeeded, and then either commit the transaction or it rollback on a failure. While none of these steps are particularly difficult to implement, wrapping every request in appropriate try/catch blocks that can handle the closing of the transaction would result in duplicated code all over the application.

The simplest way to implement this feature would instead be using Nest.js Interceptors. If you come from the express world, you will be familiar with the idea of middleware. A middleware is essentially a function that receives a request and returns a response based on that request. The power of middleware, however, comes from the fact that they can be chained together. In this way, for example, one middleware function can authorize the request, another one can transform it, and a final middleware could fetch some data and respond to the request. The problem with middleware, however, is that they only exist at one point in the middleware chain. There is no simple way to have a function that can both set up some data before the request is handled and also run some logic after the request. An interceptor, on the other hand allows for just this. By using RxJs Observables, Interceptors allow you to inject logic into any point of the request life cycle. For a transaction interceptor, this is perfect because we need to both set up the transaction before the request is handled, and either commit it or roll it back depending on the success of the request.

Implementation

To create an interceptor, all you have to do is create a class that implements the NestInterceptor interface, meaning it has a function called intercept which will receive an ExecutionContext and a CallHandler and will return an Observable. Below is an interceptor in its simplest form.

import { CallHandler, ExecutionContext, Injectable, NestInterceptor } from '@nestjs/common';
import { Observable } from 'rxjs';

@Injectable()
export class TransactionInterceptor implements NestInterceptor {

    async intercept(
        context: ExecutionContext,
        next: CallHandler
    ): Promise<Observable<any>> {
        return next.handle();
    }
}

The first step to creating the interceptor is going to be creating the transaction. Depending on what database, adaptor, or ORM you use, this part will vary, however, the concept should be the same. In this example, we are going to be using Sequelize as an ORM to create the transaction. With Sequelize, all you need to create a transaction is an established Sequelize instance with a database connection. Nest.js makes this simple as we can simply inject our Sequelize instance into the interceptor. If you are using an ORM that requires a Singleton, creating that as Singleton attached to a provider makes it easy to access throughout the Nest application.

Once that Sequelize instance is available in the intercept function, you can simply create the transaction and attach it to the request using the ExecutionContext.

@Injectable()
export class TransactionInterceptor implements NestInterceptor {

    constructor(
        @InjectConnection()
        private readonly sequelizeInstance: Sequelize
    ) { }

    async intercept(
        context: ExecutionContext,
        next: CallHandler
    ): Promise<Observable<any>> {
        const httpContext = context.switchToHttp();
        const req = httpContext.getRequest();

        const transaction: Transaction = await this.sequelizeInstance.transaction();
        req.transaction = transaction;
        return next.handle();
    }
}

At this point the transaction is available on the request, so any controller function could access it using the @Req parameter decorator; however, this is not only verbose, but is not typed by default and leads to the duplicated code of getting the transaction from the request. Instead, using Nest's createParamDecorator function, we can easily create a decorator that will get this transaction for us.

import { createParamDecorator } from '@nestjs/common';

export const TransactionParam: () => ParameterDecorator = () => {
    return createParamDecorator((_data, req) => {
        return req.transaction;
    });
};

With this decorator, any parameter you decorator in a controller function that is being intercepted by the TransactionInterceptor will receive a transaction that will be created per request.


@Get()
@UseInterceptors(TransactionInterceptor)
async handleGetRequest(
    @TransactionParam() transaction: Transaction
) {
    // make queries using your transaction here.
}

Stopping here, though, will not be any help. The transaction is getting created, but there is no logic to either commit the transaction upon success, or roll it back on errors. To do this, we can tap into the Observable returned from the handle() function. In the code below the function passed into tap() will be run when the request is handled successfully. The function passed into catchError will run when an error is thrown by the request handler.

Final Implementation

@Injectable()
export class TransactionInterceptor implements NestInterceptor {

    constructor(
        @InjectConnection()
        private readonly sequelizeInstance: Sequelize
    ) { }

    async intercept(
        context: ExecutionContext,
        next: CallHandler
    ): Promise<Observable<any>> {
        const httpContext = context.switchToHttp();
        const req = httpContext.getRequest();

        const transaction: Transaction = await this.sequelizeInstance.transaction();
        req.transaction = transaction;
        return next.handle().pipe(
            tap(() => {
                transaction.commit();
            }),
            catchError(err => {
                transaction.rollback();
                return throwError(err);
            })
        );
    }
}

With this code, any error thrown in the controller will cause the transaction to roll back and data will return to its original state. Additionally, errors are still thrown up the stack so all existing error handling/logging will continue to work.

Why use a transaction interceptor?

To start, the benefits of using transactions should be pretty clear. While we like to pretend our code doesn't fail, it inevitably will in production. If your requests make multiple writes or creates multiple objects that are related, it's important that if this process fails or is interrupted, data can be returned to a valid state. Transactions are the easiest way to ensure that and are one of the largest benefits of relational databases.

As mentioned earlier, the interceptor's main benefit is to make using transactions easier while not duplicating code. Once using the interceptor, all the logic of creating and managing the transaction is abstracted. As a developer, your only responsibility is to run your queries under that existing transaction. In this way, the repetitive and uninteresting work of creating and tearing down the transaction is removed, making developers much more likely to take advantage of the transactions that their database makes available to them.