Emulating the cloud within Booster Framework 💻🌩️

One of the cool things about Booster is that most of it's functionality sits on top of an abstract interface that expects some stuff from the cloud. The framework itself doesn't have a single call to any service from AWS, Azure, or Kubernetes. That's the job of the provider packages.

When you are developing your app, you probably don't wanna think about the very little details of each database, cloud service or whatever. Perhaps you, like me, hate even having to learn each and every library or SDK for the technology/service at hand.

Thanks to this abstraction, you just code by using Booster concepts (command, events, etc.) and forget about the rest. But what happens underneath? Let's take a look 👀

Cloud vs local development

The cloud is cool and all that jazz, but what's better than developing locally and seeing your changes instantly?

Yeah, there are things that emulate the workings of specific services, like DynamoDB, or there are folks who run their entire Kubernetes apps, with all the required processes, like MongoDB, MySQL, Redis, etc. Or even things like Serverless framework that deploy your app relatively quickly, but at the cost of maintaining a huge, messy, YAML file.

Stuff should be simpler, you shouldn't need a beefy computer to develop your app.

Due to many reasons, but along them, the ones I just described, people resolve to coding their app in the simplest way possible, probably an express server, or alike.

What if we had an express server that behaved as our app in the cloud? That's the idea with a local provider.

Implementing a Booster provider to work locally

To implement a Booster provider, you'll need to create two npm packages:

• framework-provider-<name of your environment> - This package is in charge of:
  • Provide the functions to store/retrieve data from your cloud.
  • Transform the specific objects of your cloud into Booster ones, e.g. converting an AWS event into a Booster one.
• framework-provider-<name of your environment>-infrastructure - This package is in charge of:
  • Provide a deploy function that will set all the required resources in your cloud provider and upload the code correctly, as well as a nuke function that deletes everything deployed, OR
  • Provide a start function that will start a server and all the appropriate processes in order to run the project in a specific environment. This one is the one that I'll be using for the local provider.

Given that I'm implementing the local provider, I just named them like:

• framework-provider-local
• framework-provider-local-infrastructure

To implement the local provider, I'll be using express that will act as the endpoints provided by Booster, and nedb, which is a local, filesystem implementation of a NoSQL database, with an API very similar to MongoDB. It would be the equivalent of SQLite but for NoSQL databases.

Let's start implementing the first package.

The provider interface

Booster's provider interface is a regular TypeScript interface that must have it's methods implemented, an implementation could look like this:

export const Provider = {
  events: {
    rawToEnvelopes: ...,
    forEntitySince: ...,
    latestEntitySnapshot: ...,
    store: ...,
  },
  readModels: {
    rawToEnvelopes: ...,
    fetch: ...,
    search: ...,
    store: ...,
    // ...
  },
  graphQL: {
    rawToEnvelope: ...,
    handleResult: ...,
  },
  api: {
    requestSucceeded,
    requestFailed,
  },
  // ...
}

To begin implementing the basics, let's start with rawToEnvelopes which are functions that convert from the cloud data type to the Booster one.

In the case of the local provider, the data will arrive as it is, as we are in charge of handling it with express, so the implementation is pretty simple:

export function rawEventsToEnvelopes(rawEvents: Array<unknown>): Array<EventEnvelope> {
  return rawEvents as Array<EventEnvelope>
}

export function rawReadModelEventsToEnvelopes(rawEvents: Array<unknown>): Array<ReadModelEnvelope> {
  return rawEvents as Array<ReadModelEnvelope>
}

In the case of the rawToEnvelope function for the graphQL field, we will have to get some more information from the request, like a request ID, a connection ID, or the event type, which will come in the request, to simplify things, let's ignore them:

export async function rawGraphQLRequestToEnvelope(
  request: express.Request
): Promise<GraphQLRequestEnvelope | GraphQLRequestEnvelopeError> {
  return {
    requestID: UUID.generate(),  // UUID.generate() provided by Booster
    eventType: 'MESSAGE',
    connectionID: undefined,
    value: request.body,
  }
}

With these functions implemented, we already have our endpoints connected to Booster, now we just have to teach it how to store/retrieve data!

Creating a local database

Given that we'll be using NeDB to store our Booster app data, we will need to initialize it first. We can do it in the same file as the Provider implementation:

import * as DataStore from 'nedb'
import { ReadModelEnvelope, EventEnvelope } from '@boostercloud/framework-types'

const events: DataStore<EventEnvelope> = new DataStore('events.json')
const readModels: DataStore<ReadModelEnvelope> = new DataStore('read_models.json')

NeDB uses a file for each "table", so we create two DataStores to interact with.

Now we have to implement the methods that the providers require, for example store:

async function storeEvent(event: EventEnvelope): Promise<void> {
  return new Promise((resolve, reject) => {
    events.insert(event, (err) => {
      err ? reject(err) : resolve()
    })
  })
}

async function storeReadModel(readModel: ReadModelEnvelope): Promise<void> {
  return new Promise((resolve, reject) => {
    readModels.insert(readModel, (err) => {
      err ? reject(err) : resolve()
    })
  })
}

Sadly, NeDB doesn't provide a Promise based API, and doesn't play well with promisify, so we have to wrap it manually. The implementation is pretty straightforward.

The rest of the methods are a matter of implementing the proper queries, for example:

async function readEntityLatestSnapshot(
  entityID: UUID, 
  entityTypeName: string
): Promise<EventEnvelope> {
  const queryPromise = new Promise((resolve, reject) =>
    this.events
      .find({ entityID, entityTypeName, kind: 'snapshot' })
      .sort({ createdAt: -1 }) // Sort in descending order
      .exec((err, docs) => {
        if (err) reject(err)
        else resolve(docs)
      })
  )
}

There are some other methods that can be a bit confusing, but they also act as interaction at some point, like managing HTTP responses:

async function requestSucceeded(body?: any): Promise<APIResult> {
  return {
    status: 'success',
    result: body,
  }
}

async function requestFailed(error: Error): Promise<APIResult> {
  const statusCode = httpStatusCodeFor(error)
  return {
    status: 'failure',
    code: statusCode,
    title: toClassTitle(error),
    reason: error.message,
  }
}

After implementing all the methods of the Provider, we are pretty much done with the first package, and we can hop onto the infrastructure train 🚂

Wiring everything up with an Express server

In the same case as the Provider , your Infrastructure object must conform to an interface, which in our case is a start method that initializes everything. Here we will create an express server and wire it into Booster, by calling the functions that the framework core provides.

Let's begin by initializing the express server:

export const Infrastructure = {
  start: (config: BoosterConfig, port: number): void => {
    const expressServer = express()
    const router = express.Router()
    const userProject: UserApp = require(path.join(process.cwd(), 'dist', 'index.js'))
    router.use('/graphql', graphQLRouter(userProject))
    expressServer.use(express.json())
    expressServer.use(router)
    expressServer.listen(port)
  },
}

Here we are importing user's app, in order to gain access to all the public Booster functions (typed in the UserApp type).

You can see that the only endpoint at the moment is /graphql, and that's what we are gonna configure now:

function graphQLRouter(userApp: UserApp) {
  const router = express.Router()
  this.router.post('/', async (req, res) => {
    const response = await userApp.boosterServeGraphQL(req)  // entry point
    res.status(200).json(response.result)
  })
}

And that's it, we only have to call boosterServeGraphQL on the user's app.

Because we already provided all the required methods in the Provider package, Booster has access to all the infrastructure capabilities, and it will use all of them as they need to be, no need to write more code! 🚀

That's all folks!

I'm gonna keep working on improving the local provider, like adding nice logging messages, tests, and more goodies 😉, but you can always check out the complete code in the following folders of the Booster repo:

• packages/framework-provider-local
• packages/framework-provider-local-infrastructure

Thanks for reading all of this! Have an awesome day,

Nick