DEV Community: André Eriksson

Encore.ts: a new type of framework

André Eriksson — Thu, 26 Sep 2024 18:01:00 +0000

We recently published performance benchmarks showing how Encore.ts achieves 9x request throughput compared to Express.js, and 2x compared to Fastify.

But it seems that almost every new JavaScript framework these days claims that it's "lightning fast" or "lightweight", with "clean syntax" or a "minimalistic API". Sound familiar?

Understandably, the cynicism and scepticism is relatable, and I can't really blame you if you're thinking "do we really need yet another JavaScript backend framework?".

Still, I'd like to give it a go to convince you otherwise, by explaining how Encore.ts is different. Why it isn't just another backend framework, and why this time it actually might be worth paying a bit closer attention.

Local Development Dashboard

Encore.ts focuses heavily on providing an incredible developer experience, and takes this much further than just providing clean syntax or a tiny bundle size.

When you start your Encore.ts backend you'll be greeted by Encore's Local Development Dashboard.

This dashboard provides easy access to everything you need to rapidly develop and debug your application. Check it out:

Request Tracing

Encore provides a live log of all requests, complete with a full trace of exactly what happened. This includes things like API calls, database queries, Pub/Sub messages, logging, and of course the complete request and response payloads.

API Documentation

The Local Development Dashboard provides automatic API documentation for all your API endpoints.

This is powered by Encore.ts's static analysis, which directly processes your TypeScript type definitions. As a result the API documentation is always up to date, and always accurate.

Never again worry about keeping the OpenAPI spec up-to-date.

API Explorer

Encore.ts also comes with an API Explorer that makes calling your API endpoints extremely simple. It's similar to tools like Postman, but much more powerful.

Because Encore.ts understands exactly what your API looks like it is pre-populated with all of your API endpoints and the exact fields they expect, including things like HTTP headers and query parameters.

And like Postman you can save frequently used requests, and easily share them with your team.

Architecture Diagrams

Encore.ts is designed from the ground up to support building complex, distributed systems. It supports building not just a single backend service, but an entire network of services that communicate with each other.

For more complex systems like this, the Local Development Dashboard provides a built-in architecture diagram that represents your entire system, and which services depend on each other.

It even includes infrastructure resources, like databases and Pub/Sub topics and subscriptions. More on that below.

Integrating infrastructure in application code makes development AND deployment simpler

Typical backend frameworks focus on making it easy to define API endpoints. But modern, cloud-native backends require so much more than that.

Modern backend applications make use of cloud infrastructure resources like databases, Pub/Sub topics, Cron Jobs, secrets, and so on.

Once you start breaking down your backend into multiple independent services, using Pub/Sub for event-driven communication is a must-have to ensure your system stays in sync and can scale horizontally.

It's high time for backend frameworks to step up their game and realize this is a core part of building a backend application. Encore.ts does just that, by providing built-in support for using such infrastructure resources directly in the framework.

For example, to define a Pub/Sub topic is just a few lines of code:

import { Topic } from "encore.dev/pubsub";

interface UserSignupEvent {
  userID: string;
}

export const userSignups = new Topic<UserSignupEvent>(
  "user-signup", {deliveryGuarantee: "at-least-once"});

Then just start publishing messages: await userSignups.publish({userID: "123"}).

Defining a Pub/Sub subscription is just as easy:

import { Subscription } from "encore.dev/pubsub";

const _ = new Subscription(userSignups, "notify-crm-system", {
  handler: async (event) => {
    // Notify CRM system of new signup.
  },
});

The best part of all of this is that Encore.ts's static analysis understands this, and automatically creates and configures the Pub/Sub topic when you run your application.

And when you deploy to the cloud you can easily wire up the topic to a real Pub/Sub topic in your cloud provider. Encore.ts comes with built-in integrations with AWS and Google Cloud. If you decide to use the Encore Platform it can even automatically provision the necessary infrastructure for you, directly into your own cloud account.

Benefits over Terraform/AWS CDK

If you're familiar with the cloud you've probably heard of (or used) tools like Terraform, AWS CDK or Pulumi. They're tools that allow you to describe the infrastructure you need using code, and then automatically provision that infrastructure.

Compared to clicking around in the cloud provider console this offers several benefits, like reproducibility and version control. But it also has several drawbacks:

Local development: The infrastructure setup is entirely specific to the cloud provider, making anything but the simplest setup difficult to run locally.
Error prone: Differences between local development and production can cause bugs that are difficult to catch before they cause production issues.
Cloud Provider-specific: Since code is entirely specific to your cloud provider, you end up being quite locked-in to that provider.
Production-centric: Since you tailor the infrastructure setup heavily towards your production setup, it's very difficult to have a different setup for testing or staging environments. This causes those environments to be very expensive as they mirror production unnecessarily closely. And any divergence from that causes additional complexity, leading to bugs and incidents.

Encore.ts solves all these problems by defining the infrastructure resources directly in code, in a cloud provider-agnostic way. Encore.ts provides built-in adapters for major cloud providers, so you can easily switch between them while preserving the application semantics.

And last but not least, Encore automatically sets up all the necessary infrastructure locally, so you can easily develop and test your application locally without having to write a bunch of mocks or docker-compose manifests.

TypeScript-native API validation

If you're familiar with TypeScript you might know that it needs to be transpiled into JavaScript to actually run it. This involves stripping out all the type information. As a result, all those nice type annotations you've added to the code have no effect at runtime.

This isn't normally a problem, except when it comes to user input. You might write a type-safe API endpoint definition that expects two fields to be set, and the code looks flawless. But at runtime there's nothing stopping a user from sending a request with only one of those fields set, or using the wrong field name, or sending the wrong type (a string instead of a number, for example).

This has led to a bunch of libraries that allow you to define the API schema in a way that can be used at runtime to validate the request.
Perhaps the most popular of these is Zod, but there are many others. This is great, but the syntax is much more verbose and noisy compared to the clean, concise TypeScript type syntax.

Encore.ts improves on this by using the natural TypeScript types directly. When Encore.ts compiles your application it parses the TypeScript types and generates a JSON schema from them, which it then uses to automatically validate incoming requests. As a result, you get the best of both worlds: the clean, concise TypeScript syntax, and runtime schema validation.

Encore.ts also makes it easy to define API endpoints that combine data from different sources: some fields from the request body, others from the query parameters, and yet others from the HTTP headers. It looks like this:

interface Request {
  // Maximum number of items to return. Parsed from the query parameters.
  limit?: Query<number>;

  // Custom header that must be set. Parsed from the HTTP headers.
  myHeader: Header<"X-My-Header">;

  // Regular field. Parsed from the request body.
  type: "sprocket" | "widget";
}

export const myEndpoint = api<Request, Response>(
  { expose: true, method: "POST", path: "/my/endpoint" },
  async (req) => {
    // Implementation...
  }
);

Just take a look at these two examples, comparing Zod to Encore.ts:

Zod

const headersSchema = z.object({
  "x-foo": z.string(),
});

const queryStringSchema = z.object({
  name: z.string().optional(),
  excitement: z.number().optional(),
});

const bodySchema = z.object({
  someKey: z.string().optional(),
  someOtherKey: z.number().optional(),
  requiredKey: z.array(z.number()),
  nullableKey: z.number().nullable().optional(),
  multipleTypesKey: z.union([z.boolean(), z.number()]).optional(),
  enumKey: z.enum(["Alice", "Bob"]).optional(),
});

Encore.ts

interface Schema {
  foo: Header<"x-foo">;
  name?: Query<string>;
  excitement?: Query<number>;
  someKey?: string;
  someOtherKey?: number;
  requiredKey: number[];
  nullableKey?: number | null;
  multipleTypesKey?: boolean | number;
  enumKey?: "Alice" | "Bob";
}

I know which one I'd rather write. And read. And maintain.

Try it today

Hopefully this gives you a good idea of what Encore.ts is all about, and that it's more than "just another backend framework". It's Open Source, so give it a try and let us know what you think. You can find us on Discord if you have any questions or feedback.

Fastify v5 vs v4 — vs Encore.ts

André Eriksson — Fri, 20 Sep 2024 13:54:25 +0000

Fastify, one of the most popular web frameworks for Node, just announced a major new release, version 5.0.

We were initially very excited to see if any cool new features or improvements were added, in case there was something we could take inspiration from in Encore.ts.

Unfortunately, after taking it for a spin, we felt it was quite underwhelming: the new release is extremely light on new features, but extremely heavy on breaking changes.

Let's take a look at what's new, and whether it's worth upgrading.

Fastify v5 comes with lots of breaking changes

Don't believe me? Check out the migration guide. It clocks in at almost 2000 words! It lists no less than 20 breaking changes. That's a lot of breakage to consider.

As far as we can tell, most of the breaking changes amount to "cleaning things up" in the code base.

It removes a lot of convenient shorthands for defining JSON Schemas, making schema validation more verbose and less readable.

It also removes a lot of deprecated functionality. Sometimes this is a good thing, if the functionality carries a significant maintenance burden. But in this case, a lot of the removed functionality is essentially a one-liner.

For example, reply.getResponseTime() is removed in favor of reply.elapsedTime. Maybe elapsedTime is a better name, but the cost of maintaining getResponseTime, with its one-line implementation, is zero.

Still, it's easy enough to grep a codebase for getResponseTime and fix it. Other breaking changes are more difficult.

Consider for example fastify.hasRoute() still exists, with exactly the same signature, but the behavior has changed in a subtle way. It used to support passing in a request path to evaluate whether an incoming request matches a route or not.

In Fastify v5 the method instead only supports matching the exact string given when registering the route. Migrating over to this is much more difficult, as it's not always clear whether you need to change your code or not.

Fastify v5 has very few, minor improvements

At this point I hear you say, "surely all the breaking changes were done for good reason!". Unfortunately the documentation is surprisingly sparse when it comes to the benefits of upgrading to v5.

The only new feature listed is support for the Diagnostic Channel API.
Is that worth upgrading for? For most people, probably not.

The other improvement is a modest performance boost.

From our benchmarking, Fastify v5 is about 5-10% faster than v4, depending on configuration settings and whether you use schema validation. That's nothing to sneeze at! But it's not a game changer either.

Is upgrading worth the hassle? Probably not.

It's disappointing to see a major new release of a popular framework that is so light on new features, and so heavy on breaking changes.
Particularly when the breaking changes are seemingly done to clean up internals rather than improving the user experience.

Our belief is that migrating from Fastify v4 to Fastify v5 is probably not worth it for most people.

If you care about performance or reliability, you should probably check out Encore.ts instead of spending time migrating to Fastify v5.

It's about twice as fast, and provides a lot of other benefits as well, like TypeScript-native schema validation, automatic tracing (no need to manually instrument your code with the Diagnostic Channel API!), and more.

What's more, Encore.ts takes backwards compatibility extremely seriously, and follows the Linux Kernel model when it comes to breaking changes: don't.
And Encore.ts is of course also open source.

Designing an algorithmic cloud infrastructure provisioning system

André Eriksson — Mon, 24 Oct 2022 13:05:10 +0000

Provisioning and configuring cloud infrastructure is tough. To begin with, each cloud provider offers a myriad of different products. Many of them do almost the same thing, only in subtly different ways.

For example, Google Cloud Platform lists seven official products for hosting applications, and the corresponding high-level chart compares 13 technical properties. But that's obviously just the tip of the iceberg. Combine this with a ton of database/data storage options and network configurations, and you're likely to suffer a small nervous breakdown.

The feature sprawl across the major cloud providers is hard to keep up with and the resulting combinatorial explosion can easily lead to choice paralysis. Yet what is perhaps less appreciated is the incredible minutiae you have to deal with just to deploy even the simplest of backend applications. The creative drain this represents is beastly, a mighty dragon indeed.

Since everyone loves being made aware of an annoying noise they had subconsciously blocked out, it's our great honor and privilege to bring the sheer magnitude of cloud clutter to your attention with this article. You're welcome.

But first, a cloud native digression

Our day job at Encore is building a backend development engine designed to help you create cloud native backend applications. Before you start muttering, "What does cloud native even mean?", let's just say it means applications that are designed from the ground up to leverage cloud infrastructure to its fullest. Such applications are, to a large extent, composed of cloud infrastructure services. We call those services cloud primitives.

Cloud primitives are the core building blocks of your application, such as services, databases, caches, PubSub topics, and scheduled jobs. Fully leveraging such primitives makes for incredibly scalable and reliable systems.

Unfortunately, going all in on cloud primitives has been a time-tested way to get absolutely nothing useful accomplished, at least any time soon. This is partly because it increases the complexity of your application, and partly because the developer experience is truly awful.

Models to train dragons

Freeing you from cloud complexity and poor developer experience is what we are working on at Encore. Through static analysis of your code, combined with service insights, we are able to infer your infrastructure requirements and automatically provision exactly what you need. You don't really have to do anything beyond writing your application business logic. Just sit back, relax, and watch the progress bar.

However, sometimes you have specific requirements that Encore is unable to infer automatically. You might for example want to use an existing database or SQL server. Or you may want to deploy your stuff to your own pre-existing Kubernetes cluster.

To support these use cases, and many more things, we are currently revising our infrastructure planning algorithm to allow for user-defined constraints, and many of the examples of cloud infrastructure insanity we're about to show you stemmed from this work.

So how does it work, you ask? The infrastructure computation is evaluated in three layers, starting with a very abstract representation of the infrastructure requirements and gradually adding more and more details.

We start with the Needs model, followed by the Entity model, and finally we reach the Resource model, which is the point where subjects spontaneously ~~gouge their eyes out~~ laugh uncontrollably as a coping mechanism.

The Need Model

Pushing your code to Encore kicks off our CI/CD pipeline. Aside from compiling and testing your app, the build step also runs the static analysis to extract those infrastructure needs.

From a high-level perspective, this takes the form of a graph containing all your backend primitives and the links between them. You can see a simple example on the right.

The planning algorithms then calculate the difference between the currently deployed version of the application and the recently pushed changes. We call the output of this comparison the Need Model. The Need Model is flat, cloud agnostic, and expresses the environment-independent core needs of your application.

The Entity Model

Any new needs identified are added to a high-level model we call the Entity Model. The Entity model is a hierarchical graph of your application. It contains abstract representations of cloud-specific products and their relations. For each cloud, we've modeled supported cloud products as "providers" of entities that are capable of fulfilling your (every) need.

For example, a Cloud Run Container is a provider for Services, whereas a Cloud SQL Server is a provider for databases. The providers are wrapped in parent containers that encapsulate the capabilities of their children.

Each top-level node of the Entity Model is called a Root Entity. A Root Entity can for example be a GCP organization, an AWS account, or your existing external database which fulfills the need for a specific database.

Each provider level can use the Need Model, metrics, and knowledge of the current infrastructure topology to make informed proposals of which provider/product is best suited for your needs. If you have special requirements or ~~crazy ideas~~ preferences, you can add constraints by specifying your own infrastructure preferences.

You can for example instruct the planner to deploy a specific service to a Kubernetes cluster or provide a connection string to satisfy your particular database need.

The Resource Model

The Entity Model is then further refined into what we call the Resource Model. The Resource Model is pretty much a mapping of the high-level entities to lower-level cloud provider resources. This model is what is used by our provisioning system to create and modify the cloud resources which is necessary to host your app's building blocks.

The Resource Model is computed by mapping each entity in the Entity Model to a set of resources and/or updates to shared resources. This step is a bit more complex because the resources are sometimes indirectly coupled and can in the worst case affect each other. For example, a Cloud Run Container can for example only use one Serverless VPC Access connector, which impacts how Encore configures the connection to database servers and caches.

The general idea is similar to the Entity Model though: generate a list of resources that satisfy the needs of a bunch of higher-level entities.

This is fine

In the example depicted above, we use a boring app that had a very neat mapping between the Encore needs and the resulting cloud resources. Alas, this is unfortunately not always the case. Relatively small tweaks to the Entity Model quickly balloon into extremely complex Resource Models.

To illustrate this, let's move one of our Encore Services to run on a separate Cloud Run Container. Given this new constraint, the infrastructure planner happily computes a proposal for a new Entity and Resource Model which satisfies the Need Model.

Well, that escalated quickly, really that got out of hand fast! Where did all the colorful boxes come from? Well, since the services now run on separate Cloud Run Containers, we need to configure a bunch of additional IAM bindings (plus associated roles and accounts).

Now let's say we for some obscure reason want to run one of our databases on a separate Cloud SQL Server. To make things even more interesting we may as well shove it into a separate GCP Project and ditch the Cloud SQL Proxy. After crunching your demands, the planner generates yet another infrastructure proposal, and this time it's starting to look rather unwieldy.

We're moving the database to a separate GCP Project, and since we don't use Cloud SQL Proxy, we also need to configure the Cloud Run Container to use a Serverless VPC Access connector. We also configure a Shared VPC to virtually host both databases in the same VPC. On top of this, we need to dish out some more IAM bindings, configure Global Address, etc.

This is still an extremely simple app (albeit some would argue it has silly constraints), but the infrastructure configuration has already started to balloon out of proportion.

Fortunately for those of you using Encore, you don't really need to care. Encore does all the heavy lifting for you and will soon happily cater to all (or at least most) of your obscure, exotic, and slightly lunatic needs. Unfortunately for those of you working on Encore, this is your life now.

What's next

We're currently in the midst of fine-tuning the next generation of our planning algorithm and will eventually migrate our existing, more naive models to the shiny, handsome, and all in all superior replacement. This will yield a plethora of exciting new opportunities!

To begin with, we're working on extending our visual architecture tool, Encore Flow, to also give you a fabulous overview of your provisioned cloud infrastructure. We'll also be able to better visualize changes to your infrastructure, and eventually you'll be able to effortlessly plan and migrate services, databases, or whole apps between whatever products you fancy.

And if you don't want to bother, Encore will automatically provision what's best for you, taking both cost and performance into account.

We'll tame the complexity dragon and disperse the cloud clutter, creating a brighter and sunnier future for us all.

If you're curious, why not try Encore today, using the free built-in developer cloud?

Advanced Go Concurrency

André Eriksson — Thu, 02 Jun 2022 15:33:34 +0000

Learn how to use Go's singleflight and errgroup packages, and other important design patterns for concurrency, with real-world examples.

If you've used Go for a while you're probably aware of some of the basic Go concurrency primitives:

The go keyword for spawning goroutines
Channels, for communicating between goroutines
The context package for propagating cancellation
The sync and sync/atomic packages for lower-level primitives such as mutexes and atomic memory access

These language features and packages combine to provide a very rich set of tools for building concurrent applications.
What you might not have discovered yet is a set of higher-level concurrency primitives available in the "extended standard library" available at golang.org/x/sync.
We'll be taking a look at these in this article.

Package singleflight

As the package documentation states, this package provides a duplicate function call suppression mechanism.

This package is extremely useful for cases where you are doing something computationally expensive (or just slow, like network access) in response to user activity.
For example, let's say you have a database with weather information per city and you want to expose this as an API.
In some cases you might have multiple users ask for the weather for the same city at the same time.

When that happens, wouldn't it be great if you could just query the database, and then share the result to all the waiting requests?
That's exactly what the singleflight package does!

To use it, create a singleflight.Group somewhere. It needs to be shared across all the requests to work correctly.
Then wrap the slow or expensive operation in a call to group.Do(key, fn). Multiple concurrent requests for the same key
will only call fn once, and the result will be returned to all callers once fn returns.

Here's how it looks in practice:

package weather

type Info struct {
    TempC, TempF int // temperature in Celsius and Farenheit
    Conditions string // "sunny", "snowing", etc
}

var group singleflight.Group

func City(city string) (*Info, error) {
    results, err, _ := group.Do(city, func() (interface{}, error) {
        info, err := fetchWeatherFromDB(city) // slow operation
        return info, err
    })
    if err != nil {
        return nil, fmt.Errorf("weather.City %s: %w", city, err)
    }
    return results.(*Info), nil
}

Note that the closure we pass to group.Do must return (interface{}, error) to work with the Go type system.
The third return value from group.Do, which is ignored in the example above, indicates whether the result was
shared between multiple callers or not.

Package errgroup

Another invaluable package is the errgroup package.
It is best described as a sync.WaitGroup but where the tasks return errors that are propagated back to the waiter.

This package is useful when you have multiple operations that you want to wait for, but you also want to determine
if they all completed successfully.
For example, to build on the weather example from above, let's say you want to lookup the weather for multiple cities
at once, and fail if any of the lookups fails.

Start by defining an errgroup.Group, and use the group.Go(fn func() error) method for each city.
This method spawns a goroutine to run the task. When you've spawned all the tasks you want, use
group.Wait() to wait for them to complete. Note that this method returns an error, unlike sync.WaitGroup's equivalent.
The error is nil if and only if all the tasks returned a nil error.

In practice it looks like this:

func Cities(cities ...string) ([]*Info, error) {
    var g errgroup.Group
    var mu sync.Mutex
    res := make([]*Info, len(cities)) // res[i] corresponds to cities[i]

    for i, city := range cities {
        i, city := i, city // create locals for closure below
        g.Go(func() error {
            info, err := City(city)
            mu.Lock()
            res[i] = info
            mu.Unlock()
            return err
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    }
    return res, nil
}

Here we are allocating an slice of results so that each goroutine can write to its own index.
While the above code is safe even without the mu mutex, since each goroutine is writing to its own entry in the slice, we use one anyway in case the code is changed over time.

Bounded concurrency

The code above will lookup weather information for all the given cities concurrently.
That's fine when the number of cities is small, but can cause performance issues if the number of cities is massive.
In those cases it's useful to introduce bounded concurrency.

Go makes it really easy to create bounded concurrency with the use of semaphores. A semaphore is a concurrency primitive that you might have come across if you studied Computer Science, but if not, don't worry. You can use semaphores for several purposes, but we're just going to use them to keep track of how many tasks are running, and to block until there is room for another task to start.

In Go we can accomplish this through a clever use of channels! If we want to allow up to 10 tasks to run at once,
we create a channel with space for 10 items: semaphore := make(chan struct{}, 10). You can picture this as a pipe that can fit 10 balls.

To start a new task, blocking if too many tasks are already running, we simply attempt to send a value on the channel: semaphore <- struct{}{}. This is analogous to trying to push another ball into the pipe. If the pipe is full, it waits until there is room.

When a task completes, mark it as such by taking a value out of the channel: <-semaphore. This is analogous to pulling a ball out at the other end of the pipe, which leaves room for another ball to be pushed in (another task started).

And that's it! Our modified Cities looks like this:

func Cities(cities ...string) ([]*Info, error) {
    var g errgroup.Group
    var mu sync.Mutex
    res := make([]*Info, len(cities)) // res[i] corresponds to cities[i]
    sem := make(chan struct{}, 10)
    for i, city := range cities {
        i, city := i, city // create locals for closure below
        sem <- struct{}{}
        g.Go(func() error {
            info, err := City(city)
            mu.Lock()
            res[i] = info
            mu.Unlock()
            <-sem
            return err
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    }
    return res, nil
}

Weighted bounded concurrency

And finally, sometimes you want bounded concurrency, but not all tasks are equally expensive.
In that case the amount of resources we'll consume will vary drastically depending on the distribution
of cheap and expensive tasks and how they happen to start.

A better solution for this use case is to use weighted bounded concurrency.
How this works is simple: instead of reasoning about the number of tasks we want to run concurrently,
we come up with a "cost" for every task and acquire and release that cost from a semaphore.

We can't model this with channels any longer since we need the whole cost acquired and released at once.
Fortunately the "extended standard library" comes to our rescue once more!
The golang.org/x/sync/sempahore
package provides a weighted semaphore implementation exactly for this purpose.

The sem <- struct{}{} operation is called "Acquire" and the <-sem operation is called "Release".
You will note that the semaphore.Acquire method returns an error; that is because it can be used
with the context package to abort the operation early. For the purpose of this example we will ignore it.

The weather lookup example is realistically too simple to warrant a weighted semaphore,
but for the sake of simplicity let's pretend the cost varies with the length of the city name.
Then we arrive at the following:

func Cities(cities ...string) ([]*Info, error) {
    ctx := context.TODO() // replace with a real context
    var g errgroup.Group
    var mu sync.Mutex
    res := make([]*Info, len(cities)) // res[i] corresponds to cities[i]
    sem := semaphore.NewWeighted(100) // 100 chars processed concurrently
    for i, city := range cities {
        i, city := i, city // create locals for closure below
        cost := int64(len(city))
        if err := sem.Acquire(ctx, cost); err != nil {
            break
        }
        g.Go(func() error {
            info, err := City(city)
            mu.Lock()
            res[i] = info
            mu.Unlock()
            sem.Release(cost)
            return err
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    } else if err := ctx.Err(); err != nil {
        return nil, err
    }
    return res, nil
}

Conclusion

The above examples show how easy it is to add concurrency to a Go program, and then fine-tune it based on your needs.

Feedback, corrections and suggestions on how this article can be improved are very welcome! Please reach out to me on Twitter.

Catch you in the cloud,
André

This article was originally published on the Encore Blog on 18-02-2020.

Putting the fun back into software development with Encore

André Eriksson — Mon, 30 May 2022 13:33:43 +0000

How my frustrations with building for the cloud led me to create the backend development engine I believe will transform developers' lives, and announcing our company has raised $3 million in seed funding.

I've always loved programming for how creative you can be. There's a deep joy and satisfaction in playing around with ideas and building things that delight people. That's what got me into it as a young boy. I taught myself to code at a very early age, building interactive websites and simple games. When I started playing World of Warcraft, I began building user interface modifications to improve the game for myself and my fellow players. By the time I was thirteen the mods I'd built were used by millions of people. I ended up collaborating with Blizzard – the studio behind the game – and many of the innovations I developed even made their way into the official game, which was a truly fulfilling experience.

A few years later I got my first real job as a developer and joined Spotify. This meant I had to learn anew how to build backend applications. I was used to simple setups with one or two servers, but now we were building massively scalable systems for The Cloud™. The complexity of cloud infrastructure started to dominate my work. Instead of being all about building a great product we were spending all our time provisioning, orchestrating and managing cloud services. It was all very mundane and repetitive, and a hundred times slower and more complicated compared to the days of the single server. Nothing like the creative process I used to love growing up.

At one point the complexities of cloud infrastructure resulted in what felt like several months of hell. The payments systems I was working on at Spotify started having a nasty intermittent performance problem, and were constantly at the verge of falling over. My team kept being woken up by alerts almost every night, and we were doing everything in our power to keep everything up and running so that the company could accept payments. The experience was so stressful I ended up hospitalised with gastritis. I felt strongly that it would never have happened were it not for the complex nature of large distributed systems.

I realised I had to do something for all the developers out there like me who find building for the cloud incredibly frustrating and uncreative. There are loads of tools for improving small parts of the process. But they only have a small impact on the job overall. I wanted to find a way to massively reduce the time developers spend managing the complexities of the cloud; something that would free us to get on with the fun and creative part of actually building things.

I realised the reason we spend so much time configuring cloud services is because the tools we rely on have no idea what we're trying to do. So it's up to the developer to do almost all of the work. I had the idea that in order to make a real impact, I needed to build a tool that does understand that. That's the key insight behind Encore. It's a backend development engine for building scalable cloud-based software, that has the same mental model as the developer for how things work and how they're connected.

In 2017 I started by sketching the idea out to see if there was something in it. Once I realized it could work it took over my life. It felt so exciting and important. For years I'd get home from work at 6pm, start hacking away and suddenly it would be 4am.

By early 2020 Encore was ready for early adopters so I left Spotify to work on it full-time. We open-sourced it last year and the reception blew us away. We went from having a handful of users to thousands of developers trying it out.

Today I'm excited to announce we're releasing Encore v1.0. It's my belief that Encore will transform developers' lives. Our work will become far more joyful and creative. Rather than it being 80% about configuring tools and services that have been reconfigured thousands of times before, we'll spend our days building new products that have a real impact. And that could have huge benefits for society. Imagine the exciting innovations if the world's 25+ million developers are freed up to be five times more productive!

I also hope that Encore will open things up to a more diverse range of folks than the usual bearded men. You won't need to have a deep understanding of the nuances of cloud services to develop world-changing software. Instead it will be something people can do who are expert and passionate about the real-world issues they're actually trying to solve.

I'm happy to share that we've raised $3 million in seed funding, led by Crane Venture Partners, to help us accomplish these lofty goals. With this funding round we're able to both grow our core team, and invest much more in supporting our incredible developer community. We're now able to accelerate our efforts to provide a radically improved experience for backend developers everywhere.

Catch you in the cloud,
André

This was originally published on the Encore blog 19/04/22.

Distributed tracing made simple

André Eriksson — Wed, 05 May 2021 09:23:03 +0000

Everything's on fire. That much is clear. But what's the root cause? Nobody's got a clue.

When we build distributed systems we're used to fumbling in the dark, desperately searching for the problem. Sometimes it's due to not having enough data. But paradoxically it's often due to having too much data.

The real challenge lies in making sense of it all. Unfortunately, general-purpose observability tools often have no idea how your application works. The end result is that distributed tracing, an incredibly powerful idea, largely under-delivers and is mostly useful for tracking down performance issues. That's good, but it could be so much better.

How tracing (normally) works

Tracing builds on a few pretty simple concepts.

Spans are the operations we track, with a start and an end, and references between them. They're things like a running request, or a database query. Spans can reference each other, for example to communicate that one request made an API call which results in another request. In this way we can build up a tree (or in practice a directed acyclic graph, a "DAG") of all spans.

Each span contains tags, which are key-value pairs, and events which are timestamped things that happened within the span, such as emitting log messages.

Not too bad so far, right? The devil's in the details. Because this data model is so generic, it's difficult to make sense of exactly what a span represents. Is it an API call? Is it a database query? They don't know, so in practice they render everything as one long list of spans. Not too helpful.

The other challenge is in the instrumentation itself. In order to build up this DAG of spans, we need to pass along IDs to tie everything together. This data must be carefully threaded through our whole application. How to do that in practice differs from language to language, but setting it all up is quite a nuisance and if you miss some place you'll silently get lower quality data.

In practice it ends up being lots of error-prone, boilerplate-y code. And the end result is less useful than we would like. So we figured, since Encore is all about deeply understanding how your application works using static analysis, let's try to build a better distributed tracing solution that leverages that understanding. We wanted to offer much better insights, and do it in a way that works out of the box without having to do a thing. A tall order, but we believe we've succeeded.

How tracing works with Encore

The way you write Encore applications has been carefully designed to facilitate static analysis. When you do, Encore can easily understand, among other things:

The services and APIs you have defined, and their precise request/response schemas
Precisely when and where you make an API call to another service, output structured log messages, execute database queries, and more
What infrastructure each service requires to run
Your database(s) schemas
And lots more

Using nothing but static code analysis we build up a detailed graph of your application as a distributed system.

So now, when Encore compiles your application, it queries this graph to automatically add instrumentation in the right places. Instead of having to manually propagate trace IDs, Encore comes with a slightly modified Go runtime that automatically propagates trace information for you. And since it understands exactly what every operation is, and not just a generic "span", Encore can add incredibly rich information that is unique to each type of event. Among other things:

Database queries, transactions, commits/rollbacks
API calls, complete with input/output data
Outgoing HTTP requests, along with precise timings for things like DNS resolution, TLS handshakes, and more

We also break things down further. Encore captures goroutine timings, allowing you to see at a glance when an API call runs multiple things in parallel. Including exactly what each goroutine did: database queries, API calls, log messages, and more.

Stack traces

Finally, Encore now captures complete stack traces for everything. But wait, I hear you say, won't that be incredibly slow? Nope.

Encore only collects the program counter for each stack frame, and then uses delta encoding and varint encoding. Then when it comes to viewing a trace, we map the program counters back to the files, lines, and function names using the program's symbol table stored in the binary, that we conveniently kept around from when we did our original static analysis pass.

The end result is collecting a stack trace in as little as 300ns, and each stack frame often takes 1-2 bytes to encode. This remarkable result is only possible by the tight integration between Encore and the Go runtime. It's fast enough that we can capture stack traces for pretty much everything. Database queries, API calls, log messages, and more.

Making sense of it all

Encore is all about enabling simplicity through understanding your application, and we wanted the tracing experience to reflect that. So instead of doing all this instrumentation by hand, and presenting a trace as a long list of spans, we wanted it to "just work" with no additional effort on your part, and with a UI that reflects the structure of your app.

Stack traces are great and all, but when you're building a distributed system they're often insufficient to get a complete picture of what's going on. That's what's so beautiful about our approach: the stack traces are themselves embedded in the distributed trace data.

In the end, the traces are much richer in information and at the same time faster to collect. And by understanding exactly what each span represents, we can visualize traces in a much more understandable way. And the stack traces we capture? We can correlate them across the whole system, so you can understand the exact code path across all your services that led to a particular event.

Tracing Everywhere

The final piece of the puzzle was the realization that such a powerful experience is useful for many more things than just performance analysis. That's why we made it available everywhere, and perhaps most critically, for local development.

That way, the next time you're building a new feature, and you find a bug, you can immediately open up the trace view and see exactly where the problem is. Same goes for production, your test environment, and for all your Preview Environments that Encore automatically spins up for each Pull Request you open. And that time when everything was on fire and you couldn't figure out why? It sure would be nice to immediately pinpoint where the problem was.

Why, you ask? Because it's about time we developers got better tools.

Want to play around? This stuff's all open source.

You'll go from zero to a running backend in the cloud in less than 5 minutes. With the world's most advanced tracing integration, fully open source.

Try it now