DEV Community: Simon Johansson

Next.js vs Encore.ts — When should you NOT use Next.js for Backend?

Simon Johansson — Fri, 27 Dec 2024 09:00:00 +0000

In this article I will compare Next.js with Encore. I will show you how they differ and how they compliment each other. If you have a Next.js app and want to expand the backend functionality of that app then this article is for you!

TL;DR: Code example of using Next.js with Encore.ts: https://github.com/encoredev/examples/tree/main/ts/nextjs-starter

Video version of this article, including code examples

Encore.ts vs Next.js

Encore is an Open Source framework that aims to make it easier to build robust and type-safe backends. The framework is available for both TypeScript and Go (I will focus on Encore.ts which is the TypeScript version in this article) and it has a lot of build in tools to make the development experience smoother when building everything small to large scale backends.

Like Next, Encore is completely Open Source. Read the code on GitHub, fork it or contribute to it. When building an application you can host it wherever you want (that takes Docker images, like Digital Ocean) without any hidden costs or looming quotas. And performance wise, Encore is extremely fast compared to other Node.js and Bun frameworks, including Next.js. In this benchmark we measured requests per second on a public API endpoint.

The reason why Encore is so fast is because Encore makes use of Rust under the hood, check out this article if you are interested in learning more about that: Encore.ts — 9x faster than Express.js, 3x faster than ElysiaJS & Hono

Comparison

So, how does Encore.ts and Next.js compare to each other, well I like to describe it like this:

So, we have the traditional separation between backend and frontend. The backend part is bigger just because that is the part we are interested in. Backend here is also a spectrum, going from Feature rich to Lightweight. I know, i know… This is a bit two dimensional (literally) but keep with me.

Next.js

On this scale, I would position Next.js to include all of the frontend aspects and a small part of the backend sclae. The frontend aspects are covered, you are able to do everything from static site rendering to server side components. You can even build a single page application inside Next if you want to. Next is primarily a frontend framework and the backend functionality it offers is there to make the frontend aspects better. Next is amazing and for a lot of use cases it will be everything you need, I have used Next.js for years and continue to be impressed by it.

But… The backend aspects of Next are limited. There are some things that you just can’t do and some things that you probably shouldn't do even if you can. I am thinking of things like:

You want to have multiple frontends e.g. mobile app & web page but want a single backend.
You want to offer a public API to your consumers.
The single-threaded nature of Node.js can become a performance bottleneck (not a Next.js problem specifically but still).
Next is not great if you need long-running jobs with background processing.

Connected to the last point, APIs created in Next.js are stateless. This is not necessarily a bad thing but it can become a limitation depending on the type of application you are building.

With stateless applications your API handlers are running inside lambda functions that die when the response has been sent, so you are not really able to respond to the user and then continue to process data in the background. With a stateful application you are able to do just that. Again, this does not mean that stateless/serverless is bad but it is a limitation that Next forces you into a certain route.

Encore.ts

Encore.ts on the other hand is purely a backend framework. It’s built from the ground up to be scaleable, no matter how big you application or team becomes.

Encore.ts makes it extremely easy to set up infrastructure locally so you will always be able to start the whole application by running encore run in your terminal, no matter the size of your application.
Divide your application into multiple services which can be scaled separately
Create event-driven systems where your services community through Pub/Sub in a type-safe way
Encore.ts has a Rust runtime, so you get the developer experience of writing Typescript but a huge performance boost because of Encore’s multithreaded runtime.
Have long running tasks or CronJobs running in the background.
Inspect API calls using tracing (which comes built-in and even works for your local environment).

Using Next.js with Encore.ts

So why not use Next together with Encore? Well, you should! And with Encore you can generate type-safe request clients that can be used by your Next.js app. You can think of it sort of like tRPC client if you are familiar with that.

We love using Next with Encore, encore.dev is an example of that. We use static site generation for generating our documentation to keep it snappy and fast. But we keep the backend layer of the Next.js app really thin and fetch the markdown files from an Encore app when generating the static props. That way we can leave heavier things, like generating documentation search indexes, in the Encore app.

Code example

See a full code example of using Next.js with Encore.ts in our example repo: https://github.com/encoredev/examples/tree/main/ts/nextjs-starter

Install Encore

macOS: brew install encoredev/tap/encore
Linux: curl -L https://encore.dev/install.sh | bash
Windows: iwr https://encore.dev/install.ps1 | iex

Clone the example app

encore app create --example=ts/nextjs-starter

Generating a request client

Keep the contract between the backend and frontend in sync by regenerating the request client whenever you make a change
to an Encore endpoint.

encore gen client --output=./app/client.ts --env=local

Deployment

Self-hosting

See the self-hosting instructions for how to use encore build docker to create a Docker image and configure it.

Encore Cloud Platform

Deploy your application to a free staging environment in Encore's development cloud using git push encore (this works if you have run encore app create to create your app):

git add -A .
git commit -m 'Commit message'
git push encore

You can also open your app in the Cloud Dashboard to integrate with GitHub, or connect your AWS/GCP account, enabling Encore to automatically handle cloud deployments for you.

Next.js on Vercel

The only thing you need to do is to create a new project on Vercel and point it to your newly created GitHup repo.

Conclusion

That’s it! Thank you for reading.

Give the Encore repo a star on GitHub to follow along with the updates, it will also help us out.

Post your questions below or join the Encore Discord community.

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

#node #typescript #javascript #webdev

Building and Deploying TypeScript Microservices to Kubernetes

Marcus Kohlberg for Encore ・ Dec 17

#javascript #typescript #kubernetes #tutorial

Building simple event-driven applications with Pub/Sub

Marcus Kohlberg for Encore ・ Nov 19

#javascript #webdev #typescript #node

How to use ORMs (Prisma / Drizzle / Knex.js) in a TypeScript backend built with Encore.ts

Simon Johansson — Mon, 25 Nov 2024 13:25:30 +0000

In this post, I’ll walk you through how to work with ORMs like Prisma, Drizzle and Knex.js when working with Encore.ts.

We will look at how to create and manage Databases when running your application locally but also how to go about getting an app using databases deployed to the cloud.

Video version:

Using a Database with Encore

Encore treats databases as logical resources in your code and natively supports PostgreSQL databases. But let’s dive right into some code because that will speak for itself.

import { SQLDatabase } from "encore.dev/storage/sqldb";

const db = new SQLDatabase("url", { migrations: "./migrations" });

To create a database with Encore, import SQLDatabase from Encore’s encore.dev/storage/sqldb module and call new SQLDatabase, giving it a name and a path to our migrations folder. In the migrations folder Encore expects ordinary SQL files, which is how you define the database schema.

Encore automatically handles up migrations. Each up migration runs sequentially, expressing changes in the database schema from the previous migration. The first migration typically defines the initial table structure. For instance, the first migration file for a URL shortener service might look like this:

1_create_table.up.sql

CREATE TABLE url (
    id TEXT PRIMARY KEY,
    original_url TEXT NOT NULL
);

This is how our folder structure could look like with multiple migration files:

/my-app
├── encore.app                       // ... other top-level files
│
└── url                              // url shortener service
    ├── migrations                   // database migrations (directory)
    │   ├── 1_create_table.up.sql    // first migration file
    │   └── 2_add_field.up.sql       // second migration file
    ├── url.ts                       // url service code
    └── url.test.ts                  // tests for url service

With this code in place, Encore will automatically create the database using Docker when you run the command encore run locally.

We can now start querying and inserting data into the database. With the exec method we can write SQL code for inserting data.

import { api } from "encore.dev/api";
import { randomBytes } from "node:crypto";

interface URL {
  id: string; // short-form URL id
  url: string; // complete URL, in long form
}

interface ShortenParams {
  url: string; // the URL to shorten
}

// shorten shortens a URL.
export const shorten = api(
  { expose: true, auth: false, method: "POST", path: "/url" },
  async ({ url }: ShortenParams): Promise<URL> => {
    const id = randomBytes(6).toString("base64url");
    await db.exec`
        INSERT INTO url (id, original_url)
        VALUES (${id}, ${url})
    `;
    return { id, url };
  },
);

We can even make use of template literals when writing our SQL query, allowing easy use of placeholder parameters. Encore takes care of the escaping here so there is no risk of SQL injections.

The get endpoint takes a user ID as input. We are using the queryRow method, which returns a single row. We again use a template literal to send in the ID. If no rows are found we return an error, otherwise we get the original_url from the row.

import { api, APIError } from "encore.dev/api";

// Get retrieves the original URL for the id.
export const get = api(
  { expose: true, auth: false, method: "GET", path: "/url/:id" },
  async ({ id }: { id: string }): Promise<URL> => {
    const row = await db.queryRow`
        SELECT original_url
        FROM url
        WHERE id = ${id}
    `;
    if (!row) throw APIError.notFound("url not found");
    return { id, url: row.original_url };
  },
);

Encore CLI comes with tools for you to interact with the locally running database. You can for example run encore db shell to open a psql shell to the database.

When you start your app using encore run you get access to the Local Development Dashboard. From here you can easily call your endpoints. Each call to your application results in a trace that you can inspect to see the API requests but also the database calls.

ORM

And if you don’t want to write SQL code by hand you can use ORMs like Prisma, Drizzle, or Knex if you prefer.

An ORM stands for Object–relational mapping, what this basically means is that we are using the power of object oriented programming to define a layer on top of our database.

An ORM abstracts away the communication with the database. It can speed up development and one nice thing is that you get type-safety when working with your database.

But using an ORM is not a given and you should think about if it suites your project because it can be really hard to move away from an ORM once you have committed.

But with that said, let’s look at how to use Prisma, one of the most popular ORMs for Node.js, together with Encore.

Prisma

Let’s take a look at how our schema.prisma could look like when integrating with Encore.

generator client {
  provider = "prisma-client-js"
  binaryTargets = ["native", "debian-openssl-3.0.x"]
}

datasource db {
  provider = "postgresql"
  url      = "<paste shadow db connection string here>"

}

model User {
  id      Int      @id @default(autoincrement())
  name    String
  surname String
  age Int
}

For the datasource we need to set the provider and the url to the shadow database. Prisma requires a "shadow database" for certain operations, essentially it's a second, temporary, database that is created and deleted automatically. Encore comes with built-in support for shadow databases, and by configuring Prisma to operate on Encore's shadow database, Encore.ts and Prisma won't interfere with each other.

To get the Encore shadow URI we run the following command encore db conn-uri encore_prisma_test --shadow in the terminal. Then we paste that value into the schema file. It will look something like this: "postgresql://prisma-3wv2:shadow-csub8kpuevghlr9vvci0@127.0.0.1:9500/encore_prisma_test?sslmode=disable".

Now we are ready to create a database and a PrismaClient:

import { SQLDatabase } from "encore.dev/storage/sqldb";
import { PrismaClient } from "@prisma/client";

// Define a database named 'encore_prisma_test', using the database migrations
// in the "./prisma/migrations" folder (where prisma will generate their migrations).
// Set `source` to `prisma` to let Encore know that the migrations are generated by Prisma.
const DB = new SQLDatabase('encore_prisma_test', {
  migrations: {
    path: './prisma/migrations',
    source: 'prisma',
  },
});

// Setup prisma client with connection string
const prisma = new PrismaClient({
  datasources: {
    db: {
      url: DB.connectionString,
    },
  },
});

// Select all users
const allUsers = prisma.user.findMany();

We create a database like we did earlier, but this time we get the connectionString and pass that in when creating the PrismaClient. Now we can use the prisma object to query and update data in the database.

Run npx prisma migrate dev to create new migrations if you have made any changes to the schema.prisma file. The migration files will be automatically applied by Encore.

We also add the following NPM script that will run after installing our dependencies so that Prisma sets everything up in our CI pipeline when deploying:

{
  "scripts": {
    "postinstall": "npx prisma generate --schema=users/prisma/schema.prisma"
  },
}

Drizzle

Let’s take a look at another popular ORM, Drizzle. In the database.ts file we again create a database and pass in the connectionString to drizzle:

// database.ts
import { api } from "encore.dev/api";
import { SQLDatabase } from "encore.dev/storage/sqldb";
import { drizzle } from "drizzle-orm/node-postgres";
import { users } from "./schema";

// Create SQLDatabase instance with migrations configuration
const db = new SQLDatabase("test", {
  migrations: {
    path: "migrations",
    source: "drizzle",
  },
});

// Initialize Drizzle ORM with the connection string
const orm = drizzle(db.connectionString);

// Query all users
await orm.select().from(users);

In the Drizzle config we specify the migrations folder, our schema file and that we are using a PostgreSQL database:

// drizzle.config.ts
import 'dotenv/config';
import { defineConfig } from 'drizzle-kit';

export default defineConfig({
  out: 'migrations',
  schema: 'schema.ts',
  dialect: 'postgresql',
});

And this is what a Drizzle schema file looks like:

// schema.ts
import * as p from "drizzle-orm/pg-core";

export const users = p.pgTable("users", {
  id: p.serial().primaryKey(),
  name: p.text(),
  email: p.text().unique(),
});

Run the npx drizzle-kit generate command to generate migrations. Migrations are automatically applied when you run your Encore application, so you don’t need to run drizzle-kit migrate or any similar commands manually.

Knex

Knex is a “SQL query builder”, which is somewhere in between an ORM and writing raw SQL queries:

// site.ts
import { SQLDatabase } from "encore.dev/storage/sqldb";
import knex from "knex";

// Create SQLDatabase instance with migrations configuration
const SiteDB = new SQLDatabase("siteDB", {
  migrations: "./migrations",
});

// Initialize Knex with the database connection string
const knexDB = knex({
  client: "pg",
  connection: SiteDB.connectionString,
});

// Define the Site interface
export interface Site {
  id: number;
  url: string;
}

// Query builder for the "site" table
const Sites = () => knexDB<Site>("site");

// Example queries

// Query all sites
await Sites().select();

// Query a site by id
await Sites().where("id", id).first();

// Insert a new site
await Sites().insert({ url: params.url });

We create a database like we did earlier, and pass in the connectionString to the database when creating a new Knex client. We also need to specify that we are using PostgreSQL. Then we can create a query builder by using the knexDB object, passing in the type of the object we are storing (Site) and the table name ("site").

Now we can use the Sites object to query and update data in the database.

A query builder is easier to set up but you don’t get the same level of autocompletion in your editor as with an ORM.

Deploy

So, how do we deploy this application? Well you can build your application using encore build docker image, and you get it as a docker image you can deploy anywhere you want. You will need to supply a runtime configuration where you can specify how the application should connect to the infrastructure you are using, like databases and Pub/Sub. If you don’t feel like managing this stuff manually, you can use Encore's Cloud Platform which automates setting up the needed infrastructure in your cloud account on AWS or GCP, and it comes with built-in CI/CD so you just need to push to deploy. In cloud environments, Encore automatically injects the appropriate configuration to authenticate and connect to the database, so once the application starts up the database is ready to be used. The Platform also comes with monitoring, tracing, and automatic preview environments so you can test each pull request in a dedicated temporary environment.

Running the examples yourself

If you want to play around with this application yourself you can easily do so by installing Encore and then clone an example from our examples repo. You will need to have Docker desktop installed as that is needed to create databases locally.

Installation

macOS: brew install encoredev/tap/encore
Linux: curl -L https://encore.dev/install.sh | bash
Windows: iwr https://encore.dev/install.ps1 | iex

Cloning the ORM examples

Prisma

encore app create --example=ts/prisma

Drizzle

encore app create --example=ts/drizzle

Knex.js

encore app create --example=ts/knex

Wrapping up

⭐️ Support Encore by giving the project a star on GitHub.
If you have questions or want to share your work, join the developers hangout in Encore's community on Discord.

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

#node #typescript #javascript #webdev

Building simple event-driven applications with Pub/Sub

Marcus Kohlberg for Encore ・ Nov 19

#javascript #webdev #typescript #node

TypeScript vs Go: Choosing Your Backend Language

Marcus Kohlberg for Encore ・ Nov 8

#go #typescript #javascript #webdev

Building Type-Safe Event-Driven Applications in TypeScript using Pub/Sub, Cron Jobs, and PostgreSQL

Simon Johansson — Wed, 13 Nov 2024 12:27:14 +0000

In this post, I’ll walk you through how to create an event-driven Node.js app in TypeScript. We will start with a traditional application and then take the steps needed make the services loosely coupled by making them communicate through Pub/Sub.

We will look at how run the application locally but also how to go about getting an event-driven app deployed to the cloud.

Video version:

The application

The application we will be looking at is an uptime monitoring system. We have a list of websites to monitor and a CronJob for checking every site and seeing if they are reachable or not, our application will send a notification if any of the statuses changes. The status of a newly added site will be unknown until the CronJob has checked the status of the site. This is one of the things we want to change when making our application event-driven.

Here is the full code of the finished application: https://github.com/encoredev/examples/tree/main/ts/uptime

Architecture

Here we have two architectural digrams, on the left side is our current system and on the right is how we want it to look when we are done.

In our current state, you can see that we have four services: frontend, monitor, site and slack. The filled in arrows indicate that the monitor service are calling endpoints in both the site and slack services, the monitor service has a hard dependency on those two services. We can also see that both the monitor and the site services has databases. The monitor service is the one that has the CronJob which checks the status of each site once every hour.

So, we want to introduce a site.added topic that we can publish to from the site service whenever we add a new site. We will subscribe to the site.added topic in the monitor service and then ping the website to check the status when a new site is added.

We will also want to remove the hard dependency between the monitor and the slack service by introducing a uptime-transition topic.

Benefits

Before we continue, let’s talk about why it’s a good idea to make these changes to our system.

By making these changes we make our services more loosely coupled, which almost always is a good thing. The monitor service does no longer need to know that the slack service even exists and the site service can remain independent of the monitor service.
The slack service can now be offline without affecting the monitor service. And then when it gets back online it will just read from the event queue and pick up where it left off. Are system is therefore more robust.
Both the monitor and slack service are now dependent on an abstraction rather than each other, this is called Dependency Inversion. This allows us to replace or add other notification channels like discord or email without needing to make changes to to the monitor service.

Pitfalls

But making a system event-driven is not always a good idea. Here are some things to watch out for:

It’s not an all or nothing approach! Just use Pub/Sub in places where it’s a great fit. There is no need to be dogmatic about this. Please don’t be dogmatic… WOFF!🐶
Simply replacing traditional API calls with events isn’t enough. There are essential concepts to understand when working with asynchronous queues, particularly Eventual Consistency and Idempotency. Take the time to learn about these ideas. Trust me, you will thank me later.
An event-driven system is more complex. You have more components you need to build and there are additional infrastructure requirements. Using the right tools for the job is super duper important. Debugging problems and managing your environments will be really frustrating if you don’t have the right tools.

And that is why we will be using Encore.ts to build our event-driven application. Encore.ts is an Open Source framework that is specifically designed to make it easier to build robust and type-safe distributed systems with TypeScript, exactly like the event-driven backend we’re going to build today. And it has a lot of useful built-in tools to make the development experience smoother, like a local development dashboard which we’ll look at a little later.

Now, let’s look at some code.

Adding our Pub/Sub topic

From a code perspective, a service is just another folder in your repo when working with Encore. You will most likely end up with a lot of services when building an event-driven application, so creating new services needs to be easy. This is one of the reasons why Encore is a great fit for this kind of application.

Let’s start by looking at the site service.

import { api } from "encore.dev/api";
import { SQLDatabase } from "encore.dev/storage/sqldb";
import knex from "knex";

// Site describes a monitored site.
export interface Site {
  id: number;
  url: string;
}

export interface AddParams {
  url: string;
}

// Add a new site to the list of monitored websites.
export const add = api(
  { expose: true, method: "POST", path: "/site" },
  async (params: AddParams): Promise<Site> => {
    const site = (await Sites().insert({ url: params.url }, "*"))[0];
    return site;
  },
);

// Get a site by id.
export const get = api(
  { expose: true, method: "GET", path: "/site/:id", auth: false },
  async ({ id }: { id: number }): Promise<Site> => {
    const site = await Sites().where("id", id).first();
    return site ?? Promise.reject(new Error("site not found"));
  },
);

// Delete a site by id.
export const del = api(
  { expose: true, method: "DELETE", path: "/site/:id" },
  async ({ id }: { id: number }): Promise<void> => {
    await Sites().where("id", id).delete();
  },
);

export interface ListResponse {
  sites: Site[]; // Sites is the list of monitored sites
}

// Lists the monitored websites.
export const list = api(
  { expose: true, method: "GET", path: "/site" },
  async (): Promise<ListResponse> => {
    const sites = await Sites().select();
    return { sites };
  },
);

// Define a database named 'site', using the database migrations
// in the "./migrations" folder. Encore automatically provisions,
// migrates, and connects to the database.
const SiteDB = new SQLDatabase("site", {
  migrations: "./migrations",
});

const orm = knex({
  client: "pg",
  connection: SiteDB.connectionString,
});

const Sites = () => orm<Site>("site");

This service has a few CRUD endpoints like add, get, delete and list. We are interested in the add endpoint because we want to publish an event when a new site is added. Let’s start to make our application event-driven by adding our site.added Topic. We do this by calling the Topic class, specifying the type that will be published on this topic (in this case the Site type) and we specify the delivery guarantee.

import { Topic } from "encore.dev/pubsub";

export const SiteAddedTopic = new Topic<Site>("site.added", {
  deliveryGuarantee: "at-least-once",
});

Now, in the add endpoint we can now call the .publish method on the SiteAddedTopic object.

export const add = api(
  { expose: true, method: "POST", path: "/site" },
  async (params: AddParams): Promise<Site> => {
    const site = (await Sites().insert({ url: params.url }, "*"))[0];
    await SiteAddedTopic.publish(site);
    return site;
  },
);

Using Pub/Sub with Encore type-safe so you will get compile time errors if you publish to a Topic with the incorrect parameters 🤯

Our architectural diagram now looks like this:

We are publishing to the site.added topic but we are not yet subscribing to it from the monitor service, so let’s fix that now.

Adding the subscriber

So, let’s open the monitor service.

import { api } from "encore.dev/api";
import { SQLDatabase } from "encore.dev/storage/sqldb";
import { Site } from "../site/site";
import { ping } from "./ping";
import { site, slack } from "~encore/clients";
import { CronJob } from "encore.dev/cron"; // Check checks a single site.

// Check checks a single site.
export const check = api(
  { expose: true, method: "POST", path: "/check/:siteID" },
  async (p: { siteID: number }): Promise<{ up: boolean }> => {
    const s = await site.get({ id: p.siteID });
    return doCheck(s);
  },
);

// CheckAll checks all sites.
export const checkAll = api(
  { expose: true, method: "POST", path: "/check-all" },
  async (): Promise<void> => {
    const sites = await site.list();
    await Promise.all(sites.sites.map(doCheck));
  },
);

async function doCheck(site: Site): Promise<{ up: boolean }> {
  const { up } = await ping({ url: site.url });

  const wasUp = await getPreviousMeasurement(site.id);
  if (up !== wasUp) {
    const text = `*${site.url} is ${up ? "back up." : "down!"}*`;
    await slack.notify({ text });
  }

  await MonitorDB.exec`
      INSERT INTO checks (site_id, up, checked_at)
      VALUES (${site.id}, ${up}, NOW())
  `;
  return { up };
}

async function getPreviousMeasurement(siteID: number): Promise<boolean> {
  const row = await MonitorDB.queryRow`
      SELECT up
      FROM checks
      WHERE site_id = ${siteID}
      ORDER BY checked_at DESC
      LIMIT 1
  `;
  return row?.up ?? true;
}

const cronJob = new CronJob("check-all", {
  title: "Check all sites",
  every: "1h",
  endpoint: checkAll,
});

export const MonitorDB = new SQLDatabase("monitor", {
  migrations: "./migrations",
});

Here we have a check endpoint that checks and updates the status of a single site. Inside the API handler we are calling the get endpoint in the site service. We import the service through the encore/clients folder and then we can call the endpoint just like calling a regular function, with complete type-safety. But the cool part is that under the hood, those function calls gets converted to actual HTTP calls, resulting in traces and logs.

The doCheck function looks like the function we want to call whenever a new site is added, so let’s add our Pub/Sub subscriber.

import { Subscription } from "encore.dev/pubsub";

const _ = new Subscription(SiteAddedTopic, "check-site", {
  handler: doCheck,
});

To do that we call the Subscription class, passing in the the Topic we want to subscribe to, the name of the subscription and an options object. In the options object we only need to specify the handler, the function that gets called for every new event. The doCheck function accepts a Site so there is no need to do anything more than this.

Local infrastructure

So, how does this work? Encore comes with automatic local infrastructure. When you start your Encore application locally using encore run, Encore automatically spinns upp all the infrastructure your app needs on your computer, including databases and Pub/Sub. So you don’t need to deal with writing YAML, or setting up Docker Compose, and other tools like LocalStack to run your environment.

Local Development Dashboard

Encore also comes with a built-in development dashboard. When you start your Encore app, the development dashboard is available on port localhost:9400. From here you can call your endpoints, a bit like Postman. Each call to your application results in a trace that you can inspect to see the API requests, database calls, and Pub/Sub messages.

Getting local tracing out of the box and being able to easily debug your application like this is another reason why Encore is a great choice when building event-driven apps.

The local development dashboard also includes a Service Catalog with automatic API documentation. Oh, and by the way. The pretty architectural digram from earlier in this post, that is Encore Flow which is also built into the Development Dashboard. It’s an always up-to-date representation of your system that changes in real-time as you develop. 😎

Deployment

So, how do we deploy this application? Well you can build your application using encore build, and you get it as a docker image you can deploy anywhere you want. You will need to supply a runtime configuration where you can specify how the application should connect to the infrastructure, like Pub/Sub and databases. If you don’t feel like managing this stuff manually, you can use Encore's Cloud Platform which automates setting up the needed infrastructure in your cloud account on AWS or GCP, and it comes with built-in CI/CD so you just need to push to deploy. The Platform also comes with monitoring, tracing, and automatic preview environments so you can test each pull request in a dedicated temporary environment.

Running the Uptime application yourself

If you want to play around with the Uptime application yourself you can easily do so by installing Encore and then run encore app create in your terminal. Select TypeScript and then the Uptime application in the list of starter templates. You will need to have Docker desktop installed as that is needed to create databases locally. When you have the code checked out then you can also take a look at the uptime-transition topic used by the slack service.

Wrapping up

⭐️ Support Encore by giving the project a star on GitHub.
Check out Encore’s example repo where you can find a bunch of deployable applications.
If you have questions or want to share your work, join the developers hangout in Encore's community on Discord.

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

#node #typescript #javascript #webdev

How to deploy a backend application to DigitalOcean using Docker and Encore

Marcus Kohlberg for Encore ・ Oct 23

#opensource #javascript #docker #go

TypeScript vs Go: Choosing Your Backend Language

Marcus Kohlberg for Encore ・ Nov 8

#go #typescript #javascript #webdev

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson — Fri, 01 Nov 2024 08:49:20 +0000

Node.js web frameworks — where do we even begin? With so many options out there, choosing the right one for your project can feel overwhelming.

In this post, I’ll walk you through the hottest frameworks in the Node.js ecosystem, breaking down the strengths, weaknesses, and best use cases for each one.

Whether you’re looking for speed, scalability, or simplicity, hopefully we will cover it all—so by the end, you’ll know exactly which framework is right for you.

The frameworks we will be looking at are: Nest, Elysia, Encore.ts and Hono.

Video version:

Feature overview

On a scale from most lightweight to most feature rich I would position the frameworks like this:

This does not mean that lightweight is bad, it just depends on what the needs are for your project. The lightweight nature of Hono is actually one of the selling points, with just under 14KB its perfect for deploying to Cloudflare Workers.

Encore.ts on the other hand comes with a lot of built in features, like automatic tracing out of the box and local infrastructure.

Let’s take a look at each framework, and we are going to start with Encore.ts

Encore.ts

An Open Source framework that aims to make it easier to build robust and type-safe backends with TypeScript. The framework has a lot of build in tools to make your development experience smoother and performance wise, it’s the fasters of all frameworks in this comparison.

Encore has built-in request validation. The request and response types you define in regular TypeScript are used to validate the request, both during compile and runtime. And unlike the other frameworks the actual validation is done in Rust and not in JavaScript. This makes the validation really fast, but more on that later.

import {api, Header, Query} from "encore.dev/api";

enum EnumType {
  FOO = "foo",
  BAR = "bar",
}

// Encore.ts automatically validates the request schema 
// and returns and error if the request does not match.
interface RequestSchema {
  foo: Header<"x-foo">;
  name?: Query<string>;

  someKey?: string;
  someOtherKey?: number;
  requiredKey: number[];
  nullableKey?: number | null;
  multipleTypesKey?: boolean | number;
  enumKey?: EnumType;
}

// Validate a request
export const schema = api(
  {expose: true, method: "POST", path: "/validate"},
  (data: RequestSchema): { message: string } => {
    console.log(data);
    return {message: "Validation succeeded"};
  },
);

Encore makes it easy to create and call services. From a code perspective, a service just looks like another folder in your repo. When calling an endpoint in a service it’s just like calling a regular function. But the cool part is that under the hood, those function calls gets converted to actual HTTP calls. When deploying, you can even choose to deploy your services to separate instances, for example in a Kubernetes cluster, while still having all service code in the same repo.

Import a service and call its API endpoints like regular functions

import { api } from "encore.dev/api";
import { hello } from "~encore/clients"; // import 'hello' service

export const myOtherAPI = api({}, async (): Promise<void> => {
  // all the ping endpoint using a function call
  const resp = await hello.ping({ name: "World" });
  console.log(resp.message); // "Hello World!"
});

With Encore.ts you integrate your infrastructure as type-safe objects in application code. Creating a database or a Pub/Sub topic just requires a few lines of application code. Encore.ts builds your application as a Docker image and you just supply the runtime configuration when deploying, that’s it.

Create a PostgreSQL database in one line of code

import { SQLDatabase } from "encore.dev/storage/sqldb";

const db = new SQLDatabase("userdb", {migrations: "./migrations"});
// ... use db.query to query the database.

Create Pub/Sub topics and subscriptions

import { Topic } from "encore.dev/pubsub";

export interface User { /* fields ... */ }

const signups = new Topic<User>("signup", {
  deliveryGuarantee: "at-least-once",
});

await signups.publish({ ... });

Local Development Dashboard

And its worth mentioning that even though Encore comes with a lot of features, it has 0 npm dependencies.

Hono

Hono is the creation of Yusuke Wada. He started the project 2021 because there were no good Node.js frameworks that worked well on Cloudflare Workers. Since then Hono as added support for a lot of other runtimes like Node.js, Bun and Deno.

import { Hono } from 'hono'
const app = new Hono()

app.get('/', (c) => c.text('Hono!'))

export default app

Hono is really small, the hono/tiny preset is under 13kB which makes it really suitable for deploying to Cloudflare Workers. Hono also has 0 NPM dependencies which is really impressive!

The multi-runtime selling point is interesting, the idea that no matter the Javascript runtime you will be able to run Hono. In their repo they have this concept of adapters where you can see the adjustments they make for each runtime. I think this has been huge for adoption and growing the user base but realistically for an individual user you will probably not switch runtimes once you have your app deployed to the cloud.

And even though Hono is lightweight out of the box it has a bunch of middleware, both 1st and 3rd-party that you can install to enhance your app. It’s the “add it when you want to use it”-approach that got popular with Express. This can work great as long as you app is small but with a larger app, maintaining a huge list of dependencies can be frustrating.

Elysia

Elysia, like Encore, is built for TypeScript and also offers type-safety inside your API handlers. Knowing what you are working with inside your API handlers saves you a lot of time and it’s great to not having to litter your code with type checks.

You specify the request types with the t module, which is an extension of the TypeBox validation library. Unlike Encore, the validation happens in the JavaScript layer which adds some performance overhead.

import { Elysia, t } from 'elysia'

new Elysia()
  .patch("/profile", ({ body }) => body.profile, {
    body: t.Object({
      id: t.Number(),
      profile: t.File({ type: "image" }),
    }),
  })
  .listen(3000);

Adding Swagger documentation only requires one line of code and Elysia has 1st party support for OpenTelemetry. Which is really nice, so you can easily monitor your app regardless of the platform.

Elysia is fast! But not as fast as Encore as you will see in the next section.

Nest.js

Nest.js is a bit different then the other frameworks in this comparison. Encore, Elysia and Hono tries to offer minimalistic APIs for creating endpoints and middleware and you are free to structure your business logic however you wish. Nest.js is much more opinionated and forces you to structure your code in a certain way. It offers a modular architecture that organizes code into different abstractions, like Providers, Controllers, Modules and Middleware.

Nest is designed to make it easier to maintain and develop larger applications. But whether your a fan of the opinionated structure Nest offers is at the end of the day very subjective. I would say that it might be advantageous for large-scale projects where long-term maintainability are more important than speed and simplicity. For smaller project with just a few developers the added level of abstractions will probably be overkill for most use cases. The opinionated nature of Nest also brings with it a steeper learning curve compared to the Hono, Encore and Elysia.

When using Nest you can choose to either use Express or Fastify as the underlying HTTP server framework. All the Nest functionality is added on top of that.

Performance

Speed is perhaps not the most important thing when choosing a framework, but it’s not something you can ignore. It will impact your apps responsiveness AND ultimately your hosting bill.

We have benchmarked both without and with request schema validation, the measurement is requests per second. The names in parenthesis are the request validation libraries used. Encore.ts has request validation built in.

Encore.ts is the fastest for all frameworks in our benchmark followed by Elysia, Hono, Fastify and Express. Nest uses Fastify or Express under the hood so you can expect the performance to for a Nest app to be equivalent to that, maybe a but slower as Nest adds some overhead.

How can Encore.ts be so much faster? The secret is that Encore.ts has a Rust runtime. And Rust is fast!

Encore.ts actually consists of two parts:

A user facing TypeScript part for defining APIs and infrastructure.
And under the hood, it has a multi-threaded runtime written in Rust.

The key to the performance boost is to off-load as much work as possible from the single-threaded Node.js event-loop to the Rust runtime.

For example, the Rust runtime handles all input/output like accepting incoming HTTP requests or reading from the database. Once the request or database query has been fully processed, then it gets handed over to the Node.js event-loop.

Code structure

Hono, Elysia and Encore are unopinionated in how you structure your code. And the way you create APIs and middleware are fairly similar between the frameworks.

Here is a GET endpoint for each framework. There are of course some differences, but if we squint the APIs look fairly similar. At least similar enough for this not to be the deciding factor in my opinion:

Encore.ts

interface Response {
  message: string;
}

export const get = api(
  { expose: true, method: "GET", path: "/hello/:name" },
  async ({ name }: { name: string }): Promise<Response> => {
    const msg = `Hello ${name}!`;
    return { message: msg };
  },
);

Elysia

import { Elysia, t } from "elysia";

new Elysia()
  .get(
    "/hello/:name",
    ({ params }) => {
      const msg = `Hello ${params.name}!`;
      return { message: msg };
    },
    {
      response: t.Object({
        message: t.String(),
      }),
    },
  )

Hono

import { Hono } from "hono";

const app = new Hono();

app.get("/hello/:name", async (c) => {
  const msg = `Hello ${c.req.param("name")}!`;
  return c.json({ message: msg });
});

What really makes a difference is the being able to relay on type-safety when building a robust application. Encore and Elysia offer type-safe APIs but Encore also offers compile time type-safety when working with infrastructure like Pub/Sub. With Encore, you also get compile time type-safety when calling an endpoint in another service. And if you've ever worked with a microservice architecture before you know how big of a deal that is.

Nest.js is the one that really sticks out in terms of API design. There are a lot of concepts and abstractions that goes into a Nest app. This can be both a good and a bad thing, it really depends of your preference. One thing that will be immediately apparent when looking at a Nest app is the use of decorators. Nest relies heavily on decorators, for example when using dependency injection to inject a Services into Controllers.

Nest controller

import { Controller, Get } from '@nestjs/common';
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get()
  getHello(): string {
    return this.appService.getHello();
  }
}

Personally, I am not a big fan and I know that I am not the only one out there.

Deployment & Infrastructure

All of these frameworks are similar in that they are deployable to to all mainstream cloud platforms (like Digital Ocean and Fly.io) or straight to a cloud provider like AWS or GCP.

Encore offers automatic local infrastructure. encore run starts your app and spinns up all local infrastructure, like databases and Pub/Sub topic. Forget YAML, Docker Compose, and the usual headaches.

When building your Encore app you get a runtime config where you can supply the configuration needed for connecting to the infrastructure in your cloud.

If you want to get your Encore app into the cloud quickly and don’t feel like self-serving then you can use Encore Cloud. Encore Cloud offers CI/CD and preview environments for pull requests. And if you want to, Encore Cloud can provision all the needed infrastructure in your own cloud on AWS or GCP. This means your app is not dependent on any third-party services and you have full control over all your infrastructure.

Hono sticks out in that it supports a bunch of different runtimes, you therefore have a lot of options when it comes to deployment. Deploying to Cloudflare Workers, Netlify or AWS Lambda is easy and does not require a lot of configuration.

With Nest you run the nest build command that compiles your TypeScript code into JavaScript. This process generates a dist directory containing the compiled files. Thats pretty much it, you can then use Node.js to run your dist folder.

The recommended way to deploy an Elysia application is to compile your app into a binary using the bun build command. Once binary is compiled, you don't need Bun installed on the machine to run the server.

Wrapping up

That’s it! Hopefully you know what framework to reach for in your next project.

If you want to learn more about Encore.ts, you can check out the Open Source project on GitHub: https://github.com/encoredev/encore

Recommendations:

NestJS vs Encore.ts: Choosing the Right Framework for Your TypeScript Microservices

Marcus Kohlberg for Encore ・ Oct 30

#javascript #typescript #microservices #webdev

How to deploy a backend application to DigitalOcean using Docker and Encore

Marcus Kohlberg for Encore ・ Oct 23

#opensource #javascript #docker #go

Choosing the Right Tech Stack: A Developer's Decision-Making Guide

Lucas Lima do Nascimento for Encore ・ Oct 28

#beginners #webdev #tutorial #career

Encore.ts — 9x faster than Express.js (feature comparison and migration guide)

Simon Johansson — Tue, 24 Sep 2024 07:41:33 +0000

Watch the video on YouTube:
https://www.youtube.com/watch?v=hA9syK_FtZw

Written migration guide for Express.js to Encore.ts: https://encore.dev/docs/ts/how-to/express-migration

How to make your Express.js APIs 9x faster with Encore.ts

Simon Johansson for Encore ・ Sep 10

#node #express #typescript #encore

How to let ChatGPT call functions in your app

Simon Johansson for Encore ・ Aug 14

#chatgpt #typescript #node #javascript

Fastify v5 vs v4 — vs Encore.ts

André Eriksson for Encore ・ Sep 20

#node #javascript #fastify #backend

How to make your Express.js APIs 9x faster with Encore.ts

Simon Johansson — Tue, 10 Sep 2024 11:43:42 +0000

Encore.ts is Open Source backend framework for TypeScript. This guide walks you through how to migrate an Express.js app to Encore.ts to gain type-safe APIs and 9x performance boost.

Why migrate to Encore.ts?

Express.js is a great choice for building simple APIs, but as your application grows you will likely run into limitations. There is a large community around Express.js, providing many plugins and middleware to work around these limitations. However, relying heavily on plugins can make it hard to find the right tools for your use case. It also means that you will need to maintain a lot of dependencies.

Encore.ts is an Open Source framework that aims to make it easier to build robust and type-safe backends with TypeScript. Encore.ts has 0 npm dependencies, is built with performance in mind, and has a lot of built-in features for building production ready backends. You can self-serve an Encore.ts app on any hosting service that accepts Docker containers, or use Encore Cloud Platform to fully automate your DevOps and infrastructure.

Performance

Encore.ts has its own high-performance runtime, with a multi-threaded, asynchronous event loop written in Rust. The Encore Runtime handles all I/O like accepting and processing incoming HTTP requests. This runs as a completely independent event loop that utilizes as many threads as the underlying hardware supports. The result of this is that Encore.ts performs 9x faster than Express.js.

Encore.ts handles 9x more requests/sec than Express.js

Encore.ts has 80% less response latency than Express.js

Encore.ts — 9x faster than Express.js & 3x faster than Bun + Zod

Marcus Kohlberg for Encore ・ Aug 2

#typescript #node #javascript #backenddevelopment

Built-in benefits

When using Encore.ts you get a lot of built-in features without having to install any additional dependencies:

Local Development Dashboard

Migration guide

Below we've outlined two main strategies you can use to migrate your existing Express.js application to Encore.ts. Pick the strategy that best suits your situation and application.

Forklift migration (quick start)

When you quickly want to migrate to Encore.ts and don't need all the functionality to begin with, you can use a forklift migration strategy. This approach moves the entire application over to Encore.ts in one shot, by wrapping your existing HTTP router in a catch-all handler.

Approach benefits

You can get your application up and running with Encore.ts quickly and start moving features over to Encore.ts while the rest of the application is still untouched.
You will see a partial performance boost right away because the HTTP layer is now running on the Encore Rust runtime. But to get the full performance benefits, you will need to start using Encore's API declarations and infrastructure declarations.

Approach drawbacks

Because all requests will be proxied through the catch-all handler, you will not be able to get all the benefits from the distributed tracing.
The automatically generated architectural diagrams and API documentation will not be able to show you the full picture of your application until you start moving services and APIs over to Encore.ts.
You will not be able to use the API Client Generation feature until you start defining APIs in Encore.ts.

Forklift migration step-by-step

1. Install Encore

If this is the first time you're using Encore, you first need to install the CLI that runs the local development environment. Use the appropriate command for your system:

macOS: brew install encoredev/tap/encore
Linux: curl -L https://encore.dev/install.sh | bash
Windows: iwr https://encore.dev/install.ps1 | iex

Installation docs

2. Add Encore.ts to your project



npm i encore.dev

3. Initialize an Encore app

Inside your project directory, run the following command to create an Encore app:



encore app init

This will create an encore.app file in the root of your project.

4. Configure your tsconfig.json

To the tsconfig.json file in the root of your project, add the following:



{
   "compilerOptions": {
      "paths": {
         "~encore/*": [
            "./encore.gen/*"
         ]
      }
   }
}

When Encore.ts is parsing your code it will specifically look for ~encore/* imports.

5. Define an Encore.ts service

When running an app using Encore.ts you need at least one Encore service. Apart from that, Encore.ts in not opinionated in how you structure your code, you are free to go with a monolith or microservice approach. Learn more in our App Structure docs.

In the root of your App, add a file named encore.service.ts. The file must export a service instance, by calling new Service, imported from encore.dev/service:



import {Service} from "encore.dev/service";

export default new Service("my-service");

Encore will consider this directory and all its subdirectories as part of the service.

6. Create a catch-all handler for your HTTP router

Now let's mount your existing app router under a Raw endpoint, which is an Encore API endpoint type that gives you access to the underlying HTTP request.

Here's a basic code example:



import { api, RawRequest, RawResponse } from "encore.dev/api";
import express, { request, response } from "express";

Object.setPrototypeOf(request, RawRequest.prototype);
Object.setPrototypeOf(response, RawResponse.prototype);

const app = express();

app.get('/foo', (req: any, res) => {
  res.send('Hello World!')
})

export const expressApp = api.raw(
  { expose: true, method: "*", path: "/!rest" },
  app,
);

By mounting your existing app router in this way, it will work as a catch-all handler for all HTTP requests and responses.

7. Run you app locally

You will now be able to run your Express.js app locally using the encore run command.

Next steps: Incrementally move over Encore.ts to get all the benefits

You can now gradually break out specific endpoints using the Encore's API declarations and introduce infrastructure declarations for databases and cron jobs etc. This will let Encore.ts understand your application and unlock all Encore.ts features. See the Feature-by-feature migration section for more details. You will eventually be able to remove Express.js as a dependency and run your app entirely on Encore.ts.

For more thoughts on migrating an existing backend to Encore.ts, check out our general migration guide. You can also join Discord to ask questions and meet fellow Encore developers.

Full migration

This approach aims to fully replace your applications dependency on Express.js with Encore.ts, unlocking all the features and performance of Encore.ts.

In the next section you will find a Feature-by-feature migration guide to help you with the refactoring details.

Approach benefits

Get all the advantages of Encore.ts, like distributed tracing and architecture diagrams.
Get the full performance benefit of Encore.ts - 9x faster than Express.js.

Approach drawbacks

This approach may require more time and effort up front compared to the Forklift migration strategy.

Feature-by-feature migration

Check out our Express.js compared to Encore.ts example on GitHub for all of the code snippets in this feature comparison.

APIs

With Express.js, you create APIs using the app.get, app.post, app.put, app.delete functions. These functions take a path and a callback function. You then use the req and res objects to handle the request and response.

With Encore.ts, you create APIs using the api function. This function takes an options object and a callback function. The main difference compared to Express.js is that Encore.ts is type-safe, meaning you define the request and response schemas in the callback function. You then return an object matching the response schema. In case you need to operate at a lower abstraction level, Encore supports defining raw endpoints that let you access the underlying HTTP request. Learn more in our API Schemas docs.

Express.js



import express, {Request, Response} from "express";

const app: Express = express();

// GET request with dynamic path parameter
app.get("/hello/:name", (req: Request, res: Response) => {
  const msg = `Hello ${req.params.name}!`;
  res.json({message: msg});
})

// GET request with query string parameter
app.get("/hello", (req: Request, res: Response) => {
  const msg = `Hello ${req.query.name}!`;
  res.json({message: msg});
});

// POST request example with JSON body
app.post("/order", (req: Request, res: Response) => {
  const price = req.body.price;
  const orderId = req.body.orderId;
  // Handle order logic
  res.json({message: "Order has been placed"});
});

Encore.ts



import {api, Query} from "encore.dev/api";

// Dynamic path parameter :name
export const dynamicPathParamExample = api(
  {expose: true, method: "GET", path: "/hello/:name"},
  async ({name}: { name: string }): Promise<{ message: string }> => {
    const msg = `Hello ${name}!`;
    return {message: msg};
  },
);

interface RequestParams {
  // Encore will now automatically parse the query string parameter
  name?: Query<string>;
}

// Query string parameter ?name
export const queryStringExample = api(
  {expose: true, method: "GET", path: "/hello"},
  async ({name}: RequestParams): Promise<{ message: string }> => {
    const msg = `Hello ${name}!`;
    return {message: msg};
  },
);

interface OrderRequest {
  price: string;
  orderId: number;
}

// POST request example with JSON body
export const order = api(
  {expose: true, method: "POST", path: "/order"},
  async ({price, orderId}: OrderRequest): Promise<{ message: string }> => {
    // Handle order logic
    console.log(price, orderId)

    return {message: "Order has been placed"};
  },
);

// Raw endpoint
export const myRawEndpoint = api.raw(
  {expose: true, path: "/raw", method: "GET"},
  async (req, resp) => {
    resp.writeHead(200, {"Content-Type": "text/plain"});
    resp.end("Hello, raw world!");
  },
);

Microservices communication

Express.js does not have built-in support for creating microservices or for service-to-service communication. You will most likely use fetch or something equivalent to call another service.

With Encore.ts, calling another service is just like calling a local function, with complete type-safety. Under the hood, Encore.ts will translate this function call into an actual service-to-service HTTP call, resulting in trace data being generated for each call. Learn more in our Service-to-Service Communication docs.

Express.js



import express, {Request, Response} from "express";

const app: Express = express();

app.get("/save-post", async (req: Request, res: Response) => {
  try {
    // Calling another service using fetch
    const resp = await fetch("https://another-service/posts", {
      method: "POST",
      headers: {"Content-Type": "application/json"},
      body: JSON.stringify({
        title: req.query.title,
        content: req.query.content,
      }),
    });
    res.json(await resp.json());
  } catch (e) {
    res.status(500).json({error: "Could not save post"});
  }
});

Encore.ts



import {api} from "encore.dev/api";
import {anotherService} from "~encore/clients";

export const microserviceCommunication = api(
  {expose: true, method: "GET", path: "/call"},
  async (): Promise<{ message: string }> => {
    // Calling the foo endpoint in anotherService
    const fooResponse = await anotherService.foo();

    const msg = `Data from another service ${fooResponse.data}!`;
    return {message: msg};
  },
);

Authentication

In Express.js you can create a middleware function that checks if the user is authenticated. You can then use this middleware function in your routes to protect them. You will have to specify the middleware function for each route that requires authentication.

With Encore.ts, when an API is defined with auth: true, you must define an authentication handler in your application. The authentication handler is responsible for inspecting incoming requests to determine what user is authenticated.

The authentication handler is defined similarly to API endpoints, using the authHandler function imported from encore.dev/auth. Like API endpoints, the authentication handler defines what request information it's interested in, in the form of HTTP headers, query strings, or cookies.

If a request has been successfully authenticated, the API Gateway forwards the authentication data to the target endpoint. The endpoint can query the available auth data from the getAuthData function, available from the ~encore/auth module.

Learn more in our Auth Handler docs

Express.js



import express, {NextFunction, Request, Response} from "express";

const app: Express = express();

// Auth middleware
function authMiddleware(req: Request, res: Response, next: NextFunction) {
  // TODO: Validate up auth token and verify that this is an authenticated user
  const isInvalidUser = req.headers["authorization"] === undefined;

  if (isInvalidUser) {
    res.status(401).json({error: "invalid request"});
  } else {
    next();
  }
}

// Endpoint that requires auth
app.get("/dashboard", authMiddleware, (_, res: Response) => {
  res.json({message: "Secret dashboard message"});
});

Encore.ts



import { api, APIError, Gateway, Header } from "encore.dev/api";
import { authHandler } from "encore.dev/auth";
import { getAuthData } from "~encore/auth";

interface AuthParams {
  authorization: Header<"Authorization">;
}

// The function passed to authHandler will be called for all incoming API call that requires authentication.
export const myAuthHandler = authHandler(
  async (params: AuthParams): Promise<{ userID: string }> => {
    // TODO: Validate up auth token and verify that this is an authenticated user
    const isInvalidUser = params.authorization === undefined;

    if (isInvalidUser) {
      throw APIError.unauthenticated("Invalid user ID");
    }

    return { userID: "user123" };
  },
);

export const gateway = new Gateway({ authHandler: myAuthHandler });

// Auth endpoint example
export const dashboardEndpoint = api(
  // Setting auth to true will require the user to be authenticated
  { auth: true, method: "GET", path: "/dashboard" },
  async (): Promise<{ message: string; userID: string }> => {
    return {
      message: "Secret dashboard message",
      userID: getAuthData()!.userID,
    };
  },
);

Request validation

Express.js does not have built-in request validation. You have to use a library like Zod.

With Encore.ts, request validation for headers, query params and body is. You supply a schema for the request object and in the request payload does not match the schema the API will return a 400 error. Learn more in our API Schemas docs.

Express.js



import express, {NextFunction, Request, Response} from "express";
import {z, ZodError} from "zod";

const app: Express = express();

// Request validation middleware
function validateData(schemas: {
  body: z.ZodObject<any, any>;
  query: z.ZodObject<any, any>;
  headers: z.ZodObject<any, any>;
}) {
  return (req: Request, res: Response, next: NextFunction) => {
    try {
      // Validate headers
      schemas.headers.parse(req.headers);

      // Validate request body
      schemas.body.parse(req.body);

      // Validate query params
      schemas.query.parse(req.query);

      next();
    } catch (error) {
      if (error instanceof ZodError) {
        const errorMessages = error.errors.map((issue: any) => ({
          message: `${issue.path.join(".")} is ${issue.message}`,
        }));
        res.status(400).json({error: "Invalid data", details: errorMessages});
      } else {
        res.status(500).json({error: "Internal Server Error"});
      }
    }
  };
}

// Request body validation schemas
const bodySchema = z.object({
  someKey: z.string().optional(),
  someOtherKey: z.number().optional(),
  requiredKey: z.array(z.number()),
  nullableKey: z.number().nullable().optional(),
  multipleTypesKey: z.union([z.boolean(), z.number()]).optional(),
  enumKey: z.enum(["John", "Foo"]).optional(),
});

// Query string validation schemas
const queryStringSchema = z.object({
  name: z.string().optional(),
});

// Headers validation schemas
const headersSchema = z.object({
  "x-foo": z.string().optional(),
});

// Request validation example using Zod
app.post(
  "/validate",
  validateData({
    headers: headersSchema,
    body: bodySchema,
    query: queryStringSchema,
  }),
  (_: Request, res: Response) => {
    res.json({message: "Validation succeeded"});
  },
);

Encore.ts



import {api, Header, Query} from "encore.dev/api";

enum EnumType {
  FOO = "foo",
  BAR = "bar",
}

// Encore.ts automatically validates the request schema and returns and error
// if the request does not match the schema.
interface RequestSchema {
  foo: Header<"x-foo">;
  name?: Query<string>;

  someKey?: string;
  someOtherKey?: number;
  requiredKey: number[];
  nullableKey?: number | null;
  multipleTypesKey?: boolean | number;
  enumKey?: EnumType;
}

// Validate a request
export const schema = api(
  {expose: true, method: "POST", path: "/validate"},
  (data: RequestSchema): { message: string } => {
    console.log(data);
    return {message: "Validation succeeded"};
  },
);

Error handling

In Express.js you either throw an error (which results in a 500 response) or use the status function to set the status code of the response.

In Encore.ts throwing an error will result in a 500 response. You can also use the APIError class to return specific error codes. Learn more in our API Errors docs.

Express.js



import express, {Request, Response} from "express";

const app: Express = express();

// Default error handler
app.get("/broken", (req, res) => {
  throw new Error("BROKEN"); // This will result in a 500 error
});

// Returning specific error code
app.get("/get-user", (req: Request, res: Response) => {
  const id = req.query.id || "";
  if (id.length !== 3) {
    res.status(400).json({error: "invalid id format"});
  }
  // TODO: Fetch something from the DB
  res.json({user: "Simon"});
});

Encore.ts



import {api, APIError} from "encore.dev/api"; // Default error handler

// Default error handler
export const broken = api(
  {expose: true, method: "GET", path: "/broken"},
  async (): Promise<void> => {
    throw new Error("This is a broken endpoint"); // This will result in a 500 error
  },
);

// Returning specific error code
export const brokenWithErrorCode = api(
  {expose: true, method: "GET", path: "/broken/:id"},
  async ({id}: { id: string }): Promise<{ user: string }> => {
    if (id.length !== 3) {
      throw APIError.invalidArgument("invalid id format");
    }
    // TODO: Fetch something from the DB
    return {user: "Simon"};
  },
);

Serving static files

Express.js has a built-in middleware function to serve static files. You can use the express.static function to serve files from a specific directory.

Encore.ts also has built-in support for static file serving with the api.static method.

The files are served directly from the Encore.ts Rust Runtime. This means that zero JavaScript code is executed to serve the files, freeing up the Node.js runtime to focus on executing business logic. This dramatically speeds up both the static file serving,
as well as improving the latency of your API endpoints. Learn more in our Static Assets docs

Express.js



import express from "express";

const app: Express = express();

app.use("/assets", express.static("assets")); // Serve static files from the assets directory

Encore.ts



import { api } from "encore.dev/api";

export const assets = api.static(
  { expose: true, path: "/assets/*path", dir: "./assets" },
);

Template rendering

Express.js has a built-in support for rendering templates.

With Encore.ts you can use the api.raw function to serve HTML templates, in this example we are using Handlebars.js but you can use whichever templating engine you prefer. Learn more in our Raw Endpoints docs

Express.js



import express, {Request, Response} from "express";

const app: Express = express();

app.set("view engine", "pug"); // Set view engine to Pug

// Template engine example. This will render the index.pug file in the views directory
app.get("/html", (_, res) => {
  res.render("index", {title: "Hey", message: "Hello there!"});
});

Encore.ts



import {api} from "encore.dev/api";
import Handlebars from "handlebars";

const html = `
<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8"/>
  <link rel="stylesheet" href="/assets/styles.css">
</head>
<body>
<h1>Hello {{name}}!</h1>
</body>
</html>
`;

// Making use of raw endpoints to serve dynamic templates.
// https://encore.dev/docs/ts/primitives/services-and-apis#raw-endpoints
export const serveHTML = api.raw(
  {expose: true, path: "/html", method: "GET"},
  async (req, resp) => {
    const template = Handlebars.compile(html);

    resp.setHeader("Content-Type", "text/html");
    resp.end(template({name: "Simon"}));
  },
);

Testing

Express.js does not have built-in testing support. You can use libraries like Vitest and
Supertest.

With Encore.ts you are able to call the API endpoints directly in your tests, just like any other function. You then run the tests using the encore test command. Learn more in our Testing docs.

Express.js



import {describe, expect, test} from "vitest";
import request from "supertest";
import express from "express";
import getRequestExample from "../get-request-example";

/**
 * We need to add the supertest library to make fake HTTP requests to the Express.js app without having to
 * start the server. We also use the vitest library to write tests.
 */
describe("Express App", () => {
  const app = express();
  app.use("/", getRequestExample);

  test("should respond with a greeting message", async () => {
    const response = await request(app).get("/hello/John");
    expect(response.status).to.equal(200);
    expect(response.body).to.have.property("message");
    expect(response.body.message).to.equal("Hello John!");
  });
});

Encore.ts



import {describe, expect, test} from "vitest";
import {dynamicPathParamExample} from "../get-request-example";

// This test suite demonstrates how to test an Encore route.
// Run tests using the `encore test` command.
describe("Encore app", () => {
  test("should respond with a greeting message", async () => {
    // You can call the Encore.ts endpoint directly in your tests,
    // just like any other function.
    const resp = await dynamicPathParamExample({name: "world"});
    expect(resp.message).toBe("Hello world!");
  });
});

Database

Express.js does not have built-in database support. You can use libraries like pg-promise to connect to a PostgreSQL database but you also have to manage Docker Compose files for different environments.

With Encore.ts, you create a database by importing encore.dev/storage/sqldb and calling new SQLDatabase, assigning the result to a top-level variable.

Database schemas are defined by creating migration files. Each migration runs in order and expresses the change in the database schema from the previous migration.

Encore.ts automatically provisions databases to match what your application requires. Encore.ts provisions databases in an appropriate way depending on the environment. When running locally, Encore creates a database cluster using Docker. In the cloud, it depends on the environment type:

To query data, use the .query or .queryRow methods. To insert data, or to make database queries that don't return any rows, use .exec.

Learn more in our Database docs.

Encore.ts

db.ts



import {api} from "encore.dev/api";
import {SQLDatabase} from "encore.dev/storage/sqldb";

// Define a database named 'users', using the database migrations in the "./migrations" folder.
// Encore automatically provisions, migrates, and connects to the database.
export const DB = new SQLDatabase("users", {
  migrations: "./migrations",
});

interface User {
  name: string;
  id: number;
}

// Get one User from DB
export const getUser = api(
  {expose: true, method: "GET", path: "/user/:id"},
  async ({id}: { id: number }): Promise<{ user: User | null }> => {
    const user = await DB.queryRow<User>`
        SELECT name
        FROM users
        WHERE id = ${id}
    `;

    return {user};
  },
);

// Add User from DB
export const addUser = api(
  { expose: true, method: "POST", path: "/user" },
  async ({ name }: { name: string }): Promise<void> => {
    await DB.exec`
        INSERT INTO users (name)
        VALUES (${name})
    `;
    return;
  },
);

migrations/1_create_tables.up.sql



CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL UNIQUE
);

Logging

Express.js does not have built-in support for logging. You can use libraries like Winston to log messages.

Encore.ts offers built-in support for Structured Logging, which combines a free-form log message with structured and type-safe key-value pairs. Logging is integrated with the built-in Distributed Tracing functionality, and all logs are automatically included in the active trace. Learn more in our Logging docs.

Encore.ts



import log from "encore.dev/log";

log.error(err, "something went terribly wrong!");
log.info("log message", { is_subscriber: true });

Encore.ts — 17x Faster cold starts than NestJS & Fastify

Marcus Kohlberg for Encore ・ Sep 4

#node #javascript #typescript #backenddevelopment

How to let ChatGPT call functions in your app

Simon Johansson for Encore ・ Aug 14

#chatgpt #typescript #node #javascript

Encore.ts — 9x faster than Express.js & 3x faster than Bun + Zod

Marcus Kohlberg for Encore ・ Aug 2

#typescript #node #javascript #backenddevelopment

How to let ChatGPT call functions in your app

Simon Johansson — Wed, 14 Aug 2024 09:05:21 +0000

You can now give OpenAI access to your app’s APIs when answering prompts. This means, with just a few lines of code, you can let your users interact with your API using natural language prompts, instead of a complex chain of API calls.

In this article we’ll explain how it works, when it might be relevant to use, and walk through how to build a simple implementation using Encore.

How it works

In addition to sending a text prompt to the OpenAI API (used to interact with the LLM model powering ChatGPT), you can now also send in a list of functions defined in your system that it can use. The AI will automatically decide whether to call one or multiple functions in order to answer the prompt.

It’s worth calling out, the AI doesn't execute the functions itself. Instead the model simply generates parameters that can be used to call your function, which your code can then choose how to handle, likely by calling the indicated function. This is a good thing, as your application is always in full control.

The lifecycle flow of a function call:

When would you use it?

Fetching data from various services: an AI assistant can now answer questions like “what are my recent orders?” by fetching the latest customer data from an internal system.
Taking action: an AI assistant can now take actions on the users behalf like scheduling meetings by accessing a calendar API. Adding Siri-like behaviour to your own apps can now be done in a few lines of code.
Natural language API calls: Convert user queries expressed in natural language into a complex chain of API calls. Let the LLM create the SQL queries needed to fetch data from your own database.
Extract structured data: Creating a pipeline that fetches raw text, then converts it to structured data and saves it in a database is now super easy to assemble. No more need to copy & paste the same type of data into ChatGPT.

Let’s try it

In this simple example, we’re building a conversational assistant which can help users navigate a database of books. To make the assistant helpful, we want it to be able to look up books in our database.

We’ll be using Encore.ts to quickly create the endpoints needed to interact with our app, and the OpenAI API will use these endpoints to answer the user’s prompt.

You can see the full code for the example here: https://github.com/encoredev/examples/tree/main/ts/gpt-functions

1. Install Encore and create your app

To run the example yourself, first install Encore:

macOS: brew install encoredev/tap/encore
Linux: curl -L https://encore.dev/install.sh | bash
Windows: iwr https://encore.dev/install.ps1 | iex

Next, create a new Encore app with the example code using this command:



encore app create gpt-functions-example --example=ts/gpt-functions

You will also need an OpenAI API key and set it by running:



encore secret set --type dev,local,pr,prod OpenAIAPIKey

After that you are ready to run the application:



encore run

2. Create some functions

The starting point for function calling is choosing functions in your own codebase that you’d like to enable the model to generate arguments for.

For this example, we want to allow the model to call our list, get, and search endpoints:



import { api } from "encore.dev/api";

export interface Book {
  id: string;
  name: string;
  genre: Genre;
  description: string;
}

export enum Genre {
  mystery = "mystery",
  nonfiction = "nonfiction",
  memoir = "memoir",
  romance = "romance",
  historical = "historical",
}

// Using a hardcoded database for convenience of this example
const db: Book[] = [
  {
    id: "a1",
    name: "To Kill a Mockingbird",
    genre: Genre.historical,
    description: `Compassionate, dramatic, and deeply moving, "To Kill A Mockingbird" takes readers to the roots of human behavior - to innocence and experience, kindness and cruelty, love and hatred, humor and pathos. Now with over 18 million copies in print and translated into forty languages, this regional story by a young Alabama woman claims universal appeal. Harper Lee always considered her book to be a simple love story. Today it is regarded as a masterpiece of American literature.`,
  },
  ...
];

export const list = api(
  {expose: true, method: 'GET', path: '/book'},
  async ({genre}: { genre: string }): Promise<{ books: { name: string; id: string }[] }> => {
    const books = db.filter((item) => item.genre === genre).map((item) => ({name: item.name, id: item.id}));
    return {books}
  },
);

export const get = api(
  {expose: true, method: 'GET', path: '/book/:id'},
  async ({id}: { id: string }): Promise<{ book: Book }> => {
    const book = db.find((item) => item.id === id)!;
    return {book}
  },
);

export const search = api(
  {expose: true, method: 'GET', path: '/book/search'},
  async ({name}: { name: string }): Promise<{ books: { name: string; id: string }[] }> => {
    const books = db.filter((item) => item.name.includes(name)).map((item) => ({name: item.name, id: item.id}));
    return {books}
  },
);

Worth noting here is that these are endpoints we might have in a regular CRUD service, nothing in our business logic is adapted for an LLM.

3. Describe your function to the model

We can now create a “function definition” that describes the functions to the model. This definition describes both what the functions does (and when it should be called) and what parameters are required to call each function.

You can ensure that the model calls the correct functions by providing clear guidance in the system message, using intuitive function names and detailed descriptions for functions and parameters.



import {api} from "encore.dev/api";
import {book} from "~encore/clients";
import OpenAI from "openai";
import {secret} from "encore.dev/config";

// Getting API from from Encore secrets
const apiKey = secret("OpenAIAPIKey");

const openai = new OpenAI({
  apiKey: apiKey(),
});

// Encore endpoint the receives a text prompt as a query param and returns a message as a response
export const gpt = api(
  {expose: true, method: "GET", path: "/gpt"},
  async ({prompt}: { prompt: string }): Promise<{ message: string | null }> => {

    // Using the runTools method on the Chat Completions API
    const runner = openai.beta.chat.completions
      .runTools({
        model: "gpt-3.5-turbo",
        messages: [
          {
            role: "system",
            content:
              "Please use our book database, which you can access using functions to answer the following questions.",
          },
          {
            role: "user",
            content: prompt,
          },
        ],
        // The tools array contains the functions you are exposing
        tools: [
          {
            type: "function",
            function: {
              name: "list",
              strict: true,
              description:
                "list queries books by genre, and returns a list of names of books",
              // The model will use this information to generate
              // arguments according to your provided schema.  
              parameters: {
                type: "object",
                properties: {
                  genre: {
                    type: "string",
                    enum: [
                      "mystery",
                      "nonfiction",
                      "memoir",
                      "romance",
                      "historical",
                    ],
                  },
                },
                additionalProperties: false,
                required: ["genre"],
              },
              // Calling the list endpoint in our book service 
              function: async (args: { genre: string }) => await book.list(args),
              parse: JSON.parse,
            },
          },

          ...

        ],
      })

    // Get the final message and return the message content
    const result = await runner.finalChatCompletion();
    const message = result.choices[0].message.content;
    return {message};
  },
);

In the above code we define how and when the LLM should call our list endpoint. We specify that the endpoint takes one of five predefined genres as input and that the genre argument is required.

4. Calling the example

Let’s make use of Local Development Dashboard that comes with Encore and try calling our assistant with the prompt Recommend me a book that is similar to Kill a Mockingbird to see what it responds with.

The assistent responds with:

I recommend checking out the following historical books that are similar to "To Kill a Mockingbird":

All the Light We Cannot See

Where the Crawdads Sing

These books share some similarities with "To Kill a Mockingbird" and are worth exploring

This seams like a fitting answer because both those books are also present in our database and share the same genre as “To Kill a Mockingbird”.

Encore generated a trace for us when calling the gpt endpoint. In the trace details we can see exactly which of our functions were called by the LLM to generate the final response:

The LLM extracted “To Kill a Mockingbird” from the prompt and uses that to call the search endpoint.
The search endpoint returns the ID of “To Kill a Mockingbird”.
The LLM uses the ID to call the get endpoint.
The get endpoint returns the description and the genre of the book.
The LLM uses the genre to call the list endpoint.
The list endpoint returns a list of books in the requested genre.
The LLM recommends two books in the same genre as “To Kill a Mockingbird”.

The LLM called all three of our function in order to come up with the recommendation, pretty impressive!

Function calling with Structured Outputs

By default, when you use function calling, the API will offer best-effort matching for your parameters, which means that occasionally the model may miss parameters or get their types wrong when using complicated schemas.

In August 2024, OpenAI launched Structured Outputs. When you turn it on by setting strict: true in your function definition (see above code example). Structured Outputs ensures that the arguments generated by the model for a function call exactly match the JSON Schema you provided in the function definition.

Pitfalls with Function Calling

Some things to look out for when using function calling:

The LLM with read through the data that is returned from your functions to determine the next step (return a response or keep calling functions). If your function returns big payloads then you will potentially send a lot of money for each submitted prompt. One solution to this is to limit the data returned from your functions. You can also specify max_tokens allowed for each prompt, the model will then cut off the response if the limit is reached.
Contrary to what you might think, sending in a lot functions in a single tool call will reduce the model’s ability to select the correct tool. OpenAI recommends limiting the number of functions to no more than 20 in a single tool call.

Wrapping up

Now you know how to use OpenAIs function calling feature to connect ChatGPT to external tools.

⭐️ Support the project by starring Encore on GitHub.
Discover other ways to enhance your apps with AI, check out Encore's open source Templates repo.
If you have questions or want to share your work, join the developers hangout in Encore's community on Discord.

Setting up a free & production-ready web app backend in Go (with database) in less than 100 lines

Simon Johansson — Wed, 06 Sep 2023 11:21:11 +0000

https://github.com/encoredev/examples/tree/main/meeting-notes

In this tutorial, we will create a backend in Go using Encore in less than 100 lines of code. The backend will:

Store data in a cloud SQL database
Make API calls to a third-party service
Deploy to the cloud and be publicly available (free and production-ready)

Encore is a backend development platform that helps you get production-ready Go backends up and running in the cloud, very quickly. It's ideal for both hobby projects and startups as it abstracts away all of the cloud infrastructure nonsense, and comes with free backend deployments and hosting through Encore Cloud. (And if your product becomes a huge success, you can connect Encore to AWS or GCP and deploy your backend there.)

The example app we will build is a markdown meeting notes app BUT it’s trivial to replace the specifics if you have another idea in mind (again, less than 100 lines of code).

Demo version of the app

To develop with Encore, you need the Encore CLI. It compiles your code and runs your local development dashboard:



# Mac
$ brew install encoredev/tap/encore

# Win
$ iwr <https://encore.dev/install.ps1> | iex

# Linux
$ curl -L <https://encore.dev/install.sh> | bash

Create a new app from the meeting-notes example. This will start you off with everything described in this tutorial:



$ encore app create meeting-notes --example=github.com/encoredev/example-meeting-notes

You will be asked to create a free account. This will be needed later when we use Encore Could to deploy our app.

Before running the project locally, make sure you have Docker installed and running. Docker is needed for Encore to create databases for locally running projects. Also, if you want to try the photo search functionality then you will need an API key from pexels.com/api/ (more on that below)

To run the backend locally:



$ cd you-app-name # replace with the app name you picked
$ encore run

You should see the following:

That means your local development backend is up and running! Encore takes care of setting up all the necessary infrastructure for your application, including databases. Encore also starts the local development dashboard which is a tool to help you move faster when you're developing new features.

To start the front-end, run the following commands in another terminal window:



$ cd you-app-name/frontend
$ npm install
$ npm run dev

You can now open http://localhost:5173/example-meeting-notes/ in your browser 🔥

Storing and retrieving from an SQL database

Let's take a look at the backend code. There are essentially only three files of interest, let's start by looking at note.go. This file contains two endpoints and one interface, all standard Go code except for a few lines specific to Encore.

The Note type represents our data structure:



type Note struct {
    ID       string `json:"id"`
    Text     string `json:"text"`
    CoverURL string `json:"cover_url"`
}

Every note will have an ID (uuid that is created on the frontend), Text (Markdown text content), and CoverURL (background image URL).

The SaveNote function handles storing a meeting note:



//encore:api public method=POST path=/note
func SaveNote(ctx context.Context, note *Note) (*Note, error) {
    // Save the note to the database.
    // If the note already exists (i.e. CONFLICT), we update the notes text and the cover URL.
    _, err := sqldb.Exec(ctx, `
        INSERT INTO note (id, text, cover_url) VALUES ($1, $2, $3)
        ON CONFLICT (id) DO UPDATE SET text=$2, cover_url=$3
    `, note.ID, note.Text, note.CoverURL)

    // If there was an error saving to the database, then we return that error.
    if err != nil {
        return nil, err
    }

    // Otherwise, we return the note to indicate that the save was successful.
    return note, nil
}

The comment above the function tells Encore that this is a public endpoint that should be reachable by POST on /note. The second argument to the function (Note) is the POST body and the function returns a Note and an error (a nil error means a 200 response).

The GetNote function takes care of fetching a meeting note from our database given an id:



//encore:api public method=GET path=/note/:id
func GetNote(ctx context.Context, id string) (*Note, error) {
    note := &Note{ID: id}

    // We use the note ID to query the database for the note's text and cover URL.
    err := sqldb.QueryRow(ctx, `
        SELECT text, cover_url FROM note
        WHERE id = $1
    `, id).Scan(&note.Text, &note.CoverURL)

    // If the note doesn't exist, we return an error.
    if err != nil {
        return nil, err
    }

    // Otherwise, we return the note.
    return note, nil
}

Here we have a public GET endpoint with a dynamic path parameter which is the id of the meeting note to fetch. The second argument, in this case, is the dynamic path parameter, a request to this endpoint will look like /note/123-abc where id will be set to 123-abc.

Both SaveNote and GetNote makes use of a SQL database table named note, let's look at how that table is defined.

Defining a SQL database

To create a SQL database using Encore we first create a folder named migrations and inside that folder a migration file named 1_create_tables.up.sql. The file name is important (it must look something like 1_.up.sql). Our migration file is only five lines long and looks like this:



CREATE TABLE note (
    id TEXT PRIMARY KEY,
    text TEXT,
    cover_url TEXT
);

When recognizing this file, Encore will create a note table with three columns id, text and cover_url. The id is the primary key, used to identify specific meeting notes.

Making requests to a third-party API

Let's look at how we can use an Encore endpoint to proxy requests to a third-party service (in this example photo service pexels.com but the idea would be the same for any other third-party API).

The file pexels.go only has one endpoint, SearchPhoto:



//encore:api public method=GET path=/images/:query
func SearchPhoto(ctx context.Context, query string) (*SearchResponse, error) {
    // Create a new http client to proxy the request to the Pexels API.
    URL := "https://api.pexels.com/v1/search?query=" + query
    client := &http.Client{}
    req, _ := http.NewRequest("GET", URL, nil)

    // Add authorization header to the req with the API key.
    req.Header.Set("Authorization", secrets.PexelsApiKey)

    // Make the request, and close the response body when we're done.
    res, err := client.Do(req)
    if err != nil {
        return nil, err
    }
    defer res.Body.Close()

    if res.StatusCode >= 400 {
        return nil, fmt.Errorf("Pexels API error: %s", res.Status)
    }

    // Decode the data into the searchResponse struct.
    var searchResponse *SearchResponse
    err = json.NewDecoder(res.Body).Decode(&searchResponse)
    if err != nil {
        return nil, err
    }

    return searchResponse, nil
}

Again a GET endpoint with a dynamic path parameter which this time represents the query text we want to send to the Pexels API.

The type we use to decode the response from the Pexels API looks like this:



type SearchResponse struct {
    Photos []struct {
        Id  int `json:"id"`
        Src struct {
            Medium    string `json:"medium"`
            Landscape string `json:"landscape"`
        } `json:"src"`
        Alt string `json:"alt"`
    } `json:"photos"`
}

We get a lot more data from Pexels but here we only pick the fields that we want to propagate to our frontend.

Pexels API requires an API key, as most open APIs do. The API key is added as a header to the requests (from the SearchPhoto function above):



req.Header.Set("Authorization", secrets.PexelsApiKey)

Here we could have hardcoded the API key but that would have made it readable for everyone with access to our repo. Instead, we made use of Encore's built-in secrets management. To set this secret, run the following command in your project folder and follow the prompt:



encore secret set --type dev,prod,local,pr PexelsApiKey

Creating a request client

Encore is able to generate frontend request clients (TypeScript or JavaScript). This means that you do not need to manually keep the request/response objects in sync on the frontend, huge time saver. To generate a client run:



$ encore gen client <APP_NAME> --output=./src/client.ts --env=<ENV_NAME>

You are going to want to run this command quite often (whenever you make a change to your endpoints) so having it as an npm script is a good idea:



{
...
"scripts": {
    ...
    "generate-client:staging": "encore gen client <Encore app id here> --output=./src/client.ts --env=staging",
    "generate-client:local": "encore gen client <Encore app id here> --output=./src/client.ts --env=local"
  },
}

After that you are ready to use the request client in your code. Here is an example of calling the GetNote endpoint:



import Client, { Environment, Local } from "src/client.ts";

// Making request to locally running backend...
const client = new Client(Local);
// or to a specific deployed environment
const client = new Client(Environment("staging"));

// Calling APIs as typesafe functions 🌟
const response = await client.note.GetNote("note-uuid");
console.log(response.id);
console.log(response.cover_url);
console.log(response.text);

Deploying the backend to the cloud

It’s deploy time! To get your backend deployed in the cloud all you need to do is to commit your code and push it to the encore remote:



$ git add -A .
$ git commit -m 'Initial commit'
$ git push encore

When running git push encore you will get a link to the Encore Cloud dashboard where you can view the deploy for your app and after about a minute you have a backend running in the cloud ☁️

Hosting the frontend

The frontend can be deployed to any static site hosting platform. The example project is pre-configured to deploy the frontend to GitHub Pages. Take a look at .github/workflows/node.yml to see the GitHub actions workflow being triggered on new commits to the repo:



name: Build and Deploy

on: [push]

permissions:
  contents: write

jobs:
  build-and-deploy:
    concurrency: ci-${{ github.ref }}
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: frontend

    steps:
      - name: Checkout 🛎️
        uses: actions/checkout@v3

      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '16.15.1'

      - name: Install and Build 🔧
        run: |
          npm install
          npm run build

      - name: Deploy 🚀
        uses: JamesIves/github-pages-deploy-action@v4.3.3
        with:
          branch: gh-pages
          folder: frontend/dist

The interesting part is towards the bottom where we build the frontend code and make use of the github-pages-deploy-action step to automatically make a new commit with the compiled frontend code to a gh-pages branch.

Steps to deploy to GitHub pages:

Create a repo on GitHub
In the vite.config.js file, set the base property to the name of your repo:



base: "/my-repo-name/",

Push your code to GitHub and wait for the GitHub actions workflow to finish.
Go to Settings → Pages for your repo on GitHub and set Branch to gh-pages.

Wrapping up

✨ Great job! You now have a complete app running in the cloud! ✨

You’ve learned how to build and deploy a Go backend using Encore, store data in an SQL database, and make API calls to an external service.

Encore lets you build production-ready backends with none of the boilerplate and manual work involved in setting up real cloud infrastructure. If you want to continue building more advanced applications, I recommend checking out the other Encore tutorials available.🔥

Type-safe API schemas	CORS handling	Structured logging	Authentication
Pub/Sub integrations	Secrets management	Infrastructure integrations	Database integrations
Architecture Diagrams	Local Development Dashboard	Service Catalog	TypeScript native
Debugging	Error handling	Multithreading	Request validation
Tracing	API Client generation	Automatic local infrastructure	Automated testing

Type-safe API schemas	CORS handling	Structured logging	Authentication
Pub/Sub integrations	Secrets management	Infrastructure integrations	Database integrations
Architecture Diagrams	Local Development Dashboard	Service Catalog	TypeScript native
Debugging	Error handling	Multithreading	Request validation
Tracing	API Client generation	Automatic local infrastructure	Automated testing

DEV Community: Simon Johansson

Next.js vs Encore.ts — When should you NOT use Next.js for Backend?

Encore.ts vs Next.js

Comparison

Next.js

Encore.ts

Using Next.js with Encore.ts

Code example

Install Encore

Clone the example app

Generating a request client

Deployment

Self-hosting

Encore Cloud Platform

Next.js on Vercel

Conclusion

Related articles

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

Building and Deploying TypeScript Microservices to Kubernetes

Marcus Kohlberg for Encore ・ Dec 17

Building simple event-driven applications with Pub/Sub

Marcus Kohlberg for Encore ・ Nov 19

How to use ORMs (Prisma / Drizzle / Knex.js) in a TypeScript backend built with Encore.ts

Using a Database with Encore

ORM

Prisma

Drizzle

Knex

Deploy

Running the examples yourself

Installation

Cloning the ORM examples

Wrapping up

Other related posts

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

Building simple event-driven applications with Pub/Sub

Marcus Kohlberg for Encore ・ Nov 19

TypeScript vs Go: Choosing Your Backend Language

Marcus Kohlberg for Encore ・ Nov 8

Building Type-Safe Event-Driven Applications in TypeScript using Pub/Sub, Cron Jobs, and PostgreSQL

The application

Architecture

Benefits

Pitfalls

Adding our Pub/Sub topic

Adding the subscriber

Local infrastructure

Local Development Dashboard

Deployment

Running the Uptime application yourself

Wrapping up

Other related posts

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Simon Johansson for Encore ・ Nov 1

How to deploy a backend application to DigitalOcean using Docker and Encore

Marcus Kohlberg for Encore ・ Oct 23

TypeScript vs Go: Choosing Your Backend Language

Marcus Kohlberg for Encore ・ Nov 8

Node.js Frameworks Roundup 2024 — Elysia / Hono / Nest / Encore — Which should you pick?

Feature overview

Encore.ts

Hono

Elysia

Nest.js

Performance

Code structure

Deployment & Infrastructure

Wrapping up

NestJS vs Encore.ts: Choosing the Right Framework for Your TypeScript Microservices

Marcus Kohlberg for Encore ・ Oct 30

How to deploy a backend application to DigitalOcean using Docker and Encore

Marcus Kohlberg for Encore ・ Oct 23

Choosing the Right Tech Stack: A Developer's Decision-Making Guide

Lucas Lima do Nascimento for Encore ・ Oct 28

Encore.ts — 9x faster than Express.js (feature comparison and migration guide)

Other related articles

How to make your Express.js APIs 9x faster with Encore.ts

Simon Johansson for Encore ・ Sep 10