Building a web app backend in Go (with PostgreSQL database) in <100 lines

TL;DR

💡 In this guide we will create and deploy a Go backend in less than 100 lines of code. The backend will:

• Power a markdown meeting notes app
• Store data in a cloud PostgreSQL database
• Make API calls to a third-party service

👉 See a demo version of the app here.

We will be using Encore to build our backend, it lets us declare infrastructure as objects in our Go program and then automatically deploy to the cloud.🚀

🏁 Let's go!

To make it easier to follow along, we've laid out a trail of croissants to guide your way.

Whenever you see a 🥐 it means there's something for you to do!

💽 Install Encore

Install the Encore CLI to run your local environment:

• macOS: brew install encoredev/tap/encore
• Linux: curl -L https://encore.dev/install.sh | bash
• Windows: iwr https://encore.dev/install.ps1 | iex

Create your Encore application

🥐 Create a new app from the meeting-notes example. This will start you off with everything described in this tutorial:

encore app create meeting-notes --example=github.com/encoredev/example-meeting-notes

Note: Before running the project locally, make sure you have Docker installed and running. Docker is needed for Encore to create databases for locally running projects. Also, if you want to try the photo search functionality then you will need an API key from pexels.com/api/ (more on that below)

🏁 Run your backend locally

🥐 Let’s see if it works! Start your app by running encore run.

You should see this:

You'll also see the Local Dev Dashboard (localhost:9400) open in a new tab. It gives you access to Encore's API explorer, Local tracing, architecture diagrams, and Service Catalog.

🖼 Run the frontend

🥐 To start the frontend, run the following commands in another terminal window:

cd you-app-name/frontend
npm install
npm run dev

🥐 You can now open http://localhost:5173/example-meeting-notes/ in your browser!🔥

🤔 How it works: Defining a SQL database

To create a SQL database using Encore we have created a folder named migrations and inside that folder a migration file named 1_create_tables.up.sql. (The file name is important it must look something like 1_name.up.sql).

Our migration file is only five lines long and looks like this:

CREATE TABLE note (
    id TEXT PRIMARY KEY,
    text TEXT,
    cover_url TEXT
);

When recognizing this file, Encore will create a note table with three columns id, text and cover_url. The id is the primary key, used to identify specific meeting notes.

With this code in place Encore will automatically create the database when starting encore run (locally) or on the next deployment (in the cloud).

Encore automatically injects the appropriate configuration to authenticate and connect to the database, so once the application starts up the database is ready to be used.

🤔 How it works: Storing and retrieving data from the PostgreSQL database

Let's take a look at the backend code. There are essentially only three files of interest, let's start by looking at note.go. This file contains two endpoints and one interface, all standard Go code except for a few lines specific to Encore.

The Note type represents our data structure:

type Note struct {
    ID       string `json:"id"`
    Text     string `json:"text"`
    CoverURL string `json:"cover_url"`
}

Every note will have an ID (uuid that is created on the frontend), Text (Markdown text content), and CoverURL (background image URL).

The SaveNote function handles storing a meeting note:

//encore:api public method=POST path=/note
func SaveNote(ctx context.Context, note *Note) (*Note, error) {
    // Save the note to the database.
    // If the note already exists (i.e. CONFLICT), we update the notes text and the cover URL.
    _, err := sqldb.Exec(ctx, `
        INSERT INTO note (id, text, cover_url) VALUES ($1, $2, $3)
        ON CONFLICT (id) DO UPDATE SET text=$2, cover_url=$3
    `, note.ID, note.Text, note.CoverURL)

    // If there was an error saving to the database, then we return that error.
    if err != nil {
        return nil, err
    }

    // Otherwise, we return the note to indicate that the save was successful.
    return note, nil
}

The comment above the function tells Encore that this is a public endpoint that should be reachable by POST on /note. The second argument to the function (Note) is the POST body and the function returns a Note and an error (a nil error means a 200 response).

The GetNote function takes care of fetching a meeting note from our database given an id:

//encore:api public method=GET path=/note/:id
func GetNote(ctx context.Context, id string) (*Note, error) {
    note := &Note{ID: id}

    // We use the note ID to query the database for the note's text and cover URL.
    err := sqldb.QueryRow(ctx, `
        SELECT text, cover_url FROM note
        WHERE id = $1
    `, id).Scan(&note.Text, &note.CoverURL)

    // If the note doesn't exist, we return an error.
    if err != nil {
        return nil, err
    }

    // Otherwise, we return the note.
    return note, nil
}

Here we have a public GET endpoint with a dynamic path parameter which is the id of the meeting note to fetch. The second argument, in this case, is the dynamic path parameter, a request to this endpoint will look like /note/123-abc where id will be set to 123-abc.

Both SaveNote and GetNote makes use of a SQL database table named note, let's look at how that table is defined.

📞 Making requests to a third-party API

Let's look at how we can use an Encore endpoint to proxy requests to a third-party service (in this example photo service pexels.com but the idea would be the same for any other third-party API).

The file pexels.go only has one endpoint, SearchPhoto:

//encore:api public method=GET path=/images/:query
func SearchPhoto(ctx context.Context, query string) (*SearchResponse, error) {
    // Create a new http client to proxy the request to the Pexels API.
    URL := "https://api.pexels.com/v1/search?query=" + query
    client := &http.Client{}
    req, _ := http.NewRequest("GET", URL, nil)

    // Add authorization header to the req with the API key.
    req.Header.Set("Authorization", secrets.PexelsApiKey)

    // Make the request, and close the response body when we're done.
    res, err := client.Do(req)
    if err != nil {
        return nil, err
    }
    defer res.Body.Close()

    if res.StatusCode >= 400 {
        return nil, fmt.Errorf("Pexels API error: %s", res.Status)
    }

    // Decode the data into the searchResponse struct.
    var searchResponse *SearchResponse
    err = json.NewDecoder(res.Body).Decode(&searchResponse)
    if err != nil {
        return nil, err
    }

    return searchResponse, nil
}

Again a GET endpoint with a dynamic path parameter which this time represents the query text we want to send to the Pexels API.

The type we use to decode the response from the Pexels API looks like this:

type SearchResponse struct {
    Photos []struct {
        Id  int `json:"id"`
        Src struct {
            Medium    string `json:"medium"`
            Landscape string `json:"landscape"`
        } `json:"src"`
        Alt string `json:"alt"`
    } `json:"photos"`
}

We get a lot more data from Pexels but here we only pick the fields that we want to propagate to our frontend.

Pexels API requires an API key, as most open APIs do. The API key is added as a header to the requests (from the SearchPhoto function above):

req.Header.Set("Authorization", secrets.PexelsApiKey)

Here we could have hardcoded the API key but that would have made it readable for everyone with access to our repo.

🔐 Instead, we made use of Encore's built-in secrets management.

🥐 To set this secret, run the following command in your project folder and follow the prompt:

encore secret set --type dev,prod,local,pr PexelsApiKey

📱 Creating a request client

Encore is able to generate frontend request clients (Go, TypeScript, JavaScript, and OpenAPI spec). This means that you do not need to manually keep the request/response objects in sync on the frontend, huge time saver.

🥐 To generate a client run:

encore gen client <APP_NAME> --output=./src/client.ts --env=<ENV_NAME>

You are going to want to run this command quite often (whenever you make a change to your endpoints) so having it as an npm script is a good idea:

{
...
"scripts": {
    ...
    "generate-client:staging": "encore gen client <Encore app id here> --output=./src/client.ts --env=staging",
    "generate-client:local": "encore gen client <Encore app id here> --output=./src/client.ts --env=local"
  },
}

After that you are ready to use the request client in your code. Here is an example of calling the GetNote endpoint:

import Client, { Environment, Local } from "src/client.ts";

// Making request to locally running backend...
const client = new Client(Local);
// or to a specific deployed environment
const client = new Client(Environment("staging"));

// Calling APIs as typesafe functions 🌟
const response = await client.note.GetNote("note-uuid");
console.log(response.id);
console.log(response.cover_url);
console.log(response.text);

🚀 Deploying the backend to the cloud

🥐 It’s deploy time! To get your backend deployed in the cloud all you need to do is to commit your code and push it to the encore remote:

$ git add -A .
$ git commit -m 'Initial commit'
$ git push encore

Encore will now build and test your app, provision the needed infrastructure, and deploy your application to the cloud.

After triggering the deployment, you will see a URL where you can view its progress in Encore's Cloud Dashboard.👈

It will look something like: https://app.encore.dev/$APP_ID/deploys/...

From there you can also see metrics, traces, and connect your own AWS or GCP account to use for production deployment.

🧑‍💻 Hosting the frontend

The frontend can be deployed to any static site hosting platform. The example project is pre-configured to deploy the frontend to GitHub Pages.

Take a look at .github/workflows/node.yml to see the GitHub actions workflow being triggered on new commits to the repo:

name: Build and Deploy

on: [push]

permissions:
  contents: write

jobs:
  build-and-deploy:
    concurrency: ci-${{ github.ref }}
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: frontend

    steps:
      - name: Checkout 🛎️
        uses: actions/checkout@v3

      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: "16.15.1"

      - name: Install and Build 🔧
        run: |
          npm install
          npm run build

      - name: Deploy 🚀
        uses: JamesIves/github-pages-deploy-action@v4.3.3
        with:
          branch: gh-pages
          folder: frontend/dist

The interesting part is towards the bottom where we build the frontend code and make use of the github-pages-deploy-action step to automatically make a new commit with the compiled frontend code to a gh-pages branch.

🥐 Deploy to GitHub pages

1. Create a repo on GitHub
2. In the vite.config.js file, set the base property to the name of your repo:

base: "/my-repo-name/",

1. Push your code to GitHub and wait for the GitHub actions workflow to finish.
2. Go to Settings → Pages for your repo on GitHub and set Branch to gh-pages.

🎁 Wrapping up

You’ve learned how to build and deploy a Go backend using Encore, store data in an SQL database, and make API calls to an external service. All of this in under 100 lines of code!✨

🎉 Great job - you're done!

You now have the start of a scalable Go backend app running in the cloud, complete with PostgreSQL database.

Keep building with these Open Source App Templates.👈

If you have questions or want to share your work, join the developer hangout in Encore's community Slack.👈

Comments

[Hoang Trinh]

I don't see this part in the generated code

package todo

// Create the todo database and assign it to the "tododb" variable
var tododb = sqldb.NewDatabase("todo", sqldb.DatabaseConfig{
    Migrations: "./migrations",
})

Where should I add it?

[Marcus Kohlberg]

Great question. There are two ways of defining databases with Encore, in the example program used in the tutorial it is not explicitly named, rather it is inferred from the migration file. In this example you commented, it uses the other method which is by creating a named object. Either works fine, but that is the reason you aren't seeing the sqldb.NewDatabase in the example program.

For more details see the full docs: encore.dev/docs/primitives/databas...

I will update the tutorial to be more clear!