Getting started with twirp 🦜

Twirp is a library that has all the strictly typed power of gRPC but retains the flexibility of HTTP/JSON for communication.

You define your routes, requests, and responses in one *.proto protobuf file.

Then all consumers of your API can import the protobuf file and generate their own client library in their language of choice.

Let's set up a quick and example.

First things first, you'll need to install protobuf, so on mac, start with

brew install protobuf

With that installed, you'll need to make sure you have golang setup as well as GOBIN defined; These are some reasonable defaults to add to your .zshrc:

GOBIN="~/go/bin"
PATH="$PATH:~/go/bin"

Next you'll need to install the protobuf binary plugins to generate the files for both twirp and golang;

go install github.com/twitchtv/twirp/protoc-gen-twirp@latest
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

I'm building a project called "ProxyMe" so I'll use it as an example; Let's start with a simple protobuf file to define the routes I'll start with;

syntax = "proto3";
package reaz.io.proxyme;
option go_package = "reaz.io/proxyme";

service ProxyMe {
  rpc ListProxies(ListReq) returns (ListRes);
}

message ListReq {
  string subject = 1;
}

message ListRes {
  string text = 1;
}

What's important to note is;

# We'll need this later, this is just a namespace for my projects, yours might be more like '{username}.com.{project_name}'
package reaz.io.proxyme;

Then the rest of the file goes onto describe a single ListProxies endpoint that accepts a ListReq (List request) and returns a ListReq List Response.

Now let's use twirp and protobuf to automatically scaffold some code for us;

protoc --go_out=. --twirp_out=. ProxyMe.proto

At this point, we'll have something like

├── ProxyMe.proto
└── reaz.io
    └── proxyme
        ├── ProxyMe.pb.go
        └── ProxyMe.twirp.go

2 directories, 3 files

The original file we wrote, ProxyMe.proto, and the namespace we used pointing to a .pb.go which is the go generated portion and *.twirp.go is the twirp magic that contains a HTTP service.

With that, let's make a server.go to actually implement the endpoint:

package main

import (
    "context"
    "log"
    "net/http"

    pb "proxyme/reaz.io/proxyme"
)

type ProxyMeServer struct{}

func (s *ProxyMeServer) ListProxies(ctx context.Context, req *pb.ListReq) (*pb.ListRes, error) {
    return &pb.ListRes{Text: "Hello " + req.Subject}, nil
}

const PORT = "8011"

// Run the implementation in a local server
func main() {
    twirpHandler := pb.NewProxyMeServer(&ProxyMeServer{})
    // You can use any mux you like - NewHelloWorldServer gives you an http.Handler.
    mux := http.NewServeMux()
    // The generated code includes a method, PathPrefix(), which
    // can be used to mount your service on a mux.
    mux.Handle(twirpHandler.PathPrefix(), twirpHandler)
    log.Println("Listening on http://0.0.0.0:" + PORT + twirpHandler.PathPrefix())
    err := http.ListenAndServe(":"+PORT, mux)
    if err != nil {
        log.Fatal(err)
        return
    }
}

That's it, we're mostly importing our generated code, adding a ListProxies that uses our ListReq, ListRes.

Now to run it with the following:

$ go run server.go
2022/04/18 07:07:16 Listening on http://0.0.0.0:8011/twirp/reaz.io.proxyme.ProxyMe/

With it running, we can test it using the following

curl --request POST \
  --url http://0.0.0.0:8011/twirp/reaz.io.proxyme.ProxyMe/ListProxies \
  --header 'Content-Type: application/json' \
  --data '{
    "subject": "mike"
}'

# Should see the following for a response
{
  "text": "Hello mike"
}

You can now just generate client code in other languages and use the new service!

Clients in other languages can also be generated by using the respective protoc plugins defined by their languages, for example --twirp_ruby_out.

And can use protobuf or json, all generated from one protobuf file!

Learn more about how to generate client code here.