DEV Community

Cover image for Create Production-Ready SDKs With gRPC Gateway
Speakeasy profile image Tatiana Caciur
Tatiana Caciur for Speakeasy

Posted on

Create Production-Ready SDKs With gRPC Gateway

#openapi #api

You may want to provide a RESTful API in addition to your gRPC service without the need to duplicate your code.

gRPC Gateway is a popular tool for generating RESTful APIs from gRPC service definitions.

In this tutorial, we'll take a detailed look at how to use gRPC Gateway to generate an OpenAPI schema based on a Protocol Buffers (protobuf) gRPC service definition. Then we'll use Speakeasy to read our generated OpenAPI schema and create a production-ready SDK.

If you want to follow along, you can use the gRPC Speakeasy Bar example repository.

An Overview of gRPC Gateway

gRPC Gateway is a protoc plugin that reads gRPC service definitions and generates a reverse proxy server that translates a RESTful JSON API into gRPC.

This way, you can expose an HTTP endpoint that can be called by clients that don't support gRPC. The generated server code will forward incoming JSON requests to your gRPC server and translate the responses to JSON.

gRPC Gateway also generates an OpenAPI schema that describes your API. You can use this schema to create SDKs for your API.

OpenAPI Versions

gRPC Gateway outputs OpenAPI 2.0, and Speakeasy supports OpenAPI 3.0 and 3.1. To generate an OpenAPI 3.0 or 3.1 schema, you'll need to convert the OpenAPI 2.0 schema to at least OpenAPI 3.0.

The Protobuf to REST SDK Pipeline

To generate a REST API with a developer-friendly SDK, we'll follow these three core steps:

  1. gRPC to OpenAPI: First, we will use gRPC Gateway to produce an OpenAPI schema based on our protobuf service definition. This generated schema is in OpenAPI 2.0 format.

  2. OpenAPI 2.0 to OpenAPI 3.x: Next, as gRPC Gateway's output schema is in OpenAPI 2.0 and we need at least OpenAPI 3.0 for our SDK, we will convert the generated schema from OpenAPI 2.0 to OpenAPI 3.0.

  3. OpenAPI 3.x to SDK: Finally, once we have the OpenAPI 3.0 schema, we will leverage Speakeasy to create our SDK based on the OpenAPI 3.0 schema derived from the previous steps.

By following these steps, we can ensure we have a robust, production-ready SDK that adheres to our API's specifications.

Step-by-Step Tutorial: From Protobuf to OpenAPI to an SDK

Now let's walk through generating an OpenAPI schema and SDK for our Speakeasy Bar gRPC service.

Check Out the Example Repository

If you would like to follow along, start by cloning the example repository:

git clone git@github.com:speakeasy-api/speakeasy-grpc-gateway-example.git
cd speakeasy-grpc-gateway-example
Enter fullscreen mode Exit fullscreen mode

Install Go

To generate an OpenAPI schema from a protobuf file, we'll need to install Go and protoc.

This tutorial was written using Go 1.21.4.

On macOS, install Go by running:

brew install go
Enter fullscreen mode Exit fullscreen mode

Alternatively, follow the Go installation instructions for your platform.

Install Buf

We'll use the Buf CLI as an alternative to protoc so that we can save our generation configuration as YAML. Buf is compatible with protoc plugins.

On macOS, install Buf by running:

brew install bufbuild/buf/buf
Enter fullscreen mode Exit fullscreen mode

Alternatively, follow the Buf CLI installation instructions for your platform.

Install Buf Modules

We'll use Buf modules to manage our dependencies.

cd proto
buf mod update
cd ..
Enter fullscreen mode Exit fullscreen mode

Install protoc-gen-go

Buf requires the protoc-gen-go plugin to generate Go code from protobuf files.

Install protoc-gen-go by running:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
Enter fullscreen mode Exit fullscreen mode

Make sure that the protoc-gen-go binary is in your $PATH. On macOS, you can achieve that by running the following command if the go/bin directory is not already in your path.

export PATH=${PATH}:`go env GOPATH`/bin
Enter fullscreen mode Exit fullscreen mode

Install Go Requirements 

go mod tidy
go install \
    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \
    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \
    google.golang.org/protobuf/cmd/protoc-gen-go \
    google.golang.org/grpc/cmd/protoc-gen-go-grpc
Enter fullscreen mode Exit fullscreen mode

Generate the Go Code

We'll use Buf to generate the Go code from the protobuf file.

Run the following in the terminal:

buf generate
Enter fullscreen mode Exit fullscreen mode

Buf reads the configuration in buf.gen.yaml, then generates the Go code in the proto directory.

This will generate the proto/speakeasy/v1/speakeasy.pb.go, proto/speakeasy/v1/speakeasy_grpc.pb.go, and proto/speakeasy/v1/speakeasy.pb.gw.go files.

Generate the OpenAPI Schema

Because we have the openapiv2 protoc plugin configured in our buf.gen.yaml file, Buf will generate an OpenAPI schema and save it as openapi/speakeasy/v1/speakeasy.swagger.json.

This is the OpenAPI 2.0 schema that gRPC Gateway generates by default.

Convert the OpenAPI Schema to OpenAPI 3.0

We'll use the excellent kin-openapi Go library to convert the OpenAPI 2.0 schema to OpenAPI 3.0.

In convert/convert.go, we use kin-openapi to unmarshal openapi/speakeasy/v1/speakeasy.swagger.json, then convert it to OpenAPI 3.0, then marshal it back to JSON, and finally write it to openapi/speakeasy/v1/speakeasy.openapi.json.

To do the conversion, run the following in the terminal:

go run convert/convert.go
Enter fullscreen mode Exit fullscreen mode

Create an SDK With Speakeasy

Now that we have an OpenAPI 3.0 schema, we can create an SDK with Speakeasy.

We'll use the speakeasy generate command to create an SDK for the Speakeasy Bar gRPC service.

Run the following in the terminal:

speakeasy generate sdk \
    --schema openapi/speakeasy/v1/speakeasy.openapi.json \
    --lang typescript \
    --out ./sdk
Enter fullscreen mode Exit fullscreen mode

This will create a TypeScript SDK in the sdk directory.

How To Customize the SDK

By modifying the protobuf service definition, we can customize the SDK via the generated OpenAPI schema.

We'll start with a basic example and add options to enhance the SDK.

Meet the Speakeasy Bar Protobuf Service

We'll start by taking a look at the Speakeasy Bar protobuf service definition in proto/speakeasy/v1/speakeasy.proto.

// from ./grpc-assets/speakeasy-raw.proto
Enter fullscreen mode Exit fullscreen mode

The service defines one object type, called Drink.

// from ./grpc-assets/speakeasy-raw.proto
Enter fullscreen mode Exit fullscreen mode

A service called SpeakeasyService has two methods, GetDrink and ListDrinks.

// from ./grpc-assets/speakeasy-raw.proto
Enter fullscreen mode Exit fullscreen mode

Add API Information to the Service

We'll add information about the API to the service definition using options.openapiv2_swagger from grpc.gateway.protoc_gen_openapiv2.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode

We'll add a title, description, and version to the API.

This appears in the info object in the generated OpenAPI schema.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Speakeasy requires a servers object in the OpenAPI schema. We'll add a server to the API using the host key.

Our conversion script will add the servers object to the generated OpenAPI schema.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Add Descriptions and Examples to Components

To create an SDK that offers great developer experience, we recommend adding descriptions and examples to all fields in OpenAPI components.

Speakeasy will create documentation and usage examples based on the descriptions and examples we added.

We'll start with the Drink object type.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode

We added a title, description, and example to the Drink object type.

Note that the example is a stringified JSON object.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode

We use openapiv2_field to add options to the fields in the Drink object type.

For example, we added a description, pattern, format, and example to the productCode field.

This description and example will appear in the generated OpenAPI document.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

When Speakeasy creates an SDK, this description and example will appear in the generated documentation and usage examples.

This usage example is from the TypeScript SDK's documentation.

Note how the productCode field is represented by our UUID example instead of a random string.

import { SDK } from "openapi";

(async() => {
  const sdk = new SDK();

  const res = await sdk.drinks.getDrink({
    productCode: "602a7da9-b8bb-46e6-b288-457b561029b8",
  });

  if (res.statusCode == 200) {
    // handle response
  }
})();
Enter fullscreen mode Exit fullscreen mode

Customize the OperationId

Speakeasy uses the operationId for each operation in the OpenAPI document to create method names in the SDK. By default, the operationId is the method name in the protobuf service definition.

We can customize the operationId for each method using options.openapiv2_operation.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Add Descriptions and Tags to Methods

Speakeasy uses the description and tags in the OpenAPI document to create documentation for each method in the SDK and group methods in the SDK.

We can add descriptions and tags to methods using options.openapiv2_operation.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Add OpenAPI Extensions

gRPC Gateway allows us to add OpenAPI extensions to the OpenAPI schema using the extensions key in our protobuf service definition.

For example, we can add the Speakeasy retries extension x-speakeasy-retries, which will cause the SDK to retry failed requests.

In the code example, we added the x-speakeasy-retries extension to the GetDrink method.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Add Tag Descriptions

Speakeasy uses each tag's description in the OpenAPI document to create documentation in the SDK.

We can add descriptions to tags in the protobuf definition by using options.openapiv2_swagger.

In the code example, we added a description to the drinks tag.

// from ./grpc-assets/speakeasy.proto
Enter fullscreen mode Exit fullscreen mode 
// from ./grpc-assets/speakeasy.openapi.json
Enter fullscreen mode Exit fullscreen mode

Example Protobuf Definition and SDK Generator

The source code for our complete example is available in the gRPC Speakeasy Bar example repository.

The repository contains a TypeScript SDK and instructions on how to create more SDKs.

You can clone this repository to test how changes to the protobuf definition result in changes to the SDK.

After modifying your protobuf definition, you can run the following in the terminal to create a new SDK:

buf generate && go run convert/convert.go && speakeasy generate sdk \
    --schema openapi/speakeasy/v1/speakeasy.openapi.json \
    --lang typescript \
    --out ./sdk
Enter fullscreen mode Exit fullscreen mode

Happy generating!

Top comments (0)

Read next

creuserr profile image

📨 Building your Email API with Fastmail for free!

creuser -

kmsaifullah profile image

How to Fetch Data in React JS

Khaled Md Saifullah -

ldsrogan profile image

Improving UX by using cache

Shin-Young Jung -

syncloop_dev profile image

How to Generate and Consume APIs: A Deep Dive into Swagger Codegen and OpenAPI Generator

Syncloop -