Junerey Casuga

Posted on Nov 29, 2020

Go + gRPC with Go kit

#microservices #go #tutorial

Building applications with Go can be sometimes intimidating especially when you’re just starting to learn it. Let alone transporting your APIs through gRPC. In this post, I will guide you on writing a simple Go microservice using gRPC with the help of Go kit.

What is Go kit? and why?

Go kit is the brain child of Peter Bourgon which he talked about during a FOSDEM and the Google Campus London meetup in 2015. As what he has envisioned, Go kit is a toolkit for building microservices in Go.

It provides you a set of standard libraries (packages) that are considered essentials when building microservices so you can focus on what’s important, your application’s business logic. Here is a list of packages Go kit provides

Authentication (Basic/Casbin/JWT)
Circuit Breaker (Hystrix/GoBreaker/HandyBreaker)
Logging: Provides an interface for structured logging.
Metrics: Provides an interface for service instrumentation. Has adapters for CloudWatch, Prometheus, Graphite, and more.
Rate Limiting
Service Discovery Utilities
Tracing (OpenCensus, OpenTracing, Zipkin)
Transport (HTTP/gRPC/ NATS/AWS Lambda, and more)

Other than being a toolkit, it also encourages good design principles for your application such as SOLID design principles, Domain-Driven Design (DDD), and Hexagonal architecture.

Go kit's concept

Go kit's concept is based on three major components:

Services
Endpoints
Transports

Services (Business Logic)

This is where the core business logic of your API is located. In Go kit, each service method are converted into an Endpoint.

Endpoints

An endpoint represents a single RPC method. Each Service method converts into an Endpoint to make RPC style communication between servers and clients.

Transports

In a microservice architecture, a microservice often communicate through transports such as HTTP or gRPC. A single endpoint can be exposed by multiple transports.

Let's build our microservice

For our example microservice, we will build a very simple function that would get the sum of two numbers. Before we get started, make sure that you have Protocol buffer and Go plugins for the protocol compiler installed.

To give you an overview what our project structure would look like by the end of this article, you can refer to the directory tree below.



.
├── cmd
│   └── main.go         # main entrypoint file          
├── endpoints
│   └── endpoints.go    # contains the endpoint definition
├── pb
│   ├── math.pb.go      # our gRPC generated code
│   └── math.proto      # our protobuf definitions
├── service
│   └── api.go          # contains the service's core business logic
└── transports
    └── grpc.go         # contains the gRPC transport

Protobuf

First of, we would need to create our Protobuf file inside our pb directory. Since you’re reading this, I would assume that you already know the basics of Protobuf; and the code snippet below should be pretty straightforward to understand 😄



syntax = "proto3";

option go_package = "github.com/junereycasuga/gokit-grpc-demo/pb";

service MathService {
  rpc Add(MathRequest) returns (MathResponse) {}
}

message MathRequest {
  float numA = 1;
  float numB = 2;
}

message MathResponse {
  float result = 1;
}

Once we have our Protobuf file set, we would then need to generate our gRPC code using the following command.



protoc --go_out=plugins=grpc:. pb/math.proto

Service

Next up, we will create the core business logic of our microservice. So in service/api.go , we would add our service definition by using the following code below.



package service

import (
    "context"

    "github.com/go-kit/kit/log"
)

type service struct {
    logger log.Logger
}

// Service interface describes a service that adds numbers
type Service interface {
    Add(ctx context.Context, numA, numB float32) (float32, error)
}

// NewService returns a Service with all of the expected dependencies
func NewService(logger log.Logger) Service {
    return &service{
        logger: logger,
    }
}

// Add func implements Service interface
func (s service) Add(ctx context.Context, numA, numB float32) (float32, error) {
    return numA + numB, nil
}

Endpoint



package endpoints

import (
    "context"

    "github.com/go-kit/kit/endpoint"
    "github.com/junereycasuga/gokit-grpc-demo/service"
)

// Endpoints struct holds the list of endpoints definition
type Endpoints struct {
    Add endpoint.Endpoint
}

// MathReq struct holds the endpoint request definition
type MathReq struct {
    NumA float32
    NumB float32
}

// MathResp struct holds the endpoint response definition
type MathResp struct {
    Result float32
}

// MakeEndpoints func initializes the Endpoint instances
func MakeEndpoints(s service.Service) Endpoints {
    return Endpoints{
        Add: makeAddEndpoint(s),
    }
}

func makeAddEndpoint(s service.Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (response interface{}, err error) {
        req := request.(MathReq)
        result, _ := s.Add(ctx, req.NumA, req.NumB)
        return MathResp{Result: result}, nil
    }
}

From the code above for endpoints/endpoints.go, we have the Endpoints struct that serves as a helper struct to collect all of the endpoints into a single parameter.

The MathReq and MathResp simply defines the request parameters and response values for the Add method.

makeAddEndpoint func constructs our Add endpoint wrapping the service.

And finally, MakeEndpoints func returns an Endpoints that wraps the provided server and wires in all of the expected endpoints.

Transport



package transport

import (
    "context"

    "github.com/go-kit/kit/log"
    gt "github.com/go-kit/kit/transport/grpc"
    "github.com/junereycasuga/gokit-grpc-demo/endpoints"
    "github.com/junereycasuga/gokit-grpc-demo/pb"
)

type gRPCServer struct {
    add gt.Handler
}

// NewGRPCServer initializes a new gRPC server
func NewGRPCServer(endpoints endpoints.Endpoints, logger log.Logger) pb.MathServiceServer {
    return &gRPCServer{
        add: gt.NewServer(
            endpoints.Add,
            decodeMathRequest,
            encodeMathResponse,
        ),
    }
}

func (s *gRPCServer) Add(ctx context.Context, req *pb.MathRequest) (*pb.MathResponse, error) {
    _, resp, err := s.add.ServeGRPC(ctx, req)
    if err != nil {
        return nil, err
    }
    return resp.(*pb.MathResponse), nil
}

func decodeMathRequest(_ context.Context, request interface{}) (interface{}, error) {
    req := request.(*pb.MathRequest)
    return endpoints.MathReq{NumA: req.NumA, NumB: req.NumB}, nil
}

func encodeMathResponse(_ context.Context, response interface{}) (interface{}, error) {
    resp := response.(endpoints.MathResp)
    return &pb.MathResponse{Result: resp.Result}, nil
}

In our code for transports/grpc.go, we have NewGRPCServer which makes a set of endpoints available as a gRPC MathServiceServer.

Main entrypoint

This is where we stitch things together. Here we initiate our services, endpoints and our gRPC server, then we serve it using google.golang.org/grpc package.



package main

import (

    "fmt"

    "net"

    "os"

    "os/signal"

    "syscall"

<span class="s">"github.com/go-kit/kit/log"</span>
<span class="s">"github.com/go-kit/kit/log/level"</span>
<span class="s">"github.com/junereycasuga/gokit-grpc-demo/endpoints"</span>
<span class="s">"github.com/junereycasuga/gokit-grpc-demo/pb"</span>
<span class="s">"github.com/junereycasuga/gokit-grpc-demo/service"</span>
<span class="n">transport</span> <span class="s">"github.com/junereycasuga/gokit-grpc-demo/transports"</span>
<span class="s">"google.golang.org/grpc"</span>



    

    




)

func main() {

    var logger log.Logger

    logger = log.NewJSONLogger(os.Stdout)

    logger = log.With(logger, "ts", log.DefaultTimestampUTC)

    logger = log.With(logger, "caller", log.DefaultCaller)

<span class="n">addservice</span> <span class="o">:=</span> <span class="n">service</span><span class="o">.</span><span class="n">NewService</span><span class="p">(</span><span class="n">logger</span><span class="p">)</span>
<span class="n">addendpoint</span> <span class="o">:=</span> <span class="n">endpoints</span><span class="o">.</span><span class="n">MakeEndpoints</span><span class="p">(</span><span class="n">addservice</span><span class="p">)</span>
<span class="n">grpcServer</span> <span class="o">:=</span> <span class="n">transport</span><span class="o">.</span><span class="n">NewGRPCServer</span><span class="p">(</span><span class="n">addendpoint</span><span class="p">,</span> <span class="n">logger</span><span class="p">)</span>

<span class="n">errs</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="k">chan</span> <span class="kt">error</span><span class="p">)</span>
<span class="k">go</span> <span class="k">func</span><span class="p">()</span> <span class="p">{</span>
    <span class="n">c</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="k">chan</span> <span class="n">os</span><span class="o">.</span><span class="n">Signal</span><span class="p">)</span>
    <span class="n">signal</span><span class="o">.</span><span class="n">Notify</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">syscall</span><span class="o">.</span><span class="n">SIGINT</span><span class="p">,</span> <span class="n">syscall</span><span class="o">.</span><span class="n">SIGTERM</span><span class="p">,</span> <span class="n">syscall</span><span class="o">.</span><span class="n">SIGALRM</span><span class="p">)</span>
    <span class="n">errs</span> <span class="o">&lt;-</span> <span class="n">fmt</span><span class="o">.</span><span class="n">Errorf</span><span class="p">(</span><span class="s">"%s"</span><span class="p">,</span> <span class="o">&lt;-</span><span class="n">c</span><span class="p">)</span>
<span class="p">}()</span>

<span class="n">grpcListener</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">net</span><span class="o">.</span><span class="n">Listen</span><span class="p">(</span><span class="s">"tcp"</span><span class="p">,</span> <span class="s">":50051"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
    <span class="n">logger</span><span class="o">.</span><span class="n">Log</span><span class="p">(</span><span class="s">"during"</span><span class="p">,</span> <span class="s">"Listen"</span><span class="p">,</span> <span class="s">"err"</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>
    <span class="n">os</span><span class="o">.</span><span class="n">Exit</span><span class="p">(</span><span class="m">1</span><span class="p">)</span>
<span class="p">}</span>

<span class="k">go</span> <span class="k">func</span><span class="p">()</span> <span class="p">{</span>
    <span class="n">baseServer</span> <span class="o">:=</span> <span class="n">grpc</span><span class="o">.</span><span class="n">NewServer</span><span class="p">()</span>
    <span class="n">pb</span><span class="o">.</span><span class="n">RegisterMathServiceServer</span><span class="p">(</span><span class="n">baseServer</span><span class="p">,</span> <span class="n">grpcServer</span><span class="p">)</span>
    <span class="n">level</span><span class="o">.</span><span class="n">Info</span><span class="p">(</span><span class="n">logger</span><span class="p">)</span><span class="o">.</span><span class="n">Log</span><span class="p">(</span><span class="s">"msg"</span><span class="p">,</span> <span class="s">"Server started successfully 🚀"</span><span class="p">)</span>
    <span class="n">baseServer</span><span class="o">.</span><span class="n">Serve</span><span class="p">(</span><span class="n">grpcListener</span><span class="p">)</span>
<span class="p">}()</span>

<span class="n">level</span><span class="o">.</span><span class="n">Error</span><span class="p">(</span><span class="n">logger</span><span class="p">)</span><span class="o">.</span><span class="n">Log</span><span class="p">(</span><span class="s">"exit"</span><span class="p">,</span> <span class="o">&lt;-</span><span class="n">errs</span><span class="p">)</span>



    

    




}

Source Code

You can find the source code of this example at Github

Top comments (3)

M Fariz Wisnu prananda • Sep 30 '22

thanks for sharing

Sedky Abou-Shamalah • Nov 29 '20

How does this compare to the API Gateway pattern, where all the non-business logic is handled at another layer?

Junerey Casuga • Dec 2 '20

I believe API Gateway pattern applies mostly as an infrastructure design for microservice rather than a code architecture.

IMO, Go kit's goal is to make the development of Go microservices easier/faster by providing essential packages/standard libraries rather than forcing someone on how they structure their code/infrasturcure

DEV Community