DEV Community

Cover image for go-doudou series 02: How to develop gRPC service with go-doudou
go-doudou
go-doudou

Posted on • Edited on

go-doudou series 02: How to develop gRPC service with go-doudou

#go #grpc #microservices

Since v2.0.0 version, go-doudou has been supported to develop gRPC service. The approach or workflow is the same as previous versions that is define methods of a golang interface in svc.go file first, then let go-doudou generate as much as code for you. go-doudou has a smooth learning curve, which make it very friendly to novice, especially to developers with other programming language background like Java, Nodejs or Python. go-doudou is easy, but very productive.

Here I will use a small demo to show you how the developping workflow looks like including some best practices. We will implement a Largest Remainder Method gRPC service with go-doudou, and then we will see how to test it. Full code can be cloned from https://github.com/unionj-cloud/go-doudou-tutorials/tree/master/go-stats.

Preparation

Intall Go

You should install go v1.16 or above.

Install gRPC Compiler and Plugins

Install Compiler protoc

To install the Protobuf compiler protoc, please refer to official documentation, here are the installation commands for common operating systems:

  • Ubuntu system: 
$ apt install -y protobuf-compiler
$ protoc --version # Make sure to install v3 and above
Enter fullscreen mode Exit fullscreen mode
  • Mac system, you need to install Homebrew: 
$ brew install protobuf
$ protoc --version # Make sure to install v3 and above
Enter fullscreen mode Exit fullscreen mode

If Homebrew fails to be installed on Windows systems or Mac systems, you need to download the installation package from github, unzip it, and configure the environment variables yourself.

The latest protoc download address for Windows system: https://github.com/protocolbuffers/protobuf/releases/download/v21.7/protoc-21.7-win64.zip

Mac system Intel latest protoc download address: https://github.com/protocolbuffers/protobuf/releases/download/v21.7/protoc-21.7-osx-x86_64.zip

Please find other packages in github releases.

Install Plugins

  1. Install the plugin: 
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
Enter fullscreen mode Exit fullscreen mode
  1. Configure environment variables: 
$ export PATH="$PATH:$(go env GOPATH)/bin"
Enter fullscreen mode Exit fullscreen mode

Please go to https://grpc.io/docs/languages/go/quickstart/ to find the latest version number.

Install go-doudou

  • To install go-doudou CLI

If go version < 1.17,

go get -v github.com/unionj-cloud/go-doudou/v2@v2.0.3
Enter fullscreen mode Exit fullscreen mode

If go version >= 1.17, recommend to use below command to install go-doudou cli globally

go install -v github.com/unionj-cloud/go-doudou/v2@v2.0.3
Enter fullscreen mode Exit fullscreen mode
  • To download go-doudou as dependency for your module 
go get -v -d github.com/unionj-cloud/go-doudou/v2@v2.0.3
Enter fullscreen mode Exit fullscreen mode


If you meet 410 Gone error, try to run below command, then run install command again:

export GOSUMDB=off
Enter fullscreen mode Exit fullscreen mode

After installation, if you meet go-doudou: command not found error, please configure $HOME/go/bin to ~/.bash_profile file, for example:

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:/usr/local/go/bin
PATH=$PATH:$HOME/go/bin

export PATH
Enter fullscreen mode Exit fullscreen mode

Verify go-doudou has been installed, run below command:

➜  go-doudou -v
go-doudou version v2.0.3
Enter fullscreen mode Exit fullscreen mode

Initialize Project

Now we are ready to start coding. First we should initialize our project.

go-doudou svc init go-stats -m go-doudou-tutorials/go-stats
Enter fullscreen mode Exit fullscreen mode

go-stats is project root folder path, go-doudou will create directories recursively if any directory not exists. go-doudou-tutorials/go-stats is module name, specified by -m flag.

go-doudou generated these folder and files as below.

➜  go-stats git:(master) ✗ tree -L 2
.
├── Dockerfile
├── go.mod
├── svc.go
└── vo
    └── vo.go

1 directory, 4 files
Enter fullscreen mode Exit fullscreen mode

In svc.go file there has been declared an interface for you to define methods to map protobuf rpcs. vo package is for you to define structs to map protobuf messages.

Design Service

Then we have a look the svc.go file.

There are two //go:generate directives, line 12 is for RESTful service, line 13 is for gRPC service, for convenience. If you use goland (my favourite IDE), you can just single click the green arrow to run go-doudou command.

Image description

PageUsers is an example method, now we get rid of it and write our own.

We also need to define PercentageReqVo and PercentageRespVo in vo package. Note, struct type input parameters and result parameters here in each methods must be declared in vo package as go-doudou CLI only scan the package to fetch informations for generating protobuf messages.

go-doudou name in line 7 is a small CLI tool to create or update json tag of each fields in each structs, default is lower camel case.

Generate Code

Now we can run go-doudou svc grpc command to generate proto file, server and client stub code.

➜  go-stats git:(master) ✗ tree -L 3 -a
.
├── .dockerignore
├── .env
├── .gitignore
├── Dockerfile
├── cmd
│   └── main.go
├── config
│   └── config.go
├── db
│   └── db.go
├── go.mod
├── svc.go
├── svcimpl.go
├── transport
│   └── grpc
│       ├── annotation.go
│       ├── gostats.pb.go
│       ├── gostats.proto
│       └── gostats_grpc.pb.go
└── vo
    └── vo.go

13 directories, 16 files
Enter fullscreen mode Exit fullscreen mode

We can see that there are some more folders and files generated.

  • .dockerignore : ignore **/*.local files
  • .env : dotenv configuration file
  • cmd : there is main.go file as entry point
  • config : mapping environment variables to config struct
  • db : for database connection, useless here
  • svcimpl.go : implement your own business logic
  • transport : proto file, server and client stub code

Before we dive into the code, we should run go mod tidy to download dependencies. Then we start the program to make sure everything is working correctly until now.

go-doudou is relying on go.uber.org/automaxprocs for effectly using resources, so you can see line 2.

Let's open transport/grpc/gostats.proto file and explain the code.

As we can see, all of the field names in each messages are lower camel case, which is not conventional snake case, because all our json tags in vo package are lower camel case. Of course you can use snake case both sides, but you must make them consistent with each other, otherwise you maybe have to convert protoc generated struct to vo struct manually (if you will use vo structs).

go-doudou will wrap all result parameters except error into one message like LargestRemainderRpcResponse when generating proto file and copy all comments into it.

go-doudou only supports protobuf v3 grammer.

Implement Stub

We will implement our business logic in svcimpl.go file now. Let's see what's in it now.

Code line 13 assigns a *GoStatsImpl type nil to interface type pb.GoStatsServiceServer to make sure pointer type GoStatsImpl is always implementing pb.GoStatsServiceServer interface.

We can add any fields to GoStatsImpl if necessary like external service client instance, database connection instance, any queue or pool instance, etc. There is a package level factory function NewGoStats for you to inject any dependencies and create a pointer type GoStatsImpl instance for later use.

Now let's implement LargestRemainderRpc method. Here is a piece of best practice that you don't have to implement pb.GoStatsServiceServer directly, you can choose to implement GoStats at first, then call each implemented GoStats methods in corresponding pb.GoStatsServiceServer method implementations as go-doudou doesn't support grpc-gateway and grpc-web, implementing GoStats first can let you reuse code to add RESTful endpoints when necessary in the future.

So we prefer to run go-doudou svc http -c to generate RESTful related code at first even if you don't need RESTful at the moment.

Image description

The current project structure looks like this.

➜  go-stats git:(master) ✗ tree -L 3   
.
├── Dockerfile
├── client
│   ├── client.go
│   ├── clientproxy.go
│   └── iclient.go
├── cmd
│   └── main.go
├── config
│   └── config.go
├── db
│   └── db.go
├── go.mod
├── go.sum
├── gostats_openapi3.go
├── gostats_openapi3.json
├── svc.go
├── svcimpl.go
├── transport
│   ├── grpc
│   │   ├── annotation.go
│   │   ├── gostats.pb.go
│   │   ├── gostats.proto
│   │   └── gostats_grpc.pb.go
│   └── httpsrv
│       ├── handler.go
│       ├── handlerimpl.go
│       └── middleware.go
└── vo
    └── vo.go

8 directories, 21 files
Enter fullscreen mode Exit fullscreen mode

We can see go-doudou generated httpsrv package that is for implementing RESTful. We can simply remove it or leave it there. Then let's see what changed in svcimpl.go file.

We can see line 17 and line 36-42 are new code. Line 17 is for making sure pointer type GoStatsImpl is always implementing GoStats interface. Line 36-42 are stub code for us to finish.

go-doudou provides us a helper function LargestRemainder in github.com/unionj-cloud/go-doudou/v2/toolkit/numberutils package. We omit the algorithm explain here as it is not the concern.

Now we can implement LargestRemainderRpc method like this:

We don't need to worry about manually converting from protoc generated struct to vo struct as each fields in one has the same json tag with the other, so that we can use DeepCopy function in github.com/unionj-cloud/go-doudou/v2/toolkit/copier package to help us do the conversion.

Test Service

We start our service again. Don't forget to run go mod tidy as we generated new code and imported new dependencies while we were coding.

➜  go-stats git:(master) ✗ go mod tidy && go run cmd/main.go
2022/11/23 13:18:13 maxprocs: Leaving GOMAXPROCS=16: CPU quota undefined
                           _                    _
                          | |                  | |
  __ _   ___   ______   __| |  ___   _   _   __| |  ___   _   _
 / _` | / _ \ |______| / _` | / _ \ | | | | / _` | / _ \ | | | |
| (_| || (_) |        | (_| || (_) || |_| || (_| || (_) || |_| |
 \__, | \___/          \__,_| \___/  \__,_| \__,_| \___/  \__,_|
  __/ |
 |___/
2022-11-23 13:18:13 INF ================ Registered Services ================
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF |                 SERVICE                  |         RPC          |
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF | go_stats.GoStatsService                  | LargestRemainderRpc  |
2022-11-23 13:18:13 INF | grpc.reflection.v1alpha.ServerReflection | ServerReflectionInfo |
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF ===================================================
2022-11-23 13:18:13 INF Grpc server is listening at [::]:50051
2022-11-23 13:18:13 INF Grpc server started in 1.001238ms
Enter fullscreen mode Exit fullscreen mode

Personally, I prefer to use evans to test gRPC.

➜  go-stats git:(master) ✗ evans -r repl -p 50051

  ______
 |  ____|
 | |__    __   __   __ _   _ __    ___
 |  __|   \ \ / /  / _. | | '_ \  / __|
 | |____   \ V /  | (_| | | | | | \__ \
 |______|   \_/    \__,_| |_| |_| |___/

 more expressive universal gRPC client


go_stats.GoStatsService@127.0.0.1:50051> show service
+----------------+---------------------+-----------------+-----------------------------+
|    SERVICE     |         RPC         |  REQUEST TYPE   |        RESPONSE TYPE        |
+----------------+---------------------+-----------------+-----------------------------+
| GoStatsService | LargestRemainderRpc | PercentageReqVo | LargestRemainderRpcResponse |
+----------------+---------------------+-----------------+-----------------------------+

go_stats.GoStatsService@127.0.0.1:50051> service GoStatsService

go_stats.GoStatsService@127.0.0.1:50051> call LargestRemainderRpc
<repeated> data::value (TYPE_INT32) => 20
<repeated> data::key (TYPE_STRING) => apple
<repeated> data::value (TYPE_INT32) => 30
<repeated> data::key (TYPE_STRING) => banana
<repeated> data::value (TYPE_INT32) => 40
<repeated> data::key (TYPE_STRING) => pear
<repeated> data::value (TYPE_INT32) => 
places (TYPE_INT32) => 2
{
  "data": [
    {
      "key": "apple",
      "percent": 22.22,
      "percentFormatted": "22.22%",
      "value": 20
    },
    {
      "key": "banana",
      "percent": 33.33,
      "percentFormatted": "33.33%",
      "value": 30
    },
    {
      "key": "pear",
      "percent": 44.45,
      "percentFormatted": "44.45%",
      "value": 40
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

See, we input apple 20kg, banana 30kg, pear 40kg and we want 2 digits after dot, then we get expected result that 22.22 + 33.33 + 44.45 = 100.

Summary

In this tutorial, we learned basic skill about how to develop gRPC service with go-doudou microservice framework. go-doudou not only can help you build your gRPC service to implement your business logic, but also contains full of service government features to help you build entire microservice system. go-doudou is young, but very promising, hope more and more developers can join us to make contributions.

Top comments (3)

Collapse
 
wubin1989 profile image
go-doudou •

Here is go-doudou online documentation website: go-doudou.unionj.cloud/

Collapse
 
wubin1989 profile image
go-doudou •

Done!

Collapse
 
wubin1989 profile image
go-doudou •

Sorry, this article is not finished yet...
I will finish it as soon as possible.

Read next

abathargh profile image

optimizing c structs layouts

abathargh -

ashwingopalsamy profile image

A Tiny, Dependency-Free UUID Checker for Go!

Ashwin Gopalsamy -

colindickson_78 profile image

Using Weak Pointers in Go

Colin Dickson -

johneliud profile image

FETCHING ADVENT OF CODE INPUT DYNAMICALLY IN GO

John Eliud Odhiambo -