Since v2.0.0 version, go-doudou
has been supported to develop gRPC service. The approach or workflow is the same as previous versions that is define methods of a golang interface in svc.go file first, then let go-doudou generate as much as code for you. go-doudou
has a smooth learning curve, which make it very friendly to novice, especially to developers with other programming language background like Java, Nodejs or Python. go-doudou
is easy, but very productive.
Here I will use a small demo to show you how the developping workflow looks like including some best practices. We will implement a Largest Remainder Method gRPC service with go-doudou, and then we will see how to test it. Full code can be cloned from https://github.com/unionj-cloud/go-doudou-tutorials/tree/master/go-stats.
Preparation
Intall Go
You should install go v1.16 or above.
Install gRPC Compiler and Plugins
Install Compiler protoc
To install the Protobuf compiler protoc, please refer to official documentation, here are the installation commands for common operating systems:
- Ubuntu system:
$ apt install -y protobuf-compiler
$ protoc --version # Make sure to install v3 and above
- Mac system, you need to install Homebrew:
$ brew install protobuf
$ protoc --version # Make sure to install v3 and above
If Homebrew fails to be installed on Windows systems or Mac systems, you need to download the installation package from github, unzip it, and configure the environment variables yourself.
The latest protoc download address for Windows system: https://github.com/protocolbuffers/protobuf/releases/download/v21.7/protoc-21.7-win64.zip
Mac system Intel latest protoc download address: https://github.com/protocolbuffers/protobuf/releases/download/v21.7/protoc-21.7-osx-x86_64.zip
Please find other packages in github releases.
Install Plugins
- Install the plugin:
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
- Configure environment variables:
$ export PATH="$PATH:$(go env GOPATH)/bin"
Please go to https://grpc.io/docs/languages/go/quickstart/ to find the latest version number.
Install go-doudou
- To install
go-doudou
CLI
If go version < 1.17,
go get -v github.com/unionj-cloud/go-doudou/v2@v2.0.3
If go version >= 1.17, recommend to use below command to install go-doudou cli globally
go install -v github.com/unionj-cloud/go-doudou/v2@v2.0.3
- To download go-doudou as dependency for your module
go get -v -d github.com/unionj-cloud/go-doudou/v2@v2.0.3
If you meet 410 Gone error, try to run below command, then run install command again:
export GOSUMDB=off
After installation, if you meet go-doudou: command not found
error, please configure $HOME/go/bin
to ~/.bash_profile
file, for example:
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:/usr/local/go/bin
PATH=$PATH:$HOME/go/bin
export PATH
Verify go-doudou has been installed, run below command:
➜ go-doudou -v
go-doudou version v2.0.3
Initialize Project
Now we are ready to start coding. First we should initialize our project.
go-doudou svc init go-stats -m go-doudou-tutorials/go-stats
go-stats
is project root folder path, go-doudou will create directories recursively if any directory not exists. go-doudou-tutorials/go-stats
is module name, specified by -m
flag.
go-doudou
generated these folder and files as below.
➜ go-stats git:(master) ✗ tree -L 2
.
├── Dockerfile
├── go.mod
├── svc.go
└── vo
└── vo.go
1 directory, 4 files
In svc.go
file there has been declared an interface for you to define methods to map protobuf rpcs. vo
package is for you to define structs to map protobuf messages.
Design Service
Then we have a look the svc.go
file.
There are two //go:generate
directives, line 12 is for RESTful service, line 13 is for gRPC service, for convenience. If you use goland (my favourite IDE), you can just single click the green arrow to run go-doudou command.
PageUsers is an example method, now we get rid of it and write our own.
We also need to define PercentageReqVo and PercentageRespVo in vo package. Note, struct type input parameters and result parameters here in each methods must be declared in vo package as go-doudou CLI only scan the package to fetch informations for generating protobuf messages.
go-doudou name
in line 7 is a small CLI tool to create or update json tag of each fields in each structs, default is lower camel case.
Generate Code
Now we can run go-doudou svc grpc
command to generate proto file, server and client stub code.
➜ go-stats git:(master) ✗ tree -L 3 -a
.
├── .dockerignore
├── .env
├── .gitignore
├── Dockerfile
├── cmd
│ └── main.go
├── config
│ └── config.go
├── db
│ └── db.go
├── go.mod
├── svc.go
├── svcimpl.go
├── transport
│ └── grpc
│ ├── annotation.go
│ ├── gostats.pb.go
│ ├── gostats.proto
│ └── gostats_grpc.pb.go
└── vo
└── vo.go
13 directories, 16 files
We can see that there are some more folders and files generated.
-
.dockerignore
: ignore**/*.local
files -
.env
: dotenv configuration file -
cmd
: there ismain.go
file as entry point -
config
: mapping environment variables to config struct -
db
: for database connection, useless here -
svcimpl.go
: implement your own business logic -
transport
: proto file, server and client stub code
Before we dive into the code, we should run go mod tidy
to download dependencies. Then we start the program to make sure everything is working correctly until now.
go-doudou is relying on go.uber.org/automaxprocs
for effectly using resources, so you can see line 2.
Let's open transport/grpc/gostats.proto
file and explain the code.
As we can see, all of the field names in each messages are lower camel case, which is not conventional snake case, because all our json tags in vo package are lower camel case. Of course you can use snake case both sides, but you must make them consistent with each other, otherwise you maybe have to convert protoc generated struct to vo struct manually (if you will use vo structs).
go-doudou
will wrap all result parameters except error into one message like LargestRemainderRpcResponse when generating proto file and copy all comments into it.
go-doudou
only supports protobuf v3 grammer.
Implement Stub
We will implement our business logic in svcimpl.go
file now. Let's see what's in it now.
Code line 13 assigns a *GoStatsImpl type nil to interface type pb.GoStatsServiceServer to make sure pointer type GoStatsImpl is always implementing pb.GoStatsServiceServer interface.
We can add any fields to GoStatsImpl if necessary like external service client instance, database connection instance, any queue or pool instance, etc. There is a package level factory function NewGoStats for you to inject any dependencies and create a pointer type GoStatsImpl instance for later use.
Now let's implement LargestRemainderRpc method. Here is a piece of best practice that you don't have to implement pb.GoStatsServiceServer directly, you can choose to implement GoStats at first, then call each implemented GoStats methods in corresponding pb.GoStatsServiceServer method implementations as go-doudou
doesn't support grpc-gateway and grpc-web, implementing GoStats first can let you reuse code to add RESTful endpoints when necessary in the future.
So we prefer to run go-doudou svc http -c
to generate RESTful related code at first even if you don't need RESTful at the moment.
The current project structure looks like this.
➜ go-stats git:(master) ✗ tree -L 3
.
├── Dockerfile
├── client
│ ├── client.go
│ ├── clientproxy.go
│ └── iclient.go
├── cmd
│ └── main.go
├── config
│ └── config.go
├── db
│ └── db.go
├── go.mod
├── go.sum
├── gostats_openapi3.go
├── gostats_openapi3.json
├── svc.go
├── svcimpl.go
├── transport
│ ├── grpc
│ │ ├── annotation.go
│ │ ├── gostats.pb.go
│ │ ├── gostats.proto
│ │ └── gostats_grpc.pb.go
│ └── httpsrv
│ ├── handler.go
│ ├── handlerimpl.go
│ └── middleware.go
└── vo
└── vo.go
8 directories, 21 files
We can see go-doudou
generated httpsrv
package that is for implementing RESTful. We can simply remove it or leave it there. Then let's see what changed in svcimpl.go
file.
We can see line 17 and line 36-42 are new code. Line 17 is for making sure pointer type GoStatsImpl is always implementing GoStats interface. Line 36-42 are stub code for us to finish.
go-doudou
provides us a helper function LargestRemainder in github.com/unionj-cloud/go-doudou/v2/toolkit/numberutils
package. We omit the algorithm explain here as it is not the concern.
Now we can implement LargestRemainderRpc method like this:
We don't need to worry about manually converting from protoc generated struct to vo struct as each fields in one has the same json tag with the other, so that we can use DeepCopy function in github.com/unionj-cloud/go-doudou/v2/toolkit/copier
package to help us do the conversion.
Test Service
We start our service again. Don't forget to run go mod tidy
as we generated new code and imported new dependencies while we were coding.
➜ go-stats git:(master) ✗ go mod tidy && go run cmd/main.go
2022/11/23 13:18:13 maxprocs: Leaving GOMAXPROCS=16: CPU quota undefined
_ _
| | | |
__ _ ___ ______ __| | ___ _ _ __| | ___ _ _
/ _` | / _ \ |______| / _` | / _ \ | | | | / _` | / _ \ | | | |
| (_| || (_) | | (_| || (_) || |_| || (_| || (_) || |_| |
\__, | \___/ \__,_| \___/ \__,_| \__,_| \___/ \__,_|
__/ |
|___/
2022-11-23 13:18:13 INF ================ Registered Services ================
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF | SERVICE | RPC |
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF | go_stats.GoStatsService | LargestRemainderRpc |
2022-11-23 13:18:13 INF | grpc.reflection.v1alpha.ServerReflection | ServerReflectionInfo |
2022-11-23 13:18:13 INF +------------------------------------------+----------------------+
2022-11-23 13:18:13 INF ===================================================
2022-11-23 13:18:13 INF Grpc server is listening at [::]:50051
2022-11-23 13:18:13 INF Grpc server started in 1.001238ms
Personally, I prefer to use evans to test gRPC.
➜ go-stats git:(master) ✗ evans -r repl -p 50051
______
| ____|
| |__ __ __ __ _ _ __ ___
| __| \ \ / / / _. | | '_ \ / __|
| |____ \ V / | (_| | | | | | \__ \
|______| \_/ \__,_| |_| |_| |___/
more expressive universal gRPC client
go_stats.GoStatsService@127.0.0.1:50051> show service
+----------------+---------------------+-----------------+-----------------------------+
| SERVICE | RPC | REQUEST TYPE | RESPONSE TYPE |
+----------------+---------------------+-----------------+-----------------------------+
| GoStatsService | LargestRemainderRpc | PercentageReqVo | LargestRemainderRpcResponse |
+----------------+---------------------+-----------------+-----------------------------+
go_stats.GoStatsService@127.0.0.1:50051> service GoStatsService
go_stats.GoStatsService@127.0.0.1:50051> call LargestRemainderRpc
<repeated> data::value (TYPE_INT32) => 20
<repeated> data::key (TYPE_STRING) => apple
<repeated> data::value (TYPE_INT32) => 30
<repeated> data::key (TYPE_STRING) => banana
<repeated> data::value (TYPE_INT32) => 40
<repeated> data::key (TYPE_STRING) => pear
<repeated> data::value (TYPE_INT32) =>
places (TYPE_INT32) => 2
{
"data": [
{
"key": "apple",
"percent": 22.22,
"percentFormatted": "22.22%",
"value": 20
},
{
"key": "banana",
"percent": 33.33,
"percentFormatted": "33.33%",
"value": 30
},
{
"key": "pear",
"percent": 44.45,
"percentFormatted": "44.45%",
"value": 40
}
]
}
See, we input apple 20kg, banana 30kg, pear 40kg and we want 2 digits after dot, then we get expected result that 22.22 + 33.33 + 44.45 = 100.
Summary
In this tutorial, we learned basic skill about how to develop gRPC service with go-doudou microservice framework. go-doudou
not only can help you build your gRPC service to implement your business logic, but also contains full of service government features to help you build entire microservice system. go-doudou
is young, but very promising, hope more and more developers can join us to make contributions.
Top comments (3)
Here is go-doudou online documentation website: go-doudou.unionj.cloud/
Done!
Sorry, this article is not finished yet...
I will finish it as soon as possible.