So I have been asked about gRPC quite a few times in the last couple of months. I have read about it briefly and thought it would not be hard to migrate to if need be, but I haven't gotten any hands on experience with it so far. That changes now.
So, What is gRPC?
Well the g stands for Google because they created gRPC back in 2015. RPC part stands for Remote Procedure Calls. RPC isn't exactly a new way of doing things, it is just calling a procedure from your machine on another machine. What is modern about it, is just the fact that machine in question can be located on cloud somewhere (like GCP for example).
What is also cool, gRPC uses HTTP/2. So one of the main advantages of HTTP/2 as oppose to HTTP/1.1 for example (used by REST by default) are following:
- It is binary, instead of textual
- It is fully multiplexed, instead of ordered and blocking
- It can use one connection for parallelism
- It uses header compression to reduce overhead
- It allows Server Pushing to add responses proactively into the Browser cache.
So in more laic terms: Http2 is binary so it's faster, meaning the latency is better. Serialization and deserialization of HTTP/1.1 is slower.
It also uses something called protobuf in stead of json objects for communicating. The only advantage is json is that it has bigger community (for now). Ok, maybe I'm to hard at json, it is also very simple to write, and very easy to understand for beginners.
But let's talk about protobuf. Basically think XML, but smaller, faster, and simple. You just define the .proto file (the examples will be in my app I will get in a second), and with just one command you can generate whole files and classes for yourself. So no need to worry about anything, the library does it for you.
And that's about it for the intro. Let's jump into an app.
APP
The app is pretty straight forward in the end. On frontend side, you will simply be pressing squares and circles in as fast as possible. But what's cool, is that we will be using one REST microservice the frontend will hit, and 2 gRPC microservices under it, for keeping track and fetching your highscore, and also calculating the size of next appearing objects.
The faster you are able to hit objects, the smaller they will appear next, and the slower you are, the bigger the next object will be.
So as said, game is pretty simple. And I will not really go into how frontend works as it is just one index.html file, where the hardest things are a few ajax calls to the REST microservice.
But, let's get into first gRPC microservice.
gRPC Highscore microservice
So this microservice does two things: Get Highscore and Set Highscore. Not very original but let's go with it. I never said this project was rocket science 😁
- 1) Define the .proto file
As said in the intro, gRPC uses protobuf, and to use it, we first define the .proto file. Seeing as we don't need to send anything to get the highscore, rather just read it, that part will take nothing, and the set highscore will take the score, and see if it is better than the one stored so far.
syntax = "proto3";
option go_package = "game";
service Game{
rpc SetHighScore(SetHighScoreRequest) returns (SetHighScoreResponse);
rpc GetHighScore(GetHighScoreRequest) returns (GetHighScoreResponse);
}
message SetHighScoreRequest {
double high_score = 1;
}
message SetHighScoreResponse {
bool set = 1;
}
message GetHighScoreRequest {}
message GetHighScoreResponse {
double high_score = 1;
}
Viola. As simple as it gets. Let's quickly talk about it.
The syntax part just says the version of proto we are using, and proto3 is the newest one available, so we are going with that.
option go_package just means, the .go file i creates, will be inside game package.
service is actually the bread and butter of gRPC and you can think of two rpc calls as just procedures or functions on remote machine we are trying to call. And as I said in the beginning as well, it's not using jsons, so we define two message signatures that represent what we have to send to use the gRPC. Here we have 4 message types: SetHighScoreRequest, SetHighScoreResponse, GetHighScoreRequest, GetHighScoreResponse. The only thing that I have to mention here is this funny number 1 you are seeing. That just means, in future, if we add or drop more fields, those numbers will help us save the backwards compatibility.
- 2) Run the command for autogenerating the class for us.
Well I'm using go, so my command requires just these 2 steps:
go install google.golang.org/protobuf/cmd/protoc-gen-go
and
protoc -I ./protobuf-import -I ./ ./v1/*.proto --go_out=plugins=grpc:./
and you can see it inside my Makefile. I will not go deep into Makefile, it's pretty simple so is the .sh script, but if you are curious about it, and have no experience, ping me and I'll explain it 🙂
- 3) Create the gRPC server
Well, the step 2 created this very cool highscore.pb.go file for us. Now let's use it.
I created a following structure internal_usage/server/grpc/grpc.go. The reason why it is inside interal usage, is just because, once implemented, we really shouldn't worry about it. It will always work and we can start working on other parts of the code.
But let's see what's inside this grpc.go file.
package grpc
import (
"context"
v1highscore "github.com/fvukojevic/grpc_test/m-apis/m-highscore/v1"
"github.com/pkg/errors"
"github.com/rs/zerolog/log"
"google.golang.org/grpc"
"net"
)
type Grpc struct {
address string //address where the gRPC will listen at
srv *grpc.Server
}
func NewServer(address string) *Grpc {
return &Grpc{
address: address,
}
}
var HighScore = 999999.0
func (g *Grpc) SetHighScore(ctx context.Context, input *v1highscore.SetHighScoreRequest) (*v1highscore.SetHighScoreResponse, error) {
log.Info().Msg("SetHighscore in m-highscore is called")
HighScore = input.HighScore
return &v1highscore.SetHighScoreResponse{
Set: true,
}, nil
}
func (g *Grpc) GetHighScore(ctx context.Context, input *v1highscore.GetHighScoreRequest) (*v1highscore.GetHighScoreResponse, error) {
log.Info().Msg("GetHighscore in m-highscore is called")
return &v1highscore.GetHighScoreResponse{
HighScore: HighScore,
}, nil
}
func (g *Grpc) ListenAndServe() error {
listener, err := net.Listen("tcp", g.address)
if err != nil {
return errors.Wrap(err, "Failed to open tcp port")
}
var serverOpts []grpc.ServerOption
g.srv = grpc.NewServer(serverOpts...)
v1highscore.RegisterGameServer(g.srv, g)
log.Info().Str("Address", g.address).Msg("Starting gRPC server for highscore microservice")
if err := g.srv.Serve(listener); err != nil {
return errors.Wrap(err, "Failed to start gRPC server")
}
return nil
}
As you can see, even though it looks very complex, what does it really represent? Well the functions GetHighscore and SetHigshcore are here, they were defined inside .proto file if you remember. Also we created a struct to hold our gRPC server.
And in the end, we have this ListenAndServe method, because every gRPC has to listen on a certain address(port) and be served so it can be used.
- 4) Call our server from the main
If you are familiar with go, you know every entry point is package main and main.go file. In this main.go, what do we really need to do? Well just initialize our server and serve it.
main.go
package main
import (
"flag"
grpcSetup "github.com/fvukojevic/grpc_test/m-apis/m-highscore/internal_usage/server/grpc"
"github.com/rs/zerolog/log"
)
func main() {
var addressPtr = flag.String("address", ":50051", "address where you can connect to gRPC m-highscore service")
flag.Parse()
s := grpcSetup.NewServer(*addressPtr)
if err := s.ListenAndServe(); err != nil {
log.Fatal().Err(err).Msg("Failed to start grpc server")
}
}
And that's it!.
gRPC Game Engine microservice
Even though the name sounds cool, it is just the same as highscore microservice, so I'll let you go over the code on your own. Again, it's pretty much the same
REST for communicating between frontend and backend
Well I did end up using REST in the end. Why? Well it made no sense to call gRPC directly from javascript..at least not to me. So I created a small gateway that will provide few routes, and every route it hits, will internally call gRPC and execute what it needs to.
I am using gin gonic for rest, it is a very cool and fast package and fairly simple to set up.
But my REST microservice has to be aware of gRPC running eternally. So, that created the clients.go file.
So we have our servers, highscore and game engine. But what calls them? Well that would be clients. They basically connect to the ports of the servers and communicate with them.
Here's how simple the clients.go file looks.
package domain
import (
v1gameengine "github.com/fvukojevic/grpc_test/m-apis/m-game-engine/v1"
v1highscore "github.com/fvukojevic/grpc_test/m-apis/m-highscore/v1"
"github.com/rs/zerolog/log"
"google.golang.org/grpc"
)
func NewGRPCGameServiceClient(serverAddr string) (v1highscore.GameClient, error) {
conn, err := grpc.Dial(serverAddr, grpc.WithInsecure())
if err != nil {
log.Fatal().Msgf("Failed to dial: %v", err)
return nil, err
}
log.Info().Msgf("Successfully connected to [%s]", serverAddr)
if conn == nil {
log.Info().Msg("m-highscore connection is nil")
}
client := v1highscore.NewGameClient(conn)
return client, nil
}
func NewGRPCGameEngineServiceClient(serverAddr string) (v1gameengine.GameEngineClient, error) {
conn, err := grpc.Dial(serverAddr, grpc.WithInsecure())
if err != nil {
log.Fatal().Msgf("Failed to dial: %v", err)
return nil, err
}
log.Info().Msgf("Successfully connected to [%s]", serverAddr)
if conn == nil {
log.Info().Msg("m-game-engine connection is nil")
}
client := v1gameengine.NewGameEngineClient(conn)
return client, nil
}
Just connect, see if you got any errors (your really shouldn't, unless maybe u missed a port or something), and that's it.
Now for grand finale we make our REST routes public with following code:
package main
import (
"flag"
grpcSetup "github.com/fvukojevic/grpc_test/m-apis/m-highscore/internal_usage/server/grpc"
"github.com/rs/zerolog/log"
)
func main() {
var addressPtr = flag.String("address", ":50051", "address where you can connect to gRPC m-highscore service")
flag.Parse()
s := grpcSetup.NewServer(*addressPtr)
if err := s.ListenAndServe(); err != nil {
log.Fatal().Err(err).Msg("Failed to start grpc server")
}
}
Final result
In the end, the final result should look something like this, just a simple app for clicking and tracking the speed of user.
Project resources
Github: https://github.com/fvukojevic/grpc_test
The app will work even without microservices, but they are still cool to showcase.
Again, I know this might be hard to follow with, so if you have any questions feel free to contact me with email and/or on linkedIn if you want to learn more about gRPC and we can chat about it!
My LinkedIn profile: https://www.linkedin.com/in/ferdo-vukojevic-2a9370181/
Till next time! 🚀🚀
Latest comments (6)
I’m curious what you thought “g” stands for Google? Did Google said that?
Their repo has said g is just a random name starts with “g” for each new release.
HTTP 1.1 as used by REST? Is there some restriction I don't know about that prevents using 2.0 with REST? Or are frameworks lying to me that they use 2.0 when set it on?
Protobuf instead of JSON HTTP 2.0? You know that JSON is not HTTP 1.1 specification but your own choice? Also it sounds like HTTP 2.0 is using protobuf by specification... really not the case as far as I can tell.
Post is either witten in a way it makes me confused or has misinformation
REST depends heavily on HTTP (usually HTTP 1.1). You are right probably was wrong of me to say it's not usable with HTTP/2. Just usually HTTP/1.1 as of right now.
Again inaccurate. HTTP verbs are present in HTTP 2.0. Anyone who has a chance will upgrade their servers to use 2.0. You can use protobufs with REST not restrictions their either. There's bunch of REST usuall things people do which some think are requirement.
It just so happens that HTTP was in doctoral dissertation when REST was described.
Point is using underlying protocol to communicate while REST is architectural style that depends on it so not all would satisfy the need as HTTP did.
How is me saying that REST usually goes with HTTP/1.1 inaccurate? It is the default and people are still making the transition.
You added as of right now which to me indicated that no one or barley anyone is using it. While it's just server/framework upgarde depening on the way of deploying apps and such. There's not code change involved so there's no transition in my head just ops part so to speak. But ok I get different way od expressions