DEV Community: penthaapatel

How to implement a simple gRPC service using Golang

penthaapatel — Thu, 25 Mar 2021 06:08:26 +0000

How to implement a simple RPC service using Golang

If you want a general introduction to RPC you might want to check out my previous article in this series - A concise guide to gRPC for beginners

The implemented code can be found on my GitHub Repository :
Link to GitHub repository

Before you begin

Workflow

Define protocol - Write .proto file
Compile it - generate necessary protobuf related files and client and server stubs
Write Client code
Write Server code
Run the server and client code, check if it works

A simple RPC service that can create blog posts

Overview

Client sends a request to the server to create a blog post with a given title and contents.
The server saves the newly created blog posts in an in-memory storage.
Since this particular article focuses on implementing RPC services, we use an in-memory storage to save out blog posts. This in-memory storage can be replaced with any database at the backend later.
The database can be integrated later by utilising the power of interfaces in Golang. ### Directory structure

grpcblog
├── Makefile
├── blog
│   ├── blog.pb.go
│   ├── blog.proto
│   └── blog_grpc.pb.go
├── client
│   └── client.go
├── server
│   └── server.go
├── storage
    └── storage.go

.proto file

message Blog defines the structure of the blog post and contains 2 fields - title and body.

message Blog{
   string title = 1;
   string body =2;
 }

message BlogRequest defines the structure of the client request - has 1 field - Blog.

message BlogRequest {
   Blog blog = 1;
 }

message BlogResponse defines the structure of the server response - contains 2 fields -
- id : a universally unique identifier (UUID) generated on successful creation of a new post.
- created : a boolean value set to true if CreatePost is successful; false otherwise.

 message BlogResponse{
   string id = 1;
   bool created = 2;
 }

The proto file contains a simple RPC service CreatePost - client sends a single request to the server and the server responds back with a single response. The RPC service CreatePost sends a BlogRequest from the client and the server responds with a BlogResponse.

  service BlogService{
   rpc CreatePost(BlogRequest) returns (BlogResponse) {}
 }

Compiling the .proto files

Use the following command to compile the .protofiles:

protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative blog/blog.proto

This will generate two files:

blog/blog_grpc.pb.go

blog_grpc.pb.go contains server and client stubs. The interfaces BlogServiceClient and BlogServiceServer are not implemented. These interfaces will be later implemented in our server and client code. We will have to write own own implementation of server and client.

blog/blog.pb.go

blog.pb.go contains protocol buffer code - responsible for binary serialization of data when it is transported between server and client.

Define in-memory storage

storage.go contains custom functions to implement the temporary in-memory storage.
Contains an interface BlogStorage interface with a Save() and a View() function.

type BlogStorage interface {
    Save(blog *blog.Blog, id string) error
    View()
}

type InMemoryBlogStorage struct {
    mutex sync.RWMutex
    blogs map[string]*blog.Blog
}

Blogs are saved in a map where key is the UUID and the value associated with it is the blog post.
To save the posts to a database, we can add another function like SaveToDB() that implements the BlogStorage interface.

Client code

Reads in the blog title from the console.
Reads in the blog contents from the console.
Contains the implementation of the function CreatePost that was unimplemented in BlogServiceClient interface in blog/blog_grpc.pb.go.

Server code

Creates and saves new blog posts to the in-memory storage by taking input data from the client.
Contains the implementation of the function CreatePost that was unimplemented in BlogServiceServer interface in blog/blog_grpc.pb.go.
Generates a universally unique identifier (UUID) when a new post is created and saved to the in-memory storage
Displays UUID of each new post upon successful creation of a new post

Final result

Run the server and client code in two separate terminals.
Run server

go run server/server.go

Run client - Enter title and contents for the new blog post.

go run client/client.go

And hurray! We have successfully created a new post using gRPC service.

Further reading - Some useful resources:

A concise guide to gRPC for beginners

penthaapatel — Thu, 25 Mar 2021 06:02:57 +0000

A series of simple, beginner friendly tutorials that explain the what, why and how of gRPC.

Getting started - Understanding RPC

What is RPC?

RPC aka Remote Procedure Calls - are just like functions.
These functions are executed on some remote system and hence the name.
It follows a request - response model.
A request is initiated from the client - this request is a function call with certain parameters and then, the server returns a response.
Real world application - to construct distributed, scalable, client - server based applications.

gRPC

gRPC is a technology to implement RPC APIs.

Can write servers and clients using different programming languages - a client written using Python can interact with a server written in Golang.
By default gRPC uses protocol buffers as the interface definition language (IDL) to define the RPC services and the structure of the payload messages.

Understanding protocol buffers and .proto files

What is protobuf?

Protocol buffers are a way to serialize structured data in an efficient manner.
It uses the method of binary serialization, which is faster than text based serialization methods like JSON.
The protobuf specification can be implemented in various programming languages.

What is a .proto file

A .proto file will contain the necessary definitions to see the protocol buffers in action.
A file with .proto extension will contain the definitions of messages and services. The services are analogous to procedures/functions and the messages are analogous to the data types of the parameters that can be passed to these functions.



syntax = "proto3";

message UrlRequest{
    string req = 1;
}

message UrlResponse{
    int res = 1;
}

service Url{
    rpc CallUrl(UrlRequest) returns (UrlResponse){}
}

In the above example, the first line of the file specifies that you're using proto3 syntax, CallUrl is a service, UrlRequest and UrlResponse are the message definitions. We can see that the CallUrl acts as a function with input parameter as HelloRequest and output as UrlResponse. The message definitions can have fields within them. Each field has a name and a type. Here we have req field which has the data type string in UrlRequest and res is a field whose data type is int in UrlResponse.

Important step - Compiling a .proto file

This generates helper code to implement the server and client code in the programming language of your choice.
The generated code also handles the parsing of structured data as the data is serialised into a binary format which can be transferred more efficiently as compared to text based serialisation methods like JSON, XML.

Further Reading - Language Guide (proto3)

Let's Code

Basic Workflow

Define the protocol - Write a .proto file
- Define messages
- Define RPC service procedures with their respective parameters and return types. There can be 4 types of RPC procedures:
  - simple RPC
  - server-side streaming RPC
  - client-side streaming RPC
  - bidirectional streaming RPC
- Compile .proto file - generates protobuf related code for binary serialization of data; also generates client and server stubs
Write Server and Client code using language of your choice with the help of the generated stubs
- gPRC currently supports the following languages -
  - C#
  - C++
  - Dart
  - Go
  - Java
  - Kotlin
  - Node
  - Objective-C
  - PHP
  - Python
  - Ruby
Compile and run the final application

Further reading
gRPC Documentation
gRPC vs REST: Understanding gRPC, OpenAPI and REST and when to use them in API design

Difference between functions and methods in Golang

penthaapatel — Thu, 18 Mar 2021 03:43:28 +0000

Difference between functions and methods in Golang

The words function and method are used almost interchangeably, but there are subtle differences in their implementation and usage when used in Golang. Let's see what the difference is and how its used.

Function

Functions accept a set of input parameters, perform some operations on the input and produce an output with a specific return type. Functions are independent that is they are not attached to any user defined type.

Syntax:

func FunctionName(Parameters...) ReturnTypes...

There cannot exist two different functions with the same name in the same package.

type Rectangle struct {
    Width float64
    Height float64
}

type Circle struct {
    Radius float64
}

func Area(r Rectangle) float64 {
    return 2 * r.Height * r.Width
}

func Area(c Circle) float64 {
    return math.Pi * c.Radius * c.Radius
}

Run above code in The Go Playground

The above code throws an error : Area redeclared in this block

Method

A method is effectively a function attached to a user defined type like a struct. This user defined type is called a receiver.

Syntax:

func (t ReceiverType) FunctionName(Parameters...) ReturnTypes...

There can exist different methods with the same name with a different receiver.

type Rectangle struct {
    Width  float64
    Height float64
}

func (r Rectangle) Area() float64 {
    return r.Width * r.Height
}

type Circle struct {
    Radius float64
}

func (c Circle) Area() float64 {
    return math.Pi * c.Radius * c.Radius
}

type Triangle struct {
    Base   float64
    Height float64
}

func (t Triangle) Area() float64 {
    return 0.5 * t.Base * t.Height
}

Run above code in The Go Playground

Using Multiple Linear regression to solve banknote authentication problem

penthaapatel — Fri, 11 Oct 2019 02:11:40 +0000

To predict whether a given banknote is authentic given a number of measures taken from a photograph.

Dataset used - Banknote Authentication Data Set

It is binary classification problem. The dataset consists of 5 columns as follows:

Attributes:

Variance of Wavelet Transformed image (continuous).
Skewness of Wavelet Transformed image (continuous).
Kurtosis of Wavelet Transformed image (continuous).
Entropy of image (continuous).

Labels(Target)

Output (0 for authentic, 1 for inauthentic).

Load the dataset

Load dataset into a pandas dataframe from the csv file.

Analyse the data

Using scatter matrix.

Prepare the data

Break the data (labels and attributes) into two subsets: a test set and a training set.

Create the model

Using Linear Regression.

Train the model

Train the classifier using training set.

Evaluate the model

Evaluate using test set.
Generate confusion matrix for varying threshold values.
Calculate misclassification rate.
Calculate area under ROC.

Dependence of Misclassification Error on Decision Threshold

In-sample ROC for banknote classifier

Out-of-sample ROC for banknote classifier

The entire code to this problem can be found on my github profile-
Code on GitHub - github.com/penthaapatel/BankNoteAuthentication

Contributing to Open Source Go projects on GitHub - A recipe to clone forked Go repos.

penthaapatel — Sat, 08 Jun 2019 17:49:15 +0000

Phew! It took me a while to figure out the entire process so I decided to write this post and I hope it'll further help everyone else get started. I managed to gather all the ingredients and curate the recipe to start working on an open source Golang project…so follow along.

Say you want to experiment with org's new tool which is written in Go. So, you go to get (pun intended) the repo which you have forked on your GitHub account.

Question : Why your regular go get on the forked repo won't work?
Simple Answer : All your imports will be messed up. You'll have to change all the import statements that say something like github.com/org/tool to github.com/yourGitHubID/tool to make things work and your code will run into conflicts.

go get the original org's repo. go get clones the repo into your GOPATH . -u flag updates the named packages and their dependencies.

$ go get -u github.com/org/tool

Go to the directory where you just cloned the org's tool repo

$ cd $GOPATH/src/github.com/org/tool

Let's check what remote repos we have at this point. List out the remote repositories. -v for a verbose output.

$ git remote -v
origin https://github.com/org/tool (fetch)
origin https://github.com/org/tool (push)

Add a add a new remote Git repository explicitly pointing at your fork. Here the shortname I used for the new remote is fork, you can name it whatever you like.

$ git remote add fork https://github.com/yourGitHubID/tool.git

List out the remote repositories again to check and there you see it, a new remote has been added.

$ git remote -v
fork   https://github.com/yourGitHubID/tool.git (fetch)
fork   https://github.com/yourGitHubID/tool.git (push)
origin https://github.com/org/tool (fetch)
origin https://github.com/org/tool (push)

Now that you have a remote that points to the forked repo on your GitHub account, you can start working. To start working on a new branch:

$ git checkout -b experimentbranch

Make sure you have your branch updated with org's branch. Then you can start making changes to the files on the experimentbranch git add your modified files and git commit your work after testing the new features you've added.

Push changes to your fork.

$ git push -u fork experimentbranch:experimentbranch

Then you can compare the changes you have made and submit a Pull Request to propose changes to be merged with the org's repository.

If you notice any corrections that need to be made in the post feel free to comment below! :)