RPC on the example of gRPC. When to apply and how it works

Introduction

Good day, colleagues. I am a go developer, so the examples will be mainly based on it. I would like to speculateabout the methods of interaction between services. The topic is very broad. Often we use implementations thatare not always suitable, because... We don't know where to apply this or that technology. I want to try to startclosing this gap both for myself and for people. Any comments and constructive corrections are welcome.

In this article I want to look at how gRPC works, what it can do, as well as when and why to use it.

Basic methods of interaction between services

1. REST (Representational State Transfer)

• An architectural style based on the principles described in Roy Fielding's dissertation.

• Uses standard HTTP methods (GET, POST, PUT, DELETE) to interact with resources.

• Data exchange is often in JSON or XML format.

2. RPC (Remote Procedure Call):

• A mechanism that allows functions or procedures to be called on a remote server as if they were local.

• Examples: gRPC, SOAP, JSON-RPC.

3. Message Brokers:

• An asynchronous method of exchanging messages between various system components.

• Examples: RabbitMQ, Apache Kafka, Apache ActiveMQ.

What is RPC?

Remote Procedure Call (RPC) is a communication protocol between a client and a server that allows the client tocall procedures (functions, methods) on a remote server as if they were local. This provides an abstraction fornetwork communication and allows programs to operate in a distributed environment, hiding the complexities ofdata transfer and remote operations.

Basic components of RPC

• Client: The program or component that initiates the remote procedure call.

• Server: A program or component that provides methods that can be called remotely.

• Proxy: The client uses a proxy object to call remote procedures on the server as if they were local.

• Serialization: The process of converting data and procedure parameters into a format that can be transmitted over the network (e.g. JSON format, binary format, etc.).

• Transport: A mechanism for transferring serialized data between a client and a server, such as http.

• IDL (Interface Definition Language): An interface definition language that defines the structure and signatures of remote procedures.

The RPC protocol allows developers to call functions or methods on remote servers in a way that makes the clientcode appear as if the calls are happening locally. The processes of serialization and deserialization convertdata between formats that are understandable by the client and the server.

The benefits of RPC include ease of use, ease of abstraction, and the ability to call remote procedures withoutworrying about the details of network communication.

When to use?

What is the main difference between a monolithic application and a distributed one?

A monolithic application can also have many service modules that have their own logic. Let's say one module isresponsible for users, another for products, a third for sending some messages, a fourth for the logicallycomplex calculation of bonuses and discounts.

We have one program, but it contains many separately related functionalities. They all make some "requests" toeach other. Because We have everything in one program, we highlight some suitable abstractions, connectionsbetween them, which separate the implementation and typify calls. We have implemented the interface,conditionally implemented it into our module and begin to communicate through these abstractions. In fact,everything lies on the heap (sometimes on the stack) in RAM, we pull functions by address, but in the code weseparated the implementations. The developer exhaled, you can develop on top of a well-thought-out architecture.

However, what if these modules were moved into separate service programs? These will be different programs. Oneservice cannot access another service's implementation directly through its virtual RAM. To communicate, we needto use transport communication protocols. You need to send a "request" to the service. At the request of theclient service, this service server itself will activate the required memory area and call the function.

The communication methods are different, but we still need to write human-friendly code. And the system remainsone, it simply consists of many programs, and not just one.

We ended up sitting down, drawing, and thinking about the pros and cons of approaches. A distributedmicroservice architecture was chosen to suit our needs. But what is the best way to link services? I would liketo simplify writing, make immediately readable and clean code. We want to create an abstraction interface likein a monolith. With the help of abstraction, we could call the methods we need directly. With the rightinterface, we would know exactly the implementation, and not just send requests over HTTP. Sounds like RPC. Youcan implement an interface and also invoke methods directly.

Why gRPC?

And so, we chose the RPC approach. But how can we best implement it, because there is a choice. Which toolshould you choose? Situations are different, implementations are also different. You need to know and understandwhat tools there are in principle. Let me give you some examples:

• gRPC (gRPC Remote Procedure Calls) is a framework developed by Google that provides a a powerful and efficient mechanism for implementing RPC in various programming languages, including Go. It uses Protocol Buffers to define the data structure and HTTP/2 to transfer the data. Although Protobuf is not the only way to represent data, but it is the most publicized. Появляются такие инструменты, например, как FlatBuffers, которые пытаются ускорить сериализацию данных. The gRPC also supports different languages, which makes it very flexible.

• JJSON-RPC is a simple protocol for exchanging data in JSON format between a server and a client. Go has libraries such as "github.com/gorilla/rpc/jsonrpc" or "github.com/ybbus/jsonrpc" that can be used to JSON-RPC implementation in Go.

• The net/rpc package in the Go standard library provides basic RPC support. It uses the Gob (Binary JSON) format for data serialization. Together with the net/rpc/jsonrpc package, it can provide JSON-RPC.

• Twirp is a framework for creating simple and efficient APIs using the RPC protocol. It is also based on Protocol Buffers.

Let's say I have several services on go with common models, and the only thing that can happen is that - is to add a new service also on go. And the requests will be as simple as possible, like: send message, give me a list of messages. Then I will think about choosing the standard net/rpc package.

If something more complex is required, I will have to write additional "wrappers" in net/rpc, and if I add services in other languages, I will have to implement completely different interfaces to work with them. services in other languages, I will have to implement completely different interfaces to work with them.

For such reasons, it is common to take the most popular, performance-satisfying, well-documented, multi-language and functional tool. documented, multi-lingual and functional tool. In European regions this is gRPC. In European regions, because Asia lives in its own world, with its own solutions, and they do not always resonate with the West.

What is gRPC?

gRPC is a framework, i.e. a fairly comprehensive solution. Currently, it sends data using http 2 protocol. On github I found attempts at http 3, but these solutions are not very popular. popularity.

gRPC implements its own view of http 2. Usually encryption in the form of TLS protocol is not added on top of it, but it can be added if necessary. encryption in the form of TLS protocol, but it can be added if necessary.

The data is transferred in binary format. The resulting data is smaller and more optimized than the same json (the whole json object will be converted to ASCII characters for transmission, which entails transmission of non-optimal unnecessary bytes). Serialization goes into a binary system for more optimal transmission.

For example, I want to create a user with the name 321 (so Orwellian, we call people by numbers=)). Then I need to call a method with his data.

Data in protobuf:

syntax = "proto3";

message User {
  string name = 1;
}

Request:

1. POST /path/to/resource/CreateUser HTTP/2

2. Host: example.com 

3. Content-Type: application/grpc 

4. User-Agent: grpc-go/1.42.0  

5. Content-Length: 5

6. 0a033231

Parsing by lines:

1: Indicates the request method and HTTP version (HTTP/2.0).

2: The Host header indicates the server address.

3: Indicates that the request content is gRPC.

4: User information indicating the use of gRPC on go server version 1.42.0.

5: Indicates the length of the binary body of the request in bytes.

6: Request body, presented in hexadecimal for easier readability. It contains a binary representation of theprotobuf message.

Parsing the request body:

0a: Tag and field type. In this case, 0a points to a field with tag 1 (the name field) and a byte string datatype.

03: Length of the next line in bytes. Here 03 means the next line is 3 bytes long.

3231: This is the ASCII encoded hexadecimal representation of the string"321".

Additionally, the headers are encrypted according to http 2 principles using the HPACK algorithm.

What the client and server are implemented on is not so important, because... Support is available for manypopular languages. Using the general protobuf scheme, you can implement a go server and a client in pythonwithout any difficulties, using ready-made and written solutions.

gRPC Features

I would like to briefly structure gRPC capabilities that make good use of http 2 functionality.

1. Protocol-independent serialization:

• By default, Protocol Buffers (ProtoBuf) is used to serialize data.

• Support for other serialization formats such as JSON is possible.

2. Multi-language support:

• gRPC supports many programming languages, including C++, Java, Python, Go, Ruby, C#, Node.js, and many more.

3. Simultaneous requests (Multiplexing):

• Allows you to send multiple requests and receive multiple responses simultaneously on a single connection.

4. Streaming:

• Supports both unidirectional and bidirectional data flow.

• Clients and servers can send sequences of messages.

5. Automatic code generation:

• Generation of client and server code based on the API description in the proto file (Protocol Buffers).

6. Data compression:

• Ability to compress data to reduce traffic volume.

7. SSL/TLS support:

• Ability to ensure secure data transfer using SSL/TLS.

8. Metadata library:

• Ability to send and receive metadata in request headers.

9. Cancel requests:

• Clients can cancel submitted requests, which is especially useful in asynchronous scenarios.

10. gRPC headers:

• Support for custom headers to convey additional information.

11. Additional authentication mechanisms:

• Support for authentication using tokens and other mechanisms.

12. gRPC Web:

• Ability to use gRPC in web browsers using gRPC Web.

13. Monitoring and tracing tools:

• Integration with monitoring tools such as Prometheus and tracing tools such as Jaeger.

14. Built-in status support:

• Supports standard and custom status codes.

There are examples <- using gRPC in the github repository .

A few words about gateway

gRPC can also be used to generate REST APIs using grpc-gateway. I won't stop here, I'll give an example and anexplanation of the meaning of this. Grpc-gateway generates interfaces for RPC and REST. Thus, you cansimultaneously connect to the server via RPC and REST. For example, the browser would send a REST request to getusers, and some intermediate service would call RPC methods to create those users.

I haven't found many examples of gateway implementations, so I'd like to add my own example <- .An example of generating interfaces can be found in the Makefile.

Conclusion

I believe that design topics need to be raised more often. Questions about the use of technologies, solutionsand architecture must be resolved correctly immediately, because... mistakes in fundamental elections entaillarge losses and those. duty. Whether gRPC, RPC, or distributed architecture in general is worth using dependsonly on your needs.

I will correct and supplement the information in the article and my knowledge if there is constructivecriticism. I ask you about it, I will be grateful. Have a good day. =)