DEV Community

Anurag-Rj
Anurag-Rj

Posted on

Protocol Buffers

#backend #data #java #tutorial

Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data.

How it looks: (filed number + wiretype)(value) = ( field number << 3 | wiretype)(value) precise view.

There are three terms here:

  • Serializing structured data:

Suppose you have an object in Java:

classUser {
intid=101;
Stringname="Anurag";
}
Enter fullscreen mode Exit fullscreen mode

Computers cannot directly send Java objects over a network.

So we convert the object into a format that can be transmitted or stored.

This conversion process is called S*erialization*.

Example:

{
  "id":101,
  "name":"Anurag"
}
Enter fullscreen mode Exit fullscreen mode

Protocol Buffers serialize the same data into a compact binary format instead of JSON.

  • Language-neutral:

You define the data structure once in a .proto file:

message User {
  int32 id = 1;
  string name = 2;
}
Enter fullscreen mode Exit fullscreen mode

From this single file, Protocol Buffers can generate classes for:

  • Java
  • Python
  • Go
  • C++
  • C#
  • JavaScript
  • Kotlin

So a Java service and a Python service can communicate using the same contract.

One definition → many programming languages.

Protocol buffers binary encoding

Let’s suppose:

message User {
   int32 age = 1;
}
Enter fullscreen mode Exit fullscreen mode

And suppose the value of age that is send is : 25

What protobuf actually sends

Instead of sending

{
    "age":"25"
}
Enter fullscreen mode Exit fullscreen mode

protobuf sends:

08 19

Only 2 bytes!

Let's understand where they come from.

  • Part 1: Field number - It is the just the numbering/ordering of fields that is set in “.proto” file. Like as age is at 1 then the field number is 1.
  • Part 2: Wire Type - A wire type tells protobuf what kind of data follows.
Data Type Wire Type
int32, int64, bool 0
string 2
embedded message 2

For:

int32 age = 1;
Enter fullscreen mode Exit fullscreen mode

wire type =

0
Enter fullscreen mode Exit fullscreen mode

because it's an integer.

  • Part 3: Combine Field Number + Wire Type

Protobuf stores them together:

Formula:

(field_number << 3) | wire_type
Enter fullscreen mode Exit fullscreen mode

For our example:

field_number = 1
wire_type = 0

1 << 3 | 0 = ( 1 * 2 ^ 3 ) | 0 = 8 | 0 = 1000 | 0 = 1000 = 8
So, the decimal we get is 8 we need to get it Hexadecimal representation: 08.
Enter fullscreen mode Exit fullscreen mode
  • Part 4: Encode Value

Value:

25
Enter fullscreen mode Exit fullscreen mode

25 in hexadecimal:

19
Enter fullscreen mode Exit fullscreen mode

So protobuf stores:

19
Enter fullscreen mode Exit fullscreen mode

Final Payload

Combine both parts:

08 19
Enter fullscreen mode Exit fullscreen mode

Interpretation:

08 -> Field #1, Integer
19 -> Value 25
Enter fullscreen mode Exit fullscreen mode

Top comments (0)