Serializing dynamic JSON through gRPC on nodejs

What's gRPC?

gRPC is a RPC (Remote Procedure Call) framework developed by Google that structures communication between applications using Protocol Buffers' notation on implementations on different languages or platforms, which makes gRPC a cross-platform technology.

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data. Protocol buffers provide a serialization format for packets of typed, structured data that are up to a few megabytes in size.

Approach scenario

Think we have communication between services that needs a flexible field, which we cannot expect exactly what we'll receive in it. As javascript developers, we rapidly think on plain and old JSON object, but on Protocol Buffers, things tend to be more sophisticated and structured. So, how do we represent a dynamic JSON in a Protocol Buffers definition?

The answer is the Struct type.

Struct type

The Struct type is a structured way to represent a flexible/dynamic JSON object which we need to represent the type of fields' values.

For example, think we have to send this object below in a gRPC's Struct field:

{
  name: 'John Smith',
  age: 25
}

Following the Struct type structure, we'd send it like that:

{
  fields: {
     name: {
       kind: 'stringValue',
       stringValue: 'John Smith'
     },
     age: {
       kind: 'numberValue'
       numberValue: 25
     }
  }
}

Don't worry, we don't even need to do this kind of conversion by hand, soon we'll see the pb-util lib doing that for us in the next section ;)

Hands-on

Let's say we need to develop a hypothetical email service that receives the user email and custom parameters for email formatting, but here I'll log the received payload for illustration only. Below we see the proto, server, and client-side's pieces of code.

• Proto

syntax = "proto3";
import "google/protobuf/struct.proto";

service SampleMessagingService {
  rpc SendEmail (EmailMessage) returns (Void);
}

message EmailMessage {
  string email = 1;
  google.protobuf.Struct parameters = 2;
}

• Server

function SendEmail (messagePayload, callback) {
  const { request } = messagePayload;

  console.log('Received Message Payload', {
    ...request,
    parameters: struct.decode(request.parameters)
  })

  return callback(null, {})
}

• Client

const payload = {
    email: '[e-mail]',
    parameters: struct.encode({
        name: 'John Paul',
        age: 25,
        isDev: true
    })
}

// Sending message
client.SendEmail(payload, (err, response) => {
  if (err) throw err
  console.log(response)
})

And running the server and client, we can see the result message payload on the server's console log:

That's it folks! I hope it could help. You can check the complete code here. 🚀

REFERENCES
https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/struct.proto