DEV Community: Aditya Chauhan

Protobuf vs JSON

Aditya Chauhan — Sat, 27 Jun 2026 12:08:36 +0000

Modern applications thrive on fast and efficient communication. While JSON has long been the go-to format for data exchange, many large-scale tech companies are now rethinking their choice. Companies like Atlassian, Netflix, and Google have shifted to Protocol Buffers (Protobuf) to optimize performance and reduce payload sizes across services.

Their motivation? Faster API responses, smaller data transfers, better compatibility for growing systems, and built-in schema enforcement.

In this guide, we'll break down what JSON and Protobuf are, compare their strengths and weaknesses, and walk you through practical examples in Ruby. Whether you're building microservices or optimizing existing APIs, this guide will help you decide.

1. What is JSON?

JSON (JavaScript Object Notation) is a lightweight, human-readable data format commonly used in web APIs and configs. It's based on JavaScript syntax and supported by virtually all programming languages.

Example:

{
  "name": "Alice",
  "age": 28,
  "isActive": true
}

2. Why is JSON Preferred?

JSON is often the default choice because:

Readable: Easily understood and edited by humans
Simple: No extra libraries needed in JavaScript/Node.js
Interoperable: Works across all platforms and languages
Debug-friendly: Can be logged and inspected directly
Universal: Every language has built-in or one-line library support

But JSON has limitations in performance, file size, and type safety, especially in large systems.

3. What are Protocol Buffers?

Protocol Buffers (Protobuf) is a binary serialization format developed by Google. It encodes data into compact, fast, structured bytes using .proto schema definitions.

Key features:

Compact binary format (3-10x smaller than JSON)
Faster parsing and serialization
Schema enforcement via .proto files
Built-in backward and forward compatibility
Code generation for multiple languages

Example .proto file:

syntax = "proto3";

message User {
  string name = 1;
  int32 age = 2;
  bool is_active = 3;
}

4. Protobuf vs JSON: Key Differences

Feature	JSON	Protobuf
Format	Text (UTF-8)	Binary
Human Readable	Yes	No (bytes)
Size	Larger	Smaller (3-10x)
Speed	Slower parsing	Faster parsing
Schema	Optional/loose	Required (`.proto`)
Type Safety	Weak (runtime errors)	Strong (compile-time checks)
Versioning	Manual (URL versioning)	Built-in (field numbers)
Browser-friendly	Native	Needs library

Concrete example: A list of 1000 products

JSON: ~150KB payload, ~300ms download on 3G
Protobuf: ~45KB payload, ~90ms download on 3G

5. How to Use Protobuf in Ruby

In Ruby, Protobuf is typically used with pre-generated classes.

Step 1: Define the `.proto` File

syntax = "proto3";

message Customer {
  string name = 1;
  string email = 2;
  int32 seats = 3;
}

message CreateCustomerRequest {
  string name = 1;
  string email = 2;
}

message CreateCustomerResponse {
  Customer customer = 1;
}

Step 2: Generate Ruby Code

# Install protoc compiler (platform-specific)
# macOS: brew install protobuf
# Linux: apt-get install protobuf-compiler

# Generate Ruby classes
protoc --ruby_out=app/contracts customer.proto

This generates customer_pb.rb:

# Generated by the protocol buffer compiler. DO NOT EDIT!
require 'google/protobuf'

descriptor_data = "..." # Binary schema data

Google::Protobuf::DescriptorPool.generated_pool.add_serialized_file(descriptor_data)

module MyApp
  Customer = Google::Protobuf::DescriptorPool.generated_pool.lookup("Customer").msgclass
  CreateCustomerRequest = Google::Protobuf::DescriptorPool.generated_pool.lookup("CreateCustomerRequest").msgclass
  CreateCustomerResponse = Google::Protobuf::DescriptorPool.generated_pool.lookup("CreateCustomerResponse").msgclass
end

Step 3: Use in Your Application

# Creating a message
request = MyApp::CreateCustomerRequest.new(
  name: "Acme Corp",
  email: "billing@acme.com"
)

# Accessing fields
request.name   # => "Acme Corp"
request.email  # => "billing@acme.com"

# Serialize to JSON (for HTTP APIs)
json_payload = request.to_json
# => '{"name":"Acme Corp","email":"billing@acme.com"}'

# Parse incoming JSON
raw_json = '{"name":"Acme Corp","email":"billing@acme.com"}'
parsed = MyApp::CreateCustomerRequest.decode_json(raw_json)
parsed.name  # => "Acme Corp"

# Serialize to binary (for internal services)
binary = request.to_proto
# => <Binary data>

# Parse from binary
decoded = MyApp::CreateCustomerRequest.decode(binary)

6. Real-World API Patterns

Pattern 1: Twirp (JSON over HTTP with Protobuf Schema)

Twirp is a simple RPC framework from Twitch that uses Protobuf for API contracts but communicates over HTTP with JSON.

Rules:

All endpoints use POST
Content-Type: application/json
URL pattern: /api/{service}/{method}

Example:

# Controller
class UsersController < ApplicationController
  def create
    # Decode JSON body to Protobuf object
    request = UsersPb::CreateUserRequest.decode_json(request.raw_post)

    # Business logic here...
    user = user.create!(name: request.name, email: request.email)

    # Encode response to JSON
    response = UsersPb::CreateUserResponse.new(
      user: UsersPb::Customer.new(
        name: user.name,
        email: user.email
      )
    )

    render json: response.to_json
  end
end

This gives you:

Type safety at the API boundary
Human-readable JSON for debugging
Automatic schema documentation via .proto files

Pattern 2: gRPC (Binary over HTTP/2)

For internal microservices, gRPC uses Protobuf's binary format over HTTP/2 for maximum performance.

Characteristics:

Binary payloads (smaller, faster)
HTTP/2 (multiplexing, streaming)
Strongly typed stubs in all languages
Bi-directional streaming support

7. When to Use Each

Use Protobuf when:

Building microservices that communicate frequently
Performance and bandwidth are critical (mobile, IoT)
You need strict schemas and versioning
Data payloads are large or frequent

Use JSON when:

Building browser-facing APIs
You want human-readable data for debugging
Quick prototyping and development
Working with third-party/public APIs
Team is unfamiliar with Protobuf toolchain

The Hybrid Approach (Best of Both)

Many teams use both strategically:

Protobuf binary for internal service-to-service communication (speed)
JSON for browser/client-facing APIs (readability)
The same .proto file generates both binary and JSON serializers

Example architecture:

Browser ──HTTP/JSON──▶ API Gateway ──gRPC/Protobuf──▶ User Service
                                      │
                                      └─gRPC/Protobuf──▶ Payment Service

8. Common Gotchas

1. Field Numbers Matter

In Protobuf, each field has a number (e.g., string name = 1;). These numbers:

Must never change for existing fields
Can be reused only after a field is deleted for 1+ years
Enable backward compatibility (old code ignores new fields)

2. Default Values

Protobuf has implicit defaults:

string → ""
int32 → 0
bool → false

You can't distinguish between "field not set" and "field set to default" without using optional keyword (proto3).

3. Required vs Optional

In proto3, all fields are optional by default. If you need explicit presence tracking:

optional string name = 1;  # Now you can check if it was set

9. Conclusion

Use Case	Recommendation
Human readability	JSON
Performance (internal services)	Protobuf
Browser/frontend APIs	JSON
Microservices (service mesh)	Protobuf + gRPC
RPC with schema enforcement	Protobuf + Twirp
Debugging	JSON
Schema evolution	Protobuf

Both formats have their place. JSON is perfect for human-facing interfaces and simple APIs, while Protobuf excels in high-performance, structured systems.

Mental Model for Federated GraphQL

Aditya Chauhan — Thu, 25 Jun 2026 06:43:42 +0000

Most GraphQL material explains schemas, queries, mutations, resolvers, and pagination. That is necessary, but it is not enough once the graph is split across multiple backend services. At that point, the important question becomes architectural:

How does one client request become work across multiple backend services without exposing that backend topology to the client?

.
.
.

GraphQL federation

Apollo describes federation as a way to combine multiple APIs into a single federated GraphQL API. The client sends one GraphQL request to the router, and the router coordinates the API calls needed to produce one response.

Without federation, teams usually end up with one of two uncomfortable options. Either clients learn too much about backend service boundaries, or a central API layer becomes responsible for manually stitching together everything. The first option couples frontend code to backend topology. The second often turns into a slow-moving aggregation layer owned by nobody cleanly.

Federation takes a different route. Backend teams can own separate parts of the graph as subgraphs, while clients continue to query a single API shape.

The client does not need to know which service owns users, billing, schedules, permissions, or analytics. It should only know the graph contract.

The supergraph

Subgraphs are what backend teams own. The supergraph is what clients query. The gateway uses the supergraph to decide where each part of the operation should run.

A subgraph is a GraphQL schema owned by one backend service or domain. It contributes types, fields, and relationships to the overall graph.

The supergraph is the composed result of those subgraphs. It contains the information the gateway needs to understand field ownership and build a plan for executing an operation across one or more services.

.
.
.

The gateway

It is tempting to think of a GraphQL gateway as a simple proxy in front of services. That model is too weak.

A proxy forwards requests. A federated GraphQL gateway understands the operation, validates it against the composed graph, builds a query plan, sends sub-requests to the relevant subgraphs, and assembles the response back into the shape the client requested.

Apollo Gateway's documentation explains that the gateway processes the supergraph SDL on startup. That SDL includes routing information for subgraphs, and the gateway uses it to create query plans that can execute across one or more subgraphs.

When a client asks for fields owned by a single subgraph, the gateway can send one sub-request. When the operation crosses ownership boundaries, the gateway has to coordinate multiple subgraph calls. Some of those calls may be independent. Some may depend on data returned from an earlier step.

The client still sees one GraphQL response.

.
.
.

Schema publication

In Apollo terminology, the gateway can be initialized with a supergraph SDL. A static supergraph means the gateway needs a restart to pick up schema changes. Apollo's gateway describe dynamic supergraph updates, where the gateway receives an initial supergraph and can later update it through a callback. Polling, webhooks, and file watchers are examples of update mechanisms.

It creates two separate architectural paths:

The control plane publishes and validates schema changes.
The data plane serves live GraphQL requests using the currently active supergraph.

The control plane is where subgraph schemas are registered, composed, checked, and promoted. The data plane is where client operations are planned and executed.

A field can exist in a subgraph implementation but still fail through the gateway if the composed supergraph has not been updated. A schema can compose locally but not be active in the runtime gateway. A backend service can be healthy while the graph artifact consumed by the gateway is stale.

.
.
.

Apollo Federation: federated schemas
Apollo Gateway: updating the supergraph schema

GraphQL Clicked When I Stopped Thinking About Endpoints

Aditya Chauhan — Mon, 22 Jun 2026 13:03:58 +0000

If you have built more than a couple of APIs, you have probably felt the friction of REST at scale. You ship an endpoint, the frontend team asks for one more field, you version the route, the mobile team needs a different shape of the same data, and six months later you are maintaining /v3/users/:id/full next to /v2/users/:id/summary and nobody remembers which one the Android app actually calls.

GraphQL was built to kill that exact pain. It is a query language and runtime that lets clients ask for precisely the data they need — no more, no less — from a single endpoint, against a strongly typed schema that doubles as living documentation.

GraphQL started making sense to me when I stopped treating it as "one endpoint instead of many" and started treating it as a backend-owned contract.

The client still asks for data. The response still comes back as JSON. But the important design work lives behind that request: the schema, resolvers, authorization rules, nullability choices, pagination model, and the cost of each field the backend exposes.

I recently spent time understanding GraphQL from this backend angle in a ruby production codebase. This is the mental model I wish I had before reading real GraphQL APIs.

.
.

GraphQL solves a data-shape problem

Most product screens do not naturally think in endpoints. They think in workflows.

A product screen may need viewer context, one primary resource, access state, display metadata, and the first page of related records. With fixed REST-style endpoints, teams often end up choosing between a few imperfect options:

Return too much data from a broad endpoint.
Make the frontend call several endpoints for one screen.
Add screen-specific endpoints that become hard to reuse.

GraphQL approaches this differently. The backend publishes a typed schema, and the client selects a permitted shape from that schema.

.
.
.

The schema is the public API surface

The schema answers questions like:

Which root fields exist?
Which types can clients query?
Which fields are available on each type?
Which arguments does a field accept?
Which fields require authorization?
Which fields need custom resolver logic?

Once a field is exposed, clients can start depending on it. That makes every schema field a product decision, not just an implementation detail.

.
.
.

Resolvers: Where the Work Happens

Resolvers are the part of the backend that turn fields into real values.

For a root field, a resolver may load a record using an external identifier, read request context, check access, and return an object that GraphQL can continue resolving.

For nested fields, the work can vary a lot. Some fields are simple attributes. Some are calculated methods. Some call another resolver. Some need stricter authorization than the parent object. Some load data from a join table or another service.

That is why GraphQL does not remove backend complexity. It organizes that complexity around fields.

This also explains a common production risk: a small-looking nested field can become expensive. If a resolver runs once for every item in a list and performs its own database query each time, the API can hit an N+1 problem quickly.

.
.
.

Mutations are write boundaries

Queries read data. Mutations change data.

The backend design question is not just "which method updates the record?" A mutation should define the full write contract:

What input is required?
Which object is being changed?
Who can perform the change?
What does success return?
Which validation errors can the user fix?
Which failures should be treated as system or authorization errors?
Which side effects run after success?

.
.
.

Connections make pagination explicit

Pagination was the part that initially felt the most noisy to me.

Instead of returning a plain list, GraphQL APIs often return a connection. A connection is a wrapper around one page of results.

The basic vocabulary is:

Connection: the paginated list wrapper.
Edge: one result in that list, plus metadata about the relationship.
Node: the actual object.
Cursor: the bookmark for this result.
PageInfo: metadata that tells the client whether another page exists and where to continue.

The part that made edges click for me was this:

Node is the thing. Edge is the relationship to that thing.

Imagine the same object appearing through two different relationships. The object itself is unchanged, but the metadata about each relationship may be different.

That metadata belongs to the relationship. It should not necessarily live on the object itself.

That is why edges exist. They give the API a place to put relationship-specific data, while nodes stay focused on the actual object.

.
.
.

Generated types make contract drift visible

In many modern GraphQL frontends, operation files are not just raw strings. Tooling reads the backend schema and the client operations, then generates typed variables, typed response shapes, and query or mutation hooks.

This creates useful feedback.

If the backend removes a field that an operation uses, code generation can fail. If a backend field changes from always-present to maybe-null, the generated frontend type changes. If an enum value changes, the compiler may catch places that need to be updated.

This is one of GraphQL's strongest collaboration benefits. The contract can be checked before the bug reaches a browser.

But it only works if the schema, operations, and generated files stay in sync.

For backend engineers, that means a schema change is not done just because the server boots. You also need to think about generated client impact:

Which operations select this field?
Are enum values stable?
Is this a removal or should it be a deprecation?
Will generated types change in a meaningful way?

.
.
.

GraphQL is a tradeoff, not an automatic upgrade

After this learning cycle, I do not think of GraphQL as better REST.

It is a different API design tradeoff.

GraphQL is a strong fit when product screens need precise nested data, different clients need different slices of the same domain object, and typed contracts improve frontend/backend collaboration.

It may be the wrong tool when the workflow is a simple file upload, a command with no meaningful selected response shape, a heavy async process, or an API where caching and strict cost control matter more than selection flexibility.

The main tradeoff is visibility.

With a REST endpoint, request cost is often easier to reason about from the route. With GraphQL, request cost depends on the operation shape. One endpoint can represent many different execution paths.

DEV Community: Aditya Chauhan

Protobuf vs JSON

1. What is JSON?

2. Why is JSON Preferred?

3. What are Protocol Buffers?

4. Protobuf vs JSON: Key Differences

5. How to Use Protobuf in Ruby

Step 1: Define the .proto File

Step 2: Generate Ruby Code

Step 3: Use in Your Application

6. Real-World API Patterns

Pattern 1: Twirp (JSON over HTTP with Protobuf Schema)

Pattern 2: gRPC (Binary over HTTP/2)

7. When to Use Each

Use Protobuf when:

Use JSON when:

The Hybrid Approach (Best of Both)

8. Common Gotchas

1. Field Numbers Matter

2. Default Values

3. Required vs Optional

9. Conclusion

Mental Model for Federated GraphQL

GraphQL federation

The supergraph

The gateway

Schema publication

GraphQL Clicked When I Stopped Thinking About Endpoints

GraphQL solves a data-shape problem

The schema is the public API surface

Resolvers: Where the Work Happens

Mutations are write boundaries

Connections make pagination explicit

Generated types make contract drift visible

GraphQL is a tradeoff, not an automatic upgrade

Step 1: Define the `.proto` File