DEV Community

Cover image for Building a Service Mesh From Scratch
Derek Mwale
Derek Mwale

Posted on

Building a Service Mesh From Scratch

There is a moment in every distributed system when the application stops being a collection of services and starts becoming a small country.

At first, you have three services.

An API.

An authentication service.

A database.

Everything is simple.

The API calls authentication. Authentication talks to the database. Requests move around. Logs tell you what happened. If something breaks, you open the code and fix it.

Then the system grows.

Ten services.

Twenty.

Fifty.

Suddenly, every service needs to know about retries.

Every service needs timeouts.

Every service needs authentication between services.

Every service needs metrics.

Every service needs distributed tracing.

Every service needs circuit breakers.

Every service needs traffic policies.

And every team implements these things slightly differently.

That is where the idea of a service mesh becomes interesting.

A service mesh is not simply “a proxy in front of every service.”

That description is technically convenient and architecturally incomplete.

A service mesh is a distributed infrastructure layer for controlling service-to-service communication.

The application should focus on business logic.

The mesh should handle much of the communication logic surrounding that business logic.

That distinction is powerful.

Instead of every service learning how to perform retries, enforce mutual TLS, discover other services, emit telemetry, route traffic, and survive network failures, we can move those responsibilities into infrastructure surrounding the services.

The application says:

Process this order.
Enter fullscreen mode Exit fullscreen mode

The mesh says:

How should the request reach the order service?
Should the request be encrypted?
Should it be retried?
How long should we wait?
Which version should receive it?
Should this request be traced?
Is the destination healthy?
Should traffic be shifted?
Enter fullscreen mode Exit fullscreen mode

The application remains concerned with what.

The mesh becomes concerned with how.

And today, we are going to build one.

Not a production replacement for Istio, Linkerd, or another mature service-mesh implementation.

Something more interesting.

A service mesh from first principles.

We are going to discover what actually makes a service mesh a service mesh.


The Problem We Are Actually Solving

Imagine this architecture:

                 ┌──────────────┐
                 │ API Gateway  │
                 └──────┬───────┘
                        │
              ┌─────────┴─────────┐
              │                   │
        ┌─────▼─────┐       ┌─────▼─────┐
        │   Users   │       │   Orders  │
        └─────┬─────┘       └─────┬─────┘
              │                   │
              └─────────┬─────────┘
                        │
                  ┌─────▼─────┐
                  │ Payments  │
                  └───────────┘
Enter fullscreen mode Exit fullscreen mode

It looks simple.

But imagine that Orders calls Payments.

Now ask:

What happens if Payments takes 10 seconds to respond?

What happens if Payments is temporarily unavailable?

What happens if there are three versions of Payments?

What happens if one Payments instance is unhealthy?

What happens if the network connection fails after the payment request reaches the server but before the response reaches Orders?

What happens if Orders needs to prove its identity to Payments?

What happens if we want to send 5% of traffic to Payments v2?

What happens if we need to trace:

API → Orders → Payments → Ledger
Enter fullscreen mode Exit fullscreen mode

across multiple machines?

Without a mesh, application developers often implement these behaviors directly.

That means:

Orders
 ├── retry logic
 ├── timeout logic
 ├── TLS logic
 ├── discovery logic
 ├── load balancing
 ├── metrics
 ├── tracing
 └── circuit breaker

Payments
 ├── retry logic
 ├── timeout logic
 ├── TLS logic
 ├── discovery logic
 ├── load balancing
 ├── metrics
 ├── tracing
 └── circuit breaker
Enter fullscreen mode Exit fullscreen mode

We have duplicated infrastructure logic.

A service mesh changes the architecture.

                CONTROL PLANE
        ┌─────────────────────────┐
        │ Service Registry        │
        │ Configuration           │
        │ Certificates            │
        │ Routing Policies        │
        │ Health Information      │
        └────────────┬────────────┘
                     │
          ┌──────────┴──────────┐
          │                     │
     ┌────▼────┐           ┌────▼────┐
     │ Proxy   │           │ Proxy   │
     │ Sidecar │           │ Sidecar │
     └────┬────┘           └────┬────┘
          │                     │
     ┌────▼────┐           ┌────▼────┐
     │ Orders  │──────────▶│Payments │
     └─────────┘           └─────────┘
Enter fullscreen mode Exit fullscreen mode

The application does not need to understand all of this.

The proxy does.


The Two Halves of a Service Mesh

A useful mental model is that a service mesh has two major components:

             SERVICE MESH
                  │
        ┌─────────┴─────────┐
        │                   │
   CONTROL PLANE       DATA PLANE
        │                   │
 Configuration          Requests
 Discovery              Routing
 Certificates           Retries
 Policies               Timeouts
                         Telemetry
Enter fullscreen mode Exit fullscreen mode

The control plane makes decisions.

The data plane executes them.

This separation is one of the most important ideas in modern distributed infrastructure.

The control plane might say:

orders should send 80% of traffic to payments-v1
orders should send 20% to payments-v2
timeout = 2 seconds
retry = 2 attempts
TLS = required
Enter fullscreen mode Exit fullscreen mode

The data plane receives an actual request and applies those rules.

This gives us an important principle:

The control plane should not be in the critical path of every request.

If the control plane disappears for five minutes, existing proxies should ideally continue serving traffic using their last known configuration.

That is the difference between:

control plane controls traffic
Enter fullscreen mode Exit fullscreen mode

and:

control plane carries traffic
Enter fullscreen mode Exit fullscreen mode

A good mesh does the first.


Our Architecture

Let's build a simplified architecture.

                         ┌──────────────────────┐
                         │    Control Plane     │
                         │                      │
                         │ Service Registry     │
                         │ Config Store         │
                         │ Policy Engine        │
                         │ Certificate Manager  │
                         └──────────┬───────────┘
                                    │
                        Configuration / Updates
                                    │
              ┌─────────────────────┴─────────────────────┐
              │                                           │
        ┌─────▼─────┐                               ┌─────▼─────┐
        │   Proxy   │                               │   Proxy   │
        │  Orders   │                               │ Payments  │
        └─────┬─────┘                               └─────┬─────┘
              │                                           │
        ┌─────▼─────┐                               ┌─────▼─────┐
        │  Orders   │ ───────── Service ─────────▶ │ Payments  │
        └───────────┘       Communication          └───────────┘
Enter fullscreen mode Exit fullscreen mode

Our first version needs:

  1. Service registration
  2. Service discovery
  3. Local proxy
  4. Request forwarding
  5. Load balancing
  6. Health checking
  7. Timeouts
  8. Retries
  9. Circuit breaking
  10. Routing policies
  11. Mutual TLS
  12. Telemetry
  13. Dynamic configuration

That sounds like a lot.

It is.

That is precisely why service meshes are interesting.

They sit at the intersection of networking, distributed systems, security, observability, and operating systems.


Step One: Service Registration

Before a service can communicate with another service, we need to know where it lives.

Suppose Payments has three instances:

payments-1 → 10.0.0.11:8080
payments-2 → 10.0.0.12:8080
payments-3 → 10.0.0.13:8080
Enter fullscreen mode Exit fullscreen mode

We could hardcode these addresses.

That would be terrible.

Instances disappear.

Containers restart.

IP addresses change.

Machines fail.

Instead, services register themselves.

Our registry might expose:

POST /services/register
Enter fullscreen mode Exit fullscreen mode

With:

{
  "name": "payments",
  "address": "10.0.0.11",
  "port": 8080,
  "version": "v1"
}
Enter fullscreen mode Exit fullscreen mode

The registry stores:

payments:
  - 10.0.0.11:8080
  - 10.0.0.12:8080
  - 10.0.0.13:8080
Enter fullscreen mode Exit fullscreen mode

But registration alone is not enough.

We need leases.

A service should not remain registered forever if it has died.

So we give every registration a TTL.

For example:

registration TTL = 15 seconds
heartbeat interval = 5 seconds
Enter fullscreen mode Exit fullscreen mode

The service periodically sends:

POST /services/payments/heartbeat
Enter fullscreen mode Exit fullscreen mode

If heartbeats stop, the registry eventually removes the instance.

This gives us:

service lifecycle
       │
       ├── register
       │
       ├── heartbeat
       │
       ├── heartbeat
       │
       └── disappear
              │
              ▼
         expire lease
Enter fullscreen mode Exit fullscreen mode

This is already a distributed-systems problem.

You are not really asking:

“Is this service alive?”

You are asking:

“When should the system stop believing that this service is alive?”

Those are very different questions.


Step Two: Service Discovery

Now Orders wants to call Payments.

Orders does not need to know:

10.0.0.11
10.0.0.12
10.0.0.13
Enter fullscreen mode Exit fullscreen mode

It asks the control plane:

GET /services/payments
Enter fullscreen mode Exit fullscreen mode

The control plane returns:

{
  "service": "payments",
  "instances": [
    {
      "address": "10.0.0.11",
      "port": 8080,
      "version": "v1"
    },
    {
      "address": "10.0.0.12",
      "port": 8080,
      "version": "v1"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

But there is another important optimization.

The proxy should cache discovery information.

We do not want this:

Every request
     │
     ▼
Control Plane
     │
     ▼
Service Instance
Enter fullscreen mode Exit fullscreen mode

That would turn the control plane into a bottleneck.

Instead:

                 Control Plane
                      │
                configuration
                      │
                      ▼
                 Local Proxy
                      │
              ┌───────┼───────┐
              ▼       ▼       ▼
           request request request
              │       │       │
              └───────┴───────┘
Enter fullscreen mode Exit fullscreen mode

The proxy maintains a local view of the service topology.


Step Three: The Sidecar Proxy

Now we reach the heart of the architecture.

Every service gets a proxy.

┌─────────────────────────┐
│         Orders          │
│                         │
│   application           │
│                         │
└───────────┬─────────────┘
            │
            │ localhost
            ▼
┌─────────────────────────┐
│      Orders Proxy       │
│                         │
│ discovery               │
│ routing                 │
│ retries                 │
│ timeout                 │
│ TLS                     │
│ metrics                 │
└───────────┬─────────────┘
            │
            │ network
            ▼
       Payments Proxy
            │
            ▼
       Payments App
Enter fullscreen mode Exit fullscreen mode

The application sends requests to its local proxy.

The proxy forwards them.

This is the sidecar model.

The application can simply call:

http://payments/pay
Enter fullscreen mode Exit fullscreen mode

while the proxy decides how that logical destination maps to an actual instance.

This is where the mesh becomes transparent.


Step Four: Intercepting Traffic

How does the application automatically send traffic through the proxy?

There are multiple approaches.

One is explicit configuration.

The application sends requests to:

localhost:15001
Enter fullscreen mode Exit fullscreen mode

The proxy then handles routing.

Another approach uses network-level traffic interception.

For example:

Application
     │
     ▼
iptables / eBPF / networking layer
     │
     ▼
Proxy
     │
     ▼
Destination
Enter fullscreen mode Exit fullscreen mode

The application thinks it is connecting directly to another service.

The networking layer redirects the connection.

This is one of the reasons service meshes can feel almost magical.

The application has not changed.

The network behavior has.


Step Five: The Forwarding Engine

Our proxy needs a simple pipeline.

Conceptually:

request
   │
   ▼
parse
   │
   ▼
identify destination
   │
   ▼
load configuration
   │
   ▼
select instance
   │
   ▼
apply timeout
   │
   ▼
apply retry policy
   │
   ▼
send request
   │
   ▼
collect telemetry
   │
   ▼
response
Enter fullscreen mode Exit fullscreen mode

A simplified proxy API might look like:

Proxy.forward(request):
    destination = resolve(request.host)

    policy = policy_for(destination)

    instance = load_balancer.choose(destination)

    response = send(
        instance,
        timeout=policy.timeout
    )

    if should_retry(response, policy):
        retry()

    return response
Enter fullscreen mode Exit fullscreen mode

But distributed systems immediately make this more complicated.

What counts as a retryable failure?

A TCP connection failure?

HTTP 503?

HTTP 429?

A timeout?

What if the request reached the server?

This is where engineering gets interesting.


Retries Are Dangerous

Retries sound harmless.

They are not.

Imagine:

Orders → Payments
Enter fullscreen mode Exit fullscreen mode

Orders sends:

POST /charge
Enter fullscreen mode Exit fullscreen mode

Payments successfully charges the customer.

But the response is lost.

Orders sees:

timeout
Enter fullscreen mode Exit fullscreen mode

The proxy retries.

Now Payments receives:

POST /charge
Enter fullscreen mode Exit fullscreen mode

again.

The customer may be charged twice.

Therefore:

A retry policy is not simply a networking feature. It is an application semantics problem.

Safe retries usually require idempotency or knowledge that the operation is safe to repeat.

For example:

Idempotency-Key: payment-12345
Enter fullscreen mode Exit fullscreen mode

Payments can recognize duplicate attempts.

Our mesh can enforce conservative retry policies, but it cannot magically determine whether every operation is semantically safe.

That boundary matters.


Load Balancing

Suppose we have:

payments-1
payments-2
payments-3
Enter fullscreen mode Exit fullscreen mode

Our proxy needs to choose one.

The simplest strategy is round robin:

request 1 → payments-1
request 2 → payments-2
request 3 → payments-3
request 4 → payments-1
Enter fullscreen mode Exit fullscreen mode

Another option is weighted routing:

payments-v1 → 90%
payments-v2 → 10%
Enter fullscreen mode Exit fullscreen mode

Or least connections:

payments-1 → 14 active
payments-2 → 4 active
payments-3 → 9 active
Enter fullscreen mode Exit fullscreen mode

Choose:

payments-2
Enter fullscreen mode Exit fullscreen mode

We can define an interface:

LoadBalancer.choose(instances)
Enter fullscreen mode Exit fullscreen mode

and implement:

RoundRobin
Weighted
Random
LeastConnections
ConsistentHash
Enter fullscreen mode Exit fullscreen mode

Now routing becomes policy rather than application code.


Health Checking

A registered service is not necessarily a healthy service.

We therefore need active health checks.

The proxy can periodically call:

GET /health
Enter fullscreen mode Exit fullscreen mode

If the response is:

200 OK
Enter fullscreen mode Exit fullscreen mode

we consider the instance healthy.

If it repeatedly fails:

failure count = 1
failure count = 2
failure count = 3
Enter fullscreen mode Exit fullscreen mode

we temporarily remove the instance from the load-balancing pool.

But health checking introduces another distributed-systems tradeoff.

A service can be healthy from one machine's perspective and unreachable from another.

There is no universal magical concept of “network health.”

Health information is contextual.

Our system should therefore treat health as a signal, not absolute truth.


Circuit Breaking

Now imagine Payments is broken.

Orders keeps sending requests.

Requests fail.

Retries make things worse.

Traffic increases.

The failing service becomes even more overloaded.

We need a circuit breaker.

The classic model is:

             failures
                │
                ▼
        ┌────────────────┐
        │     CLOSED     │
        └───────┬────────┘
                │
          failure threshold
                │
                ▼
        ┌────────────────┐
        │      OPEN      │
        └───────┬────────┘
                │
            cooldown
                │
                ▼
        ┌────────────────┐
        │   HALF-OPEN    │
        └───────┬────────┘
                │
         ┌──────┴──────┐
         ▼             ▼
      success        failure
         │             │
         ▼             ▼
      CLOSED          OPEN
Enter fullscreen mode Exit fullscreen mode

Suppose:

failure threshold = 5
cooldown = 30 seconds
Enter fullscreen mode Exit fullscreen mode

After five consecutive failures, the proxy stops sending traffic.

Instead, it fails quickly.

This protects the system from cascading failures.

A service mesh is therefore not just routing infrastructure.

It becomes a failure-management layer.


Timeouts

Every network request should have a deadline.

Without a timeout:

Orders
   │
   └──── waiting forever ────▶ Payments
Enter fullscreen mode Exit fullscreen mode

One blocked dependency can consume all available threads.

Instead:

deadline = 2 seconds
Enter fullscreen mode Exit fullscreen mode

After two seconds:

timeout
Enter fullscreen mode Exit fullscreen mode

But timeout configuration is subtle.

Suppose:

API timeout = 5 seconds
Orders timeout = 4 seconds
Payments timeout = 3 seconds
Database timeout = 2 seconds
Enter fullscreen mode Exit fullscreen mode

The request's total budget must make sense.

A sophisticated mesh can propagate deadlines.

For example:

X-Request-Deadline: 2026-09-18T19:00:05Z
Enter fullscreen mode Exit fullscreen mode

Every downstream service receives the remaining budget.

This creates a powerful concept:

A distributed request should have a distributed deadline.

Otherwise, downstream services may continue working after the caller has already given up.


Mutual TLS

Now we have a security problem.

How does Payments know that a request really came from Orders?

IP addresses are not identities.

We need cryptographic identity.

This is where mutual TLS enters.

Instead of:

Orders ───── TLS ────▶ Payments
Enter fullscreen mode Exit fullscreen mode

we have:

Orders Proxy
     │
     │ client certificate
     ▼
Payments Proxy
     │
     │ server certificate
     ▼
validated identity
Enter fullscreen mode Exit fullscreen mode

Both sides authenticate each other.

Conceptually:

Orders
  │
  │ "I am orders"
  │
  ▼
Certificate Authority
  │
  ▼
certificate
Enter fullscreen mode Exit fullscreen mode

The control plane can issue identities.

For example:

spiffe://mesh/services/orders
spiffe://mesh/services/payments
Enter fullscreen mode Exit fullscreen mode

Now authorization can become identity-based.

We can define:

orders → payments = allowed
users → payments = denied
analytics → payments = read-only
Enter fullscreen mode Exit fullscreen mode

This is much stronger than trusting network location.


Certificate Rotation

Certificates expire.

Therefore, the control plane needs certificate rotation.

A proxy might receive:

certificate lifetime = 1 hour
Enter fullscreen mode Exit fullscreen mode

Before expiration, it requests a new certificate.

Proxy
 │
 ├── certificate valid
 │
 ├── renew
 │
 ├── receive new certificate
 │
 └── continue traffic
Enter fullscreen mode Exit fullscreen mode

The application does not need to know.

This is another example of infrastructure absorbing operational complexity.


Authorization Policies

Now let's build a simple policy model.

{
  "source": "orders",
  "destination": "payments",
  "methods": ["POST"],
  "path": "/charge",
  "action": "allow"
}
Enter fullscreen mode Exit fullscreen mode

The proxy evaluates the request:

Who are you?
       │
       ▼
orders

Where are you going?
       │
       ▼
payments

What are you doing?
       │
       ▼
POST /charge

Policy?
       │
       ▼
ALLOW
Enter fullscreen mode Exit fullscreen mode

This gives us a policy engine.

The application no longer needs to implement every network authorization rule.


Routing Rules

Suppose Payments v2 is ready.

We don't want to send 100% of traffic to it immediately.

Our control plane can configure:

payments-v1 = 95%
payments-v2 = 5%
Enter fullscreen mode Exit fullscreen mode

The proxy receives:

POST /charge
Enter fullscreen mode Exit fullscreen mode

and chooses:

v1
Enter fullscreen mode Exit fullscreen mode

or:

v2
Enter fullscreen mode Exit fullscreen mode

according to the configured weights.

This enables canary deployments.

We could gradually move:

99 / 1
95 / 5
90 / 10
75 / 25
50 / 50
0 / 100
Enter fullscreen mode Exit fullscreen mode

without changing the application.

That is one of the most compelling features of a service mesh.


Observability

Distributed systems create a nasty debugging problem.

A user reports:

“The checkout is slow.”

Where is the latency?

Gateway
   │ 40ms
   ▼
Orders
   │ 200ms
   ▼
Payments
   │ 1500ms
   ▼
Ledger
Enter fullscreen mode Exit fullscreen mode

Without distributed tracing, this can be painful.

The mesh can automatically generate telemetry.

Every request can receive a trace identifier:

trace-id = 8af7c91
Enter fullscreen mode Exit fullscreen mode

Then:

API
 └── trace 8af7c91
      └── Orders
           └── Payments
                └── Ledger
Enter fullscreen mode Exit fullscreen mode

We can measure:

request count
error rate
latency
retry count
timeouts
connection failures
circuit state
Enter fullscreen mode Exit fullscreen mode

The proxy knows about communication because communication is literally its job.


Metrics

Our proxy can expose:

mesh_requests_total
mesh_request_errors_total
mesh_request_duration_seconds
mesh_retries_total
mesh_timeouts_total
mesh_circuit_open_total
Enter fullscreen mode Exit fullscreen mode

Then dashboards can show:

Payments

Requests/sec:        4,200
Error rate:          0.8%
p95 latency:         180ms
p99 latency:         740ms
Retries:             124
Enter fullscreen mode Exit fullscreen mode

Now infrastructure becomes observable without forcing developers to manually instrument every HTTP client.


Distributed Tracing Context

A proxy can propagate tracing metadata.

For example:

traceparent: 00-4bf92f3577b34da6-00f067aa0ba902b7-01
Enter fullscreen mode Exit fullscreen mode

The important idea is not the specific header.

It is propagation.

incoming request
       │
       ▼
proxy extracts context
       │
       ▼
proxy creates child span
       │
       ▼
downstream request
       │
       ▼
destination proxy
       │
       ▼
destination application
Enter fullscreen mode Exit fullscreen mode

This allows a distributed operation to become one observable execution graph.


Dynamic Configuration

One of the most important control-plane capabilities is dynamic configuration.

Suppose we change:

timeout: 2s
Enter fullscreen mode Exit fullscreen mode

to:

timeout: 1s
Enter fullscreen mode Exit fullscreen mode

We should not need to restart every proxy.

The control plane publishes a configuration update.

Control Plane
      │
      ├──────▶ Proxy A
      │
      ├──────▶ Proxy B
      │
      ├──────▶ Proxy C
      │
      └──────▶ Proxy D
Enter fullscreen mode Exit fullscreen mode

Each proxy updates its local configuration.

This is where a service mesh starts looking less like a collection of proxies and more like a distributed operating system for services.


Configuration Versioning

Dynamic configuration introduces another problem.

What if updates arrive out of order?

Imagine:

version 10
version 11
version 12
Enter fullscreen mode Exit fullscreen mode

Proxy receives:

12
11
Enter fullscreen mode Exit fullscreen mode

It must not roll backward.

Therefore configuration should have versions.

For example:

{
  "version": 12,
  "routes": [...]
}
Enter fullscreen mode Exit fullscreen mode

The proxy only accepts:

new_version > current_version
Enter fullscreen mode Exit fullscreen mode

This seems trivial.

At scale, details like this become the difference between deterministic infrastructure and mysterious distributed behavior.


The Control Plane Should Be Eventually Consistent

Here is a philosophical shift.

We often want configuration to be instantly consistent everywhere.

But that can be expensive.

Suppose there are:

10,000 proxies
Enter fullscreen mode Exit fullscreen mode

A configuration update must propagate.

There will be some period where:

Proxy A → v12
Proxy B → v12
Proxy C → v11
Proxy D → v11
Enter fullscreen mode Exit fullscreen mode

This is normal.

The important question is:

Is temporary inconsistency safe?

If yes, eventual propagation can be perfectly reasonable.

The control plane can therefore operate using a model where configuration converges.

That is a fundamental distributed-systems idea:

We do not always need everyone to know everything immediately. We need the system to converge toward the correct state.


Failure of the Control Plane

Now let's kill our control plane.

Everything should not immediately collapse.

Existing proxies should have:

cached service endpoints
cached policies
cached certificates
cached routing configuration
Enter fullscreen mode Exit fullscreen mode

Therefore:

Control Plane DOWN
        │
        ▼
Existing Proxy
        │
        ├── cached routing
        ├── cached endpoints
        ├── cached policy
        └── cached certificates
        │
        ▼
traffic continues
Enter fullscreen mode Exit fullscreen mode

Some capabilities may degrade.

New services may not register.

Configuration changes may not propagate.

Certificates may eventually need renewal.

But existing traffic should remain operational for as long as possible.

This is an extremely important availability principle:

The control plane should fail independently from the data plane.


Backpressure

There is another problem.

Suppose Payments can process:

5,000 requests/sec
Enter fullscreen mode Exit fullscreen mode

but Orders sends:

20,000 requests/sec
Enter fullscreen mode Exit fullscreen mode

Something has to give.

Without backpressure, queues grow until memory disappears and latency explodes.

A proxy can implement connection limits:

max concurrent requests = 1,000
Enter fullscreen mode Exit fullscreen mode

Additional requests may receive:

HTTP 503
Enter fullscreen mode Exit fullscreen mode

or be queued within controlled limits.

The goal is not to make every request succeed.

The goal is to prevent overload from becoming systemic failure.


Connection Pooling

Opening a new TCP connection for every request is wasteful.

Our proxy can maintain connection pools.

Payments connection pool

conn-1
conn-2
conn-3
conn-4
conn-5
Enter fullscreen mode Exit fullscreen mode

Requests reuse existing connections.

For HTTP/2, multiple requests can share a connection.

The mesh therefore becomes responsible for transport optimization.

Again:

The application says:

Call Payments.
Enter fullscreen mode Exit fullscreen mode

The mesh decides:

Which connection?
Which instance?
Which protocol?
Which timeout?
Which policy?
Enter fullscreen mode Exit fullscreen mode

A Simple Proxy State Machine

At this point, our proxy has a lot of state.

We can model it explicitly:

┌───────────────┐
│ INITIALIZING  │
└───────┬───────┘
        │
        ▼
┌───────────────┐
│ SYNCING       │
│ CONFIGURATION │
└───────┬───────┘
        │
        ▼
┌───────────────┐
│ READY         │
└───────┬───────┘
        │
        ├──────────────┐
        │              │
        ▼              ▼
   CONFIG UPDATE    CONTROL PLANE
                    DISCONNECTED
        │              │
        └──────┬───────┘
               ▼
             READY
Enter fullscreen mode Exit fullscreen mode

The proxy should distinguish between:

no configuration
Enter fullscreen mode Exit fullscreen mode

and:

control plane temporarily unavailable
Enter fullscreen mode Exit fullscreen mode

The first might mean:

do not serve traffic
Enter fullscreen mode Exit fullscreen mode

The second might mean:

continue using cached state
Enter fullscreen mode Exit fullscreen mode

These distinctions matter.


Building the Data Plane

A minimal implementation can have these modules:

proxy/
├── listener
├── router
├── discovery
├── load_balancer
├── health
├── retry
├── timeout
├── circuit_breaker
├── tls
├── policy
├── telemetry
└── connection_pool
Enter fullscreen mode Exit fullscreen mode

The request flow becomes:

Listener
   │
   ▼
Router
   │
   ▼
Policy
   │
   ▼
Discovery
   │
   ▼
Load Balancer
   │
   ▼
Circuit Breaker
   │
   ▼
Retry + Timeout
   │
   ▼
TLS Connection
   │
   ▼
Destination
Enter fullscreen mode Exit fullscreen mode

We can implement this in Rust, Go, C++, Java, or another language suited to network services.

Rust is particularly interesting for this kind of system because the proxy is fundamentally a highly concurrent network program.

But the architecture matters more than the language.


Building the Control Plane

Our control plane could expose APIs such as:

POST /services/register
POST /services/heartbeat

GET /services/:name

POST /routes
GET /routes/:service

POST /policies
GET /policies/:service

POST /certificates/issue
Enter fullscreen mode Exit fullscreen mode

Internally:

Control Plane
│
├── Registry
├── Config Store
├── Policy Engine
├── Certificate Authority
└── Event Bus
Enter fullscreen mode Exit fullscreen mode

The event bus allows updates to propagate.

For example:

route changed
     │
     ▼
event: ROUTE_UPDATED
     │
     ├──▶ Proxy A
     ├──▶ Proxy B
     ├──▶ Proxy C
     └──▶ Proxy D
Enter fullscreen mode Exit fullscreen mode

Watching Instead of Polling

A naive proxy might ask:

GET configuration
Enter fullscreen mode Exit fullscreen mode

every five seconds.

That works.

But it is inefficient.

A better approach is a watch stream:

Proxy ───────────────▶ Control Plane
         subscribe

Control Plane
      │
      │ configuration update
      ▼
Proxy
Enter fullscreen mode Exit fullscreen mode

The proxy maintains a long-lived connection.

Whenever configuration changes, the control plane pushes an update.

This reduces polling overhead and improves propagation speed.

But now we must handle:

connection dropped
reconnect
resume
missed updates
version synchronization
Enter fullscreen mode Exit fullscreen mode

Again, distributed systems.


The Mesh as a Distributed State Machine

At this point, something deeper becomes visible.

A service mesh is not merely networking software.

It is a distributed state machine.

The control plane maintains a model of:

services
instances
identities
policies
routes
health
configuration
Enter fullscreen mode Exit fullscreen mode

The proxies maintain local replicas of relevant state.

The system continuously tries to converge.

                 GLOBAL STATE
                      │
              ┌───────┴────────┐
              ▼                ▼
          Proxy A          Proxy B
          local state       local state
              │                │
              └───────┬────────┘
                      │
                   traffic
Enter fullscreen mode Exit fullscreen mode

This is why building a service mesh forces you to understand distributed systems.

You are designing:

  • failure detection
  • replication
  • consistency
  • identity
  • state propagation
  • timeouts
  • retries
  • fault isolation
  • concurrency
  • observability

The network is only the surface.


The Hardest Problem: Failure

The happy path is easy.

request
   ↓
proxy
   ↓
healthy instance
   ↓
response
Enter fullscreen mode Exit fullscreen mode

The interesting architecture begins when everything goes wrong.

What if:

DNS fails?
Enter fullscreen mode Exit fullscreen mode

What if:

certificate expires?
Enter fullscreen mode Exit fullscreen mode

What if:

control plane disappears?
Enter fullscreen mode Exit fullscreen mode

What if:

proxy crashes?
Enter fullscreen mode Exit fullscreen mode

What if:

service registration becomes stale?
Enter fullscreen mode Exit fullscreen mode

What if:

configuration versions diverge?
Enter fullscreen mode Exit fullscreen mode

What if:

the request reaches the destination but the response is lost?
Enter fullscreen mode Exit fullscreen mode

What if:

all healthy instances become overloaded?
Enter fullscreen mode Exit fullscreen mode

What if:

a retry storm begins?
Enter fullscreen mode Exit fullscreen mode

A service mesh is essentially a machine for making these failures predictable.


Retry Storms

Consider:

API
 │
 ▼
Orders
 │
 ▼
Payments
Enter fullscreen mode Exit fullscreen mode

Payments starts returning errors.

Orders retries twice.

But 100 Orders requests are active.

Suddenly:

100 original requests
+
200 retries
=
300 Payments requests
Enter fullscreen mode Exit fullscreen mode

Now Payments becomes even more overloaded.

Retries have amplified the failure.

This is called a retry storm.

Our mesh should therefore support:

retry budgets
jitter
exponential backoff
maximum attempts
deadline propagation
Enter fullscreen mode Exit fullscreen mode

Instead of:

retry immediately
retry immediately
retry immediately
Enter fullscreen mode Exit fullscreen mode

we use something like:

attempt 1
   │
   └── 50ms
attempt 2
   │
   └── 100ms + jitter
attempt 3
Enter fullscreen mode Exit fullscreen mode

Even better, retries should be bounded by an overall request deadline.


Why Jitter Matters

Imagine 10,000 requests fail simultaneously.

If every proxy retries exactly after:

100ms
Enter fullscreen mode Exit fullscreen mode

then all 10,000 requests arrive together.

Again.

Instead:

100ms + random jitter
Enter fullscreen mode Exit fullscreen mode

spreads them out.

request A → 103ms
request B → 117ms
request C → 94ms
request D → 128ms
Enter fullscreen mode Exit fullscreen mode

Randomness becomes a stability mechanism.

That is one of the weirdest things about distributed systems:

Sometimes adding randomness makes a system more predictable.


Security Boundaries

A mesh also changes the security model.

Instead of trusting the internal network, we can define:

identity → policy → action
Enter fullscreen mode Exit fullscreen mode

For example:

service:orders
    ↓
identity verified
    ↓
policy evaluated
    ↓
POST /payments allowed
Enter fullscreen mode Exit fullscreen mode

This moves us toward zero-trust networking.

But the mesh is not automatically secure just because it uses TLS.

We still need to secure:

control plane APIs
certificate authority
configuration distribution
proxy administration
service registration
identity issuance
Enter fullscreen mode Exit fullscreen mode

If an attacker compromises the control plane, the consequences can be enormous.

The control plane is therefore part of the security perimeter.


What Happens When the Proxy Dies?

This is one of the uncomfortable questions.

If the application depends on the sidecar:

Application → Proxy → Network
Enter fullscreen mode Exit fullscreen mode

what happens when:

Proxy = dead
Enter fullscreen mode Exit fullscreen mode

Depending on the deployment architecture, application networking may fail.

Therefore proxies need:

fast startup
resource limits
health checks
automatic restart
minimal memory footprint
Enter fullscreen mode Exit fullscreen mode

And the platform needs to understand that the proxy is infrastructure.

A service mesh introduces another moving part.

That is an important tradeoff.


The Cost of Abstraction

Service meshes solve problems.

They also create problems.

You now have:

application
proxy
control plane
configuration
certificates
telemetry
service registry
Enter fullscreen mode Exit fullscreen mode

instead of:

application
Enter fullscreen mode Exit fullscreen mode

This means:

more CPU
more memory
more latency
more operational complexity
more debugging layers
Enter fullscreen mode Exit fullscreen mode

If you have three services, you may not need a service mesh.

If you have hundreds of services operated by multiple teams, the economics change.

The lesson is not:

Every distributed system needs a service mesh.

The lesson is:

When communication complexity becomes systemic, communication deserves its own infrastructure layer.


A Minimal End-to-End Request

Let's put everything together.

Suppose Orders sends:

POST /charge
Enter fullscreen mode Exit fullscreen mode

The request enters the Orders proxy.

1. Identity

The proxy identifies the source:

orders
Enter fullscreen mode Exit fullscreen mode

2. Destination

The router determines:

payments
Enter fullscreen mode Exit fullscreen mode

3. Authorization

Policy engine checks:

orders → payments
Enter fullscreen mode Exit fullscreen mode

Result:

ALLOW
Enter fullscreen mode Exit fullscreen mode

4. Discovery

Proxy finds:

payments-v1
payments-v2
Enter fullscreen mode Exit fullscreen mode

5. Routing

Policy says:

v1 = 90%
v2 = 10%
Enter fullscreen mode Exit fullscreen mode

Proxy chooses:

payments-v2
Enter fullscreen mode Exit fullscreen mode

6. Load balancing

Three v2 instances exist.

Proxy chooses:

payments-v2-3
Enter fullscreen mode Exit fullscreen mode

7. Circuit breaker

Circuit is:

CLOSED
Enter fullscreen mode Exit fullscreen mode

Proceed.

8. Timeout

Deadline:

2 seconds
Enter fullscreen mode Exit fullscreen mode

9. TLS

Proxy establishes an authenticated encrypted connection.

10. Telemetry

Proxy records:

trace_id
latency
status
destination
retry_count
Enter fullscreen mode Exit fullscreen mode

11. Response

Payments responds:

200 OK
Enter fullscreen mode Exit fullscreen mode

The proxy records the result and returns it to Orders.

The application sees:

200 OK
Enter fullscreen mode Exit fullscreen mode

It does not need to know about most of the machinery.

That is the entire point.


A Better Mental Model

Many people think of a service mesh like this:

service + proxy
Enter fullscreen mode Exit fullscreen mode

I prefer this:

        SERVICE MESH

   ┌──────────────────────┐
   │ Communication Policy │
   └──────────┬───────────┘
              │
      ┌───────▼───────┐
      │ Control Plane │
      └───────┬───────┘
              │
       configuration
              │
 ┌────────────┴────────────┐
 ▼                         ▼
Proxy                     Proxy
 │                         │
Service                   Service
 │                         │
 └──────────network────────┘
Enter fullscreen mode Exit fullscreen mode

The proxy is the executor.

The control plane is the brain.

The network is the environment.

The application is the organism using the environment.

This framing makes the architecture easier to reason about.


What We Have Built

Our service mesh now has:

✓ Service discovery
✓ Service registration
✓ Health checking
✓ Local proxies
✓ Load balancing
✓ Timeouts
✓ Retries
✓ Circuit breakers
✓ Dynamic routing
✓ Mutual TLS
✓ Identity
✓ Authorization
✓ Metrics
✓ Distributed tracing
✓ Configuration propagation
✓ Connection pooling
✓ Backpressure
Enter fullscreen mode Exit fullscreen mode

And yet, this is still only the beginning.

A production-grade mesh introduces even more concerns:

HTTP/2
gRPC
HTTP/3
WebSockets
TCP proxying
DNS handling
xDS-style APIs
certificate rotation
multi-cluster networking
fault injection
traffic mirroring
locality-aware routing
outlier detection
ambient networking
eBPF acceleration
resource isolation
configuration validation
control-plane federation
Enter fullscreen mode Exit fullscreen mode

The rabbit hole goes deep.

Very deep.


The Bigger Idea

There is a broader software architecture lesson hidden inside the service mesh.

Applications keep accumulating responsibilities.

At some point, we recognize that certain responsibilities are not really application responsibilities.

Authentication becomes identity infrastructure.

Logging becomes observability infrastructure.

Databases become persistence infrastructure.

Containers become runtime infrastructure.

And service-to-service communication becomes mesh infrastructure.

This is a recurring pattern in computer science.

A problem becomes common enough that we stop solving it independently inside every application.

We build a platform.

The service mesh is one expression of that idea.


Build the Mesh to Understand the Network

You do not need to build a complete production service mesh to learn something valuable.

Build the smallest possible one.

Start with:

service registry
      ↓
service discovery
      ↓
proxy
      ↓
round-robin load balancing
Enter fullscreen mode Exit fullscreen mode

Then add:

health checks
Enter fullscreen mode Exit fullscreen mode

Then:

timeouts
Enter fullscreen mode Exit fullscreen mode

Then:

retries
Enter fullscreen mode Exit fullscreen mode

Then:

circuit breaking
Enter fullscreen mode Exit fullscreen mode

Then:

TLS
Enter fullscreen mode Exit fullscreen mode

Then:

identity
Enter fullscreen mode Exit fullscreen mode

Then:

routing policies
Enter fullscreen mode Exit fullscreen mode

Then:

telemetry
Enter fullscreen mode Exit fullscreen mode

Then:

dynamic configuration
Enter fullscreen mode Exit fullscreen mode

Each feature teaches you something different.

Service discovery teaches you distributed state.

Retries teach you failure semantics.

Circuit breakers teach you cascading failures.

TLS teaches you machine identity.

Routing teaches you traffic management.

Telemetry teaches you observability.

Dynamic configuration teaches you eventual consistency.

And proxying teaches you something even more fundamental:

the network itself can become programmable infrastructure.


Final Architecture

Our finished conceptual system looks like this:

                         ┌──────────────────────────┐
                         │       CONTROL PLANE      │
                         │                          │
                         │ Service Registry         │
                         │ Configuration Store      │
                         │ Routing Engine            │
                         │ Policy Engine             │
                         │ Certificate Authority     │
                         │ Event Distribution         │
                         └────────────┬─────────────┘
                                      │
                       Configuration / Identity
                                      │
             ┌────────────────────────┴────────────────────────┐
             │                                                 │
      ┌──────▼──────┐                                  ┌───────▼──────┐
      │ Orders Proxy│                                  │Payments Proxy│
      │             │                                  │              │
      │ Discovery   │                                  │ Discovery    │
      │ Routing     │                                  │ Routing      │
      │ LB          │                                  │ LB           │
      │ Retry       │                                  │ Retry        │
      │ Timeout     │                                  │ Timeout      │
      │ Circuit     │                                  │ Circuit      │
      │ TLS         │                                  │ TLS          │
      │ Telemetry   │                                  │ Telemetry    │
      └──────┬──────┘                                  └──────┬───────┘
             │                                                │
      ┌──────▼──────┐                                  ┌──────▼───────┐
      │   Orders    │ ─────────── network ───────────▶│   Payments   │
      └─────────────┘                                  └──────────────┘
Enter fullscreen mode Exit fullscreen mode

That diagram looks like networking.

But underneath it is something much deeper.

It is distributed systems theory turned into infrastructure.

A service mesh is a system that takes all the ugly questions surrounding network communication and turns them into explicit mechanisms:

Who are you?
Where are you?
Are you healthy?
Should I trust you?
Should I send traffic to you?
How much traffic?
For how long?
What happens if you fail?
Should I retry?
Should I stop retrying?
Can I encrypt this?
Can I observe this?
Can I change the behavior without redeploying?
Enter fullscreen mode Exit fullscreen mode

Those questions exist whether you build a mesh or not.

The difference is where you answer them.

Without a mesh, they tend to leak into application code.

With a mesh, they can become infrastructure policy.

And that is the real reason to build one from scratch.

Not because the world needs another service mesh.

But because building one forces you to understand what happens between your services.

And that space between services is where distributed systems actually live.

Code is easy when everything is local.

The interesting engineering begins when your code has to cross a network.

That is where latency appears.

That is where packets disappear.

That is where identities matter.

That is where retries become dangerous.

That is where failures become contagious.

And that is where architecture stops being a diagram and starts becoming a machine.

Build the proxy.

Build the control plane.

Break the network.

Kill the services.

Drop the connections.

Expire the certificates.

Send too much traffic.

Watch everything fail.

Then make it recover.

Because if you can build a service mesh from scratch, you are no longer just learning how services communicate.

You are learning how large software systems survive reality.

Top comments (0)