The Go Gateway: Confessions of a Professional Middleman

This is Part 2 of "The Muse of Microservices," a series on the system design behind Polyhymnia, a deliberately overengineered random quote generator. If you haven't met the whole cast yet, Part 1 covers the architecture end to end — worth reading first.

Inside Polyhymnia's single HTTP endpoint — orchestration vs choreography, deadline budgets, retry loops, and the art of turning three gRPC calls into one honest JSON response.

One endpoint, zero excuses

The entire public surface of Polyhymnia is a single route:

GET /api/quote

No path parameters, no query strings, no body. And yet everything the system does routes through it, because the Go service in front of it isn't a proxy — it's an orchestrator. It's the one component in the whole stack that knows, and enforces, the correct order of operations: fetch every ID, hand them to the randomizer, fetch the winning quote. Nothing downstream is trusted to figure that sequence out on its own, and nothing downstream is reachable from outside the cluster at all. Every other service in Polyhymnia speaks gRPC exclusively; Go is the only translator standing between "the internet" and "the mesh."

That's a real, deliberate pattern with a name: the API gateway. Its value isn't glamorous — it's a bulkhead. The public contract (plain HTTP, JSON) is decoupled from the internal contract (gRPC, protobuf), which means the internal services are free to change their wire format, their language, even their topology, without a single line of frontend code ever noticing.

Orchestration, chosen on purpose

There are two broad ways to coordinate a multi-step workflow across services. Choreography has each service react to events as they arrive — nobody's in charge, the sequence emerges from a chain of triggers. Orchestration puts one component in the driver's seat, explicitly calling each step and deciding what happens if one fails.

Polyhymnia picks orchestration, and the gateway's handler reads almost like pseudocode because of it: fetch the ID list, pass it to the randomizer, fetch the chosen quote, encode the response. Three sequential RPCs, one after another, all driven from a single function.

That choice has real tradeoffs, and Polyhymnia's small enough that you can see both sides clearly:

• You get it: a request's entire lifecycle is traceable in one place. Reading the handler top-to-bottom tells you exactly what happened for any given request — there's no event bus to reconstruct, no implicit ordering to reverse-engineer.
• You pay for it: the gateway is now coupled to the internal shape of the workflow. If a third backend service joined the pipeline tomorrow, the gateway's code — not some independent event contract — is what has to change.

For a three-step, single-consumer pipeline like this one, orchestration is the right call. It stops being the right call once you have a dozen services and every team wants to own their slice of the sequence independently — which is exactly the kind of judgment call real system design is made of.

A deadline budget, not a stopwatch per hop

Here's a detail that's easy to skim past and genuinely worth sitting with:

ctx, cancel := context.WithTimeout(r.Context(), callTimeout) // 5 seconds
defer cancel()

That one context is threaded through all three downstream calls. It is not "5 seconds per RPC" — it's a single 5-second budget for the entire request, shared across GetAllIds, SelectRandomId, and GetQuoteById combined. If the first call to Rust takes 4.5 seconds for whatever reason, the C++ engine and the second Rust call are left splitting the remaining half-second between them, whether they like it or not.

This is deadline propagation, and it's one of those concepts that looks academic right up until you've been paged for a service that kept retrying a downstream call for 30 seconds because nobody told it the caller had already given up 29 seconds ago. Polyhymnia's version is about as simple as deadline propagation gets — one shared context.Context, no per-hop sub-budgets — but the underlying idea, that a deadline should travel with a request rather than being reinvented at every hop, is exactly the idea production systems lean on.

Retrying like you mean it (sort of)

Before the gateway serves a single request, it has to actually connect to both backends:

const (
    maxRetries    = 30
    retryInterval = 500 * time.Millisecond
)

dialGrpc loops up to 30 times, sleeping 500ms between attempts, using grpc.WithBlock() so the dial call won't return until a connection actually succeeds (or every attempt is exhausted, at which point the gateway logs a fatal error and exits). That's roughly a 15-second window for the Rust and C++ services to become reachable before the gateway gives up on life entirely.

It's worth naming precisely what this is and isn't. It's a fixed-interval retry with a blocking dial — simple, predictable, and completely adequate for a small number of startup-order dependencies. It is not exponential backoff, and it has no jitter. That's a fine tradeoff here: with only two dependencies and a bounded startup window, a fixed 500ms cadence isn't going to hurt anyone. The moment you're retrying against a service under real load with many callers doing the same thing at once, a fixed interval turns into a synchronized herd hammering the same endpoint every 500ms — which is precisely the scenario exponential backoff with jitter exists to prevent. Good instinct to have in your back pocket even when, like here, the simpler tool is the right one for the job.

An error taxonomy that actually means something

The gateway's error handling is short, but it draws a distinction a lot of production code gets lazy about:

if len(idList.GetIds()) == 0 {
    writeError(w, http.StatusNotFound, "no quotes available")
    return
}

...versus:

if err != nil {
    writeError(w, http.StatusBadGateway, "failed to fetch quote ids")
    return
}

An empty result and a failed call are not the same kind of problem, and the gateway refuses to collapse them into one generic "something went wrong" response. A database with zero rows is a legitimate state of the world — the request succeeded, there was simply nothing to return, so it's a 404. A gRPC call that errors out means a dependency is broken or unreachable — that's a 502 Bad Gateway, correctly blaming the infrastructure rather than the caller. A request with the wrong HTTP method gets its own 405. Three distinct failure modes, three distinct status codes, and a frontend that can actually tell them apart if it ever needs to.

It's a small thing. It's also exactly the kind of small thing that separates an API that's pleasant to build against from one that returns 500 for everything and makes every client guess.

The localhost trap

Both the gateway's dial addresses and the Rust service's listen address are pinned to explicit IPv4:

rustDbAddr    = "127.0.0.1:50051"
cppEngineAddr = "127.0.0.1:50052"

The code comment explains why, and it's a genuinely useful bit of networking trivia: on some systems, localhost resolves to 127.0.0.1 (IPv4) while a server that bound to [::1] (IPv6-only) is listening on a different address entirely. The result is a maddening "connection refused" error against a server you can plainly see is running. Pinning both sides to the same explicit address family sidesteps the ambiguity completely.

File that one away — it's the kind of bug that costs an afternoon the first time and takes ten seconds to recognize every time after. It'll also come back with a twist in Part 4, once these same services move into separate Docker containers and "127.0.0.1" stops meaning what everyone assumed it meant.

What's next

The gateway's job is to ask the right questions in the right order and translate the answers into JSON. It doesn't know how a random ID actually gets picked, and it definitely doesn't know how the database it's querying decided to keep itself safe. Part 3 goes downstream into both of those services — Rust's data-ownership model and C++'s spectacularly unnecessary approach to picking a number — where the real personality of this project lives.