Your frontend knows too much.
It knows the hostname of the user service, the orders service, the payments service, the notifications service. It knows that service A wants a Bearer token and service B wants an API key in a custom header. It knows the rate limit for each one is different. It knows they all return errors in slightly different shapes. And every time you add a new service or change an auth flow, you're shipping a mobile update or redeploying the client.
That's the mess. That's what happens when clients talk directly to a dozen backends with no coordination layer between them.
The moment you break a monolith into services, you inherit this problem. Most teams solve it by accident first. They hardcode service URLs, duplicate auth logic, and scatter retry policies across three frontend codebases. Then they solve it deliberately, usually after something breaks in production.
🧠 What an API gateway actually is
An API gateway is a single entry point that sits between your clients and your backend services. Every request from every client hits the gateway first. The gateway figures out which service should handle it, and forwards it there.
That's it. The client stops knowing your internal topology.
Here's the before and after:
BEFORE (clients wired directly to services):
Mobile App ──→ User Service
Mobile App ──→ Orders Service
Mobile App ──→ Payments Service
Web App ────→ User Service
Web App ────→ Orders Service
Web App ────→ Notifications Service
AFTER (gateway in the middle):
Mobile App ──→ ┌─────────────┐ ──→ User Service
Web App ────→ │ API Gateway │ ──→ Orders Service
Partner API ─→ └─────────────┘ ──→ Payments Service
──→ Notifications Service
The client knows one hostname. One auth scheme. One error format. The gateway handles the rest.
The things it centralizes
This is where a gateway earns its keep. It takes a pile of cross-cutting concerns that would otherwise be duplicated across every service and puts them in one place.
Request routing is the obvious one. Path-based, host-based, header-based. A request to /api/users/123 goes to the user service, /api/orders goes to the orders service. The gateway also load balances across instances of each service.
But the one that matters most is authentication. If you validate tokens in twelve services, that's twelve implementations of the same logic. Twelve chances to get the parsing wrong, miss a clock-skew edge case, or forget to check token expiry. Do it once at the gateway instead.
A big caveat here though. Validating the token at the gateway handles authentication. It does not replace authorization inside your services. The gateway confirms "this is a valid user." Your service still needs to enforce "this user is allowed to access this resource." Teams miss this constantly.
Beyond routing and auth, a gateway typically handles:
- Rate limiting and quotas (per client, per endpoint, per plan)
- TLS termination so internal traffic can stay on plain HTTP
- Request and response transformation (stripping headers, reshaping payloads)
- Response aggregation, where one client call triggers multiple backend calls and the gateway stitches the responses together (this is basically the Backend for Frontend pattern, or BFF)
- Caching for responses that don't change often
- Observability: every request flows through one place, so you get logging, metrics, and trace correlation for free
That's a lot of duplicated work removed from your services.
⚡ How a request flows through it
Concrete walkthrough:
- Client sends HTTPS request to
api.yourapp.com - Gateway terminates TLS, decrypts the request
- Gateway matches the path/method against its routing table
- Gateway validates the auth token (JWT signature, expiry, audience)
- Gateway checks the rate limit for this client
- Gateway rewrites the path or headers if needed (maybe strips
/api/v2prefix) - Gateway forwards the request to the matched backend service
- Service processes and returns a response
- Gateway optionally transforms or aggregates the response
- Gateway emits metrics (latency, status code, route)
- Gateway returns the response to the client
The whole thing adds maybe 1-5ms of latency per hop. Usually worth it. Sometimes not.
Gateway, load balancer, reverse proxy
These three overlap heavily and confuse everyone. Here's the short version:
| Component | Layer | Main job | API-aware? |
|---|---|---|---|
| API gateway | L7 (HTTP) | Route, authenticate, transform API requests | Yes |
| Load balancer | L4 or L7 | Distribute traffic across instances | Not typically |
| Reverse proxy | L7 | Forward client requests to backend servers | Sometimes |
Honestly, an API gateway is a reverse proxy with API-specific features bolted on. And the same product (Kong, Envoy, Traefik) often does all three jobs depending on how you configure it. The distinction between forward and reverse proxies gets its own post.
Other names you'll run into: AWS API Gateway and Apigee on the managed side.
North-south versus east-west
A gateway handles north-south traffic. That's requests coming into your system from the outside world. Clients hitting your APIs.
East-west traffic is service-to-service communication inside your system. That's handled by a service mesh (Istio, Linkerd) typically using sidecar proxies attached to each service. Different problem, different tool.
So teams that buy a service mesh thinking it'll solve their client-facing API problems are buying the wrong thing. And teams that try to route internal service calls through their API gateway are creating a bottleneck where none needs to exist.
🛠️ The anti-pattern that ruins gateways
Here's what kills a gateway over time: business logic creeping in.
It starts small. Someone adds a response transformation that strips a field for mobile clients. Then a conditional: if the user is on plan X, add this header. Then someone puts an order-validation rule in the gateway config because it was "easier than deploying the orders service."
Now you have a distributed monolith. The gateway config is a codebase. The gateway team is a deployment bottleneck for every other team. Every feature change requires a gateway deploy.
Routing, auth, rate limiting, and cross-cutting concerns belong in the gateway. Domain logic does not. Full stop.
When you don't need one
The title promised honesty, so here it is.
A single service or a monolith gains nothing from a gateway. You're adding a network hop, an operational dependency, and a component that must be highly available because if the gateway is down, everything is down. That's a single point of failure unless you deploy it with redundancy, which is more infrastructure to manage.
Small teams feel this the hardest. You're now operating and monitoring an extra piece of infrastructure that needs health checks, autoscaling, and its own deployment pipeline. For two services, that's overhead you don't need.
And managed gateways (AWS API Gateway, for instance) charge per request. At low volume that's fine. At high volume, the bill gets surprising fast.
If you have one or two services, skip the gateway. Put it on your architecture roadmap for when you hit five or six services and the client coupling starts hurting. Not before.
📌 Takeaways
- An API gateway is a single L7 entry point that shields clients from your internal service topology
- Authentication belongs at the gateway, but fine-grained authorization still belongs in each service
- The moment business logic enters your gateway config, you're building a distributed monolith
- A gateway handles north-south ingress; a service mesh handles east-west traffic between services
- If you only have one or two services, you don't need one yet
Keep reading
- Understanding CORS: Why Your API Request Failed
- OAuth 2.0 and OpenID Connect: What "Sign in with Google" Actually Does
Where else to find me
I keep all my posts and projects over at arnavsharma.dev.
Top comments (0)