DEV Community

Preecha
Preecha

Posted on

Service Mesh vs API Gateway: The Only Guide You’ll Ever Need

Modern microservice systems usually need two traffic-control layers: an API gateway for external client traffic and a service mesh for internal service-to-service traffic. The key implementation decision is not always “service mesh vs API gateway,” but where each one belongs in your architecture.

Try Apidog today

In this guide, you’ll compare API gateways and service meshes by traffic scope, responsibilities, deployment model, and operational use cases. You’ll also see practical configuration examples and how API design/testing tools like Apidog fit into either approach.

What Is Service Mesh vs API Gateway?

Before choosing a tool, separate the two traffic paths in your system:

  • North-south traffic: external clients calling internal services
  • East-west traffic: internal services calling other internal services

API gateways primarily manage north-south traffic. Service meshes primarily manage east-west traffic.

What Is an API Gateway?

An API gateway is the single entry point for client requests into your microservices system.

Use it to control how external consumers access your APIs.

Common API gateway responsibilities include:

  • Authentication and authorization
  • Request routing
  • Request aggregation
  • Rate limiting and throttling
  • Protocol translation, such as REST to gRPC
  • API versioning
  • Monitoring, logging, and analytics
  • Developer portal support

Typical flow:

Client / Mobile App / Partner
        |
        v
   API Gateway
        |
        v
 Internal Services
Enter fullscreen mode Exit fullscreen mode

An API gateway is usually placed at the edge of your system before requests reach internal services.

What Is a Service Mesh?

A service mesh is an infrastructure layer for managing communication between internal services.

Instead of making every service implement networking concerns directly, a service mesh usually uses lightweight sidecar proxies to intercept service-to-service traffic.

Common service mesh responsibilities include:

  • Service discovery
  • Internal load balancing
  • Mutual TLS between services
  • Traffic splitting
  • Canary releases
  • A/B testing
  • Retries and timeouts
  • Circuit breaking
  • Distributed tracing
  • Per-service observability

Typical flow:

Service A  <-->  Sidecar Proxy  <-->  Sidecar Proxy  <-->  Service B
Enter fullscreen mode Exit fullscreen mode

A service mesh is most useful when internal service communication becomes difficult to secure, observe, or control manually.

Why the Difference Matters

Choosing the wrong layer creates operational complexity.

Use the distinction to decide where to implement each concern:

  • Put external API security at the gateway.
  • Put internal mTLS in the mesh.
  • Put client-facing rate limits at the gateway.
  • Put service-to-service retries and timeouts in the mesh.
  • Put developer onboarding and API documentation near the gateway.
  • Put distributed tracing and internal traffic policy in the mesh.

Service Mesh vs API Gateway: Key Differences

Dimension API Gateway Service Mesh
Traffic scope North-south East-west
Primary users External clients, partners, frontend apps Internal services
Placement Network edge Inside the cluster/service runtime
Authentication API keys, OAuth, JWT Service identity, mTLS
Rate limiting Common Sometimes available
Request transformation Common Not a primary use case
Protocol translation Common Not a primary use case
Service discovery Basic Advanced
Load balancing Basic to moderate Advanced
Traffic splitting Limited to edge traffic Fine-grained internal traffic
Observability API-level analytics Per-service metrics and tracing
Resilience patterns Limited Retries, timeouts, circuit breaking
Developer portal Common Not applicable

Where They Overlap

API gateways and service meshes can both support:

  • Authentication and authorization
  • Routing
  • Load balancing
  • Observability
  • Traffic policy

The difference is the target boundary.

For example:

  • An API gateway may validate an external client’s JWT.
  • A service mesh may enforce mTLS between orders-service and payments-service.

They solve related problems at different layers.

When to Use an API Gateway

Use an API gateway when you need to expose services to external consumers.

Common cases:

  • Public APIs
  • Partner APIs
  • Mobile or web app backends
  • API monetization
  • Centralized authentication
  • API key management
  • Rate limiting
  • API documentation and onboarding
  • Request transformation or aggregation

Example architecture:

Mobile App
   |
   v
API Gateway
   |
   +--> users-service
   +--> orders-service
   +--> payments-service
Enter fullscreen mode Exit fullscreen mode

A SaaS product exposing REST APIs to web and mobile clients would typically use an API gateway for authentication, versioning, routing, and usage analytics.

When to Use a Service Mesh

Use a service mesh when internal service communication needs stronger control.

Common cases:

  • Many microservices communicating inside Kubernetes
  • Service-to-service encryption
  • mTLS enforcement
  • Canary releases
  • Blue-green deployments
  • Fine-grained traffic splitting
  • Distributed tracing
  • Internal retries and timeouts
  • Circuit breaking

Example architecture:

orders-service
   |
   v
payments-service
   |
   v
billing-service
Enter fullscreen mode Exit fullscreen mode

With a service mesh, these calls can be encrypted, traced, retried, and controlled without each application implementing that logic manually.

When to Use Both

In many production microservice architectures, you use both:

External Client
      |
      v
 API Gateway
      |
      v
 Internal Service A
      |
      v
 Service Mesh
      |
      v
 Internal Service B
Enter fullscreen mode Exit fullscreen mode

A practical split looks like this:

Concern Use API Gateway Use Service Mesh
External authentication ✅
API keys ✅
OAuth/JWT validation for clients ✅
Developer portal ✅
External rate limiting ✅
Internal mTLS ✅
Service-to-service authorization ✅
Internal retries ✅
Circuit breaking ✅
Canary traffic between services ✅
Distributed tracing ✅

This layered approach gives you edge-level control plus internal service reliability.

Practical Examples

Example 1: E-Commerce Platform

Use an API gateway for customer-facing endpoints:

  • Login
  • Checkout
  • Product search
  • Partner API access

Use a service mesh for internal calls between:

  • Inventory service
  • Payment service
  • Recommendation service
  • Order service

Implementation split:

Customer
   |
   v
API Gateway
   |
   +--> checkout-service
            |
            v
       Service Mesh
            |
            +--> inventory-service
            +--> payment-service
            +--> recommendation-service
Enter fullscreen mode Exit fullscreen mode

Example 2: API Monetization

Use an API gateway for:

  • Developer portal
  • API key management
  • Usage tracking
  • Billing integration
  • Client-level quotas

Use a service mesh for:

  • Secure billing-service communication
  • Internal analytics calls
  • Resilient core service communication

Example 3: Canary Deployments

Use an API gateway to route some external traffic to a new API version.

Use a service mesh to manage more granular internal rollout behavior.

Example:

API Gateway:
  90% traffic -> v1 API
  10% traffic -> v2 API

Service Mesh:
  checkout-service -> payment-service-v1: 95%
  checkout-service -> payment-service-v2: 5%
Enter fullscreen mode Exit fullscreen mode

Example 4: Protocol Translation

Use an API gateway when external clients need one protocol but internal services use another.

Example:

External REST request
        |
        v
API Gateway
        |
        v
Internal gRPC service
Enter fullscreen mode Exit fullscreen mode

The service mesh can still secure and observe internal gRPC traffic, but protocol translation is typically handled by the gateway.

API Gateway Configuration Example

Here is a simplified Kong-style configuration for rate limiting and API key authentication.

apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: rate-limited-api
route:
  strip_path: true
  protocols:
    - https
plugin:
  - name: rate-limiting
    config:
      minute: 100
      policy: redis
  - name: key-auth
    config:
      key_names:
        - x-api-key
Enter fullscreen mode Exit fullscreen mode

This type of policy belongs at the API gateway because it protects external-facing APIs from unauthorized or excessive client traffic.

Service Mesh Configuration Example

Here is a simplified Istio VirtualService for internal routing and retries.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews-routing
spec:
  hosts:
    - reviews
  http:
    - match:
        - sourceLabels:
            app: productpage
      route:
        - destination:
            host: reviews
            subset: v2
      retries:
        attempts: 3
        perTryTimeout: 2s
        retryOn: 5xx
Enter fullscreen mode Exit fullscreen mode

This policy belongs in the service mesh because it controls internal service-to-service behavior.

Implementation Checklist

Use this checklist when deciding what to deploy.

Start With an API Gateway If You Need To

  • Expose APIs to public clients
  • Authenticate external consumers
  • Manage API keys
  • Enforce client quotas
  • Apply rate limits
  • Publish API documentation
  • Support API versioning
  • Transform requests or responses
  • Translate protocols

Add a Service Mesh If You Need To

  • Encrypt service-to-service traffic
  • Enforce internal service identity
  • Add retries and timeouts consistently
  • Use circuit breaking
  • Perform canary releases between services
  • Capture distributed traces
  • Improve internal traffic observability
  • Manage many microservices at scale

Use Both If You Need Layered Control

A common production setup:

Client
  -> API Gateway
  -> Kubernetes Service
  -> Service Mesh Sidecar
  -> Internal Microservice
Enter fullscreen mode Exit fullscreen mode

In this model:

  • The gateway manages external API access.
  • The mesh manages internal service reliability and security.

Best Practices

Do Not Use a Service Mesh as a Full API Gateway

A service mesh is not designed for external API management features such as:

  • Developer portals
  • API key onboarding
  • Client-facing API documentation
  • Protocol mediation for external consumers
  • API monetization workflows

Do Not Overload the API Gateway

Some gateways provide mesh-like capabilities, but avoid using the gateway as the main control plane for all internal traffic at scale.

Internal service communication often needs:

  • Service identity
  • mTLS
  • Per-hop retries
  • Per-service metrics
  • Fine-grained traffic splitting

Those are service mesh concerns.

Apply Security at Both Layers

Use layered security:

External client security:
  API Gateway

Internal service security:
  Service Mesh
Enter fullscreen mode Exit fullscreen mode

For example:

  • Validate external JWTs at the gateway.
  • Enforce mTLS between internal services with the mesh.

Design APIs Before Deploying Traffic Policies

Before configuring a gateway or mesh, define:

  • API contracts
  • Request/response schemas
  • Error formats
  • Versioning rules
  • Authentication requirements
  • Test cases

This keeps implementation consistent across gateway-managed APIs and mesh-managed service calls.

Where Apidog Fits

Whether you use an API gateway, service mesh, or both, Apidog can support the API lifecycle around that architecture.

Use Apidog for:

  • API design and documentation
  • Mocking client-to-service calls
  • Testing service-to-service APIs
  • Managing API versions
  • Collaborating across backend, frontend, and QA teams

For an API gateway workflow, you can design and document APIs before exposing them through the gateway.

For a service mesh workflow, you can model and test internal service contracts before deploying traffic policies such as retries, timeouts, and canary routing.

Conclusion

API gateways and service meshes solve different parts of the microservices networking problem.

Use an API gateway for external API access, authentication, rate limiting, documentation, and protocol handling.

Use a service mesh for internal service communication, mTLS, retries, traffic splitting, and deep observability.

In many modern systems, the best implementation is both: an API gateway at the edge and a service mesh inside the cluster.

Top comments (0)