“You have multiple backend instances. If one instance goes down, how will Apigee X automatically route traffic to another healthy instance?”
This sounds like a simple interview question.
But if you've worked with Apigee, you know there is more to it than just saying:
“I'll configure multiple Target Servers.”
The interviewer is really testing whether you understand load balancing, failover, health monitoring, retries, and high availability.
In this article, we'll build the solution step by step using a simple real-world scenario.
By the end, you'll understand:
- What Target Servers are
- How load balancing works in Apigee X
- How
MaxFailureshelps with failover - Why Health Monitoring is important
- How Apigee brings a recovered server back into rotation
- When to use
RoundRobin,Weighted, andLeastConnections - How retry can affect transactional APIs
- Common mistakes to avoid
- How to answer this question confidently in an interview
The Problem: What Happens When a Backend Server Goes Down?
Imagine you have an API exposed through Apigee X:
Client
|
v
Apigee X
|
v
Payment API
|
+----------------+
| |
v v
Backend Server 1 Backend Server 2
Now your application grows.
You don't want just two backend instances. You want three:
Backend Server 1
Backend Server 2
Backend Server 3
Why?
Because if one server fails, the other servers can continue handling requests.
For example:
Apigee X
|
Load Balancer
/ | \
↓ ↓ ↓
S1 S2 S3
✓ ✓ ✓
Everything is healthy.
But suddenly:
Apigee X
|
Load Balancer
/ | \
↓ ↓ ↓
S1 S2 S3
✓ ❌ ✓
DOWN
We don't want Apigee to continue sending requests to Server 2.
Instead:
Apigee X
|
Load Balancer
/ \
↓ ↓
S1 S3
✓ ✓
That's the problem we're solving.
And Apigee X has built-in support for load balancing and failover across multiple backend server instances. ([Google Cloud Documentation][1])
Think of Target Servers Like a Restaurant
Here's an easy analogy.
Imagine you're running a busy restaurant.
You have three cashiers:
Cashier 1
Cashier 2
Cashier 3
Customers shouldn't have to know which cashier they're going to.
The manager decides where each customer should go.
If Cashier 2 suddenly stops working, the manager stops sending customers there.
That's essentially what we're doing with Apigee.
Customers
↓
Restaurant Manager
↓
Cashier 1 | Cashier 2 | Cashier 3
In our API architecture:
API Clients
↓
Apigee X
↓
Load Balancer
↓
Target Server 1 | Target Server 2 | Target Server 3
The Target Server represents the backend destination, while the LoadBalancer determines how traffic is distributed among those destinations.
What Is a Target Server in Apigee X?
A Target Server allows you to separate your backend server configuration from your API proxy configuration.
Instead of putting a concrete backend URL directly inside your TargetEndpoint, you define a named Target Server.
For example:
payment-server-1
|
+--> backend-1.example.com:8080
Another:
payment-server-2
|
+--> backend-2.example.com:8080
And another:
payment-server-3
|
+--> backend-3.example.com:8080
Google describes Target Servers as a way to decouple concrete backend URLs from the TargetEndpoint configuration. ([Google Cloud Documentation][1])
You can read more in the official Apigee load balancing documentation.
Step 1: Create Multiple Target Servers
Let's assume we have three backend instances:
| Target Server | Backend |
|---|---|
payment-server-1 |
10.10.1.10:8080 |
payment-server-2 |
10.10.1.11:8080 |
payment-server-3 |
10.10.1.12:8080 |
Conceptually:
payment-server-1 → 10.10.1.10:8080
payment-server-2 → 10.10.1.11:8080
payment-server-3 → 10.10.1.12:8080
The important thing is that the API proxy doesn't need to hardcode these backend URLs directly.
Step 2: Configure the Load Balancer
Now we tell Apigee:
“I have multiple backend servers. Distribute requests between them.”
Inside the TargetEndpoint:
<TargetEndpoint name="default">
<HTTPTargetConnection>
<LoadBalancer>
<Algorithm>RoundRobin</Algorithm>
<Server name="payment-server-1"/>
<Server name="payment-server-2"/>
<Server name="payment-server-3"/>
</LoadBalancer>
</HTTPTargetConnection>
</TargetEndpoint>
That's the basic configuration.
The official Apigee documentation supports three load-balancing algorithms:
RoundRobinWeightedLeastConnections
RoundRobin is the default algorithm. ([Google Cloud Documentation][1])
How Does Round Robin Work?
Think of three people standing in a queue.
Instead of sending every request to the first person, Apigee distributes them one after another.
Request 1 → Server 1
Request 2 → Server 2
Request 3 → Server 3
Request 4 → Server 1
Request 5 → Server 2
Request 6 → Server 3
So:
Apigee X
|
Round Robin
|
+------+------+------+
| | |
↓ ↓ ↓
S1 S2 S3
This works well when your backend instances have roughly similar capacity.
But What If Server 2 Goes Down?
Now we reach the interesting part.
Suppose:
Server 1 → Healthy ✓
Server 2 → DOWN ❌
Server 3 → Healthy ✓
Without failure handling, requests could still be sent toward Server 2.
That's obviously not what we want.
This is where MaxFailures comes into play.
Step 3: Configure MaxFailures
We can configure:
<LoadBalancer>
<Algorithm>RoundRobin</Algorithm>
<Server name="payment-server-1"/>
<Server name="payment-server-2"/>
<Server name="payment-server-3"/>
<MaxFailures>5</MaxFailures>
</LoadBalancer>
Here, we're telling Apigee to remove a target from rotation after the configured failure threshold is reached.
For example:
Server 2
Failure
↓
Failure
↓
Failure
↓
Failure
↓
Failure
↓
Removed from rotation
Traffic then continues through the healthy servers:
Server 1 ←→ Server 3
The important detail is that Apigee's definition of a failure is primarily a case where it doesn't receive a response from the target. An HTTP response such as 404 or 500 normally counts as a response rather than a connection failure. If you want particular HTTP status codes to count as failures, you can configure ServerUnhealthyResponse. ([Google Cloud Documentation][1])
For example:
<LoadBalancer>
<Algorithm>RoundRobin</Algorithm>
<Server name="payment-server-1"/>
<Server name="payment-server-2"/>
<Server name="payment-server-3"/>
<MaxFailures>5</MaxFailures>
<ServerUnhealthyResponse>
<ResponseCode>500</ResponseCode>
<ResponseCode>502</ResponseCode>
<ResponseCode>503</ResponseCode>
</ServerUnhealthyResponse>
</LoadBalancer>
Now those configured response codes are also treated as failures.
A Small but Important Detail About MaxFailures
This is something many developers miss.
The default value of:
<MaxFailures>0</MaxFailures>
means Apigee does not remove a Target Server from rotation based on the failure count.
So if you're designing failover, don't simply create multiple Target Servers and assume Apigee will automatically remove unhealthy ones.
You need to configure the failure behavior appropriately. ([Google Cloud Documentation][1])
Step 4: Add Health Monitoring
Now imagine Server 2 has been removed.
A few minutes later, the infrastructure team fixes it.
Server 2 is healthy again.
What happens now?
This is where Health Monitoring becomes extremely useful.
Think about the restaurant analogy again.
The manager doesn't permanently ban a cashier just because they temporarily left.
The manager periodically checks:
“Are you ready to work again?”
That's essentially what a health monitor does.
Server 2 DOWN
↓
Removed from rotation
↓
Health Monitor checks
↓
Server 2 becomes healthy
↓
Added back into rotation
Apigee health monitoring can actively poll backend Target Servers using either TCP or HTTP checks. When the target becomes healthy again, Apigee can automatically return it to rotation without redeploying the proxy. ([Google Cloud Documentation][1])
TCP Health Monitor
For a basic connectivity check, you can use a TCP monitor.
For example:
<HealthMonitor>
<IsEnabled>true</IsEnabled>
<IntervalInSec>5</IntervalInSec>
<TCPMonitor>
<ConnectTimeoutInSec>10</ConnectTimeoutInSec>
</TCPMonitor>
</HealthMonitor>
This essentially asks:
“Can I establish a TCP connection to this backend?”
If the connection fails, the target's failure count is incremented.
If the health check succeeds, the target can become healthy again.
HTTP Health Monitor
Sometimes just checking whether a port is open isn't enough.
Your server might accept TCP connections but the actual application could still be unhealthy.
For example:
TCP connection → ✓
Application → ❌
Database connection → ❌
Dependencies → ❌
In that case, an HTTP health endpoint is often more useful.
For example:
GET /health
Expected response:
HTTP/1.1 200 OK
A simplified HTTP monitor configuration can look like:
<HealthMonitor>
<IsEnabled>true</IsEnabled>
<IntervalInSec>5</IntervalInSec>
<HTTPMonitor>
<Request>
<Verb>GET</Verb>
<Path>/health</Path>
<ConnectTimeoutInSec>10</ConnectTimeoutInSec>
<SocketReadTimeoutInSec>30</SocketReadTimeoutInSec>
</Request>
<SuccessResponse>
<ResponseCode>200</ResponseCode>
</SuccessResponse>
</HTTPMonitor>
</HealthMonitor>
Now Apigee isn't simply asking:
“Is the server reachable?”
It's asking:
“Is the application responding to its health check correctly?”
Putting It All Together
Now we can build the complete configuration.
<TargetEndpoint name="default">
<HTTPTargetConnection>
<LoadBalancer>
<!-- Distribute traffic across healthy targets -->
<Algorithm>RoundRobin</Algorithm>
<!-- Backend instances -->
<Server name="payment-server-1"/>
<Server name="payment-server-2"/>
<Server name="payment-server-3"/>
<!-- Remove a target after repeated failures -->
<MaxFailures>5</MaxFailures>
</LoadBalancer>
<!-- Continuously check backend health -->
<HealthMonitor>
<IsEnabled>true</IsEnabled>
<IntervalInSec>5</IntervalInSec>
<HTTPMonitor>
<Request>
<Verb>GET</Verb>
<Path>/health</Path>
<ConnectTimeoutInSec>10</ConnectTimeoutInSec>
<SocketReadTimeoutInSec>30</SocketReadTimeoutInSec>
</Request>
<SuccessResponse>
<ResponseCode>200</ResponseCode>
</SuccessResponse>
</HTTPMonitor>
</HealthMonitor>
</HTTPTargetConnection>
</TargetEndpoint>
The exact health endpoint, timeout values, and failure threshold should be chosen based on your application's behavior rather than blindly copying these numbers.
So What Happens During a Real Failure?
Let's visualize the entire lifecycle.
Everything is healthy
Apigee X
|
LoadBalancer
|
+----------+----------+
| | |
↓ ↓ ↓
S1 S2 S3
✓ ✓ ✓
Traffic:
S1 → S2 → S3 → S1 → S2 → S3
Server 2 fails
Apigee X
|
LoadBalancer
|
+----------+----------+
| | |
↓ ↓ ↓
S1 S2 S3
✓ ❌ ✓
Apigee detects failures.
Once the configured failure threshold is reached:
S2
↓
Marked unavailable
↓
Removed from rotation
Traffic becomes:
S1 → S3 → S1 → S3
Server 2 recovers
The Health Monitor detects that Server 2 is healthy again.
S2
↓
Health check succeeds
↓
Returned to rotation
And we're back to:
S1 → S2 → S3 → S1 → S2 → S3
That's the complete failover lifecycle.
What About Retry?
There's one more concept worth understanding:
Retry.
By default, Apigee's TargetEndpoint retry behavior is enabled.
Retries can occur for response failures such as I/O errors or HTTP timeouts, and can also be configured to react to status codes specified under ServerUnhealthyResponse. Apigee requires at least two Target Servers for retry to work. ([Google Cloud Documentation][1])
Conceptually:
Client
|
↓
Apigee
|
↓
Server 1
|
X Connection failure
|
↓
Another available target
|
↓
Server 2
|
✓
This can improve resilience.
But there's an important catch.
⚠️ Be Careful With Retry on Payment APIs
Imagine this API:
POST /payments
The request reaches Server 1.
Server 1 successfully processes the payment.
But before Apigee receives the response, the network connection fails.
Apigee doesn't know whether the payment was processed.
If the request is retried against Server 2:
Server 1
|
+--> Payment processed ✓
|
X Response lost
Apigee
|
+--> Retry
|
↓
Server 2
|
+--> Same payment request
Now you have a potential duplicate transaction.
That's why idempotency is extremely important for transactional APIs.
For example:
POST /payments
Idempotency-Key: payment-request-12345
The backend can use that key to recognize duplicate requests.
So when designing a highly available payment API, don't think about retry in isolation.
Think:
Load Balancing
+
Failover
+
Retry
+
Idempotency
What If All Backend Servers Go Down?
You can also configure a dedicated fallback Target Server.
For example:
Primary Servers:
S1
S2
S3
Fallback:
S4
Configuration:
<LoadBalancer>
<Algorithm>RoundRobin</Algorithm>
<Server name="payment-server-1"/>
<Server name="payment-server-2"/>
<Server name="backup-server">
<IsFallback>true</IsFallback>
</Server>
</LoadBalancer>
The fallback server isn't used during normal load balancing.
It becomes available when all the other target servers have been removed from rotation. Only one Target Server can be configured as the fallback server. ([Google Cloud Documentation][1])
So:
Normal:
S1 ←→ S2
S1 DOWN
S2 DOWN
↓
Fallback S3
This can be useful for emergency or disaster-recovery scenarios.
RoundRobin vs Weighted vs LeastConnections
Choosing the load-balancing algorithm depends on your backend architecture.
1. RoundRobin
S1 → S2 → S3 → S1 → S2 → S3
Use it when backend instances have roughly similar capacity.
2. Weighted
Suppose:
Server 1 → Weight 5
Server 2 → Weight 3
Server 3 → Weight 2
The servers don't necessarily receive equal traffic.
This makes sense when your backend infrastructure has different capacities.
For example:
S1 → 8 CPU
S2 → 4 CPU
S3 → 2 CPU
You may want the stronger server to handle more traffic.
3. LeastConnections
Instead of simply taking turns, Apigee can distribute traffic based on active connections.
Conceptually:
S1 → 10 connections
S2 → 4 connections
S3 → 7 connections
A new request can be directed toward the server with fewer active connections.
This can be useful when requests have significantly different processing times.
Apigee supports RoundRobin, Weighted, and LeastConnections. ([Google Cloud Documentation][1])
Common Mistakes to Avoid
❌ Mistake 1: Using Only One Target Server
What's the point of load balancing if there's only one server?
<LoadBalancer>
<Server name="server-1"/>
<MaxFailures>5</MaxFailures>
</LoadBalancer>
If that single server is removed from rotation, there is nowhere else to send traffic.
Google specifically documents using a single Target Server with non-zero MaxFailures as an anti-pattern. ([Google Cloud Documentation][2])
❌ Mistake 2: Using MaxFailures Without Thinking About Recovery
MaxFailures determines when a server should be removed from rotation.
For automatic recovery, configure a Health Monitor.
Google recommends using MaxFailures > 0 with a Health Monitor so that a recovered Target Server can automatically return to rotation. ([Google Cloud Documentation][1])
❌ Mistake 3: Assuming Every HTTP 500 Is Automatically a Target Failure
This is a subtle one.
By default, receiving an HTTP response—even a 500—means Apigee received a response from the target.
If you want certain HTTP status codes to count toward target failure handling, configure ServerUnhealthyResponse.
❌ Mistake 4: Blindly Retrying POST Requests
Retries can be dangerous for operations such as:
Payments
Orders
Money transfers
Booking creation
Always consider whether the operation is idempotent before enabling or relying on retries.
❌ Mistake 5: Treating TCP Health as Application Health
A server can accept TCP connections while its application is unhealthy.
For critical applications, consider an HTTP health endpoint that verifies the application is actually ready to process requests.
Best Practices for Apigee X Backend Failover
Here are the practices I'd follow in a production environment.
1. Use Multiple Target Servers
Don't put all your availability expectations on a single backend.
Target Server 1
Target Server 2
Target Server 3
This gives Apigee multiple destinations for failover.
2. Configure a Meaningful Health Check
Prefer an application-level health endpoint when appropriate:
GET /health
The endpoint should provide a meaningful indication that the application can actually serve traffic.
3. Tune MaxFailures Carefully
Don't blindly use:
<MaxFailures>1</MaxFailures>
or:
<MaxFailures>100</MaxFailures>
The right value depends on:
- Backend stability
- Traffic volume
- Expected transient failures
- Business criticality
- Recovery time
4. Design Retry With Idempotency
For read operations such as:
GET /customers
retry is usually easier to reason about.
For operations such as:
POST /payments
POST /orders
you need to think carefully about duplicate processing.
5. Monitor the Failover Behavior
Don't just configure failover and forget about it.
Test scenarios such as:
Server 1 DOWN
Server 2 DOWN
Server 3 DOWN
Server recovery
Network timeout
Connection refused
Backend 500
Backend 503
Then verify that Apigee behaves as expected.
The Architecture at a Glance
Here's the complete picture:
API Client
|
↓
APIGEE X
|
API Proxy
|
TargetEndpoint
|
LoadBalancer
|
+---------------+---------------+
| | |
↓ ↓ ↓
Target Server 1 Target Server 2 Target Server 3
| | |
↓ ↓ ↓
Backend #1 Backend #2 Backend #3
✓ ❌ ✓
|
MaxFailures
|
↓
Remove from pool
|
Health Monitor
|
↓
Backend recovers
|
↓
Back into rotation
The mental model is simple:
Multiple Target Servers → Load Balancer → Detect Failure → Remove Unhealthy Target → Continue Traffic → Detect Recovery → Add Target Back
🎯 Interview-Ready Answer
If an interviewer asks:
“How will you configure multiple backend instances in Apigee X so that if one goes down, traffic is routed to another instance?”
Here's the answer I'd give:
“I would create multiple Target Servers in the Apigee environment, with each Target Server pointing to a different backend instance. Then, in the TargetEndpoint, I would configure a LoadBalancer referencing those Target Servers. Depending on the requirement, I can use RoundRobin, Weighted, or LeastConnections.
For failover, I would configure MaxFailures so that an unhealthy Target Server is removed from rotation after the configured failure threshold. I would also configure a Health Monitor so Apigee can actively check the backend and automatically return the Target Server to rotation once it becomes healthy again.
If required, I can configure a dedicated fallback Target Server using IsFallback. For transactional APIs such as payment APIs, I would also carefully evaluate retry behavior and use idempotency to prevent duplicate transactions.”
That answer demonstrates much more than simply knowing what a Target Server is.
It shows that you understand high availability and production API architecture.
Conclusion
Configuring multiple backend instances in Apigee X isn't just about adding multiple URLs.
The real solution combines several concepts:
Target Servers
↓
Load Balancer
↓
Failure Detection
↓
MaxFailures
↓
Health Monitor
↓
Automatic Failover
↓
Automatic Recovery
And depending on your use case:
Retry
+
Fallback Server
+
Idempotency
+
Monitoring
The key takeaway is this:
Target Servers define where your API can go. The LoadBalancer decides where traffic should go. MaxFailures helps remove unhealthy targets, while Health Monitoring helps bring recovered targets back into rotation.
Once you understand that flow, this entire Apigee X interview question becomes much easier.
And more importantly, you can apply the same architecture to real-world systems where availability and resilience matter.
💬 Your Turn
Have you implemented backend failover using Apigee X Target Servers?
Did you use:
- RoundRobin?
- Weighted?
- LeastConnections?
- Health Monitoring?
- A fallback server?
Share your experience or questions in the comments. I'd love to hear how you're handling backend high availability in your Apigee projects.
If you're learning Apigee X, API management, API security, or API traffic management, follow along for more practical, interview-focused articles.
Further Reading
For the implementation details, Google's official documentation is the best reference:
Top comments (0)