Day 22: Client-side load balancing with Spring Cloud LoadBalancer

Back on Day 19, Eureka turned a hardcoded http://localhost:8081 into a name: order-service now calls inventory-service and the registry resolves it. But a name doesn't resolve to an instance — it resolves to a list of them. So there's a question I'd quietly been letting the framework answer for me: when there are three copies of inventory-service running, who decides which one a given request hits? Today was about answering that on purpose.

Two places to balance

There are two spots you can put that decision. Server-side load balancing puts a dedicated proxy in front — a cloud ELB, nginx, a Kubernetes Service. The caller knows one address, sends there, and the proxy picks an instance and forwards. Clients stay dumb, but there are two network hops and the proxy is another box to run, scale, and keep highly available. Client-side load balancing moves the choice into the caller: it fetches the instance list from the registry and picks one itself, then calls it directly. One hop, no proxy box — the price is that the balancing logic lives in every caller, which is a complete non-issue in an all-Spring fleet where it's just a library.

That library is Spring Cloud LoadBalancer, the successor to the now-retired Netflix Ribbon. It's been on my classpath since the Eureka starter arrived, silently doing round-robin for me. Today I stopped relying on the default and configured it explicitly, because it's built around two clean extension points: where the instance list comes from, and how one instance is chosen.

The list, and the pick

The list is a ServiceInstanceListSupplier — a small pipeline of decorators you assemble with a builder. It's discovery-driven end to end; there are no hardcoded URLs anywhere.

@Bean
public ServiceInstanceListSupplier inventoryServiceInstanceListSupplier(
        ConfigurableApplicationContext context) {
  return ServiceInstanceListSupplier.builder()
      .withBlockingDiscoveryClient()                // the list, from Eureka
      .withBlockingHealthChecks(new RestTemplate()) // keep only instances reporting UP
      .withCaching()                                // cache the resolved list briefly
      .build(context);
}

The pick is a ReactorLoadBalancer<ServiceInstance>. Spring ships RoundRobinLoadBalancer (walks the list in order — the even, predictable default) and RandomLoadBalancer (uniform random — even only in expectation, but stateless). I exposed both behind a property so switching is config, not a recompile:

@Bean
public ReactorLoadBalancer<ServiceInstance> inventoryLoadBalancer(
        Environment env, LoadBalancerClientFactory factory) {
  String name = env.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
  String strategy = env.getProperty("orderhub.loadbalancer.strategy", "round-robin");
  var supplier = factory.getLazyProvider(name, ServiceInstanceListSupplier.class);
  return "random".equalsIgnoreCase(strategy)
      ? new RandomLoadBalancer(supplier, name)
      : new RoundRobinLoadBalancer(supplier, name);
}

The gotcha: per-service config must not be scanned

This tripped me up until I read the fine print. A load-balancer config class must not be picked up by your component scan — if it were a normal @Configuration in a scanned package, its beans would apply to every load-balanced client. Spring Cloud gives each client its own isolated child context, and you attach a config to one client by name:

@LoadBalancerClient(name = "inventory-service",
                    configuration = InventoryLoadBalancerConfig.class)

So InventoryLoadBalancerConfig is a plain class — deliberately no @Configuration, no @Component — and the component scan skips it. Spring Cloud instantiates it in the child context for just that one client. The default RoundRobinLoadBalancer bean is @ConditionalOnMissingBean, so declaring my own makes it back off cleanly.

Making it visible

Load balancing is invisible by design, which makes it annoying to prove. So I gave inventory-service a GET /api/inventory/instance that returns its own port and Eureka instance id, added a Feign method for it, and a GET /api/inventory-lb/distribution?calls=12 on order-service that fires N load-balanced calls and tallies which instance answered. Run two instances (SERVER_PORT=8091 and SERVER_PORT=8092, same jar), and round-robin comes back a clean 6/6. Flip the strategy to random and it goes lumpy. Kill one instance and its health check drops it from rotation within seconds — traffic rebalances onto the survivor with no restart of the caller. Two other pieces round it out: health checks close Eureka's ~90s eviction gap on the caller's side, and LoadBalancer retries can fail a request over to the next instance (idempotent GETs only — writes still lean on the Day 16 idempotency keys).

I tested the strategy the way you should — offline. A fixed instance list fed into a real RoundRobinLoadBalancer, asserting nine calls split evenly across three instances, plus a check that the config builds round-robin by default and random on request. No registry, no live instances, no Docker.

The reactor is still five modules, and mvn clean install stays green.

Full 3-tier walkthrough (with a live client-side load-balancer sim — pick a strategy, fire requests, kill an instance and watch it rebalance): https://dev48v.infy.uk/orderhub.php
Code: https://github.com/dev48v/order-hub-from-zero