Session affinity is a simple idea: route a client to the same backend node it used
last time, so state that lives on that node (a cache, a connection, an in-memory
session) doesn't have to move. Assign once, store the assignment, reuse it on every
subsequent request. It's a standard pattern, and on its own it's fine.
The failure mode isn't in the assignment logic. It's in what happens after the
assignment specifically, in never asking the question "when does this stop being
true?"
The setup
A routing layer assigns each client to a specific backend instance and persists
that assignment in a table a row per client, pointing at a node. Every future
request for that client looks up the row first: if it exists, route there; if not,
run a "pick the least-loaded node" selection and write a new row.
This is correct, as far as it goes. The problem is what "least-loaded" is measured
against: a live count of how many clients are currently assigned to each node. That
count comes from counting rows in the same table.
The gap
Nothing ever deleted a row. Not on client disconnect, not on any timer, not on any
background sweep. An assignment, once made, was permanent.
Each individual row is completely correct the client it points to really was
assigned to that node, and routing them back there really is the right behavior for
an active client. Nothing here throws an error. Nothing fails a test. No alert
threshold gets crossed. The system does exactly what it was built to do, every
single time.
And it degrades anyway. A node that happens to pick up more assignments early
maybe it just came online first, maybe it got a slightly heavier initial burst
accumulates them permanently. Its "load," as measured by row count, can only ever
go up. It never goes down, because clients who went quiet hours or days ago are
still sitting in the table, still being counted, still making that node look busier
than it actually is to every future "least-loaded" decision.
Given enough time and enough churn, you get a small number of nodes stuck
permanently "full" by this measure, while others sit comparatively empty not
because of any real capacity shortage, but because the accounting never forgot
anyone.
This is the part worth sitting with: the failure has no direct symptom pointing
back at its cause. What you actually observe is imbalance, or unexplained
pressure alerts on specific nodes, or oscillating autoscaling behavior downstream of
that imbalance. None of it says "assignment table." You have to trace the actual
data pull real row counts, look at how old the entries are before the cause is
visible at all.
State that's correct but never expires is a silent failure class
Generalize this past routing tables specifically: any system that writes state on
an event but never removes it on the absence of a corresponding event has this
shape. Caches with no eviction, sessions with no expiry, locks with no timeout,
subscriptions with no unsubscribe path. Every individual write is right. The
aggregate, over time, is wrong and it's wrong in a way that produces symptoms far
downstream of the actual bug, which is what makes it genuinely hard to trace back.
The fix, and why TTL specifically
Add a TTL to the assignment, refreshed on every successful use. An active client's
assignment keeps renewing itself and never expires mid-session. An idle client's
assignment ages out, and their next request correctly re-runs the "least loaded"
selection against current, meaningful data.
The choice of TTL over an explicit delete-on-disconnect is deliberate, not
arbitrary. A disconnect event is one clean path but it's not the only way a
client goes quiet. Crashes, closed tabs, dropped connections, network partitions
none of these fire a clean disconnect event. If cleanup depends on that one path
firing, cleanup only works for the cases that were never really the problem. TTL is
robust to every reason a client goes silent, because it doesn't depend on being
told it just requires the client to prove it's still active by showing up again
before the clock runs out.
One more detail worth knowing if you're building this on DynamoDB specifically:
native TTL is best-effort, not instantaneous. AWS documents it as typically
completing within a window, not guaranteed by any specific deadline. If your
correctness depends on an expired row actually being gone by the time you next
read it, you can't rely on the background sweep alone. The read path itself has to
check the TTL field and treat an expired-but-not-yet-swept row as already gone,
independent of whether DynamoDB has physically deleted it yet. Native TTL becomes
housekeeping it keeps the table from growing unbounded not the mechanism your
actual correctness depends on.
Two clocks, not one, if there's local state involved
If the same client also has some local state on the node an idle timeout that
frees up a resource on that specific machine after inactivity that's a second,
separate clock, doing a different job, and it should almost always be shorter than
the routing TTL. The local timeout frees something cheap and reversible: a slot on
one machine. The routing TTL frees something bigger: the decision about which
machine gets that client at all. If the routing TTL is shorter than the local
timeout, you can end up expiring a routing decision while the local resource is
still actively in use reshuffling a client that never actually went idle. The
routing clock has to outlast the local one, not match it.
The takeaway
If a piece of state gets written on one event and there's no code path that removes
it on the absence of a future event, that's worth a second look not because it's
obviously wrong, but because it's the specific shape of bug that produces no error,
passes every individual-write test, and shows up later as something that looks
completely unrelated.
Top comments (0)