Long-lived connections between clouds rarely fail loudly. A database session, a gRPC stream, or a streaming inference connection passes every short test, deploys cleanly, and then drops in production after a few quiet minutes — no reset packet, no clear error, just a client that retries and a server that thought the peer had gone away. Most of the time nothing is broken. Something in the middle simply forgot the connection existed.
Every hop keeps its own clock
An idle timeout is a housekeeping rule: if a stateful device sees no packets for a set period, it deletes the connection entry to free memory. Firewalls, NAT gateways, load balancers, and proxies all do this, each with its own default, and the defaults rarely match.
That mismatch is the trap. A connection that crosses two clouds may pass through a NAT gateway in one, a firewall in the middle, and a load balancer in the other. The effective timeout for the whole path is not an average — it is the shortest timeout on any hop. Tune keepalives to the client and server alone and the middle hop still wins.
Vendor defaults also change over time and differ by protocol, so copying a number from an old runbook is risky. Treat any specific value as something to verify in the current documentation for the exact resource type in the path, not as a constant.
Why short tests miss it completely
Health checks and load tests keep connections busy, so no entry ever goes idle long enough to expire. The failures appear instead in the quiet paths: a standby replication link, a connection pool holding spare connections, a long polling client, or a streaming response that pauses while a model thinks.
Connection pools make it worse. A pool hands out the oldest idle connection first in some configurations, which is precisely the connection most likely to have been forgotten in the middle. The first query after a quiet period fails, the retry on a fresh connection succeeds, and the incident looks intermittent.
Build a timeout inventory for the path
For each long-lived cross-cloud flow, list every stateful hop in order and record three things: its idle timeout for the protocol in use, whether that timeout can be raised, and what happens on expiry — silent drop or reset. Silent drops are far harder to debug because neither endpoint learns promptly that the path is gone.
Then set one rule for the flow: application activity must happen well before the shortest timeout on the path. A common working margin is to send something at less than half the shortest interval, so a single missed beat does not kill the connection.
TCP keepalive is not an application heartbeat
TCP keepalive probes are cheap and often disabled by default, and their default intervals on many systems are measured in hours — far longer than most middlebox timeouts. If keepalive is the plan, it must be set deliberately on the socket, per connection, to an interval below the path minimum.
Even then, keepalive only proves the network path is alive. It does not prove the application on the other side is responsive. Protocols that carry their own ping or heartbeat frames are usually the better answer: the heartbeat exercises the application as well as the path. Use keepalive as a backstop, not as the only signal.
Test for quiet, not just for load
Add an idle soak test to pre-production checks. Open the connection exactly as production does, leave it completely idle for longer than the suspected shortest timeout, then send a real request on the same connection — not a new one. If that request hangs or fails while a fresh connection works, a middle hop is expiring the entry.
Repeat the test after any change to NAT, firewall, or load balancer configuration, and record the measured behaviour next to the documented timeout. The measured value is the one to tune against.
A short checklist
- List every stateful hop for each long-lived cross-cloud flow.
- Record the idle timeout and expiry behaviour for each hop, verified against current vendor documentation.
- Identify the shortest timeout — that number governs the whole path.
- Set application heartbeats to less than half that interval; set TCP keepalive deliberately as a backstop.
- Configure connection pools to validate or recycle connections idle beyond a safe threshold.
- Run an idle soak test after every network change, and keep the result with the runbook.
None of this requires new tooling. It requires treating time as part of the network design, in the same way address space and routing already are.
Teams that design these paths day to day — including the work described at CloudVerg around VergMesh private connectivity and VergRail — tend to put the timeout inventory in the same document as the routing design, so a future change to either one prompts a review of the other. However the estate is built, the principle holds: if a connection matters, something on the path should never be allowed to forget it quietly.
Top comments (0)