Our login codes go out through an SMS provider. When we send too fast it answers 429 with a Retry-After header, and our sender waits that long before trying again. For three years the header held a number of seconds, usually one or two.
On Monday the fifteenth of June the provider moved its API behind a new gateway, and the header started holding a date instead: Mon, 15 Jun 2026 08:02:31 GMT. That was not a change of contract. HTTP has always allowed Retry-After to be either a delay in seconds or a date, and the provider's documentation said so. Our client only knew the first form. It parsed the header as an integer, the parse threw, and the exception handler, written to be safe, fell back to a delay of zero.
So every 429 was followed at once by the same request. Forty sender workers did this together during the busiest half hour of the week, when people sign in at the start of work. Within four minutes the provider's abuse protection suspended our account for an hour. Nobody could receive a login code, and anybody whose session had expired over the weekend could not sign in.
What makes this worth writing down is that the provider was right on every point. They asked us to wait, in a form we had agreed to accept, and we ignored them in a way that looked to them exactly like an attack.
Our client now reads both forms of Retry-After, converts a date into a delay against our own clock, and clamps the result between one second and ten minutes. A header we cannot parse means a long wait with jitter, never zero, and the raw value goes into the log. Retries from every worker draw from one shared budget, so a provider slowing down cannot multiply our traffic. Contract tests for each HTTP integration now cover every form the specification allows for each header we read, not just the one the partner happens to send today. And login codes now fail over to a second SMS provider after two minutes.
A default in an error handler is a decision made for the worst moment. Ours decided that when we did not understand the other side, we should ask again immediately.
– Sergey Shinder
Top comments (0)