The gate said no, and it said when to come back.
HOLD frequency_cap retry_after 2026-09-15T12:00:00Z
So the caller came back at 12:00:00Z, and the gate said no, and it said to come back at 12:00:00Z.
Nobody was in an infinite loop in production, because this was a test suite and the package had not shipped yet. But the shape of it is the thing I want to write about, because I have written rate limiting more than once and I had never tested the part that broke.
The package, briefly
I was building a decision service for follow-up sequences. The idea is small: your n8n or Make scenario already knows how to send a message, so this is the thing it asks first. It answers SEND, SKIP, HOLD, STOP or ESCALATE, with a reason and the evidence it read.
Four acts: the case, a signal that lands between steps, the same step asked twice after a restart, and a quiet window read on the contact's clock.
One of the rules it enforces is a frequency cap. At most N messages on a channel per contact inside a sliding window, counted across every sequence that contact is in, because two campaigns that each respect their own limit still add up to two messages for the person receiving them.
The cap is about eleven lines. Here is the shape of it.
for cap in sequence.caps:
if cap.channel is not step.channel:
continue
since = now - cap.window
sends = ledger.sends_for_contact(case.contact_id, cap.channel, since)
if len(sends) < cap.limit:
continue
oldest = min(s.decided_at for s in sends)
return Hold(
reason=FREQUENCY_CAP,
retry_after=oldest + cap.window,
)
Read it again and it still looks right. One send at 12:00 on Monday, a cap of one per day, so the window clears at 12:00 on Tuesday. That is exactly what oldest + cap.window says.
The other half, in another file
The ledger decides what counts as inside the window.
def sends_for_contact(self, contact_id, channel, since):
return [
r for r in self._records
if r.contact_id == contact_id
and r.channel is channel
and r.decided_at >= since # <- here
and r.status != "failed"
]
>=.
At 12:00 on Tuesday, since is 12:00 on Monday. The Monday send has decided_at == since. Monday is greater than or equal to Monday, so the send is still in the window, so the count is still one, so the cap still holds, so the gate computes oldest + window again and returns the same instant it just refused.
The two pieces of code were written twenty minutes apart, in two files, and each one is defensible alone. The interval is closed at the bottom in one and treated as open at the bottom in the other. A window that includes its own lower boundary can never be escaped by waiting exactly one window.
The whole bug in one line: the instant the refusal points at is the instant the rule still refuses.
Why the suite did not catch it
This is the part I actually want to talk about, because the test that should have caught it was already there. Here it is as I first wrote it:
def test_the_cap_holds_the_second_send_inside_the_window():
harness.send_and_confirm("s1")
harness.clock.advance(timedelta(hours=1))
verdict = harness.decide("s2")
assert verdict.kind is VerdictKind.HOLD
assert verdict.reason is Reason.FREQUENCY_CAP
Green. It asserts that the cap refuses. It says nothing at all about the instruction attached to the refusal.
And that is how almost every rate limiter I have read is tested. There is a test for "it lets the first one through", a test for "it blocks the second one", and usually a test for "after enough time it lets another through", written with a comfortable margin: advance a day and an hour, or a day and a minute, and assert SEND. Mine had that one too. timedelta(days=1, minutes=1) passed happily, because a minute past the boundary is not the boundary.
The margin is what hides it. Nobody tests the boundary, because the boundary feels like an off-by-one you would notice, and because advancing time by a convenient amount is what a person types when the point of the test is something else.
The test that catches it
The fix in the test is not more assertions about the hold. It is to obey the hold.
def test_the_hold_says_exactly_when_the_window_clears():
harness.send_and_confirm("s1")
harness.clock.advance(timedelta(hours=1))
held = harness.decide("s2")
assert held.retry_after == T0 + timedelta(days=1)
harness.clock.set(held.retry_after) # do exactly what it told me
assert harness.decide("s2").kind is VerdictKind.SEND
Set the clock to the value the system handed back. Not a second later. Not a comfortable minute later. The exact instant, taken from the response rather than typed by hand.
That test fails on the original code, and the failure message is the whole story: HOLD frequency_cap where a SEND was expected, at the timestamp the system itself chose.
The fix was one character. >= became >, with a comment explaining why the interval is half open, and the boundary test pinned it.
# Strictly after: at exactly `since` the send has aged out of the window.
# With >= here, retry_after (oldest + window) lands on an instant that is
# still capped, and a caller that obeys it gets held again, forever.
if r.decided_at > since and r.status != "failed":
The general version
When a gate refuses and returns a retry time, it has made a promise, and the promise is written in a different place from the rule that produced it. Nothing in the type system, and nothing in a normal test suite, forces those two places to agree.
So the class of bug is not "off by one in a comparison". It is an output that no test treats as an input. retry_after goes out to the caller and comes back as the caller's behaviour, and if your suite never closes that loop, the value is decorative.
The practice I took from it is short: for any value your system returns as advice, write the test that follows the advice. Retry after. Next page cursor. Suggested chunk size. The Location header on a 202. Each of those is a round trip through your own API, and each of them is usually asserted for shape and never for effect.
It generalises past time, too. A paginator that returns a cursor and a rate limiter that returns a delay are the same thing: a machine that refuses now and describes the conditions under which it will not refuse. If you never feed the description back in, the description is a comment.
While I was there
The same suite caught a second, quieter version of the same theme.
The service returns a reason on every verdict, from a fixed vocabulary, and the reason string goes into the audit log. So I wrote a test that reads the engine source and asserts that every member of the enum is actually produced somewhere.
def test_every_reason_is_reachable_from_a_verdict_kind():
documented = {r.value for r in Reason}
engine = (SRC / "engine.py").read_text()
unused = {r for r in documented if f"Reason.{r.upper()}" not in engine}
assert not unused, f"reasons never produced by the engine: {sorted(unused)}"
It failed immediately on sequence_exhausted. I had declared a reason for "every step of this sequence is settled" and never written the code that could return it. A caller reading my enum would have built a branch for a verdict that could not arrive.
That one was not a fix, it was a missing feature: nothing in the service answered "the sequence is over", which is exactly what a cron loop needs to know to stop asking. It is why the package now has an advance(case_id) call at all.
Dead vocabulary in a public enum is a small lie about your contract. A test can notice it, and the test is six lines.
The repo
The package is sequence-gate, MIT, Python, no network and no sleeps in the suite. It ships three deliberately wrong implementations alongside the right ones, and the tests run them and watch them fail: a stop check that runs after the send, a ledger keyed on the provider's message id that double-sends after a replay, and quiet hours evaluated on the server's clock that texts someone in Kolkata at 03:12.
Both bugs in this post are in the README, in a section called "bugs found while building it", which I think is the most useful section a repository like that can have.
If you keep one line from this: the timestamp your system returns is not documentation, it is an instruction, and something should be executing it.
Vinicius Pereira
vinimabreu.dev · github.com/vinimabreu
Top comments (0)