The problem
On-call tools can already ring you. PagerDuty's phone alert is a recording that asks you to press 4 to acknowledge and 6 to resolve, and you still have to get up and find a laptop to learn what actually broke.
What Ringbolt does
Any monitor posts a plain JSON alert. Policy decides whether it's worth waking somebody, going by the service's severity threshold, its quiet hours and whether the same problem already rang a phone. If it is, Ringbolt places a call through the CALL-E API to whoever is on the rota. On the call you can ask what broke and what else is affected, and then say what to do. The whole exchange lands in one audit record: the transcript, the decision, who authorized it, what ran, and the state before and after.
It runs on Cloudflare Workers, with one Durable Object per incident holding the state machine and the escalation timer, and D1 for storage.
The idea that shaped it
Handing a phone call the power to change production is only reasonable if the refusals are strict, so most of the design is about what does not run. CALL-E delivers its terminal webhook unsigned, so Ringbolt never trusts it and fetches the call again under its own key before acting. The decision has to validate against a schema, clear a confidence floor and name an action the service's policy already allows. For anything destructive, the confirmation phrase has to appear as whole words in a transcript turn from the responder, and "don't roll it back" does not count as "roll it back".
A call where the responder was never heard is refused outright, whatever the confidence score says. The demo carries an example of exactly that: a call that came back completed, at high confidence, with a valid decision to change production, where not one word from the person who answered was transcribed. It never ran.
What I learned
A create that times out isn't a create that didn't happen. On 22 August my build created 23 separate call tasks in half an hour, and each one is billed whether it connects or not. Two rules came out of that. A failed create is retried once under the same idempotency key, because a repeat under a fresh key bills as a second call to the same person. And there's now a hard ceiling of three real calls in ten minutes, because every one of those 23 was a different logical call, so no per-call check could have stopped any of them.
Transcription is lossy in both directions. The transcript check will sometimes refuse a call someone really did authorize. I took that trade on purpose, because the refusal falls toward calling the next person on the rota and never toward an unauthorized change.
The last one came from the provider's region table, which lists the Netherlands with English. The call setup I used was still refused, with "Calls to the Netherlands in English are not supported for this call setup". That's why the public demo runs the same loop against a local stand-in of their API, and why one contract suite runs every case against both the stand-in and the real adapter, so the two can't drift apart. There are 555 tests in total.
Try it
The hosted demo has a service called dockside that exists to be broken. Press the button and watch the alert, the call, the transcript and the rollback. Nothing you press there can ring a phone.
- Demo: https://ringbolt.taranity.com
- Code: https://github.com/voyagi/ringbolt
- Video: https://www.youtube.com/watch?v=EIm87clsks8
- The same walkthrough in words: https://github.com/voyagi/ringbolt/blob/main/examples/demo-transcript.md
Built for the CALL-E hackathon.
Top comments (0)