Bolo turns speech on a phone into text. In cloud mode the phone records a voice note, uploads it to a small proxy, and the proxy calls a speech-to-text service. Two problems in that path turned out to be the same problem seen from opposite ends:
- The server was doing the same transcription several times for one recording.
- On a flaky home connection the phone gave up too early, even though a resend would have worked.
The fix for the second one is only safe because of the fix for the first. This post walks through both.
Disclosure: I'm the developer of Bolo, and this post is published from my account. It was written and published by an AI agent working on my behalf.
Problem 1: an idempotency key that never matched
The proxy already had an idempotency guard. The client sent an X-Bolo-Request-Id header, and a repeat of the same ID was supposed to return the first result instead of calling the speech service again.
The logs from 31 real dictations on one phone showed it collapsed nothing. One recording had been uploaded four times, with identical bytes, two of the uploads 44 ms apart. Each upload carried a fresh request ID, because the app minted a new UUID per call. Four IDs, four upstream calls, four quota charges. About 13% of the uploads in that sample were redundant.
The root cause on the phone was a race between two code paths that could both send the same recording, and that was fixed in the app. But an app build that is already installed cannot be patched from the server, so the server needed its own net.
Key on what the work is, not on what the client calls it
The question idempotency answers is "have I already done this piece of work?". For a transcription, the work is defined by who sent it and what audio it is. The request ID is just a label the client chose.
So the key became the device plus a SHA-256 of the uploaded bytes:
const serverAudioHash = crypto.createHash('sha256').update(req.file.buffer).digest('hex');
const suppliedAudioHash = String(req.headers['x-bolo-audio-sha256'] || '').toLowerCase();
if (suppliedAudioHash && suppliedAudioHash !== serverAudioHash) {
return res.status(400).json({ error: 'audio hash mismatch' });
}
const idempotencyKey = `${deviceId}:audio:${serverAudioHash}`;
const execution = await transcriptionIdempotency.execute(
idempotencyKey,
serverAudioHash,
() => transcribe(req.file /* , ... */)
);
Two choices worth pointing out:
- The server computes the hash itself. The client may send its own hash, and a mismatch is rejected with 400, which catches a truncated upload. But the key is always built from the server's hash, so a client cannot claim someone else's cached result by sending a made-up hash.
- The device is part of the key. Two different users dictating the same short phrase do not share a cache entry, even if their audio happened to be byte-identical.
The request ID is still logged on every line. It just no longer decides identity.
The store: share the promise, cache only success
class IdempotencyStore {
constructor({ ttlMs = 10 * 60 * 1000 } = {}) {
this.ttlMs = Math.max(1000, Number(ttlMs) || 10 * 60 * 1000);
this.entries = new Map();
}
async execute(key, audioHash, operation) {
this.cleanup();
const existing = this.entries.get(key);
if (existing) {
if (existing.audioHash !== audioHash) {
const error = new Error('Request ID was already used for different audio');
error.statusCode = 409;
throw error;
}
return { value: await existing.promise, deduplicated: true };
}
const entry = { audioHash, expiresAt: Date.now() + this.ttlMs, promise: null };
entry.promise = Promise.resolve().then(operation);
this.entries.set(key, entry);
try {
return { value: await entry.promise, deduplicated: false };
} catch (error) {
// Failed attempts are retryable with the same key; only completed work is cached.
this.entries.delete(key);
throw error;
}
}
cleanup(now = Date.now()) {
for (const [key, entry] of this.entries.entries()) {
if (entry.expiresAt <= now) this.entries.delete(key);
}
}
}
What makes this work for the "two uploads 44 ms apart" case:
- The entry is stored before the work finishes. The second upload finds the first one's promise and awaits it. It does not start a second upstream call, and it gets the same text as soon as the first finishes.
- Failure removes the entry. If the speech service errors, the next attempt with the same audio runs fresh. Caching an error would turn one bad moment into ten minutes of failures.
- A ten-minute TTL. Long enough to cover any realistic retry storm from one recording, short enough that the map stays small. It is in memory, so a restart clears it, which is acceptable: the worst case is one repeated call.
- The 409 branch is a safety check left from the request-ID design: the same key with different audio is a contradiction. With the key now derived from the audio hash it should never fire, and if it ever does, it means the key construction has a bug.
Problem 2: giving up on a link that was only flaky
A later report came from a home connection: a 24-second note timed out, the single automatic retry died after 5 seconds, and a manual resend went through and was transcribed in 1.5 seconds. The audio was fine and the server was fine. The client simply did not resend patiently enough.
With the server-side key in place, resending the same audio is harmless. If the earlier upload did land, the resend gets the cached text: no second speech call, no second quota charge. So the app can afford to try harder:
object TranscriptionResend {
const val MAX_ATTEMPTS = 4 // including the first
val BACKOFF_MS = listOf(2_000L, 5_000L, 10_000L) // before attempts 2, 3, 4
const val RESEND_WINDOW_MS = 90_000L // no new attempt after this
fun isTransient(e: Throwable): Boolean = when (e) {
is VoxException -> false // the app's own "this request is wrong"
is java.net.UnknownHostException -> true
is java.io.IOException -> !isCallCancelled(e)
is retrofit2.HttpException -> e.code() in TRANSIENT_HTTP
else -> false
}
// 500-504 from the proxy or nginx; 520-524 are Cloudflare's "origin did not answer in time".
private val TRANSIENT_HTTP = setOf(500, 502, 503, 504, 520, 521, 522, 523, 524)
// OkHttp reports its own cancel (the user tapped stop) as IOException("Canceled").
private fun isCallCancelled(e: java.io.IOException) =
e.message.equals("Canceled", ignoreCase = true)
}
And the loop:
suspend fun <T> run(
nowMs: () -> Long = System::currentTimeMillis,
sleep: suspend (Long) -> Unit = { delay(it) },
block: suspend (attempt: Int) -> T
): T {
val startedAt = nowMs()
var attempt = 1
while (true) {
try {
return block(attempt)
} catch (e: Exception) {
currentCoroutineContext().ensureActive()
val pause = BACKOFF_MS.getOrElse(attempt - 1) { BACKOFF_MS.last() }
val outOfTime = nowMs() - startedAt + pause > RESEND_WINDOW_MS
if (!isTransient(e) || attempt >= MAX_ATTEMPTS || outOfTime) throw e
attempt++
sleep(pause)
}
}
}
The rules it encodes:
- Only link failures are retried. A 4xx, or the app's own validation error, means resending the same thing gives the same answer.
-
A user's cancel is never retried.
ensureActive()rethrows if the coroutine was cancelled, and OkHttp's own "Canceled" IOException is treated as final. The original bug report included "I pressed retry and it cancelled", so a stop must mean stop. - A time budget, not only a count. Four attempts with growing pauses, but no new attempt once 90 seconds have passed since the first. A person waiting for text will not wait forever.
- Clock and sleep are parameters, so the whole policy is a fast unit test with a fake clock.
A read timeout that scales with the note
One more piece: a fixed 30-second read timeout cut off slow but healthy answers for long notes. The timeout is now 30 s plus half the note's length, capped at 2 minutes, applied by an OkHttp interceptor that reads the duration header the upload already carries:
fun readTimeoutMsFor(audioDurationMs: Long): Long =
(30_000L + audioDurationMs.coerceAtLeast(0L) / 2).coerceIn(30_000L, 120_000L)
Other calls keep the client's default timeouts.
Takeaways
- An idempotency key should describe the work, not the request. For uploads, a content hash plus the caller's identity is often the right key.
- Compute the hash on the server; treat any client-supplied hash as a check, not as the key.
- Store the in-flight promise so concurrent duplicates share one call, and cache only successful results.
- Server-side idempotency is what makes aggressive client resends safe. Build it first.
- Retry link failures only, never a user's cancel, and bound retries by time as well as by count.
Bolo is on Google Play as "Bolo AI Voice Typing & PC Sync" (search "Bolo AI"): https://play.google.com/store/apps/details?id=com.ashwar.bolo&utm_source=devto&utm_medium=article&utm_campaign=2026-w45-idempotency. Cloud dictation has a limited free cloud trial, and you can also bring your own OpenAI or Gemini key, or use offline on-device dictation.
Top comments (1)
Does the cache key also include the requested language or transcription settings? If I resend the same recording after correcting the language selection, device + audio hash alone looks like it could return the earlier transcript for ten minutes.