A batch job has processed 47 of 50 audio files when one worker disappears. What should happen when the job starts again?
Restarting all 50 wastes work. Trusting a counter in the parent process is risky if that process also restarted. The interesting question is: where can the next run find authoritative evidence that a file is already done?
The Sub-Agent Orchestrator Actor example answers that with a parent actor, a child actor per file, and Telnyx KV. It is a TypeScript sample built on @telnyx/edge-runtime 0.15.2, with a local demo that can simulate a worker failure.
The write order matters
The parent OrchestratorAgent starts a job and spawns TranscriberAgent children. Each child handles one audio URL. In live mode, a child sends that URL to Telnyx's /v2/ai/audio/transcriptions endpoint. In the default demo mode, transcription is mocked and no real SMS is sent.
After processing a file, the child writes its outcome to KV before calling back to the parent. The per-file key has the shape job/<jobId>/file/<fileId>; the parent also maintains an attempt record under job/<jobId>/attempt/<fileId>.
That ordering closes an important gap. If a child finishes its transcription but goes away before reporting completion, a later reconciliation can see the KV result and adopt it. The notification to the parent is useful for progress, but it is not the only record of completed work.
Resume with the same job ID
The HTTP entry point routes POST /jobs to a parent actor named for jobId. The response distinguishes three cases:
| Status | Meaning |
|---|---|
STARTED |
This job ID is new. |
RESUMED |
The job exists and its state is reconciled with KV. |
ALREADY_DONE |
The completed job is a no-op, even after the actor has cleaned itself up. |
During reconcile(), the parent reads per-file KV records and the attempt ledger. It leaves completed files alone, checks which workers are still running, and replaces missing or stuck workers when attempts remain. MAX_CHILD_ATTEMPTS bounds that recovery; exhausted files remain failures in the final scorecard rather than silently disappearing.
The sample exposes GET /api/jobs/:jobId for the job record. GET / is a clinic-themed demonstration of the batch and failure scenario, while GET /console shows the actor names, attempts, and recovery state for developers.
Run the failure scenario locally
The repo includes a local host and an in-memory recovery test, so you can inspect the behavior before deploying an Edge function:
git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/sub-agent-orchestrator-actor
npm ci
cp .env.example .env
npm run typecheck
npm run test:recovery
npm run dev
Open http://localhost:8787/ for the demonstration, or /console for the engineering view. The example's .env.example sets DEMO_MODE=true, so this flow uses mock transcripts and logs the operator SMS instead of sending it. You can also submit a job to the local server:
curl -X POST http://localhost:8787/jobs \
-H 'Content-Type: application/json' \
-d '{"jobId":"demo-batch-1","hangOnce":["file-2"],"stuckTimeoutSeconds":6}'
hangOnce makes the first attempt for file-2 fail in the demo; the replacement attempt can then complete. Reusing demo-batch-1 lets you observe the resume or already-done path. The README covers live-mode credentials and deployment with npm run deploy.
This is a sample, not a patient-data production system. Before adapting the pattern to real recordings, review authentication, data retention, access to transcripts, retry behavior, and how the audio URLs are supplied.
The reusable idea is the write order: persist a result before reporting it, then reconcile from persisted evidence after an interruption.
Resources
Top comments (0)