My cleanup code was cancelling itself (an asyncio gotcha)

Every call that ended in silence was vanishing.

No transcript. No recording. And the session sat in our dashboard marked Active forever, like a guest who missed every hint that the party was over.

It was picky, too. Agent says goodbye? Saved. User hangs up? Saved. Caller goes quiet and the agent ends the call? Gone. 9 out of 9 in a day, while the other 24 calls were perfect.

That's not flaky. That's a code path.

The setup

I build real-time voice AI agents. When a caller goes quiet, the agent checks in ("Are you still there?") a few times, then ends the call. That runs as a background task:

async def _inactivity_loop(self):
    try:
        await asyncio.sleep(self.user_away_timeout)
        for _ in range(3):
            await self.say_check_in()
            await asyncio.sleep(self.check_interval)
        await self.end_call(reason="presence_timeout")
    except asyncio.CancelledError:
        logger.debug("inactivity task cancelled")

Every way of ending a call goes through one shared cleanup:

async def end_call(self, reason):
    await self._perform_cleanup()

async def _perform_cleanup(self):
    logger.info("Cleanup started")
    self._cancel_inactivity_task()   # stop the silence timer
    await self.session.aclose()
    await self.save_call_data()
    logger.info("Cleanup completed")

def _cancel_inactivity_task(self):
    if self._inactivity_task:
        self._inactivity_task.cancel()

Looks responsible. Call's ending, stop the timer.

Now ask one question: which task is running the cleanup when the call ends from silence?

What cancel() actually does

Task.cancel() doesn't stop a task on the spot. It sets a flag that says "next time you hand control back to the event loop, throw CancelledError in instead."

Normally you cancel a different task, parked on some await. But a task can cancel itself too. No error, no warning. It just keeps running until its next await, and then the error lands.

A delayed self-destruct, basically.

So on the silence path:

1. The inactivity task calls end_call(). Awaiting a coroutine doesn't create a new task, so we're still inside the inactivity task.
2. Cleanup calls .cancel() on the inactivity task. On itself.
3. Next line, await self.session.aclose(), suspends. CancelledError arrives.
4. It unwinds back to _inactivity_loop, gets caught, and is logged at debug. Which production doesn't print.

Call data never saved. The logs showed Cleanup started and then nothing at all.

30-line repro

import asyncio

class Session:
    def __init__(self, guard=False):
        self.events, self.task, self.guard = [], None, guard

    async def watchdog(self):
        await asyncio.sleep(0.1)
        await self.cleanup()

    async def cleanup(self):
        self.events.append("cleanup_started")
        if self.task and (not self.guard or self.task is not asyncio.current_task()):
            self.task.cancel()
        await asyncio.sleep(0)            # any real await
        self.events.append("call_data_saved")

async def run(guard):
    s = Session(guard)
    s.task = asyncio.create_task(s.watchdog())
    try:
        await s.task
    except asyncio.CancelledError:
        s.events.append("cancelled")
    print(f"guard={guard}: {s.events}")

asyncio.run(run(False))
asyncio.run(run(True))

guard=False: ['cleanup_started', 'cancelled']
guard=True: ['cleanup_started', 'call_data_saved']

Even await asyncio.sleep(0) is enough. One yield and the save is gone.

Why goodbye and hang-up worked

Their cleanup ran in a different task. Goodbye runs in the agent's tool-call task, hang-up spawns its own. So the cancel hit the sleeping inactivity task, not themselves.

Same method, same .cancel(). Correct for two callers out of three. That's why reviewing the method alone would never catch it.

The fix

def _cancel_inactivity_task(self):
    task = self._inactivity_task
    if task and task is not asyncio.current_task():
        task.cancel()

If cleanup runs inside that task, there's nothing to cancel. It finishes on its own.

The cleaner long-term design: a watchdog shouldn't tear down the thing that owns it. Let it signal (an asyncio.Event, a queue), and have one owner end the call. And no, asyncio.shield() doesn't help here. Your own await still raises.

The real villain: swallowing CancelledError

The self-cancel caused it. The debug log hid it.

CancelledError inherits from BaseException on purpose, so except Exception won't eat it. It's a control signal, not an error. Swallow it and callers think you stopped when you didn't, asyncio.timeout() and TaskGroup get confused, and real failures go silent.

except asyncio.CancelledError:
    logger.info("inactivity task cancelled")
    raise

A cleanup that stops halfway is never a debug-level event.

Why sessions stayed stuck forever

Losing data is one bug. A session stuck forever means every backstop failed too. Ours:

• A fallback "session ended" report existed, but sat after the cancelled await, and its API route wasn't deployed on dev anyway.
• LiveKit's room_finished webhook would have closed it, but dev had no webhook configured.
• expires_at only filtered lists. It never changed the status.

If your expiry doesn't change state, it isn't expiry. It's a rug.

One testing lesson

My first test reproduced the pattern and passed. But the bug lived in how the real class wired things together. The test that matters drives the real silence path with tiny timeouts and asserts on outcomes ("data saved, status ended"), not on "cleanup() was called". Cleanup was called. It just never finished.

Grep your code for this today

• Every .cancel() in shared cleanup: can the target be the caller?
• Every except asyncio.CancelledError: does it re-raise?
• Background timers that call teardown: should they signal instead?
• Success logs: do they come after the work?

The fix was one if. Finding it took a day, thanks to one debug log line.