Mic Doesn't Return Even After Reconnecting — It Was the Old Microphone Marked for Disposal That Was Enabled

📝 Originally published (in Japanese) at forge.workstyle.tech.

Operating a Voice Dialogue Avatar Embedded in a Website

The avatar connects when the page is opened and responds when spoken to. Behind the scenes, WebRTC connects the browser and voice pipeline.

However, after restarting the server, the avatar stopped responding.

👤 (speaking)
🤖 (no response)
👤 (speaking again)
🤖 (no response)

Visually, everything seems fine. The avatar is standing, and the microphone button is still ON. Visitors won't notice that it's "broken" until they reload the page.

Initially, There Was No Reconnection Mechanism

The first thing I checked was that there was no code to reconnect after a disconnection.

The connection process is only called once when the page is opened, and there is no path to reconnect after a disconnection. This means that not only server restarts but also events like subway rides, Wi-Fi switching, or waking up from sleep can cause the same issue. With each deployment, visitors who happen to be speaking to the avatar at that moment will be met with silence.

I added a reconnection process with exponential backoff.

const waits = [1000, 2000, 4000, 8000, 15000, 30000];
const wait = waits[Math.min(retryRef.current, waits.length - 1)];
retryRef.current += 1;
setTimeout(() => reconnectRef.current(), wait);

It will not timeout after the last 30 seconds, but repeat indefinitely. I intentionally did this because visitors are likely to leave the page open, rather than coming back later. Since the cost of reconnection is almost zero, there's no reason to give up.

After deploying and restarting the server, I got this log.

[widget] Connection lost — reconnection in 1 second (1st attempt)

...but it stopped there. The first log appeared, but the second one didn't. It didn't reconnect.

Non-Idempotent Function Was Called as If It Were Idempotent

The connection process started like this.

const connect = useCallback(async () => {
    if (pcRef.current) return;      // ← here
    ...

If an old connection object remains, it will return without doing anything. As a function called only once when the page is opened, this guard is correct to prevent double connections.

However, when reconnecting, it's precisely the "old connection" that remains. The connect() function was quietly returning without doing anything every time. No error, no log.

I changed it to call disconnect() before reconnecting. This time, it connected successfully.

15:09:43  Server restart
15:10:16  Voice pipeline detected disconnection
15:10:28  Reconnection (12 seconds later)

The conversation history remained intact. Since the conversation ID is stored in the browser's sessionStorage, reconnection allows the conversation to "continue from where it left off."

The Microphone Alone Didn't Come Back

Even though it reconnected, speaking to the avatar yielded no response. The microphone button was still ON.

I confirmed this on the server-side log, where I had implemented a diagnostic that outputs the volume and "human voice-like" detection every 30 seconds.

[vad] Last 30 seconds: 1500 judgments, 0 speeches, 0 near-misses (average voice 0.00, volume 0.00)

The volume remained 0.00. No sound was reaching the server. The microphone track is obtained when connecting and disabled by default. The button is enabled when pressed. So, after reconnection, if the microphone was originally ON, it needs to be re-enabled.

I Read the Source and Missed It Three Times

This is where the embarrassing part begins. I read the code, formed a hypothesis, fixed it, and missed it three times.

Suspected cause	Reason	Result
Limit on usage	There's a process that mutes the microphone when the limit is reached	The limit wasn't being applied
8-second status check misjudging	I had added a process to verify the reconnection later	It was unrelated
Reconnection process's internal order being swapped	Asynchronous processes were running in parallel	This was also incorrect

All three times, I "read the source and chose a plausible explanation." Each one seemed to make sense when reading the code, but none of them were actually happening.

On the fourth attempt, I gave up and added diagnostic logs to see who was muting the microphone and when.

One Log Line Solved It

The returned log was this.

[widget] After microphone recovery: {called: true, enabled: [true]}
[voicepipe] Obtaining new microphone track (default is disabled)    ← ★this is later
[widget] 2 seconds after microphone recovery: {enabled: [true], returned: true}

The microphone was being enabled before obtaining the new track.

In other words, the track I was enabling was the old track that was about to be discarded. After that, connect() created a new track, which started disabled by default.

It wasn't that someone was muting it later; the order was simply reversed. My three hypotheses were all about "who muted it," but no one was muting it.

The Premature Reason Was a Lying State Variable

Then, why did the enablement run before the new track was obtained? The condition was written like this.

if (voicepipe.status === 'connected') { ... }   // enable when connected

The update of the connection status was like this.

pc.oniceconnectionstatechange = () => {
  const s = pc.iceConnectionState;
  if (s === 'connected' || s === 'completed') {
    setStatus('connected');
  } else if (s === 'failed' || s === 'disconnected' || s === 'closed') {
    if (s === 'failed') setStatus('failed');    // ← only when failed
    ...

The status remains 'connected' even when 'disconnected' or 'closed'. The status is only updated to 'failed' when it actually fails.

So, the condition "enable when connected" was already met immediately after disconnection. I was waiting, but I wasn't actually waiting.

This state variable isn't wrong. It's reasonable not to immediately set the status to 'failed' when disconnected, as it might recover. The problem is that I used this variable, which has a specific meaning, to judge "is it currently connected?"

The fix was simple. Instead of looking at the state variable, I waited for the connection process to complete.

voicepipe.disconnect();
await voicepipe.connect();                              // wait for completion
if (micOnRef.current) voicepipe.setMicEnabled(true);     // then enable

The connect() function returns after obtaining a new track and completing the offer/answer process, so at this point, I can be sure to get the new track. I changed from "looking at the status and judging" to "executing in order."

It Worked

I confirmed it by restarting the server.

15:10:28  Reconnection (45 seconds after server restart / 12 seconds after disconnection detection)
15:17:42  [vad] Volume 0.51, 0 speeches      ← microphone is alive and waiting
15:18:12  [vad] 31 speeches, voice 0.03        ← responded to speech
15:17:49  [Talk TTS] 'Hello. What brings you here today?'

As a byproduct, I can now determine the microphone's status from the server-side log alone.

	Volume
Microphone OFF	0.00
Microphone ON	around 0.5

After realizing this, I no longer need to ask visitors to open their browser's console. I can verify the issue from the server-side log when they report that the microphone isn't responding despite being turned on.

Takeaways

• Be careful not to call non-idempotent functions as if they were idempotent. The if (already exists) return; guard is correct for initialization, but it becomes a "quietly do nothing" function when called from the "retry" path. Since it doesn't produce an error, it takes time to discover.
• State variables might only be looking at a part of the failures. The status variable this time was meant to indicate "failed or not," not "is it currently connected?" The name looks like the latter, but the processes that use this variable as a condition will all be wrong. If you want to wait for something, it's more reliable to await the completion rather than looking at the state.
• Hypotheses formed by reading the source can be wrong. I was wrong three times. Diagnostic logs solved it in one attempt. Code only teaches you "what can happen," not "what happened." If you come up with three plausible hypotheses, it's likely that you can't figure it out just by reading the code.
• Reconnection is not just about server restarts. Before implementing reconnection, I organized the causes of disconnections.
  • Deployment, pod restart, node replacement
  • Wi-Fi ↔ mobile network switching, subway, elevator
  • Smartphone sleep and wake (background connections may be terminated)
  • Intermediate NAT or TURN timeouts (may drop after prolonged silence)

While server-side issues can be reduced with operation, the last three cannot be reduced. Investing in "reconnect even if disconnected" rather than "don't disconnect" is the right approach.