📝 Originally published (in Japanese) at forge.workstyle.tech.
TL;DR MediaRecorder delivers recording data in chunks, and processing them is asynchronous. If stopping closes the file before pending chunk sends finish, the final data is lost. Silent audio and a stop check that misses the studio avatar's voice can also cause empty files or cut speech.
Conclusion First
The reason a roughly 190-second recording with MediaRecorder lasted only 92 seconds was that the system didn't wait for the final chunk processing upon stopping, no audio data was sent during silence, and the stop logic did not account for the end of the studio-side audio. I implemented fixes to wait for the chunk transmission to complete, stream silence using a ConstantSource, and ensure the recording continues until the studio-side audio track has actually finished. After these fixes, a 60-second recording of silence was saved correctly with 60.49 seconds of video and 60.48 seconds of audio.
Why the last data was missing after stopping the recording
The reason for the missing last data was that the main side was notified of the recording stop before finishing processing the chunks.
MediaRecorder divides the recorded data into chunks and passes them on. Even after receiving a chunk, the process of converting its contents to a byte array is asynchronous (a method that proceeds to the next process without waiting for completion). Right before stopping, the last chunk had arrived, but I didn't wait for the processing to finish and notified the main side to stop the recording.
The main side receives the stop instruction and closes the file. Even if the chunk processing is completed afterwards, the destination for writing is already closed. Although it seems like the recording was working until the end, the last data is not saved.
In the fix, I processed the chunks in the order they were received and sent the next one after the previous chunk's transmission was completed. When stopping, I wait until all pending transmissions are finished. What was necessary was not the fact that the stop was called, but that the processing started before the stop was completed.
Why My 60-Second Voiceless Recording Resulted in a 0-Byte File
The reason the voiceless recording came out as a 0-byte file was that no data was flowing through the audio channel.
When I examined the case where I recorded for 60 seconds without any voice, I found that no audio data was being transmitted through the audio track used for the recording until a voice actually entered. With the audio input empty, the MediaRecorder did not write anything, resulting in a 0-byte file even after stopping the recording.
I had assumed that only the video was being captured. In reality, the recording input included an audio track (the audio data intended to be included in the recording), and this audio channel remained empty during periods of silence.
I connected a ConstantSource (an audio source that continuously outputs a constant value) to the audio channel, outputting a value of 0. This ensures that silent audio data is always flowing. The purpose is to continue passing the audio track to the recording even during moments when no voice is present. I discussed this mechanism in a previous article. What I discovered this time is that waiting for 60 seconds without any voice resulted in a 0-byte file, and that the root cause was the lack of incoming data on the audio channel. Even if the intent is to record video only, when using a recording setup that combines an audio track, the state of the audio input can significantly affect the final output.
Why the last voice of the studio-side avatar was cut off
The reason why the last word was cut off in the middle is that the recording stop judgment was not looking at the voice of the studio-side avatar.
The stop judgment was looking at the voice sounding in the avatar window on the desktop. However, the voice of the avatar that appears only on the broadcast screen is on the studio side. The window being looked at by the stop judgment does not have the end time of that voice.
I thought that if the side that determines the stop of the recording judges that "it's quiet now", it would also think that the voice on the studio side has ended. However, that premise was incorrect. Even just before stopping the recording, there were cases where the voice on the stage was still sounding. In fact, in a scene where the last word lasted for about 34 seconds, it was cut off after about 2 seconds.
So, I made the studio side have the end time of the voice for each avatar. When stopping the recording, it waits until the time when the last voice ends plus 1.5 seconds. However, this waiting time has an upper limit of 120 seconds. I also extended the main side's save wait time from 15 seconds to 150 seconds. This is to prevent the main side's wait from being cut off first while the studio is waiting for the end of the voice.
How the Recording Time and Audio Length Changed After Fixes
After the fixes, the video and audio lengths were almost aligned, and the final subtitles fit within the video.
Recording 60 seconds without speaking resulted in a video length of 60.49 seconds and an audio length of 60.48 seconds. A recording with six dialogue exchanges resulted in a video length of 171.42 seconds and an audio length of 171.37 seconds.
The alignment was achieved by revisiting the processing at three points before stopping the recording: waiting for chunk transmission, streaming silence in the audio path during silent periods, and waiting for the avatar's voice to end in the studio before deciding to stop. Fixing only one of these would still result in gaps in the remaining paths.
The recording time and the saved video length are not always the same. It was necessary to track not only the time when the stop was called but also the ongoing processes before and after, as well as which audio the termination judgment was monitoring.
Key takeaways
- Process chunks in order and wait for all pending sends before stopping.
- Feed silent data into the audio path with ConstantSource when no one is speaking.
- Use the studio avatar's voice end time in the stop check.
- Cap the studio wait at 120 seconds and extend the main save wait from 15 to 150 seconds.
Top comments (0)