I work on AuCut at W Lab. It is an audio editor for iPhone and Android: trim a recording, pull the audio out of a video, convert between MP3, M4A and WAV, change the volume or the speed. Everything is processed on the phone and no file is uploaded. That one rule shaped most of the architecture, and this post is about where it got awkward.
One Flutter UI, two native engines, and FFmpeg
The UI is Flutter with Riverpod: one controller and one screen per tool under lib/features, shared services under lib/core. Dart doesn't touch audio samples. It talks to native code over one MethodChannel for commands (trim, extract, convert, get duration, cancel) and one EventChannel for progress.
Behind that channel are two different engines:
- On iOS, AVFoundation, mostly
AVAssetExportSession. - On Android, Media3
Transformer.
And beside both, FFmpeg, through the LGPL audio-only build of FFmpegKit. The platform APIs alone turned out not to be enough, for the reasons below.
Neither platform will write an MP3
Both phones play MP3 all day. Neither ships an MP3 encoder you can call. AVFoundation has none, and neither does Android's stock codec set. So "save as MP3" is a two-step job: the native side exports M4A, then FFmpeg re-encodes it with libmp3lame. The progress bar is weighted so the two halves look like one job.
WAV is the opposite surprise. It is the simplest format there is, and neither export API will produce it. AVAssetExportSession has no WAV preset, and Transformer can't write one. Both are done by hand:
- iOS: read with
AVAudioFile, convert withAVAudioConverter, write 16-bit PCM. - Android: decode to raw PCM with
MediaExtractorandMediaCodec, then put a 44-byte WAV header in front of it.
The rule I ended up with: the platform does what it does well (cutting, pulling a track out of a video, AAC), and FFmpeg does what the platform can't.
Trimming without re-encoding when possible
A trim is a time range on an export. On iOS that is a CMTimeRange on the export session. The session uses the passthrough preset when the source codec can be copied straight into an M4A container, which is fast and loses nothing, and falls back to an AAC re-encode when it can't. On Android it is a clipping configuration on the media item.
If the user also asked for MP3 output, a volume change or a fade, a second FFmpeg pass follows the native cut.
Cutting by ear, to a tenth of a second
A waveform is too coarse to cut speech by eye. You can't see where a word ends. So the editor is built around listening:
- Dragging a handle on the waveform is continuous.
- Two arrow buttons under each handle move it 0.1 s per tap. Hold one and it repeats, then speeds up to 1 s steps.
- A jog bar under each handle gives fine control over ±5 seconds from where you touch it.
- Every time a cut point moves, the app plays two seconds around it: the first two seconds of what will be kept after a start point, the last two before an end point. You hear exactly what the cut will sound like.
The waveform is 100 bars of RMS, extracted natively by the audio_waveforms plugin. I paint the bars myself with a CustomPainter behind a RepaintBoundary, because the plugin's own widget repainted every bar on each drag frame. Seeks while scrubbing are throttled to one per 60 ms. A seek is async, and queueing one per pointer event just piles them up.
Speed without pitch, pitch without speed
Both are FFmpeg filters. atempo changes tempo and keeps pitch. asetrate changes both together. So a pitch shift is asetrate by the pitch ratio, then an atempo that cancels the tempo change it caused:
'aresample=$rate', 'asetrate=${(rate * k).round()}', 'aresample=$rate',
..._atempoChain(speed / k) // k = 2^(semitones / 12)
Older FFmpeg builds only accept 0.5 to 2.0 per atempo instance, so larger changes are chained.
Normalising volume uses FFmpeg's loudnorm, which measures loudness the EBU R128 way, not by peak. One trap: loudnorm upsamples to 192 kHz internally, so the filter chain has to resample back down or the output file balloons.
Files that arrive from other apps
Much of the real use starts in another app: a voice memo, a video in Photos, an audio message from a chat. On iOS the app declares itself a viewer for audio and movie types. On Android it has intent filters for send and view. Either way the native side copies the file into a temp directory, queues its path, and Dart drains the queue and asks which tool to open it in.
Two details mattered there. On a cold launch, the file can arrive before Flutter is listening, so the Dart side drains the queue once at start-up and the iOS side sweeps its inbox when asked. And chat apps hand over OPUS, OGG and FLAC, which the native trim path won't take, so anything that isn't M4A, MP3 or WAV is re-encoded to M4A by FFmpeg on the way in.
Temp files carry a timestamp in their name and are cleaned up after 24 hours.
Small things that cost real time
- Reading a file's duration does file I/O and can stall on a large or cold file, so it runs off the main thread on both platforms. An unreadable asset reports a NaN duration, and converting that to an
Intin Swift traps. - Android's
EventSinkmay only be called on the UI thread. A decode loop on a background thread has to post its progress. - A stray back swipe used to throw away a long export without a word. Now a guard asks first, and leaving a tool cancels its job. Riverpod providers outlive their screen, so without that a job kept running where nobody could see it.
- The strings are in eight languages, and a test fails if any language is missing a key or has an extra one.
What I'd do the same way again
Keep one narrow channel between Dart and native, with every operation reporting progress and accepting a cancel. Let each platform's own exporter do the cutting. And accept early that an audio app needs FFmpeg for the formats and filters the platforms leave out.
AuCut is on the App Store as "AuCut: Audio Cutter & Editor" and on Google Play as "adTrim: Offline Audio Tools": https://w-lab.app/apps/aucut/
Top comments (0)