DEV Community

Cover image for Don't Embed a Live Matplotlib Canvas in Your Desktop App: Render the Waveform as a Bitmap
WgeorgeAssistantIA
WgeorgeAssistantIA

Posted on

Don't Embed a Live Matplotlib Canvas in Your Desktop App: Render the Waveform as a Bitmap

If you search for "audio waveform Python GUI", the usual advice is to embed a Matplotlib canvas (FigureCanvasTkAgg, FigureCanvasQTAgg) in your window.

It works. It also has a catch that only shows up when the user resizes the window.

In VoxCut, a desktop tool that removes silences from audio, I kept Matplotlib for drawing the waveform. What I changed is how it is embedded: the waveform is rendered once, off-screen, into a bitmap, and the window only ever shows that bitmap. Here is the pattern, and why it fixed the resize glitches.

The problem with a live canvas

A live Matplotlib canvas installs its own resize handler. Every time the widget changes size, it re-renders the whole figure into the Agg buffer, synchronously, on the UI thread.

When a user drags a window corner, the OS fires a stream of <Configure> events. Each one triggers a full Matplotlib render. I measured one render of a small waveform strip (1,200 × 72 px, 1,400 peak values) at about 37 ms on median, 64 ms at worst on my dev machine. That is already two to four frames at 60 Hz, per event, blocking the thread that repaints the rest of the window.

In VoxCut this showed up as trails and ghosting while resizing and scrolling: the window could not repaint because the UI thread was busy re-rendering a 72-pixel-high strip.

Step 1: reduce the audio to ~1,400 numbers

Before drawing anything, you don't want raw samples. You want an envelope: one value per horizontal bin.

VoxCut asks FFmpeg for a mono, low-rate raw stream and reduces it on the fly:

cmd = ["ffmpeg", "-v", "quiet", "-i", str(input_file),
       "-vn", "-ac", "1", "-ar", "8000", "-f", "s16le", "-"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
Enter fullscreen mode Exit fullscreen mode

Then it reads the pipe in 128 KB chunks, so memory stays flat whatever the file length:

chunk = proc.stdout.read(131072)
arr = np.abs(np.frombuffer(data[: n * 2], dtype="<i2").astype(np.float32))
Enter fullscreen mode Exit fullscreen mode

Each bin keeps the maximum absolute value of its samples. The total number of bins is fixed (1,400 points), and the bin size is total_samples // points. At the end the array is normalized to 0–1.

Two details matter:

  • 8 kHz is plenty for a visual envelope. You are not listening to it, you are drawing it.
  • The loop carries a leftover byte when a chunk ends in the middle of a 16-bit sample, and a cur_max / cur_count pair across chunk boundaries, so bins are correct even when they straddle two reads.

On my machine, the NumPy part for one hour of 8 kHz PCM (440 chunks of 128 KB) costs about 0.1 s. The real cost is FFmpeg decoding the file, not the reduction.

Step 2: render once, off-screen, to a bitmap

Matplotlib stays, but with the Agg backend and no live canvas:

import matplotlib
matplotlib.use("Agg")
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg
from PIL import Image

fig = Figure(figsize=(w_px / 100, h_px / 100), dpi=100, facecolor=bg)
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
x = np.arange(len(peaks))
ax.fill_between(x, peaks, -peaks, lw=0)

canvas = FigureCanvasAgg(fig)
canvas.draw()
w, h = canvas.get_width_height()
img = Image.frombuffer("RGBA", (w, h), canvas.buffer_rgba(), "raw", "RGBA", 0, 1).copy()
Enter fullscreen mode Exit fullscreen mode

The result is a plain PIL image, shown in a label. A bitmap in a label is resized and scrolled natively by the toolkit, with none of the smearing you get when a widget re-renders itself on every event.

A bonus: Matplotlib is heavy to import (about 2 seconds in a frozen executable for VoxCut), so it is loaded lazily, the first time a file is opened, and the window appears faster at startup.

Step 3: stretch while dragging, re-render when it stops

You still want a sharp waveform at the final size. The trick is to do two different things depending on whether the user is still dragging:

def _on_configure(self, event=None):
    w = self._avail_width()
    # Pure vertical scroll also fires <Configure>: width unchanged -> do nothing
    if self._last_rendered_w is not None and abs(self._last_rendered_w - w) < 4:
        return

    # 1. Instant: stretch the existing bitmap (no Matplotlib call)
    stretched = self._raw_pil_img.resize((w, h_phys), Image.Resampling.BILINEAR)
    self._lbl.configure(image=ctk.CTkImage(stretched, size=(w / scale, H)))

    # 2. Debounced: re-render sharply once the drag has stopped
    if self._render_job is not None:
        self.after_cancel(self._render_job)
    self._render_job = self.after(250, self._rerender)
Enter fullscreen mode Exit fullscreen mode

Three things make this work:

  1. Ignore events where the width didn't change. Scrolling the page vertically also emits <Configure> (the widget's relative position changed). A 4 px threshold on the width filters those out.
  2. A PIL resize is cheap. I measured a bilinear resize of the strip at about 3 ms median (6 ms worst), against 37 ms for a full Matplotlib render. During the drag the user sees a slightly soft bitmap that follows the mouse.
  3. A 250 ms debounce (after / after_cancel) triggers the single sharp re-render once the window stops moving.

Step 4: the playhead is not part of the image

The moving cursor and the hover timecode tooltip are separate widgets placed on top of the bitmap, not pixels in it:

self._cursor = ctk.CTkFrame(self._lbl, width=2, fg_color=text_color, corner_radius=0)

def set_cursor(self, frac):
    self._cursor.place(relx=min(1.0, max(0.0, frac)), y=0, relheight=1.0)
Enter fullscreen mode Exit fullscreen mode

Moving a 2-pixel-wide frame with place(relx=...) never touches the waveform image. The waveform is rendered when the data or the width changes, and not once per playback tick. That is the same idea as "layering" in a game engine, using ordinary toolkit widgets.

What I would take from this

  • Don't let a plotting library own your resize loop. Matplotlib is fine as a renderer; it is a poor fit as a live widget in a window that gets resized and scrolled.
  • Reduce first, draw second. A fixed number of bins (here 1,400) decouples the drawing cost from the file length.
  • Use a cheap approximation while the user is moving things, and the exact result when they stop.
  • Keep moving things (cursor, tooltip) out of the expensive layer.

A caveat on the numbers: they come from a deliberately throttled development machine, from my own micro-benchmark (30 renders, 60 resizes), not from a rigorous study. Treat them as orders of magnitude, and measure on your own hardware.

VoxCut is the app this pattern comes from. If you want to see the result, it is at voxcutpro.com.

Top comments (0)