DEV Community

Cover image for How Bastillion Streams Hundreds of MB of SSH Session Replay Without Melting the Browser
Loophole, LLC
Loophole, LLC

Posted on

How Bastillion Streams Hundreds of MB of SSH Session Replay Without Melting the Browser

Bastillion is a self-hosted SSH gateway: it centralizes SSH key management and terminal access behind one web console, so you're not scattering keys and ad-hoc bastion boxes across your infrastructure. The feature we spent the most time on recently is session audit and replay, and the interesting part wasn't the recording, it was making the replay not fall over.

Bastillion multi-session web terminal

Why replay, not just logging

Compliance frameworks like PCI DSS, HIPAA, SOC 2, and ISO 27001 all want the same thing: a privileged-access audit trail that shows who ran what, where, and when. A scrollback log is fine for a quick grep, but when you need to reconstruct exactly what an admin did on a production box six weeks ago, character-by-character replay of the actual terminal session is a lot more convincing than a text dump.

So Bastillion records every keystroke and every byte of terminal output per session, and lets you scrub through it later like a video, except it's a real terminal.

The problem: sessions get big

A short debugging session might be a few KB. A long-running one, someone tailing logs, running a build, leaving a pane open overnight, can produce hundreds of megabytes of raw terminal output. The naive implementation (load the whole session into memory, hand it to the browser as one blob, let the DOM render it) works great in a demo and falls over immediately in the real world:

  • The server has to buffer the entire session before responding.
  • The browser has to hold the whole thing in memory and reflow the DOM as it renders.
  • Seeking to "near the end" of a huge session means waiting for everything before it.

The fix: stream it

Instead of materializing the full session, Bastillion streams the recorded output incrementally: the server reads and pushes chunks from the database as the client asks for them, and the browser renders progressively instead of waiting for a complete payload. Memory stays flat regardless of session size, because at any point in time you're only holding the chunk you're currently rendering, not the whole recording.

This sounds simple in hindsight, but it's the difference between a feature that works on your test VM and one that survives someone's 300MB overnight session in production.

Cleaning up the noise

Raw terminal output is full of ANSI/VT100 control sequences: color codes, cursor movement, screen clears, alternate screen modes. Recorded verbatim, replaying it naively means re-executing all of that control logic just to render old output, which is both slow and fragile (get a sequence wrong and the replay UI breaks). Bastillion strips these control sequences from the replayed output, trading "pixel-perfect terminal emulation" for "reliable, readable playback," which is the right trade for an audit tool where the goal is what happened, not what it visually looked like.

A bug that only shows up under load

While productionizing this, we also found and fixed a server-side cleanup process that could enter an infinite request loop under certain conditions, the kind of bug that's invisible in a quick test and only shows up once real session volume hits the retention/cleanup path. Audit data has to be trustworthy, so a background job that could spin instead of complete was a real problem, not just a performance nit.

Where this sits in the bigger picture

This shipped as part of a broader modernization: Bastillion now runs as a single self-contained JAR (embedded Jetty, built-in HTTPS, no external app server), rebuilt on Java 21 and Jakarta EE 11, with Ed25519/Ed448 SSH key support and a hardened auth layer (CSRF protection with constant-time comparison, PBKDF2 password hashing with a legacy fallback so existing users aren't forced to reset credentials, per-IP login throttling). Session audit and replay is enabled by default with a 90-day retention window out of the box.

Bastillion is built by Loophole, LLC and is source-available under the Prosperity Public License, free to run for up to 5 systems. Code's on GitHub if you want to see how the streaming layer is wired up.

Top comments (0)