TestFly Part 14: Step Traces — Per-Step ARIA/DOM Snapshots for Visual Debugging

TL;DR: TestFly 1.1.0 adds per-step snapshots to the trace: every step now records the
accessibility tree (or, with dom/both, the pruned page HTML) as it looked when the step
started, and a failed test also records the page after its last step with a before/after diff.
New config keys tracing.snapshots, tracing.snapshotMaxKb and tracing.snapshotOnFailureAfter
control what's captured, StepRecord carries the new fields, and TraceRecorder.saveIfEnabled
writes the trace — now from TestNG, JUnit 5, and Cucumber, with the HTML report linking to it
from every test.

What Is a Step Trace, and Why Snapshot Every Step?

A trace is one self-contained HTML file — target/traces/{ClassName}/{testMethod}-trace.html —
that replays a test's step timeline: step names, offsets, screenshots, the final failure
screenshot, the error and stack trace, and the video if recording is on. You open it in any
browser; it needs no server.

Screenshots tell you what the page looked like. They don't tell you what was actually in the
DOM — whether a button was disabled, whether an element had aria-hidden, whether the markup you
expected was even there. That's the gap per-step snapshots close: alongside each step's
screenshot, the trace now embeds a text snapshot of the page exactly as StepSnapshotCapture
saw it the moment the step began.

# testfly.yml
tracing:
  enabled: true

With tracing on, every step gets a snapshot by default — no extra code in the test.

What Does Each Step's Snapshot Contain?

Capture is driven by StepSnapshotCapture.captureForStep(), called once per step. It checks
tracing.enabled, reads the tracing.snapshots mode, and only touches the browser if a driver
session is actually alive — API and database tests, or steps logged before a browser exists,
simply get no snapshot:

// StepSnapshotCapture.java (simplified)
public static Snapshots captureForStep() {
    TestFlyConfig.Tracing tracing = tracing();
    if (tracing == null || !tracing.isEnabled()) {
        return Snapshots.NONE;
    }
    String mode = tracing.getSnapshots();
    boolean aria = mode.equals("aria") || mode.equals("both");
    boolean dom  = mode.equals("dom")  || mode.equals("both");
    WebDriver driver = liveDriver();
    if (driver == null) {
        return Snapshots.NONE;
    }
    int maxKb = tracing.getSnapshotMaxKb();
    return new Snapshots(aria ? captureAria(driver, maxKb) : null,
                          dom  ? captureDom(driver, maxKb)  : null);
}

StepRecord — the object that already held a step's name, offset, status and screenshot — grew
two new fields plus one that's filled in later, for a failed test's final state:

// StepRecord.java
public StepRecord(String name, long offsetMs, String status, String screenshotBase64,
                   String ariaSnapshot, String domSnapshot) { ... }

public String getAriaSnapshot()      { return ariaSnapshot; }
public String getDomSnapshot()       { return domSnapshot; }
public String getAfterAriaSnapshot() { return afterAriaSnapshot; } // set only on failure

aria captures the same YAML text ariaSnapshot() prints — small, readable, unaffected by CSS
class churn. dom captures the page HTML, pruned of scripts, styles and decoration by
DomPruner; it's bigger but shows markup the accessibility tree leaves out, like a CSS class
you're debugging.

How Do You Configure What Gets Captured?

# testfly.yml
tracing:
  enabled: true
  snapshots: both             # none | aria | dom | both — default: aria
  snapshotMaxKb: 256           # longest snapshot kept per step, in KB — default: 256
  snapshotOnFailureAfter: true # also snapshot the page after the last step of a failed test

TestFlyConfig validates tracing.snapshots eagerly — anything other than none, aria, dom
or both throws IllegalArgumentException at config-load time, not mid-run. A snapshot longer
than snapshotMaxKb is cut and ends with … truncated (N more characters), so one verbose page
never bloats the whole trace file.

What Does the Trace HTML Actually Show You?

Click a step in the timeline and the screenshot panel opens with tabs: ARIA (before step),
DOM (before step), and — only on the last step of a failed test — ARIA (after failure) and a
computed Diff tab. The diff is a plain line-based set difference between the before and after
YAML, rendered client-side in the trace's inlined JavaScript (lineDiff() in TraceRecorder):
lines only in the "before" snapshot are marked -, lines only in "after" are marked +. No
external diff library, no network call — the trace is one file you can email or drop in Slack.

Step blocks in the timeline also get a small 🌳 marker when they carry a snapshot, next to the
📷 marker for a screenshot, so you can see at a glance which steps have something to click into.

Does This Slow Down or Risk Failing My Tests?

Capturing a snapshot is one script call per step — a few milliseconds on a typical page — and it
only runs when tracing.enabled is on. StepSnapshotCapture wraps every capture in a
try/catch that returns null on failure rather than propagating: a page that can't be read, a
closed driver, or a flaky script call never fails the test, it just leaves that step without a
snapshot. TraceRecorder.saveIfEnabled itself is wrapped the same way:

// TraceRecorder.java
@TestFlyApi(since = "1.1.0")
public static void saveIfEnabled(String testId, String testName, boolean isPassing) {
    try {
        TestFlyConfig.Tracing tracing = TestFlyContext.getConfig().getTracing();
        if (tracing == null || !tracing.isEnabled()) return;
        if (isPassing && !tracing.isCaptureOnPass()) return;
        save(testId, testName);
    } catch (Exception ignored) {
    }
}

Does This Work With Cucumber?

Yes — as of 1.1.0, CucumberHooks calls the same trace machinery TestNG and JUnit 5 use, so
Gherkin scenarios get a trace file too, snapshots included, with no change to your feature files
or step definitions. The HTML test report also links directly to each test's trace from its
detail panel, so you go from a red row in the report to the exact step and snapshot that shows
what went wrong, in one click.

FAQ

Do I need to add code to my tests to get per-step snapshots?

No. With tracing.enabled: true, every step captures a snapshot automatically according to
tracing.snapshots (default aria). There's no per-step API to call.

What happens if a test has no browser, like an API test?

Nothing is captured. StepSnapshotCapture checks for a live WebDriver session before taking a
snapshot and returns no snapshot if none exists — it never creates or recreates a driver just to
capture one.

Can a snapshot capture failure cause my test to fail?

No. Every capture path is wrapped so an exception is swallowed and logged at FINE level, never
rethrown. A snapshot is best-effort; it never affects the pass/fail verdict.

Is the diff tab a real structural diff of the accessibility tree?

No — it's a line-based set difference between the "before" and "after" ARIA YAML text, computed
in the trace's own inline JavaScript. It's enough to spot what appeared or disappeared, not a
semantic tree diff.