DEV Community

Henry Parker
Henry Parker

Posted on

Building ReproSpec: Portable Evidence Bundles and Offline Replay for Node.js

#opensource #docker #testing #javascript

The Maintainer Problem

Open-source maintainers face a dual crisis: a high volume of incomplete or platform-dependent bug reports ("works on my machine"), and a growing security risk when triaging pull requests that require running untrusted test suites locally.

When a bug report is filed, maintainers often must choose between setting up a matching VM environment or triaging based on static text logs.

The Information Gap in Ordinary Logs

Ordinary logs are useful, but they usually do not include a standardized integrity manifest, a portable environment summary, and a documented local replay workflow in one bundle.

ReproSpec is an experimental CLI and evidence-bundle format designed to bridge this gap.

1. Bundle Structure

A ReproSpec bundle contains the following schema layout (reprospec.bundle/v0.1-alpha):

  • manifest.json: Metadata, schema version, and security exceptions.
  • command.json: Captured command parameters, exit code, and duration.
  • environment.json: Node runtime details, scrubbed environment variables.
  • stdout.txt / stderr.txt: Process logs.
  • integrity.json: SHA-256 hash index of the declared files.
  • EVIDENCE_CARD.md: A human-readable markdown report card.

2. Capture & Redaction Workflow

When you run reprospec capture -- <command>, ReproSpec:

  • Spawns the command and records its output streams.
  • Slices each of stdout and stderr streams individually to enforce a strict 10MB limit.
  • Runs a regex redaction processor to remove supported credential patterns (such as ghp_ tokens) from stdout, stderr, environment keys, and command arguments.

⚠️ Argument Immutability: ReproSpec executes the captured command with its original arguments to ensure program behavior is unchanged. Redaction is applied only to the persisted evidence representation.

3. Integrity Limitations

Verification re-calculates hashes against integrity.json.

⚠️ Important Limit: This only detects changes relative to the SHA-256 hashes recorded in the bundle manifest. A party able to rewrite the entire bundle could also recompute those hashes. It does not prove author identity or captured trace truthfulness.

4. Replay Sandbox Parity & Boundaries

  • Immutable Source Snapshot: Capture stores the required source content inside the evidence bundle under snapshot/, with relative paths, sizes, and SHA-256 hashes in snapshot.json and the bundle integrity manifest.
  • Writable Disposable Execution: Before replay, ReproSpec verifies the immutable snapshot and copies it into a new ephemeral execution workspace. Only that disposable copy is mounted writable, so ordinary build and test commands can create dist, coverage, cache, and temporary files.
  • No Original Checkout Mount: The original checkout and evidence bundle are not mounted into replay. allowWorkspaceMount is disabled and blocked in the Alpha.2 replay path.
  • Output Handling: Replay records stdout, stderr, the exit-code oracle result, and files created, modified, or deleted in the execution workspace. The workspace and generated files are removed after success, failure, or timeout; generated files are never added back to the evidence snapshot.
  • Dependency Isolation: An ephemeral build container uses npm ci with network access to fetch packages based on package-lock.json. It runs without host credentials, home-directory mounts, Docker socket access, or inherited variables.
  • Offline Replay: The actual replay container runs offline with --network none.
  • Host Check: The CLI compares git status before and after replay, failing if any host changes occur.

5. Lessons from the Three-Repository Pilot

During early testing against real repositories, we encountered several portability limits:

  1. Vercel's ms package: Replay failed because Jest 30 preset resolution (ts-jest) failed to resolve correctly relative to the isolated workspace root under Node 20.
  2. dotenv package: The exact public commit c0e32b8267b69a438bc0cc31345f73b5e2f037db passed the Node.js 20 baseline and Alpha.2 clean-room replay. Its build and test workflow generated dist and .tap content inside the writable ephemeral execution workspace; those files were tracked and discarded without changing the original checkout or immutable bundle snapshot.

6. Supported and Unsupported Alpha Scope

Supported Scope

  • Runtime: Node.js 20+
  • Languages: Pure JavaScript or TypeScript repositories (ordinary single-package structures).
  • Package Manager: npm with standard package-lock.json.
  • Operating Systems: macOS and Linux hosts.

Unsupported or Unverified Scope (Alpha)

  • npm workspaces and monorepos.
  • Yarn, pnpm, and non-npm package managers.
  • Runtimes other than Node.js (Python, Go, etc.).
  • Windows hosts.
  • Reuse of native host-compiled binaries inside replay.
  • Active network connections or remote endpoint access during replay.

7. Execution Commands and Outputs

To capture, verify, and replay an execution:

# Capture command execution
node bin/reprospec.js capture --output my-bundle -- node -e "console.log('original run')"
Enter fullscreen mode Exit fullscreen mode

Output:

[reprospec] Capturing: node -e console.log('original run')
[reprospec] Working directory: <temporary-workspace>
[reprospec] Started at: 2026-07-31T15:46:09.188Z

[reprospec] Bundle written to: <temporary-workspace>/my-bundle
[reprospec] Exit code: 0
[reprospec] Status: PASS
[reprospec] Removed environment variables: <redacted>

✅ Bundle written to: <temporary-workspace>/my-bundle
Enter fullscreen mode Exit fullscreen mode 
# Verify bundle integrity
node bin/reprospec.js verify my-bundle
Enter fullscreen mode Exit fullscreen mode

Output:

── ReproSpec Bundle Verification ──────────────────────
Bundle: <temporary-workspace>/my-bundle
Status: ✅ INTEGRITY_VERIFIED — The bundle manifest and SHA-256 file hashes were verified.

File Details:
  ✅ stdout.txt: OK
  ✅ stderr.txt: OK
  ✅ command.json: OK
  ✅ environment.json: OK
  ✅ integrity.json: OK
───────────────────────────────────────────────────────
Enter fullscreen mode Exit fullscreen mode 
# Replay inside Docker sandbox
node bin/reprospec.js replay my-bundle
Enter fullscreen mode Exit fullscreen mode

Output:

── ReproSpec Replay Sandbox ───────────────────────────
Bundle: <temporary-workspace>/my-bundle
Docker: v29.5.3
────────────────────────────────────────────────────────

[reprospec] Running replay container: docker run --detach --name reprospec-replay-<id> --network none --user 1000:1000 --memory 1g --cpus 1.0 --pids-limit 128 --security-opt no-new-privileges=true --read-only --env REPROSPEC_REPLAY=1 --env HOME=/tmp -v <temporary-execution-workspace>:/workspace --workdir /workspace --tmpfs /tmp:rw,noexec,nosuid,size=64m --stop-timeout 60 node:20-alpine node -e console.log('original run')

── Replay Result ───────────────────────────────────────
Status:           REPLAY_VERIFIED
Expected Exit:    0
Container Exit:   0
Oracle Match:     ✅ YES
Execution Mode:   writable-ephemeral-snapshot
Files Created:    <tracked>
Files Modified:   <tracked>
Files Deleted:    <tracked>
Workspace Cleanup: ✅ removed

Container Output:
original run

───────────────────────────────────────────────────────
Enter fullscreen mode Exit fullscreen mode

8. Specific Technical Feedback Request

We are looking for developer feedback on:

  • How you handle test suites requiring local databases or external configurations offline.
  • Caching policies for npm packages to prevent native binary mismatches.

Check out the repository and prerelease:

9. AI-Assistance Disclosure

Disclosure: This codebase and documentation were prepared with the assistance of agentic AI coding models during development planning, clean-room auditing, and public candidate preparation.

Top comments (0)