I built a flight recorder for AI sessions

#javascript #typescript #ai #devtools

Token costs kept showing up on my bill and I couldn't tie them to specific sessions. When something behaved unexpectedly my options were basically "add more console.log" and hope I could reproduce it. Not great.

So I built an AI Flight Recorder.

The idea

Flight recorders on planes capture everything that happens so that events can be reconstructed after the fact. Same concept here: wrap your existing client once, then every prompt, token, tool call, and completion gets captured automatically to a .flight file.

Setup is one line

import OpenAI from "openai";
import { FlightRecorder, wrapOpenAI } from "@ai-flight-recorder/sdk";

const fr = new FlightRecorder();
const openai = wrapOpenAI(new OpenAI(), fr);

// Your existing code doesn't change

Sessions save as .flight files, plain JSON that you can commit to git, attach to a bug report, or hand off to a teammate.

What you get

The DevTools app lets you replay any session at variable speed (0.25x to 8x) with a full timeline: prompts going in, tokens streaming out, tool calls, completions. It also breaks down cost per request so you can see exactly where your token budget is going.

There's a VS Code extension too if you'd rather not leave your editor.

The .flight format

I wanted these files to be readable without special tooling, so they're plain JSON with a version envelope:

{
  "version": "1",
  "exportedAt": 1722960000000,
  "session": {
    "id": "abc-123",
    "status": "ended",
    "events": [
      { "type": "prompt", "model": "gpt-4o", "prompt": "..." },
      { "type": "token", "token": "Hello", "index": 0 },
      { "type": "completion", "totalTokens": 142, "estimatedCost": 0.000284 }
    ]
  }
}

Each event has a timestamp, which is what makes the replay accurate.

Try it

If you have questions about the format design or how the replay timing works, I'm around in the comments.

DEV Community