Build an Agent Roadmap Tracker with TracePilot

What we're building: a small Node.js script that reads your GitHub roadmap issues, tracks which tracks are gated, and traces every decision so you can debug why a status flipped.

Prerequisites

• Node.js 18+
• A GitHub personal access token (read access to your repo)
• A TracePilot API key — free at tracepilotai.com
• npm install @octokit/rest tracepilot-sdk

Step 1 — Scaffold the project

mkdir roadmap-tracker && cd roadmap-tracker
npm init -y
npm pkg set type=module
npm install @octokit/rest tracepilot-sdk dotenv

Create .env:

GITHUB_TOKEN=ghp_...
GITHUB_REPO=Q00/ouroboros
TRACEPILOT_API_KEY=tp_live_...

Step 2 — Fetch roadmap issues

Roadmap sequencing lives in issues #920–#960. We pull them with Octokit.

// fetch.js
import { Octokit } from '@octokit/rest';
import 'dotenv/config';

const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
const [owner, repo] = process.env.GITHUB_REPO.split('/');

export async function fetchRoadmapRange(start = 920, end = 960) {
  const issues = [];
  for (let n = start; n <= end; n++) {
    try {
      const { data } = await octokit.issues.get({ owner, repo, issue_number: n });
      issues.push({
        number: data.number,
        title: data.title,
        state: data.state,
        labels: data.labels.map(l => l.name),
        updated: data.updated_at,
      });
    } catch (err) {
      if (err.status !== 404) throw err;
    }
  }
  return issues;
}

Run it:

node -e "import('./fetch.js').then(m => m.fetchRoadmapRange().then(console.log))"

You'll get a list of issues with their state and labels. Good. Now we need to understand what's gated.

Step 3 — Derive track status

A track is "gated" when its issue is open and carries a blocked or gate label. Everything else is either done or in-flight.

// status.js
export function deriveTrackStatus(issue) {
  const labels = new Set(issue.labels);
  if (issue.state === 'closed') return 'complete';
  if (labels.has('blocked') || labels.has('gate')) return 'gated';
  if (labels.has('wip') || labels.has('in-progress')) return 'in-flight';
  return 'planned';
}

export function summarize(issues) {
  return issues.reduce((acc, i) => {
    const s = deriveTrackStatus(i);
    acc[s] = (acc[s] || 0) + 1;
    return acc;
  }, {});
}

Step 4 — Wire it together

// index.js
import { fetchRoadmapRange } from './fetch.js';
import { deriveTrackStatus, summarize } from './status.js';

const issues = await fetchRoadmapRange();
const withStatus = issues.map(i => ({ ...i, status: deriveTrackStatus(i) }));

console.table(withStatus.map(({ number, title, status }) => ({ number, title, status })));
console.log('Summary:', summarize(issues));

You now have a working roadmap tracker. Copy-paste, run, done.

Step 5 — Adding observability

Here's where things get real. When the roadmap says Track A is "gated" but the warden comment says it's actually complete, you need to know why. That's what TracePilot is for.

Install:

npm install tracepilot-sdk

One line change — wrap your fetch + derive pipeline in a trace:

// index.js
import { TracePilot } from 'tracepilot-sdk';
import { fetchRoadmapRange } from './fetch.js';
import { deriveTrackStatus, summarize } from './status.js';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY);

await tp.startTrace('agentos-roadmap-warden');

const issues = await tp.wrapToolCall(
  'github-fetch-roadmap',
  () => fetchRoadmapRange(),
  null,
  1
);

const withStatus = issues.result.map(i => ({ ...i, status: deriveTrackStatus(i) }));

const { spanId } = await tp.wrapToolCall(
  'derive-track-status',
  async () => summarize(issues.result),
  null,
  2
);

console.table(withStatus.map(({ number, title, status }) => ({ number, title, status })));
console.log('Summary:', summarize(issues.result), '· span', spanId);

That's it. Every run now emits a trace with the exact issue payloads, the derived statuses, and the parent-child relationship between fetch and derive. Open tracepilotai.com/dashboard and you'll see the whole run.

Why this matters for a roadmap

Roadmaps drift. An issue gets closed, a label gets removed, the warden comment says one thing and the API says another. Without traces you're diffing JSON by eye at 11pm.

With TracePilot you can:

• See the exact issue payload that produced a wrong status.
• Fork the derive step with a mutated label set and replay — no re-fetching 40 issues.
• Catch silent regressions when a label rename flips ten tracks from gated to planned overnight.

For a meta-SSOT like #961 where a single mis-derived status can unblock the wrong track, that's the difference between a 30-second fix and a two-hour investigation.

Next steps

• Add a --since flag so you only trace issues updated in the last 24h.
• Persist summaries to a JSON file and diff against the previous run — wire the diff into a wrapToolCall('roadmap-drift-check', ...).
• Schedule it with a cron or GitHub Action, and let TracePilot alert you when gated count jumps by more than 3.
• Extend deriveTrackStatus to read the warden comment body and cross-check it against labels — then trace both signals so you can see which one lied.

You got this. Ship the tracker, then let the traces tell you what broke.

---

Debugging AI agents shouldn't feel like reading The Matrix.
Join other engineers who are building reliable autonomous workflows in our community: TracePilot Discord