Redact the Stack Before the Model Sees It

A raw stack trace is closer to a memory dump than a comment. You should not hand it to a model, a Slack thread, or a ticket until secrets are gone and the payload has a hard size. The useful move is a local digest pipeline: redact, cap, call, validate, then keep a receipt.

That order is the article. Skip a step and you leak a session token, stall on a dark host, or publish a paragraph that never names the failing function.

The paste reflex

You hit a 500 in staging. The log pane dumps forty frames, a request id, a customer email, and a JWT that someone printed "just this once." You want five lines for the channel. A chat box looks faster than reading.

It is not faster when the paste still holds an Authorization header. It is not faster when the model invents a root cause because the trace was chopped mid-frame. And it is not faster when the HTTP call hangs while your terminal still looks busy.

Treat the model like Redis sitting across the network. You would not send Redis an unbounded blob without a timeout. You would not log the raw command with passwords intact. Give the digest the same manners.

A compiler pass, not an incident bot

You are not wiring a pager. You are building a small compiler pass that turns noisy stderr into a tiny JSON object. The object names a likely surface, keeps two or three frames you still trust, and refuses to guess when the input is too thin.

Keep the CLI boring. Read a file. Write digest.json and receipt.json. Exit non-zero when redaction, budget, health, or schema fails. Humans still own the outage. Weekend-grade hosts go dark without a status page. Plan for that silence in the client, not in a postmortem.

Redaction is the first pass

Think of redaction as a linter that runs before prettier. If the linter fails, nothing else runs. You do not "see what the model says" with a live cookie still in the text.

The filter below is a starting net, not a compliance program. It strips common secret shapes, emails, and bearer tokens. It also drops query strings from URLs because reset links hide there. Sample code only; extend the rules for your shop.

export const SECRET_SHAPES = [
  { name: 'bearer', re: /bearer\s+[a-z0-9._\-+=\/]+/gi, to: 'bearer [redacted]' },
  { name: 'jwt', re: /\beyJ[a-zA-Z0-9_-]+\.[a-zA-Z0-9_-]+\.[a-zA-Z0-9_-]+\b/g, to: '[jwt]' },
  { name: 'email', re: /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi, to: '[email]' },
  { name: 'key_eq', re: /\b(api[_-]?key|secret|password|token)\s*[:=]\s*\S+/gi, to: '$1=[redacted]' },
  { name: 'aws_akid', re: /\bAKIA[0-9A-Z]{16}\b/g, to: '[aws-key-id]' },
  { name: 'query', re: /https?:\/\/[^\s]+/gi, to: (m) => m.replace(/\?.*$/, '?[query-redacted]') }
];

export function redact(raw) {
  let text = String(raw);
  const hits = [];
  for (const rule of SECRET_SHAPES) {
    const found = text.match(rule.re);
    if (found && found.length) hits.push({ name: rule.name, count: found.length });
    text = text.replace(rule.re, rule.to);
  }
  return {
    text,
    hits,
    originalChars: String(raw).length,
    redactedChars: text.length
  };
}

Run that function on a fixture you know contains a fake JWT. If the output still contains eyJ, you do not call the model. That check is the security story for this CLI.

node --input-type=module -e "
import fs from 'node:fs';
import { redact } from './redact.mjs';
console.log(JSON.stringify(redact(fs.readFileSync('trace.sample.txt','utf8')), null, 2));
"

Budget the frames, not the novel

After redaction you still have a wall of text. Quiet endpoints often swallow overflow instead of naming it. They also invent frames that never arrived.

Keep a character budget and a frame budget. Count lines that look like frames. Keep the exception header plus the top frames. Drop the rest on purpose and record the drop count. These caps are sample defaults, not measured optima.

export const BUDGET = {
  maxChars: 8000,
  maxFrames: 12,
  reserve: 1200
};

const FRAME_RE = /^\s*at\s+/;

export function clipTrace(text, budget = BUDGET) {
  const lines = text.split(/\r?\n/);
  const header = [];
  const frames = [];
  for (const line of lines) {
    if (FRAME_RE.test(line)) frames.push(line);
    else if (frames.length === 0) header.push(line);
  }
  const keptFrames = frames.slice(0, budget.maxFrames);
  let body = [...header, ...keptFrames].join('\n');
  const room = budget.maxChars - budget.reserve;
  if (body.length > room) body = body.slice(0, room);
  return {
    body,
    keptFrames: keptFrames.length,
    droppedFrames: Math.max(0, frames.length - keptFrames.length),
    chars: body.length
  };
}

A dropped frame is a first-class field. If eighteen frames fall on the floor, the digest must say the picture is incomplete. Otherwise you will debug a ghost that only existed in the prompt prefix.

A character cut is a blunt knife. Tokenizers will disagree about where the last frame ends. Tune against the endpoint's current documented limit, not a number copied from an old blog.

Write a receipt before you wait

The last time you stared at a spinner, you could not tell a DNS miss from a long completion. Observability people solved this years ago with spans. Steal the idea in a short JSON file.

A receipt stores phase, timings, redaction hits, dropped frames, and HTTP status. It never stores the raw trace. You write it before the probe, after the probe, and after the complete call, so a crash still leaves a breadcrumb.

import fs from 'node:fs/promises';

export async function writeReceipt(path, span) {
  const row = { name: 'error_digest', ts: Date.now(), ...span };
  await fs.writeFile(path, JSON.stringify(row, null, 2));
  return row;
}

const PROBE_MS = 3000;
const CALL_MS = 15000;

export async function probe(url) {
  const t0 = Date.now();
  const ctrl = new AbortController();
  const timer = setTimeout(() => ctrl.abort(), PROBE_MS);
  try {
    const res = await fetch(url, { method: 'GET', signal: ctrl.signal });
    return { ok: res.ok, status: res.status, ms: Date.now() - t0 };
  } finally {
    clearTimeout(timer);
  }
}

export async function complete(url, payload) {
  const t0 = Date.now();
  const ctrl = new AbortController();
  const timer = setTimeout(() => ctrl.abort(), CALL_MS);
  try {
    const res = await fetch(url, {
      method: 'POST',
      signal: ctrl.signal,
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify(payload)
    });
    const text = await res.text();
    return { ok: res.ok, status: res.status, ms: Date.now() - t0, text };
  } finally {
    clearTimeout(timer);
  }
}

Point both URLs at environment variables. Do not bake a hostname into the module. A closed port should fail the probe in three seconds, not three hours.

export MODEL_URL="http://127.0.0.1:8080/v1/complete"
export MODEL_HEALTH_URL="http://127.0.0.1:8080/health"
curl -sS -o /tmp/health.json -w "%{http_code} %{time_total}\n" "$MODEL_HEALTH_URL"

A GET on a health route does not prove inference. It only proves you should not wait on a dead socket. That is still the difference between a short failure and a wasted afternoon.

Freeze the digest shape

Prose summaries drift. One run returns a haiku. The next returns a lecture. Your ticket template cannot parse either.

Ask for JSON and then distrust it. The sample contract is small on purpose. Import it from tests the same way the CLI does.

export const DIGEST_SPEC = {
  required: ['surface', 'confidence', 'frames', 'notes', 'incomplete'],
  surfaces: new Set(['app', 'db', 'http', 'queue', 'unknown']),
  confidences: new Set(['low', 'medium', 'high']),
  maxNotes: 3,
  maxNote: 160,
  maxFrames: 5
};

export function parseDigest(raw, spec = DIGEST_SPEC) {
  let data;
  try {
    data = JSON.parse(raw);
  } catch {
    throw new Error('digest_not_json');
  }
  for (const key of spec.required) {
    if (!(key in data)) throw new Error(`digest_missing_${key}`);
  }
  if (!spec.surfaces.has(data.surface)) throw new Error('digest_surface');
  if (!spec.confidences.has(data.confidence)) throw new Error('digest_confidence');
  if (typeof data.incomplete !== 'boolean') throw new Error('digest_incomplete_type');
  if (!Array.isArray(data.frames) || data.frames.length > spec.maxFrames) {
    throw new Error('digest_frames');
  }
  if (!Array.isArray(data.notes) || data.notes.length === 0 || data.notes.length > spec.maxNotes) {
    throw new Error('digest_notes');
  }
  for (const note of data.notes) {
    if (typeof note !== 'string' || note.length < 8 || note.length > spec.maxNote) {
      throw new Error('digest_note_len');
    }
  }
  if (data.confidence === 'high' && data.incomplete === true) {
    throw new Error('digest_confident_sketch');
  }
  return data;
}

If incomplete is true, you may still print the digest, but you label it as a sketch. High confidence plus a clipped trace is an argument waiting to happen. Reject that pair in code so Slack does not have to.

A driver that refuses to skip steps

Save this as digest.mjs. Feed it a file path. It should never post the original bytes. Sample driver only; it is not a published latency study.

import fs from 'node:fs/promises';
import { redact } from './redact.mjs';
import { clipTrace } from './clip.mjs';
import { probe, complete } from './net.mjs';
import { parseDigest } from './parse.mjs';
import { writeReceipt } from './receipt.mjs';

const url = process.env.MODEL_URL;
const healthUrl = process.env.MODEL_HEALTH_URL || url;
if (!url) throw new Error('MODEL_URL missing');

const raw = await fs.readFile(process.argv[2] || 'trace.txt', 'utf8');
const scrubbed = redact(raw);
if (scrubbed.text.includes('eyJ')) throw new Error('jwt_still_present');

const clipped = clipTrace(scrubbed.text);
await writeReceipt('receipt.json', {
  phase: 'probe',
  hits: scrubbed.hits,
  clipped
});

const health = await probe(healthUrl);
if (!health.ok) {
  await writeReceipt('receipt.json', { phase: 'health_fail', health, clipped });
  throw new Error('health_fail');
}

const call = await complete(url, {
  instruction: 'Return JSON that matches the digest spec. Do not invent frames.',
  trace: clipped.body,
  droppedFrames: clipped.droppedFrames
});

await writeReceipt('receipt.json', {
  phase: 'called',
  health,
  httpStatus: call.status,
  ms: call.ms,
  clipped
});

if (!call.ok) throw new Error(`complete_http_${call.status}`);
const digest = parseDigest(call.text);
if (clipped.droppedFrames > 0) digest.incomplete = true;

await fs.writeFile('digest.json', JSON.stringify(digest, null, 2));
await writeReceipt('receipt.json', {
  phase: 'done',
  digest,
  clipped,
  hits: scrubbed.hits
});
console.log(JSON.stringify({ digest, receipt: 'receipt.json' }, null, 2));

node digest.mjs ./fixtures/trace-jwt.txt
cat receipt.json
cat digest.json

The driver is the policy. Policy left in a chat window evaporates by Tuesday. If the phase never leaves probe, you do not have a summary. You have a dark host and a timestamp.

Where a free model server belongs

This pipeline only needs a complete URL that returns text. MonkeyCode offers free model access and a free server option, which is enough to exercise the CLI against a remote host you do not have to stand up yourself.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Swap the URL tomorrow and the redactor still matters. Do not assume a quota, a model name, or an uptime promise in the runbook. Read current product docs before you plan any capacity. The CLI remains useful if you point it at a process on localhost instead.

Fixtures argue; demos applaud

Demos always work. Fixtures pick fights. Keep three files next to the parser and run them on every change.

trace-jwt.txt holds a fake JWT and an email. trace-shallow.txt holds an exception with one frame. trace-noisy.txt holds thirty frames and a connection string. Label the checks unexecuted until you run them on your machine.

import test from 'node:test';
import assert from 'node:assert/strict';
import { redact } from './redact.mjs';
import { clipTrace } from './clip.mjs';
import { parseDigest, DIGEST_SPEC } from './parse.mjs';

test('strips jwt and email', () => {
  const raw = [
    'user [e-mail]',
    'token eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.aaa.bbb',
    '    at boom (/app/x.js:3:1)'
  ].join('\n');
  const out = redact(raw);
  assert.equal(out.text.includes('jane@'), false);
  assert.equal(out.text.includes('eyJ'), false);
});

test('drops extra frames', () => {
  const lines = ['Error: boom', ...Array.from({ length: 30 }, (_, i) => `    at f${i} (/app/a.js:${i}:1)`)] ;
  const clipped = clipTrace(lines.join('\n'));
  assert.equal(clipped.keptFrames, 12);
  assert.equal(clipped.droppedFrames, 18);
});

test('rejects a confident sketch', () => {
  const raw = JSON.stringify({
    surface: 'app',
    confidence: 'high',
    frames: ['at boom (/app/x.js:3:1)'],
    notes: ['Looks like a null deref in boom()'],
    incomplete: true
  });
  assert.throws(() => parseDigest(raw, DIGEST_SPEC));
});

node --test redact.test.mjs clip.test.mjs parse.test.mjs

If the JWT test ever fails, you stop shipping the CLI. Rotating staging secrets because someone pasted a trace is more expensive than one angry fixture.

Signals and moves

Signal	Move
Redactor reports zero hits on a fixture that should have secrets	Fail the eval; do not call the model
eyJ still present after redact	Abort the process
droppedFrames > 0	Force incomplete=true
Health probe abort or non-OK	Skip complete; keep the receipt
Body is not JSON	Do not write digest.json
confidence=high and incomplete=true	Reject the object
HTTP 429	Exit; do not retry in a loop
HTTP 5xx	Record status, then exit

Keep that table next to the parser, not in a chat history. The model will not remember it between runs. The driver refuses to hammer a 429. One failure is a receipt, not a loop.

What this will not save you from

This pipeline does not find root cause. It compresses what you already logged. Wrong frames in, confident nonsense out. A schema pass does not prove the note is true.

Regex redaction misses custom secret formats. If your shop uses unusual prefixes, extend the rule list and add a fixture that would have burned you last quarter. A GET health check does not prove the weights loaded. It only proves the port answered.

Do not send production traces that include customer payloads, health data, or credentials, even after this filter. Do not use this as an unattended incident responder. Do not use it when you need a written latency SLA. If your org blocks outbound model calls, run nothing. If the file still looks like a dump of .env or /etc, delete the file and fix the logger.

After the receipt exists

Read receipt.json before you trust digest.json. If phase is still probe or health_fail, the channel does not need a story. It needs a reachable host, or it needs you to read the remaining frames yourself.

If you already have a complete endpoint handy, point this CLI at it and keep the redactor on by default. The interesting part is not the model. The interesting part is that the stack never left your machine in the shape it arrived.