Recursive Issue Bots Are a Trap

Here's what's breaking. You point a script at a repo. It finds "low hanging fruit." It files an issue. That issue says only the author can solve it. Then the script runs again, finds another fruit, files another issue. Forever.

Sound familiar? The SecureBananaLabs issue #743 is basically a spec for a self-replicating ticket machine. And if you build it naively, you'll wake up to 400 open issues and a maintainer who's blocked you.

Let me show you why it goes wrong, then how to actually build it.

Why naive automation fails

The obvious implementation is a cron job:

0 * * * * python find_fruit.py && python file_issue.py

find_fruit.py scans for TODOs, FIXMEs, missing tests, unhandled errors. file_issue.py opens a ticket for each hit.

Three things kill you:

1. Duplicates. The same TODO gets filed every hour. GitHub has no dedupe. You need to check existing issues before creating one, and the check has to be reliable.

2. No state. The script doesn't remember what it already filed. Restart the container, lose the memory, refile everything.

3. Claimed issues. The issue body says "only the issue author can solve this." Fine. But your bot is the author. So now your bot owns 200 issues it will never solve. That's not low hanging fruit. That's a graveyard.

Manual fix that actually works

Before you automate anything, get the dedupe right. This is the part everyone skips.

import hashlib
import json
import os
from github import Github

gh = Github(os.environ["GITHUB_TOKEN"])
repo = gh.get_repo("SecureBananaLabs/bug-bounty")
STATE_FILE = "filed.json"

def load_state():
    if not os.path.exists(STATE_FILE):
        return {}
    with open(STATE_FILE) as f:
        return json.load(f)

def fruit_id(path, line, kind):
    key = f"{path}:{line}:{kind}"
    return hashlib.sha256(key.encode()).hexdigest()[:16]

def already_filed(fid, state):
    if fid in state:
        return True
    # Also check GitHub itself — state file can drift
    for issue in repo.get_issues(state="all", labels=["auto-fruit"]):
        if fid in issue.body:
            return True
    return False

def file_issue(fid, title, body, state):
    issue = repo.create_issue(
        title=title,
        body=f"{body}\n\n<!-- fruit-id: {fid} -->",
        labels=["auto-fruit"],
    )
    state[fid] = issue.number
    with open(STATE_FILE, "w") as f:
        json.dump(state, f)
    return issue

The <!-- fruit-id: --> comment is the trick. It's invisible in rendered markdown but greppable by the API. That's your idempotency key.

Now the scanner:

import re
from pathlib import Path

PATTERNS = [
    (re.compile(r"#\s*TODO:?\s*(.+)"), "todo"),
    (re.compile(r"#\s*FIXME:?\s*(.+)"), "fixme"),
    (re.compile(r"raise NotImplementedError"), "notimpl"),
]

def scan(root="."):
    for path in Path(root).rglob("*.py"):
        if ".venv" in str(path) or "node_modules" in str(path):
            continue
        for i, line in enumerate(path.read_text().splitlines(), 1):
            for pat, kind in PATTERNS:
                m = pat.search(line)
                if m:
                    yield {
                        "path": str(path),
                        "line": i,
                        "kind": kind,
                        "text": m.group(1).strip() if m.groups() else kind,
                    }

Run it once. Look at the output. It'll be garbage. Half the TODOs are stale, a third are in vendored code, and the rest are things the maintainer already knows about.

Filter hard:

def is_worth_filing(fruit, repo_age_days=30):
    # Skip anything older than the repo's active window
    # Skip generated files
    if any(p in fruit["path"] for p in ("migrations/", "generated/", "vendor/")):
        return False
    # Skip trivial
    if len(fruit["text"]) < 10:
        return False
    return True

The recursion problem

The issue says "automate recursively." That's the dangerous part. If your bot files an issue, then scans for work on that issue, it will file issues about issues. Infinite regress. You'll DoS yourself.

Cap it. Explicitly.

MAX_DEPTH = 1
MAX_OPEN_ISSUES = 5

def should_continue(state):
    open_count = len(list(repo.get_issues(state="open", labels=["auto-fruit"])))
    if open_count >= MAX_OPEN_ISSUES:
        return False
    return True

Five open bot-issues max. When the author closes one, the bot can file the next. That's the recursion you actually want: bounded, gated on human action.

Where this gets painful

The state file drifts. Someone edits an issue body, your fruit-id comment gets stripped, you refile. GitHub rate limits you at 5000 requests/hour for authenticated calls, and get_issues(state="all") is expensive on a busy repo. You end up caching, and now you have a cache invalidation problem on top of a dedupe problem.

This is the point where most people give up and just... don't automate it.

The one-line version

This is what TracePilot is for. Instead of hand-rolling state files and dedupe logic, you wrap the agent that decides what to file:

import { TracePilot } from 'tracepilot-sdk';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY);

await tp.startTrace('fruit-finder');

const { result, spanId } = await tp.wrapOpenAI(
  () => openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'system', content: 'You find one actionable bug. Reply with JSON.' },
      { role: 'user', content: scanOutput }
    ]
  }),
  messages
);

Now every scan is a trace. When the bot files a duplicate, you open the dashboard, find the span where it picked that fruit, fork it, change the input, replay. No re-running the whole scan. No redeploying the cron job.

The dedupe bug that took you three hours to find? You see it in the trace in thirty seconds. The model picked the same TODO because your scanner passed it the same file twice. Obvious once you can see the input.

The hook

Here's the thing nobody tells you about recursive issue bots: the hard part isn't the recursion. It's proving why the bot filed what it filed, three weeks later, when the maintainer asks.

Logs won't save you. State files won't save you. Only the full execution trace will.

Build the dedupe first. Cap the recursion. Then instrument the decision, not just the output.

---

Debugging AI agents shouldn't feel like reading The Matrix.
Join other engineers who are building reliable autonomous workflows in our community: TracePilot Discord