With four user-facing apps, I centralized all bug reports into one place. Only after aggregating them did I realize that the severity ratings were based on different criteria across the four apps. By standardizing the criteria and laying them out on a single screen, I was able to hand them over to multiple Claude instances and have bugs already reported get squashed in a single day. This article explains how operational overhead concentrates here when you run multiple commercial apps.
I currently have four apps with users: LiveTR, which provides real-time voice translation; OLTranslator, which translates screens into Japanese on the fly; Kikoeru, an English listening tool; and AuctionBOT, a Discord bot running on my own server that manages guild auctions. The more apps you have, the more you worry about "is everything working correctly" rather than developing new features—because real users are using them.
Overview: BugHub polls each app's admin read API every ~3 minutes, deduplicates by signature, and displays them on a dashboard and /ai endpoint under the same criteria
Apps I can observe vs. apps I cannot
Of the four apps, I can only track the behavior of one myself. AuctionBOT is a Discord bot running on my own server, so I can investigate what's happening to some extent. The other three (OLTranslator, LiveTR, and Kikoeru) run on users' devices and environments. I don't have the same environment at hand, and some bugs cannot be reproduced in a single environment.
So I added that familiar prompt to all apps: "Send information to improve quality." When I was a user, I'd just click OK without thinking. Now on the developer side, I see how valuable it is. Most bugs aren't something you find yourself; they come up from users' environments. For what I can't observe on my own, I rely on user reports.
Reports scattered across different places per app
I added a reporting mechanism to all apps, but then reports piled up in different locations for each app. When I'm running four apps by myself plus AI, I couldn't see how many issues each app had side by side. There was no place to check which app was having problems before fixing them.
So I built an aggregation system. It's a read-only internal container running on my home server, named BugHub. It does one simple thing: it polls each app's admin read API exposed within the LAN every ~3 minutes and lays everything out on a single dashboard. All the app needs to do is expose one "LAN-internal, Bearer-authenticated read API." Bugs are returned grouped by fingerprint. The fingerprint is a hash of "which module, which type, and what message template"—it doesn't include variable values, so bugs with the same root cause get the same hash. Each bug comes with severity, a message template (variable values and personal information masked), cumulative occurrence count, last occurrence time, and status (unresolved or resolved).
On BugHub's side, if the data matches this format (called the signature format), ingestion is just one line.
// Apps that output this format just need to add one line to the source list
const SOURCES = [
{ name: "AuctionBOT", api: AUCTION_API },
{ name: "OLTranslator", api: OLTR_API },
{ name: "LiveTR", api: LIVETR_API },
// Only Kikoeru had a different log format, so I inserted a small adapter
{ name: "Kikoeru", api: KIKOERU_API, adapt: kikoeruAdapter },
];
LiveTR, OLTranslator, and AuctionBOT were aligned to the same format from the start. Only Kikoeru had a different logging style, so I inserted one small conversion adapter. That put all four side by side.
Different criteria became visible side by side
What I noticed after laying them out was that the severity of each report was based on different criteria per app. What one app calls "critical" might be equivalent to "warning" in another. Only when they are arranged under the same criteria does a list appear where you can tackle them from the top.
So I unified the severity labels into four levels. The app assigns them, and BugHub uses them as-is without re-evaluation.
| Label | Meaning |
|---|---|
| fatal | The app stops working or doesn't function. Most severe. |
| high | The app runs but causes actual harm to users (malfunctions, data loss, etc.). One step below fatal. |
| warn | Not ideal but still working. Should be fixed. |
| info | Needs fixing but no solution available now, or reference info just for recording. |
The definition is just four lines, but the implementation affects each app's reporting mechanism itself. I had to review which events were assigned to which level across all four apps. As a result, each app underwent a revision that both fixed bugs and rebuilt the reporting mechanism.
How to pass common specifications to AI
Once the labels were unified, the next question was how to pass these common specifications to the party doing the fixing. The one fixing isn't me; it's AI. Different VS Code instances and different Claude instances are set up for each project, so explaining the specifications verbally each time would defeat the purpose of unifying.
So I made this into a single page. Calling GET /ai returns a single Markdown document written for AI consumption. Its pillars are as follows:
- The four-level severity criteria (the ones above)
- Fix procedure: ① Get unresolved items → ② Fix the relevant app's project → ③ Deploy to production and confirm it doesn't recur → ④ If confirmed, record the resolution on the app side. Don't record the resolution first. Don't touch anything you're not confident about.
- A list of unresolved items at that point (with a resolution command for each item)
In addition, the same page includes a table of "where to record the resolution" for each app and common specifications for "how to integrate a new app."
I wrote before about the mechanism for letting AI fix and deploy. What I added this time is the part that aggregates reports from multiple apps under the same criteria and makes them available via a single URL. If I tell AI to "make this app aggregatable to BugHub," the AI implements the app's API according to the common specifications on this page. Integrating a new app also requires only passing the same single URL.
Resolutions are recorded on the app side, not in BugHub. The app side is the source of truth; BugHub just copies and displays that state. Even if you delete something in BugHub, it will be overwritten by the app's state on the next poll, and if the app detects a recurrence, it will automatically revert to "unresolved."
Same /ai URL passed to multiple VS Code/Claude instances, each fetching their assigned app, fixing, deploying, verifying, and recording the resolution. Resolutions are recorded on the app side.
Handing it over to multiple Claude instances and getting it done in a day
Once everything was in place, I started clearing out the reported bugs. I set up Claude in multiple VS Code instances, passed the /ai URL to all of them, and instructed: "Look at this and fix it." Since they all see the same list under the same criteria, each Claude picks up unresolved items, fixes its assigned app, deploys, verifies, and records the resolution. I ran this in parallel, and it was largely cleared up in that one day.
There is no LLM inside the polling container. The ~3-minute polls and the instant alerts for surges or new issues all run on pure threshold rules. AI is involved only on the outside—on the bug-fixing side and the weekly Codex summary.
Where the most effort goes now
When you personally run multiple commercial apps, the operational overhead concentrates here. The time spent handling incoming reports grows larger than the time for adding new features. I wrote before about the speed of building, and as a follow-up, the same kind of organization is needed on the maintenance side. That's where things stand now.
Now, unresolved items for all four apps are lined up on a single screen under the same criteria, and by passing a single URL, AI can go and fix them from there. That prompt "Send information to improve quality" that I absentmindedly clicked OK to as a user—this is how it gets used on the developer side.
Top comments (0)