DEV Community: Ashish Mishra

Automated First-Pass PR Reviews - Build, Buy, or Use Your Coding Agent’s Cloud?

Ashish Mishra — Fri, 24 Jul 2026 02:52:25 +0000

TL;DR: I wanted every new PR to get an automatic first-turn brief — summary, risks, security/performance smells — before I deep-dive. After comparing SaaS review bots, building a custom agent, and using Cursor’s cloud agents from a GitHub Action, I chose the Action + cloud-agent path because I already pay for Cursor Pro+, I wanted the pipeline in my repo, and I cared more about a trustworthy first pass than a fully productized review platform.

The problem I was actually solving

Not “replace human review.” Not “ship a full AppSec program.”

I wanted a first turn: when a PR opens, something reads the change and leaves a structured brief so I’m not starting cold. Ideal shape:

What is this PR trying to do?
Bugs / logic risks worth noticing early
Security smells (secrets, auth, injection-shaped code)
Performance smells (hot paths, N+1s, wasteful work)
A clear signal: looks fine / needs changes / needs discussion

Later I also want staging previews — but that’s usually a separate pipeline. This piece is about the analysis brief.

I already had: GitHub repos + a Cursor Pro+ subscription. I didn’t have a settled opinion on build vs buy vs “use the agent product I already pay for.”

What “good enough” first-pass analysis can (and can’t) do

Almost every AI review path — bot, DIY agent, or cloud coding agent — is strong at reasoning over the diff and nearby code:

Usually catches	Usually needs extra tools
Logic bugs, edge cases, off-by-ones	Live CVE / dependency vulns (`npm audit`, Snyk, etc.)
Code smells, complexity, inconsistency	Binary / artifact scanning
Security patterns (secrets in code, missing checks)	Formal coverage gates (unless tests are run and parsed)
Performance anti-patterns in visible code	Org-wide policy engines and compliance attestations

So the industry split is less “AI vs not AI” and more:

Deterministic CI — lint, tests, build, secret scan, dependency audit
AI first-pass — narrative brief + judgment calls
Human — merge decision
Preview/staging — clickable environment (often orthogonal)

Treat the AI layer as a sharp intern’s notes, not the only gate.

Industry pattern underneath every option

Whatever logo is on the bot, the system is always:

Listen  →  something wants a review (PR open, /review, Slack)
Think   →  model + tools gather context and reason
Speak   →  comment on the PR (or Slack, or both)

If you build it yourself, you own all three.

If you buy a product, they own Think (and often Speak).

If you use a coding agent’s cloud runtime, you often own Listen/Speak and rent Think.

The main roads people take

1. Buy a PR review product (e.g. CodeRabbit and peers)

What you get: Install a GitHub App, get inline/threaded comments, summaries, sometimes learning from your repo style, dashboards, team defaults.

Strengths

Fastest path to “PRs get reviewed comments”
Productized UX (inline nits, severity, ignore rules)
Less prompt-engineering on day one

Tradeoffs

Another vendor in the trust/security review
Opinionated comment style; tuning is product settings, not “your agent”
Overlap with tools you may already pay for (IDE agents, CI scanners)
Cost scales with seats/repos/PR volume in their pricing, not your existing AI plan

Best when: You want a dedicated review product and don’t care that Think lives outside your coding agent.

2. Build your own Listen → Think → Speak system

Classic DIY: GitHub webhook or Actions → call a model API with tools (fetch PR, get file, maybe gh) → post a comment. Optional Slack: paste a PR link, get a thread reply.

Strengths

Full control of prompt, severity bar, and where results go
Can mix models, scanners, and internal APIs
Clear architecture; easy to explain to security teams

Tradeoffs

You maintain tool loops, auth, retries, prompt drift
Large diffs need careful context management
Slack needs an immediate ACK (3s) and async work
You’re on the hook for reliability (“why didn’t the bot run?”)

Best when: Review is a core workflow you want to own, or you must stay inside strict data boundaries.

3. Use your coding agent’s cloud features (e.g. Cursor cloud agents / Automations)

Newer path: the same class of agent you use in the IDE can run in the cloud against a PR — either via native Automations (PR opened → comment) or via a thin GitHub Action that starts a cloud agent through an API and posts the result.

Strengths

Think quality close to the agent you already trust day to day
Can reuse one subscription’s cloud usage (for me: Pro+)
Automations minimize glue; Actions keep the trigger in .github/workflows

Tradeoffs

Usage is token/spend based — deep reviews can run 10–15+ minutes and burn budget faster than IDE chat
API-started agents may need the Action to Speak (post the comment) even if Think succeeded
Org repos need GitHub App + Actions permissions sorted
Not a full substitute for dependency CVE scanners

Best when: You already live in that agent daily and want the first-pass voice to match.

4. Hybrid (what most mature teams quietly do)

Secret scan + npm audit / SAST in CI (Speak: check failures)
AI brief on PR open (Speak: one summary comment)
Preview deploy URL from hosting/CI (Speak: environment link)
Humans for merge

Best when: You want seatbelts and a narrative first turn.

Comparison at a glance

Path	Time to first useful comment	Ownership	Cost shape	Feels like “my agent”?	First-pass summary quality
SaaS review bot	Hours	Low	Per product pricing	Low–medium	High UX, fixed personality
DIY agent system	Days–weeks	High	Model API + your time	High	As good as your prompt/tools
Cloud coding agent + Automation	Hours	Medium	Included in agent plan usage	Medium–high	High if prompt is good
Cloud coding agent + GitHub Action	Half day–days	High	Plan usage + CI minutes	High	High if prompt is good
Hybrid	Ongoing	Medium	CI + AI + host	Medium	Best overall risk coverage

What I chose — and why

I went with GitHub Action → Cursor cloud agent → PR comment.

Not because SaaS bots are bad. Not because DIY is “more pure.” Because of my constraints:

I already pay for Cursor Pro+. I wanted the first-pass brain to be the same one I trust in the editor, not a second personality I have to learn to ignore.
I wanted a brief, not a product. One structured comment before I dive in — summary, bugs, security, performance, verdict — was enough for v1.
I wanted the trigger in my repo. A workflow file I can read, diff, and tweak beats a dashboard full of vendor toggles (for me). Automations were a strong alternative; I still preferred Option-B-style glue I own.
I accepted the honest limits. Pattern-level security/perf in the brief; real CVE scanning stays in CI later. Staging deploys stay a separate preview pipeline.
Build-from-scratch DIY was attractive in research, but I didn’t need to own the tool loop on day one when a cloud agent already knows how to explore a repo.

What the decision felt like in practice

Listen: pull_request opened / ready for review (skip drafts)
Think: cloud agent with prUrl — no analysis on the Actions runner itself
Speak: Action posts with gh pr comment --repo … (because the runner never checked out git)

The surprising lesson: people ask “how does it analyze if Actions didn’t checkout?” — it doesn’t analyze there. The cloud agent clones elsewhere. The Action is a remote control.

The painful lesson: deep first-pass reviews are slower and hungrier than IDE chat. Budget them. Read your usage dashboard after the first few runs.

The quality lesson: the prompt is the product. A generic “review this” gets generic notes. A forced structure (summary → risks → security → performance → verdict) is what makes the comment useful before you dive in.

How I’d advise someone still deciding

Ask yourself, in order:

Do I want a productized review UX (inline threads, team policies)? → Start with a SaaS bot; add CI scanners beside it.
Do I already pay for a coding agent with cloud/Automations? → Try native Automation or a thin Action before building a custom tool loop.
Do I need Slack-first ChatOps or weird internal tools? → DIY Listen/Speak; rent any Think backend.
Do I need CVE/policy gates? → Those are CI jobs either way; don’t make the LLM the only security control.
Do I need staging? → Preview deploys are usually CI/CD; let the AI brief link to the URL later.

My answer to (2) was yes, so I didn’t buy another review logo — I aimed the agent I already have at GitHub and made the first turn automatic.

Takeaways

A first-pass PR brief is a different job from full human review or full AppSec.
Every approach is Listen → Think → Speak; you’re only choosing who owns each box.
Buy for speed and review UX; build for control; reuse your coding agent’s cloud when the subscription and trust are already there.
Hybrid with deterministic scanners + AI summary + human merge is still the grown-up default.
If you automate cloud agents from Actions, own the comment step, expect multi-minute runs, and treat prompt design as the real feature.

I didn’t want more noise in the PR timeline. I wanted a calm first paragraph from something that already thinks like I do when I’m in the editor — waiting for me before I dive in.

From "You Have a Bug" to "Here's the Root Cause" - Adding AI Code Analysis to My App Review Pipeline

Ashish Mishra — Tue, 21 Jul 2026 01:33:50 +0000

TL;DR: I built an app review pipeline that classifies bugs and crashes. It was useful but stopped at "what's wrong." I extended it to answer "where in the code" and "what to change" — using a PydanticAI agent with codebase exploration tools, then made the analysis engine pluggable so it can use Grok Build, Claude Code, or OpenAI Codex as backends. Here's how I got there, what broke along the way, and how the architecture ended up.

Where I Started

I have a CLI tool called AppPulse that monitors my app's reviews and crash data. Every morning it:

Pulls new reviews from Google Play / App Store
Classifies each one with an LLM — bug, crash, feature request, performance, praise
Correlates reviews with crash data from Sentry/Firebase
Sends me a digest

apppulse run
# → "12 new reviews: 3 bugs (1 critical), 2 feature requests, 7 praise"

apppulse reviews --category bug
#  ID │ Rating │ Summary                              │ Severity
#  42 │  ★☆☆☆☆ │ Photo upload crashes for >10MB images │ critical

This was genuinely useful. I knew what users were complaining about and which crashes were affecting the most people. But the pipeline stopped at classification. When I saw "Photo upload crashes for >10MB images," I still had to:

Open the project in my IDE
Search for upload-related code
Cross-reference the stacktrace from Sentry
Mentally map the review to the actual code
Figure out what to change

For one bug, that's fine. For a batch of 5 critical bugs on a Monday morning, it's a lot of manual work before I've even started fixing anything.

I wanted the pipeline to go further — to take the classified review, point to the specific files and lines causing the issue, and hand me a brief I could act on immediately.

The First Version: A PydanticAI Agent with Codebase Tools

The core idea was simple: give an LLM agent access to my codebase through read-only tools and let it investigate the bug, just like I would.

Why PydanticAI

I considered LangGraph, LangChain, and rolling my own agent loop. PydanticAI won for three reasons:

Structured output by default. I needed the analysis to come back as a validated Pydantic model — not free-form text I'd have to parse. PydanticAI auto-validates the LLM output against the schema and retries with correction instructions if it's wrong.
Minimal dependency footprint. AppPulse is a lightweight CLI. I didn't want to drag in LangChain's dependency tree for a single-agent linear workflow.
It already supports the LLM providers I use — OpenAI, Anthropic, Ollama. No new API keys.

The Output Model

Every analysis produces a validated AnalysisBrief:

class AnalysisBrief(BaseModel):
    issue_type: str              # bug | crash | feature_request
    summary: str                 # one-paragraph assessment
    root_cause: str | None       # for bugs/crashes
    feature_rationale: str | None # for feature requests
    affected_files: list[AffectedFile]
    proposed_changes: list[ProposedChange]
    complexity: str              # small | medium | large
    risks: list[str]             # potential side effects
    testing_notes: str           # what tests to write/update

This is the contract between the analysis engine and the rest of the pipeline. Whether the analysis runs in-process or via an external coding agent, it always produces the same structure.

Four Built-in Tools

The agent gets four tools — all read-only, all operating on the local codebase:

@analysis_agent.tool
async def tool_search_code(ctx: RunContext[AnalysisDeps], keyword: str, file_extensions: str = "") -> str:
    """grep/ripgrep search — finds where relevant code lives."""
    return search_code(ctx.deps.code_path, keyword, file_extensions)

@analysis_agent.tool
async def tool_read_file(ctx: RunContext[AnalysisDeps], file_path: str, start_line: int = 0, end_line: int = 0) -> str:
    """Read source with line range, capped at 500 lines, path-validated."""
    return read_file(ctx.deps.code_path, file_path, start_line, end_line)

@analysis_agent.tool
async def tool_list_files(ctx: RunContext[AnalysisDeps], directory: str = ".", pattern: str = "") -> str:
    """Directory listing with sizes."""
    return list_files(ctx.deps.code_path, directory, pattern)

@analysis_agent.tool
async def tool_find_symbols(ctx: RunContext[AnalysisDeps], name: str) -> str:
    """Regex-based definition search — classes, functions, methods."""
    return find_symbols(ctx.deps.code_path, name)

Every tool validates paths against code_path to prevent directory traversal — no ../../etc/passwd games. The agent can explore but never escape the project root.

The Problem with V1: Too Many API Calls

The first version worked, but the agent was expensive. It would make 40-50 tool calls — listing directories, searching for broad terms, reading entire files — before settling on a hypothesis. At gpt-4o-mini rates that's still cheap, but it hit the API request limit I'd set and sometimes never produced output.

Optimization 1: The Repo Map

The biggest waste was the agent discovering project structure. It would call list_files recursively, trying to understand where things lived before it could even start investigating.

The fix: generate a repo map upfront and include it in the system prompt. The repo map is a compact text overview of every source file with its top-level symbols (classes, functions) extracted via regex:

src/services/
  ImageUploadService.java — ImageUploadService, compress, validateSize, uploadToS3
  UserService.java — UserService, getProfile, updatePreferences
src/utils/
  ImageUtils.java — ImageUtils, downscale, getExifOrientation

This gives the agent a "GPS" before it makes any tool calls. Instead of blindly listing directories, it can read the repo map and go straight to ImageUploadService.java when investigating an upload crash.

Caching the Repo Map

Generating the repo map means walking the entire source tree and reading every file for definitions. For a medium project (~500 files), that's 100-500ms. Fine for a one-off, but I didn't want to pay that cost on every analysis.

The cache strategy uses a git-based fingerprint:

def _compute_fingerprint(root: Path) -> str:
    # Git repos: HEAD commit + dirty-file count (~5ms)
    head = subprocess.run(["git", "rev-parse", "HEAD"], ...)
    status = subprocess.run(["git", "status", "--porcelain"], ...)
    dirty_count = len(status.stdout.strip().splitlines())
    return hashlib.sha256(f"{commit}:{dirty_count}".encode()).hexdigest()[:16]

Cache hit: ~0ms. Cache miss (new commit or uncommitted changes): regenerate. The cache self-invalidates on every git operation that changes the working tree.

Optimization 2: Progressive Tool Gating

Even with the repo map, the agent would sometimes keep exploring forever — finding tangential files, searching for related patterns, going down rabbit holes. Sound familiar? It's what I do when debugging at 2am.

The fix is a funnel that progressively restricts the agent's tools:

async def _prepare_tools(ctx: RunContext[AnalysisDeps], tool_defs: list[ToolDefinition]):
    tool_call_count = count_tool_calls(ctx.messages)

    if tool_call_count >= 18:
        return []           # No tools — force the agent to produce output

    if tool_call_count >= 12:
        # Only read_file — no more searching, refine what you already found
        return [td for td in tool_defs if td.name not in EXPLORATION_TOOLS]

    return tool_defs        # Full access during exploration phase

Phase	Tool Calls	Available Tools
Exploration	0-12	All 4 tools
Deep-dive	12-18	`read_file` only
Forced output	18+	None

This dropped the average analysis from 40+ tool calls to 8-12, with no noticeable quality difference. The agent just gets to the point faster.

The Bigger Step: Pluggable Analysis Backends

The in-process PydanticAI agent worked well for straightforward bugs. But for complex, multi-file issues — or when I wanted deeper analysis — I kept wishing I could point a full coding agent like Grok Build or Claude Code at the codebase and just get back a structured brief.

All of these tools support headless mode:

# Grok Build
grok -p "analyze this bug..." --cwd /path/to/code --tools "read_file,grep,list_dir" --yolo

# Claude Code
claude -p "analyze this bug..." --cwd /path/to/code --dangerously-skip-permissions

# OpenAI Codex
codex --quiet --approval-mode full-auto "analyze this bug..."

The challenge: each agent returns its output differently (JSON, plain text, structured conversations), and I still needed a validated AnalysisBrief at the end.

The Two-Stage Pipeline

The solution is a two-stage pipeline for external backends:

Stage 1: Coding Agent → Raw Markdown
Stage 2: Cheap LLM (structurer) → Validated AnalysisBrief

Stage 1 runs the coding agent with a prompt that asks it to produce a markdown analysis with specific sections (Summary, Root Cause, Affected Files, Proposed Changes, etc.). The agent uses its own tools to explore the codebase — it's much better at this than my four built-in tools because it has full access to grep, file reading, and code understanding.

Stage 2 is a single, cheap gpt-4o-mini call that extracts the markdown into the AnalysisBrief Pydantic model. No tools, no exploration — just structured extraction. Costs fractions of a cent.

The raw markdown from Stage 1 is saved as an inspectable artifact at ~/.apppulse/analyses/raw/. If the structured output looks wrong, I can check exactly what the coding agent found.

The Backend Architecture

I wanted to be able to add new backends without touching the core analysis flow. The architecture uses a lazy-loaded registry with a strategy pattern:

# backends/__init__.py
BACKEND_REGISTRY: dict[str, str] = {
    "builtin":     "apppulse.code_analysis.backends.builtin:BuiltinBackend",
    "grok":        "apppulse.code_analysis.backends.grok:GrokBackend",
    "claude_code": "apppulse.code_analysis.backends.claude_code:ClaudeCodeBackend",
    "codex":       "apppulse.code_analysis.backends.codex:CodexBackend",
}

def get_backend(name: str) -> BaseAnalysisBackend:
    module_path, class_name = BACKEND_REGISTRY[name].rsplit(":", 1)
    module = importlib.import_module(module_path)
    return getattr(module, class_name)()

Backends are imported lazily — if you never use Grok, grok.py is never loaded. Adding a new backend is three steps:

Create a module that subclasses ExternalAgentBackend
Implement _build_command() — return the argv list
Add one line to BACKEND_REGISTRY

Here's the entire Grok backend:

class GrokBackend(ExternalAgentBackend):
    name = "grok"
    display_name = "Grok Build"
    cli_command = "grok"

    def _build_command(self, prompt, cwd, max_turns):
        return [
            "grok", "-p", prompt,
            "--cwd", cwd,
            "--tools", "read_file,grep,list_dir",   # read-only
            "--yolo",
            "--output-format", "json",
            "--max-turns", str(max_turns),
        ]

Everything else — prompt assembly, subprocess management, output parsing, raw artifact saving, and the Stage 2 structurer call — is handled by the ExternalAgentBackend base class. The concrete backend only knows how to build its CLI command.

Switching Backends

In config.yaml:

code_analysis:
  enabled: true
  backend: "grok"        # swap to "builtin", "claude_code", or "codex"
  max_turns: 15           # how many exploration turns for external agents

The setup wizard auto-detects which CLIs are installed on your PATH and lets you pick:

Available analysis backends:
  1. builtin     — Built-in (PydanticAI) — lightweight, no extra tools needed
  2. grok        — Grok Build — headless mode, read-only tool access
  3. claude_code — Claude Code — deep exploration, 200K context
Choose a backend [1]:

What the Output Looks Like

Whether using the builtin or an external backend, the output is the same validated structure:

$ apppulse analyze 42

Analyzing review #42 against MyApp Android codebase...
  Code path: /Users/me/projects/myapp
  Issue type: bug
  Backend:   grok

  > Launching Grok Build agent...
  ... Agent exploring codebase (this may take 30-60s)...
  OK Raw analysis saved to ~/.apppulse/analyses/raw/MyApp_bug_20250710_091523.md
  > Structuring analysis into brief...
  OK Analysis complete

┌──────────────────────────────────────────────────────────────┐
│  CODE ANALYSIS — Review #42                                   │
│  Photo upload crashes for >10MB images                        │
│  Type: bug │ Complexity: medium                               │
├──────────────────────────────────────────────────────────────┤
│  ROOT CAUSE                                                   │
│  ImageUploadService.compress() loads the full bitmap into     │
│  memory before compression. Images >10MB exceed the heap      │
│  limit on devices with ≤4GB RAM → OutOfMemoryError.           │
│                                                               │
│  AFFECTED FILES                                               │
│  ImageUploadService.java (lines 45-89) — compress() method    │
│  ImageUtils.java (lines 12-35) — no input size validation     │
│                                                               │
│  PROPOSED CHANGES                                             │
│  1. Add BitmapFactory.Options.inSampleSize for downsampling   │
│  2. Add file size validation before compress()                │
│  3. Use streaming compression instead of in-memory byte array │
│                                                               │
│  RISKS                                                        │
│  • Downsampling reduces image quality for server-side use     │
│  • Need to test across API levels 26-34                       │
└──────────────────────────────────────────────────────────────┘

With --output file, it saves the analysis as markdown with a ready-to-paste prompt section at the bottom — hand it directly to a coding agent to implement the fix.

Watch Out For

Config nesting. The code_analysis block goes at the top level of your config YAML, not nested under an app entry. I made this mistake myself — the parser reads code_analysis from the root, so nesting it under apps[] means it silently gets ignored and falls back to defaults.
prepare_tools is called every turn. It receives the full message history, so counting tool calls is O(n) over all messages. For 25 turns this is negligible, but if you remove the request limit, it could get slow on very long conversations.
External agent stdout formats vary. Claude Code with --output-format json wraps its response in a JSON object; Codex prints plain text. The ExternalAgentBackend._extract_text() method handles both, but if you add a new backend, check what it actually writes to stdout.
Model name prefix matters. PydanticAI wants openai-chat:gpt-4o-mini (not just gpt-4o-mini). Without the prefix, you get a deprecation warning and potentially the wrong provider.

What I'm Thinking About Next

This pipeline keeps evolving. Some things on the radar:

Per-app backend config. Right now code_analysis is global. If you have multiple apps, you might want Grok for one and builtin for another. The config structure needs to support this.
Batch analysis. apppulse analyze --category bug --severity critical to analyze all critical bugs in one run instead of one-by-one.
Analysis history. Tracking which analyses led to successful fixes — creating a feedback loop that improves future analysis quality.
Auto-analyze in pipeline. Option to run analysis on all critical bugs during apppulse run, so the morning digest includes code-level briefs, not just classification.

Takeaways

Start with classification, then add analysis. Getting reviews categorized (bug vs. feature request vs. praise) is the foundation. Code analysis is the layer on top — it's valuable but depends on good classification first.
Give the agent a map before tools. The repo map cut tool calls by 60%. An LLM exploring a codebase blind will waste most of its budget on orientation. Front-load structure.
Funnel, don't limit. Progressive tool gating works better than a hard request limit. The agent naturally transitions from exploration to deep-dive to output, rather than being cut off mid-investigation.
Make the backend a strategy, not a hard dependency. The two-stage pipeline (explore → structure) decouples the exploration engine from the output format. Today it's Grok and Claude Code; tomorrow it could be any agent that can read files and write markdown.
Save raw artifacts. The raw markdown from external agents is invaluable for debugging bad analyses. If the structured output looks wrong, the raw artifact tells you whether the problem was Stage 1 (bad exploration) or Stage 2 (bad extraction).

AppPulse is a personal project I keep iterating on. The code analysis pipeline is the part I'm most excited about — it's the difference between "you have 3 critical bugs" and "here's exactly where they are and what to change." If you're building something similar, the pluggable backend pattern and two-stage pipeline might save you some architecture headaches.

I built a very cheap App Review analysis pipeline that also finds Bugs in my app code

Ashish Mishra — Mon, 20 Jul 2026 19:18:20 +0000

I got tired of reading app reviews manually every morning, so I built a command-line tool that pulls reviews from Google Play and the App Store, classifies them (bug, feature request, crash, praise), and sends me a daily summary. It costs about $0.50/month. In version 2, I added the ability to point it at my source code and get an analysis: which files are causing the bug, what to change, and how complex the fix is.

If you need to checkout code for your own setup : https://github.com/Mr-Ashish/AppPulse

The Problem

I have a few apps on Google Play. Every morning, I'd open the Play Console, scroll through new reviews, and try to figure out:

Is this a bug or a complaint?
Is this the same issue three other people mentioned yesterday?
Does this match any of the crashes I'm seeing in Firebase?

About 15 minutes of work. But it's unfocused work. Scanning text, making mental classifications, switching between tabs. On days with 20+ reviews, I'd either rush through them or skip the whole exercise.

I wanted a morning briefing:

"You got 12 new reviews. 3 are bugs (1 critical, photo upload crashes on large images). 2 are feature requests. 7 are praise. The critical bug matches a Firebase crash that affected 312 users this week."

So I built one.

Version 1: Classify and Digest

The Flow

Google Play ──┐
               → Pull new reviews→Classify review → Save to database
App Store ───┘         │             (using AI)              
                         │                                           
Firebase/Sentry ──────
---------------------------------------------------------------    
     ↓

Daily Digest                                                                             ┌──────────────────┐
│ 3 bugs (1 critical)
│ 2 feature requests
│ 7 praise
│ Top crash: OOM in
│  ImageUploadService
└──────────────────┘
      ↓         ↓
     Terminal   Slack

Classification

Each review goes through an LLM (a language model; I use OpenAI's gpt-4o-mini, their cheapest option). The LLM reads the review text and categorizes it:

Category	Meaning	Example review
bug	Something broken	"Upload button does nothing after I pick a photo"
crash	App closes or freezes	"App crashes every time I open settings"
feature_request	User wants something new	"Please add dark mode"
performance	Slow, laggy, battery drain	"App drains 20% battery in an hour"
praise	Positive feedback	"Best app I've used for this!"
complaint	General dissatisfaction	"Used to be good, now it's terrible"

Along with the category, the LLM extracts:

Severity (critical, major, or minor)
Keywords (the specific feature or area mentioned, e.g., "photo upload", "settings screen")
Functional area (which part of the app is affected)

Reviews go to the LLM in batches of 10, so one API call classifies 10 reviews at once. That's what keeps the cost low.

Cost

For an app with about 500 reviews per week:

Component	Monthly Cost
Google Play API	Free
App Store Connect API	Free
Firebase/Sentry API	Free (free tier)
OpenAI classification (~2,000 reviews/month)	~$0.50
Storage (SQLite, local file)	Free
Total	~$0.50/month

If you use Ollama (runs AI models on your own computer), the cost drops to $0.00/month.

The Tech Stack

Python , the whole tool is a Python command-line app
Click , a library for building command-line interfaces (handles apppulse run, apppulse reviews, etc.)
SQLAlchemy + SQLite , stores everything in a single local database file at ~/.apppulse/data.db
Rich , makes the terminal output clean with tables, colors, and panels
OpenAI / Anthropic / Ollama , LLM providers for classification (you pick one during setup)

No servers. No cloud infrastructure. No Docker. Run pip install and a 5-minute setup wizard.

A Typical Morning

$ apppulse run
# Pulling reviews for MyApp Android... 12 new reviews
# Classifying... done (0.8s)
# Pulling crash data from Firebase... 3 active crashes
# Correlating reviews with crashes... 1 match found

# ── Daily Digest ───────────────────────────────────────
#  12 new reviews: 3 bugs (1 critical), 2 feature requests, 7 praise
#
#  🔴 Critical: Photo upload crashes for >10MB images
#     → Matches Firebase crash: OutOfMemoryError (312 users affected)
#     → 3 reviews mention this issue
#
#  🟡 Major: Search doesn't update after applying filters
#     → 2 reviews
#
#  💡 Feature requests: Dark mode (2 requests)

Five seconds. I know what matters. No tab-switching, no scrolling through the Play Console.

Version 2: From Classification to Code Analysis

Version 1 told me what users were experiencing. It didn't tell me where in my code the problem lived or what to change.

For the critical bug above, "Photo upload crashes for >10MB images," I'd still need to:

Open the project
Search for upload-related code
Look at the Firebase stacktrace
Connect the dots
Figure out a fix

So I built a second stage: code analysis.

The Mechanism

                    apppulse analyze 42
                         │
                         ▼
        ┌────────────────────────────────┐
        │    Load the review from the    │
        │    database + crash data       │
        └───────────────┬────────────────┘
                        │
                        ▼
        ┌────────────────────────────────┐
        │    AI agent explores your      │
        │    source code with read-only  │
        │    tools (search, read files,  │
        │    find function definitions)  │
        └───────────────┬────────────────┘
                        │
                        ▼
        ┌────────────────────────────────┐
        │    Produces an Analysis Brief  │
        │    • Root cause                │
        │    • Affected files + lines    │
        │    • Proposed changes          │
        │    • Complexity estimate       │
        │    • Testing notes             │
        └────────────────────────────────┘

You point AppPulse at your app's source code folder during setup. Running apppulse analyze 42 (where 42 is a review ID) triggers the AI agent to:

Read the review and any matched crash data
Get a structural overview of your project (a map of every file and its key classes/functions)
Search the code for relevant keywords, error messages, class names, the feature area mentioned in the review
Read the specific files that look promising
Produce a structured analysis with file paths, line numbers, and proposed changes

The Output

┌──────────────────────────────────────────────────────────────┐
│  CODE ANALYSIS — Review #42                                   │
│  Photo upload crashes for >10MB images                        │
│  Type: bug │ Complexity: medium                               │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  ROOT CAUSE                                                   │
│  ImageUploadService.compress() loads the full image into      │
│  memory. Images >10MB exceed the memory limit on phones       │
│  with ≤4GB RAM, causing the app to crash.                     │
│                                                               │
│  AFFECTED FILES                                               │
│  ImageUploadService.java (lines 45-89) — compress() method    │
│  ImageUtils.java (lines 12-35) — no file size check           │
│                                                               │
│  PROPOSED CHANGES                                             │
│  1. Resize large images before loading them into memory       │
│  2. Add a file size check before compression starts           │
│  3. Use streaming compression instead of loading everything   │
│     into memory at once                                       │
│                                                               │
│  TESTING NOTES                                                │
│  Test with 1MB, 10MB, and 50MB images on a low-memory device  │
└──────────────────────────────────────────────────────────────┘

Instead of a category label, I get the file, the line, what to change, and how to test it.

Pick Your Analysis Engine

The analysis engine is pluggable. I built it with four options:

Backend	Description	Good for
Built-in	A lightweight AI agent that runs inside AppPulse, using PydanticAI (a Python framework for building AI agents with structured output)	Quick analyses, small codebases
Grok Build	xAI's coding assistant in headless mode	Fast, targeted exploration
Claude Code	Anthropic's coding assistant in headless mode	Deep multi-file investigations
OpenAI Codex	OpenAI's coding assistant	Sandboxed, autonomous mode

The external backends (Grok, Claude Code, Codex) use a two-stage approach:

The coding assistant explores the codebase and writes its findings as a markdown document
A cheap AI call ($0.001) extracts that document into the structured analysis format

Consistent output regardless of which AI did the exploration. Adding a new backend takes about 20 lines of code.

Switching is one line in the config file:

code_analysis:
  enabled: true
  backend: "grok"    # or "builtin", "claude_code", "codex"

Lessons

1. Classify before you analyze

Categorization (bug vs. feature request vs. praise) determines everything downstream. Code analysis needs good classification as input, or it wastes time investigating a feature request as if it were a bug.

2. Cheap AI models handle most of this

The classification pipeline runs on gpt-4o-mini, OpenAI's cheapest model. For categorizing app reviews, which are 1-3 sentences, it's accurate enough. Expensive models are overkill.

3. Give the AI a map of your codebase

The biggest analysis quality improvement came from generating a project map upfront, a compact overview of every file and its key components. Without the map, the AI spent most of its budget figuring out where things lived. With it, the AI goes to the relevant files on its first move. A GPS for the codebase.

4. Save raw output

Every analysis saves the AI's raw output as a markdown file. If the structured summary looks off, I can check what the AI found and pinpoint whether the problem was in the exploration or the extraction.

5. No infrastructure required

The whole tool runs as a single command-line app. The database is a SQLite file on disk. The cache is a text file. No server, no Docker container, no cloud deployment. I run apppulse run by hand or on a daily schedule.

Up Next

Batch analysis , analyze all critical bugs at once instead of one at a time
Per-app configuration , different analysis settings for different apps
Auto-analysis , run code analysis on critical bugs during the daily pipeline, so the morning digest includes root causes alongside classifications
Tracking outcomes , connect analyses to fixes, so the tool can learn which investigations led to successful resolutions

The classification half took a weekend. Code analysis took another. The pipeline costs less per month than a cup of coffee, and it replaced 15 minutes of unfocused tab-switching with a 5-second terminal readout.

Your Team Can Build Dashboards — But Can You Share Them Safely?

Ashish Mishra — Mon, 20 Jul 2026 19:17:14 +0000

TL;DR: Non-tech teammates can already build useful internal dashboards — often as a single HTML file with AI help. The real problems are sharing, secrets, and control. We solved that with a small publish app, two repositories, and a clear rule: code goes through review; passwords never go into Git.

The Problem

More teams can create internal tools than ever before. Someone in support, ops, or finance asks an AI assistant for “a dashboard that shows X,” downloads one HTML file, and it works on their laptop.

Then reality hits:

How do we share it? Emailing files and “open this on your machine” does not scale.
Where do API keys go? Those keys unlock company systems. Pasting them into a file that gets copied around is a security incident waiting to happen.
Who checked it? One edit can break the dashboard — or worse, leak data. Someone should review changes before the whole company uses them.
Who can open the link? It should be company people only, not anyone on the internet who finds the URL.

So the issue is not “can we build dashboards?” It’s can we publish them like a product: safely, repeatedly, without forcing every creator to learn Git.

What We Were Aiming For

Think of it as internal websites for ops dashboards, not a full developer platform.

Goal	In plain terms
Easy publish	Drag a file into a simple UI — no terminal
Safe secrets	Keys live in a locked vault (database), not in the file people share
Review before live	Changes go through a pull request (a proposed change someone can approve)
Company-only access	Sign in with work Google account
Room to grow	Same design can later support bots, coding agents, and stricter permissions

The Overall Idea

  Creator + AI                Publish app                 Two stores
 ─────────────               ────────────                ──────────
  One HTML file  ──upload──►  Company login               ┌─ Git: redacted HTML (via PR)
                              Save settings               └─ Database: config + secrets
                              Strip secrets
                                      │
                                      ▼
                              After review + merge
                                      │
                                      ▼
                              Shareable company link
                                      │
                                      ▼
                              Server calls APIs with secrets
                              (keys never shown in the page)

In one sentence: people upload a dashboard; the app saves safe settings in a database, sends a cleaned-up copy of the HTML for review in Git, and only after approval serves a company login–protected link — while secrets stay on the server.

Why Two Repositories? (The Original Thinking)

This was the biggest structural choice. Early on it was tempting to keep everything in one place: the publish website and all the dashboard HTML together.

Here’s why we didn’t.

What each repo is for

Repository	What’s inside	Who cares
Platform repo	The publish website, login, “serve this dashboard” logic, secret proxy	Engineers who ship the product
Dashboards repo	Only the HTML (and a tiny list of setting names, not values)	Reviewers, automated checks, and later AI agents that edit dashboards

Why split them

1. Different jobs, different risk

Platform code is “how publishing works.” Dashboard HTML is “what the business sees.” Mixing them means a dashboard tweak and a security fix compete in the same history. Splitting them makes reviews clearer: Is this a product change or a dashboard change?

2. Secrets and HTML must not travel together

If HTML and the app live in one pile, it’s too easy for keys to end up next to the files people download and copy. A dedicated dashboards repo has a hard rule: only cleaned-up HTML. Real keys never belong there.

3. Review that non-engineers can still understand

A pull request on a dashboards-only repo looks like: “here’s the new support board page.” Reviewers aren’t wading through unrelated website code. That matches how the team actually works — ops owns the dashboard content; engineering owns the platform.

4. Safer automation later

We knew we might want Slack bots or coding agents to open dashboard updates. Giving an automation access to only the dashboards repo is much safer than giving it keys to the whole platform.

5. Cleaner automated checks

Checks on dashboards can be simple: “is there one HTML file?”, “does this look like a leftover API key?”, “is the file too big?” Those checks don’t need to understand the full website.

Tradeoff we accepted

Two repos	One repo
Slightly more setup	Simpler at first
Clear ownership and safer automation	Faster to start, messier later
Platform releases don’t churn when dashboards change	Every dashboard edit looks like a product change

Original thinking in short: treat dashboards like content and the publish app like product. Content gets a content repo. Product gets a product repo. Secrets get neither — they get the database.

How Publishing Works (For Humans)

Sign in with your work Google account.
Upload one HTML file (title, optional team label).
The app reads any preview settings used for local testing, saves them for that dashboard, and strips secrets out of the copy that goes to Git.
It opens a pull request — a proposed change — on the dashboards repo.
Someone reviews and merges.
The owner marks it live.
Teammates open a share link, sign in, and use the dashboard.

Updates follow the same path: new upload → update the open pull request (or open a new one) → merge → live.

Security — What Can Go Wrong, and How This Helps

Risk	What it looks like in real life	How the design responds
Keys in the file	HTML emailed around with an API key inside	Keys saved encrypted in the database; Git only gets placeholders
Keys in the browser	Anyone can “View Source” and steal a key	Page only gets non-secret config; the server calls external APIs on your behalf
Open link on the internet	Company data on a URL with no login	Company Google sign-in before viewing
Silent go-live	Broken or unsafe change hits everyone at once	Pull request + merge before live
One mega-repo blast radius	A bot or leak touches the platform and every dashboard	Two repos; automation can be limited to dashboards only

The simple rule we tell creators:

Config (folder IDs, public URLs) can appear on the page. Secrets (API keys, tokens) never do — the server holds them.

The same HTML can still work on a laptop for preview: it uses a local preview block when it’s not on the platform, and platform-provided config when it is. Creators aren’t maintaining two different apps.

Architecture at a Glance

                    ┌─────────────────────────────────────┐
                    │           Platform app              │
   Creator ───────► │  Upload · Settings · Google login   │
                    │  Serve link · Secret proxy          │
                    └───────────┬─────────────┬───────────┘
                                │             │
                    save config │             │ redacted HTML
                    + secrets   │             │ as a pull request
                                ▼             ▼
                         ┌──────────┐   ┌──────────────┐
                         │ Database │   │ Dashboards   │
                         │ (locked) │   │ Git repo     │
                         └────┬─────┘   └──────┬───────┘
                              │                │
                              │   after merge  │
                              └───────┬────────┘
                                      ▼
                               Teammate opens
                            share link (+ login)

Three ideas to remember:

Git holds the dashboard pages (after review).
The database holds settings and secrets (encrypted).
The platform app is the only door: login, upload, serve, and proxy.

Tradeoffs (Honest Ones)

Choice	Upside	Downside
Pull request before live	Safer, reviewable history	Not “instant publish”
Secrets in the database + server proxy	Keys stay off pages and out of Git	Slightly more setup for API connections
Two repositories	Clear ownership, safer bots later	Two places to understand
Company sign-in only	Fits internal tools	Not for public / customer-facing sites
Single HTML dashboards first	Matches how people actually create with AI today	Multi-file apps come later

We deliberately deferred finer team permissions, Slack bots, and heavy hosting complexity. The skeleton — two repos, secrets out of Git, company login, review before live — is what has to be right first.

Infrastructure (Small Picture)

You don’t need a huge cloud setup for this.

Need	Typical choice
Run the publish website	A small hosted web service (for example on Google Cloud or AWS)
Database for settings and secrets	Managed Postgres
Keep the platform’s own production secrets safe	The cloud provider’s secret store

Traffic is light for an internal pilot — tens of dashboards, a handful of people using any one at a time. Cost is usually dominated by the database staying on, not by the website. Google Cloud and AWS both work; pick wherever your company already lives.

Takeaways

The bottleneck isn’t building dashboards — it’s sharing them without leaking keys or skipping review.
Treat dashboard HTML as content and the publish website as product: that’s why two repositories beat one pile of everything.
Security for humans: keys in the vault, pages for viewers, review before live, company login on the link.
Start simple on infrastructure; invest first in the rules of the road — where HTML lives, where secrets live, who can open the URL.

If your non-tech teams are already creating internal dashboards, you don’t need to stop them — you need a front door that makes the safe path the easy path.