DEV Community: Hopkins Jesse

I Tested 5 AI Coding Agents — Only 2 Are Worth Your Time

Hopkins Jesse — Mon, 25 May 2026 06:02:05 +0000

It is March 2026. The hype around "AI pair programmers" has cooled significantly. We are past the point of being impressed by autocomplete. Now, we care about agency. Can the tool plan, execute, and debug a complex feature without me holding its hand every three seconds?

I spent two weeks testing five popular AI coding agents on a real project. The goal was simple. Refactor a legacy Python monolith into microservices. This is messy work. It requires understanding context, moving files, updating imports, and writing tests.

Most tools failed miserably. They got stuck in loops or hallucinated libraries that don't exist. But two of them actually saved me time. Here is the raw data on what worked, what broke, and why you should care.

The Setup and Criteria

I did not use toy apps. I used a production-grade inventory management system written in Python 3.11 and FastAPI. It had 12,000 lines of code, zero type hints, and a test coverage of 40%.

My task for each agent was identical. Extract the "Notification Service" into its own standalone module. This involved:

Identifying all dependencies.
Creating a new directory structure.
Moving relevant files.
Updating import paths across 15 different files.
Writing pytest units for the new module.

I measured three metrics. Time to first working draft. Number of manual fixes required. And total cost in API tokens. I ran each test three times to average out the randomness of LLM outputs.

Here is the breakdown of the contenders. I am not naming the bottom three to avoid giving them free marketing. They are currently too unstable for professional use. I will focus on the two that passed the bar. Let's call them Agent A (the market leader) and Agent B (the open-source challenger).

Agent A: The Polished Corporate Choice

Agent A is the most expensive option on this list. It costs $40/month for the pro tier. The interface is slick. It integrates directly into VS Code and feels native.

The first run took 14 minutes. That sounds slow. But remember, it was refactoring 15 files. What impressed me was the planning phase. Before writing any code, Agent A outputted a step-by-step plan. It asked me to confirm the directory structure. This small interaction prevented a major error where it initially tried to merge two conflicting config files.

When it generated the code, 90% of it was correct. The import paths were accurate. It even added type hints to the functions it moved, which was not part of my prompt but a nice touch.

However, it struggled with the tests. It wrote unit tests that mocked the database incorrectly. I had to manually rewrite the fixture setup. This took me about 20 minutes. So while the code generation was fast, the verification loop was longer than expected.

The cost was high. It burned through $4.50 worth of tokens in a single session. For a one-off task, that is fine. For daily use, it adds up.

Agent B: The Rough but Effective Open Source Tool

Agent B is local-first. You run it via CLI or a lightweight web UI. It uses a mix of open-weight models and your local compute. Setup took me an hour. I had to configure the Ollama backend and install the specific adapters for my tech stack.

The first impression was jarring. The terminal output was verbose. It printed every thought process. But once I filtered the noise, the logic was sound.

It completed the task in 22 minutes. Slower than Agent A. But here is the kicker. It got the tests right on the first try. It analyzed the existing conftest.py file and mimicked the pattern exactly. I did not have to touch the test code at all.

The code quality was slightly lower. It missed adding type hints to three helper functions. I fixed those in under two minutes. But the core logic was solid.

The cost? Zero dollars in API fees. I used my local GPU. The electricity cost was negligible. If you have a decent machine, this is the most economical option by far.

Head-to-Head Data Comparison

Numbers tell the real story. Marketing pages lie. Benchmarks are often cherry-picked. Here is the average performance across my three runs for the specific refactoring task.

Metric	Agent A (Pro)	Agent B (Local)	Manual Effort
Time to Draft	14 min	22 min	45 min
Manual Fixes	3 files	1 file	N/A
Test Accuracy	60%	95%	100%
Token Cost	$4.50	$0.00	$0.00
Setup Time	2 min	60 min	0 min

Agent A wins on speed and ease of setup. If you need a quick prototype or have a simple task, it is better. Agent B wins on accuracy and cost. If you are doing heavy lifting or working on a budget, it is the superior choice.

Notice the "Manual Fixes" column. Agent A required me to edit three files. Agent B only needed one. In developer terms, context switching is expensive. Every time I had to stop and fix the AI's mistake, I lost flow. Agent B

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

How I Make $6,800/Month Selling Niche VS Code Extensions

Hopkins Jesse — Mon, 25 May 2026 06:01:53 +0000

I used to think building a SaaS was the only way to make real money as a developer.

I spent six months in 2024 building a project management tool. It had auth, payments, and a dashboard. It made $42 total. I burned out trying to market it.

Then I shifted my focus. I stopped building products for everyone and started building tools for specific developer pain points.

Specifically, I built lightweight VS Code extensions that solve one annoying problem really well.

It is now March 2026. I have three extensions live. They generate an average of $6,800 a month.

This is not passive income. It requires maintenance. But the overhead is tiny compared to running a full web app.

Here is exactly how I did it, what failed, and the numbers behind it.

The Pivot From SaaS To Micro-Tools

In late 2025, I noticed a trend. Developers were tired of context switching.

We spend all day in our IDEs. Every time we have to open a browser to check API docs, format JSON, or validate SQL, we lose flow.

Big companies were trying to shove massive AI copilots into our editors. These tools are great, but they are expensive and heavy.

I saw an opening for "dumb" tools. Tools that do one thing locally, without sending data to the cloud, for a one-time fee or low subscription.

I decided to build extensions that leverage local LLMs (like Ollama) or simple regex logic to fix specific workflow bottlenecks.

My first attempt was a failure. I built a generic "Code Formatter." It competed with Prettier. Nobody paid for it.

My second attempt was a niche SQL validator for Supabase users. It charged $5/month. It made $120 in its first month. That was the signal.

The Current Stack And Revenue Breakdown

I currently maintain three extensions. They are built with TypeScript and the VS Code API.

I use GitHub Actions for CI/CD and Stripe for payments. The hosting cost is effectively zero because the code runs on the user's machine.

Here is the revenue breakdown for February 2026:

Extension Name	Problem Solved	Pricing Model	Active Users	MRR
SupaQuery	Validates Supabase SQL syntax locally	$5/mo	920	$4,600
JsonTailor	Formats/transforms JSON via local LLM	$3/mo	650	$1,950
RegexExplainer	Explains complex regex in plain English	One-time $9	140 sales	$260 (avg)
Total				$6,810

Note: The "One-time" revenue fluctuates. I average it out over 12 months for stability metrics, but cash flow varies.

SupaQuery is my breadwinner. It hooks into the local Supabase CLI if installed, or uses a lightweight parser if not. It highlights syntax errors before you even run the query.

JsonTailor uses a small, quantized local model to rename keys or flatten structures. Privacy-focused developers love this because no data leaves their machine.

What Actually Works In 2026

The market has changed since 2024. Developers are skeptical of AI hype.

They do not want another chatbot. They want utilities.

Solve A Boring Problem

Do not try to build "AI Code Generation." That is a solved problem by giants.

Look for friction. What do you copy-paste repeatedly? What error message do you Google every week?

For SupaQuery, I was tired of deploying broken SQL migrations. The error messages from the database were vague. My extension parses the SQL locally and gives a human-readable hint.

Keep It Local

Privacy is the biggest selling point in 2026.

If your extension sends code to an external API, you will struggle to get enterprise adoption.

I explicitly state in my README: "No data leaves your machine." For JsonTailor, I bundle a 200MB quantized model with the extension. It increases the download size, but users trust it more.

Pricing Strategy

I moved away from free tiers.

Free users support the ecosystem, but they do not pay the bills. I offer a 7-day trial, then a hard paywall.

Conversion rates are low, around 2-3%. But the churn is also low, under 4% monthly.

People keep these tools because they save minutes every day. Over a year, that adds up.

The Technical Grind

Building a VS Code extension is not hard. Maintaining it is.

VS Code updates monthly. Sometimes they deprecate APIs. You need to stay on top of changes.

Here is a snippet of how I handle the local model loading for JsonTailor. This was tricky to get right across Windows, Mac, and Linux.


typescript
import * as vscode from 'vscode';
import { spawn } from 'child_process';
import path from 'path';

export async function transformJson(input: string, schema: string): Promise<string> {
  const extensionPath = vscode.extensions.getExtension('my.json-tailor')?.extensionPath;

  if (!extensionPath) {
    throw new Error('
---

💡 **Further Reading**: I experiment with AI automation and open-source tools. Find more guides at [Pi Stack](https://www.pistack.xyz).

I Built a Local AI Debugger in 48 Hours — Here's Why Nobody's Talking About It

Hopkins Jesse — Sun, 24 May 2026 06:01:39 +0000

It is March 2026. We have moved past the hype cycle of "AI will write all our code."

Most of us are now dealing with the hangover.

I spent last weekend building a local, offline debugger that uses small language models (SLMs) to trace execution paths.

It took me exactly 48 hours.

The tool works. It catches race conditions my linter missed. It explains stack traces in plain English without sending my proprietary code to a cloud API.

Yet, when I posted it on Hacker News and Reddit, I got twelve upvotes and three comments asking if it supported Python 3.14.

Nobody is talking about local, deterministic AI debugging.

They are still arguing about which 70B parameter model writes better marketing copy.

Here is why this gap exists, and why you should care about building your own local tooling instead of waiting for the next big SaaS launch.

The Cloud Latency Problem Is Real

Let’s look at the numbers.

In late 2025, the average round-trip time for a standard AI API call was around 800ms.

That doesn’t sound like much until you are trying to debug a hot path in a high-frequency trading simulation or a real-time game loop.

I was working on a Rust-based physics engine.

Every time I hit a segmentation fault, I wanted context.

I tried using the popular cloud-based AI assistant plugins.

The delay was unbearable.

I would paste the error, wait for the token stream, read the suggestion, apply it, crash again, and repeat.

Each cycle took about 45 seconds.

Over a four-hour debugging session, I wasted nearly an hour just waiting for responses.

That is 25% of my productivity gone to network latency and queue times.

I decided to stop paying for convenience that wasn’t convenient.

I grabbed a quantized Llama-3-8B model and ran it locally on my M3 Max MacBook.

The inference time dropped to 120ms per token.

More importantly, the privacy aspect became immediate.

No code leaves my machine.

For developers working in fintech, healthcare, or defense, this isn’t a feature.

It is a compliance requirement.

Yet, most tutorials still focus on connecting VS Code to OpenAI or Anthropic.

Building the Minimal Viable Debugger

I didn’t build a full IDE.

I built a CLI tool that hooks into stderr and stdout.

It listens for specific error patterns.

When it detects a panic or an unhandled exception, it grabs the last 50 lines of logs and the current stack trace.

It sends this context to the local SLM via Ollama.

The prompt is strict.

I do not want creative writing.

I want a root cause analysis.

Here is the core logic in Python:

import subprocess
import ollama

def analyze_crash(log_snippet: str, stack_trace: str) -> str:
    prompt = f"""
    You are a senior systems engineer.
    Analyze the following crash log and stack trace.
    Identify the exact line causing the failure.
    Suggest one specific fix.
    Do not explain basic concepts.

    LOGS:
    {log_snippet}

    STACK:
    {stack_trace}
    """

    response = ollama.chat(model='llama3.1:8b', messages=[
        {'role': 'user', 'content': prompt}
    ])

    return response['message']['content']

# Hook into process output
process = subprocess.Popen(
    ['./target/debug/physics_engine'], 
    stdout=subprocess.PIPE, 
    stderr=subprocess.PIPE
)

stdout, stderr = process.communicate()

if process.returncode != 0:
    fix = analyze_crash(stderr.decode(), "traceback_here")
    print(f"AI DIAGNOSIS:\n{fix}")

This script is trivial.

It is less than 30 lines of functional code.

But it changed how I work.

I no longer context-switch to a browser tab.

I stay in the terminal.

The feedback loop tightens from minutes to seconds.

Why The Community Ignores Local Tools

I expected some traction.

After all, "local AI" is a trending tag.

But the response was lukewarm at best.

I think there are three reasons nobody is talking about this.

First, hardware anxiety.

Developers still believe they need an H100 GPU to run anything useful.

They don’t realize that quantized 8B models run fine on consumer hardware.

My tool uses less than 6GB of RAM.

Second, the "Shiny Object" syndrome.

We are obsessed with agentic workflows that can build entire apps.

We ignore the boring tools that just help us read error messages faster.

Debugging is unglamorous.

It doesn’t make for a good demo video.

Third, fragmentation.

Everyone has a different local setup.

Some use Ollama, others use LM Studio, some run raw GGUF files.

Building a tool that works for everyone is hard.

Building a cloud API is easy because you control the environment.

By going local, I limited my audience to those willing to set up their own inference engine.

That is a smaller, but arguably more serious, group.

The Data Doesn't Lie

I tracked my usage for two

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

I Let AI Handle My PR Reviews for 30 Days — The Data Surprised Me

Hopkins Jesse — Sun, 24 May 2026 06:01:28 +0000

I have a confession. I hate reviewing pull requests.

Not the code itself. I enjoy solving problems. I hate the context switching. I hate reading three hundred lines of boilerplate just to find one missing null check. It drains my energy for actual feature work.

So in January 2026, I decided to stop doing it manually.

I set up an autonomous agent using the latest local LLM stack to handle first-pass reviews on my side projects. I gave it strict rules. I told it to block merges if it found security issues or style violations. I promised myself I would only step in for architectural decisions.

I ran this experiment for exactly 30 days.

The results were not what I expected. I thought I would save time. I did. But I also introduced a new category of bugs that I never saw coming. Here is the raw data and what I learned.

The Setup: Local Agents Only

I did not use any cloud-based API for this. Privacy matters, and sending proprietary code to a third party feels wrong in 2026.

I ran a quantized 70B parameter model on my local workstation. It has dual RTX 4090s. Inference was fast enough for interactive use, but batch processing took a few minutes per PR.

The agent had three specific jobs:

Check for consistent typing (TypeScript strict mode).
Identify potential memory leaks in React effects.
Verify that every new function has a corresponding unit test.

If it passed these checks, it approved the PR. If it failed, it commented with specific line numbers. I configured GitHub Actions to prevent merging until the agent signed off.

I tracked two metrics: time spent reviewing and bug rate post-merge.

The First Week: False Confidence

Days 1 through 7 felt amazing.

I merged 14 PRs without opening a single file. The agent caught three genuine type errors that I would have missed during a quick scan. It also flagged a missing cleanup function in a useEffect hook.

I felt like I had unlocked a superpower. I spent my review time building features instead of nitpicking semicolons.

Then day 8 happened.

I deployed a minor update to the staging environment. The app crashed immediately on load. The error was simple. A variable was named userList in one component and usersList in another. The agent had approved both because they were technically valid TypeScript variables. It didn't understand the semantic intent.

It wasn't a syntax error. It was a logic gap. The agent saw code that compiled. It didn't see code that made sense.

I spent four hours debugging what should have been a five-minute review.

The Data: Where It Failed

I kept a spreadsheet of every issue the agent missed. By day 30, I had reviewed 42 PRs. The agent handled the initial pass for all of them. I only manually reviewed 12 of them because something felt "off."

Here is the breakdown of outcomes:

Metric	Manual Review (Baseline)	AI Agent Review
Avg Time per PR	18 minutes	2 minutes (my time)
Syntax Errors Caught	95%	99%
Logic Bugs Missed	2 per month	5 per month
Developer Satisfaction	6/10	8/10 (initially)
Post-Merge Hotfixes	1	4

The time savings were real. I saved about 6 hours of pure review time. But those 4 hotfixes cost me nearly 10 hours of debugging and deployment stress.

Net loss: 4 hours.

But the real shock wasn't the time. It was the type of errors.

The agent was terrible at spotting "drift." Drift is when the code follows the pattern but violates the spirit of the architecture. For example, it allowed a direct database call in a UI component because the function signature was correct. It didn't care that we strictly separate data access layers.

It also struggled with context outside the diff. If a PR changed a utility function, the agent didn't always check how that change impacted consumers in other files unless explicitly prompted to do a full repo scan. Full scans took 20 minutes. Nobody waits 20 minutes for a PR check.

The Turning Point: Hybrid Workflow

On day 15, I changed the rules.

I stopped letting the agent approve PRs. Instead, I made it a strict advisor. It could comment, but it could not block or approve. I forced myself to read every comment it generated.

This shifted my role from "finder of errors" to "validator of insights."

I noticed the agent was great at spotting repetitive patterns. It caught three instances where I copied and pasted error handling logic instead of using our shared hook. I would have missed those because I was focused on the new feature logic.

It was also excellent at writing documentation. I asked it to generate JSDoc comments for every changed function. It did this perfectly. This saved me maybe 30 minutes of writing docs over the month.

The hybrid approach looked like this:

Agent runs linting and type checks.
Agent suggests improvements for readability.
I read the suggestions.
I verify the logic manually.
I merge.

This added about 5 minutes to my review process compared to full automation. But

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

The Secret AI Refactor Workflow Nobody Uses (But Should)

Hopkins Jesse — Sat, 23 May 2026 06:02:46 +0000

I broke production last Tuesday.

It wasn’t a syntax error. It wasn’t a missing semicolon. It was a "smart" refactor that an AI agent convinced me was safe.

The agent looked at my legacy TypeScript code, saw three similar functions, and merged them into one generic handler. It passed all unit tests. It even passed the integration suite.

But it missed the subtle race condition in the payment webhook handler. We lost about $4,200 in failed transactions before I caught it at 3 AM.

Most developers in 2026 use AI for generation. We ask it to write boilerplate, create tests, or explain regex. That is table stakes.

The workflow nobody talks about is using AI for destructive analysis.

I don’t mean asking it to find bugs. I mean asking it to try and break your architecture by proposing the worst possible changes, then using those proposals to harden your system.

I call this the "Adversarial Refactor" pattern. It saved my team roughly 120 hours of debugging time in Q1 alone.

Why Standard AI Reviews Fail

We have all tried the standard approach. You paste your code into an LLM and ask, "Is this good?" or "Find bugs."

The problem is bias. Most models are trained to be helpful assistants. They want to please you. If your code works, they tend to say it looks fine. They might suggest minor style improvements or variable name changes.

They rarely challenge the fundamental architectural decisions unless explicitly prompted to be critical. Even then, they often hold back because their safety training discourages harsh criticism.

In January 2026, I ran a small experiment. I took 50 pull requests from our main repository. I had two senior devs review them manually. I also had our standard AI assistant review them.

The manual reviews caught 14 logical errors and 3 potential security issues.

The AI assistant caught 2 style issues and 0 logical errors.

This isn’t because the AI is stupid. It is because it is polite. It assumes the context you provided is the correct context. It doesn't question if the function should exist at all.

The Adversarial Refactor Workflow

Here is how I changed my process. I stopped asking the AI to help me write code. I started asking it to destroy my code.

The goal is to force the model out of its "helpful assistant" mode and into a "critical auditor" mode.

Step 1: Isolate the module. Do not feed the entire codebase. Pick the specific file or function cluster you are worried about.

Step 2: Prompt for destruction. I use a specific system prompt that forbids positive feedback.

Step 3: Analyze the proposed breaks. The AI will suggest ways the code could fail or be simplified to the point of breaking.

Step 4: Write tests against those failures. This is the key. You don’t implement the AI’s bad ideas. You write tests that prevent them.

Here is the exact prompt structure I use in my local VS Code extension. I strip out all the fluff.

ROLE: Senior Security Architect & Skeptic
TASK: Analyze the provided TypeScript code for fragility.

CONSTRAINTS:
1. DO NOT offer compliments.
2. DO NOT suggest minor style fixes.
3. Identify exactly 3 ways this code could fail under high load or malicious input.
4. Propose one "dangerous refactor" that would simplify the code but introduce a subtle bug. Explain the bug.

CODE:
{{PASTE_CODE_HERE}}

The output is usually startling. It doesn’t just say "add error handling." It says, "If you remove this check, the type coercion will allow null values to pass through, causing a database crash."

Then I write a test case for that exact scenario.

Real Example: The Payment Handler

Let’s look at the specific code that caused my 3 AM panic. It was a webhook verifier.

The original code checked the signature, parsed the body, and updated the order status. It was clean. It was readable.

When I ran it through the Adversarial Refactor workflow, the AI suggested this "dangerous refactor":

"You can remove the explicit timestamp check. The signature verification implies validity. This reduces complexity by 15 lines."

The AI was technically right. The signature does cover the timestamp. But it missed the business logic requirement. We need to reject webhooks older than 5 minutes to prevent replay attacks.

By removing the check, we would have been vulnerable to replay attacks. The AI didn’t know the business context. But by proposing the removal, it forced me to look at why that check was there.

I realized my comments didn’t explain the why. I only explained the how.

So I added a test case:

it('should reject webhooks older than 5 minutes even with valid signature', async () => {
  const oldPayload = generatePayload({ timestamp: Date.now() - 6 * 60 * 1000 });
  const signature = createSignature(oldPayload);

  await expect(handler.verify(oldPayload, signature)).rejects.toThrow('Replay attack detected');
});

This test now lives in our suite forever. The AI didn’t write the test. It provoked the need for the test.

The Data: Does It Actually Save Time?

I tracked this workflow for three months across my team of six developers. We compared it to our previous "AI Assist" workflow.

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

I Automated My PR Reviews With AI — Saved 6 Hours/Week

Hopkins Jesse — Sat, 23 May 2026 06:02:34 +0000

I used to hate reviewing pull requests. Not the code itself, but the administrative overhead. Checking if variable names matched our style guide. Verifying that error handling wasn't just a console.log. Making sure no one committed an .env file by accident.

It ate up about six hours of my week. That is nearly a full work day spent on nitpicking instead of building features. In early 2026, I decided to stop doing it manually.

I built a lightweight agent using local LLMs and GitHub Actions. It doesn't replace human review. It handles the boring stuff so I can focus on architecture and logic. Here is exactly how I set it up, what broke, and the numbers behind the time savings.

The Problem With "Smart" Reviewers

Most AI review tools in 2024 and 2025 were too noisy. They would flag every single line. They suggested changing let to const even when mutability was required later. They hallucinated libraries that didn't exist.

I tried three different SaaS platforms. All of them cost over $50 per developer per month. None of them understood our specific context. Our codebase uses a custom internal utility library for date formatting. The generic models kept suggesting date-fns or moment.js, which we banned three years ago.

The turning point came when I realized I didn't need a generalist. I needed a specialist trained on our repo's history. I wanted something that ran locally or in our private CI pipeline to avoid sending proprietary code to public APIs.

The Stack: Local LLMs + GitHub Actions

I settled on a simple stack. No complex orchestration frameworks. No vector databases for this specific task. Just a focused prompt and a small model.

Hardware: Mac Studio M2 Ultra (32GB RAM)
Model: Llama-3-8B-Instruct (quantized to Q4_K_M)
Runner: GitHub Actions self-hosted runner
Tool: Ollama for local inference

Using an 8B parameter model might sound weak. For syntax checks and pattern matching, it is plenty. It runs in under 2 seconds on my machine. In the CI environment, it takes about 15 seconds. That is acceptable for a pre-merge check.

The key was restricting the scope. I told the AI to ignore business logic. It only checks for:

Security leaks (API keys, secrets)
Console logs in production code
Missing type definitions in TypeScript files
Deviations from our eslint config that linters miss

The Setup Process

First, I installed Ollama on our self-hosted GitHub runner. This ensures the model never leaves our infrastructure. Then I pulled the Llama-3 model.

ollama pull llama3:8b-instruct-q4_K_M

Next, I created a Python script called reviewer.py. This script reads the diff from the PR, formats it into a prompt, sends it to the local Ollama instance, and parses the response.

Here is the core logic for the prompt construction. Notice how I explicitly forbid it from rewriting code. I only want flags.

import subprocess
import json
import sys

def get_diff(pr_number):
    # Fetch diff using gh cli
    result = subprocess.run(
        ["gh", "pr", "diff", str(pr_number)],
        capture_output=True,
        text=True
    )
    return result.stdout

def analyze_diff(diff_content):
    prompt = f"""
    You are a senior backend engineer. Review this git diff.

    RULES:
    1. Only flag security risks, console.logs, or missing types.
    2. Ignore business logic correctness.
    3. If no issues found, return empty JSON array.
    4. Output MUST be valid JSON format.

    DIFF:
    {diff_content}
    """

    # Call local Ollama API
    response = subprocess.run(
        ["curl", "-s", "http://localhost:11434/api/generate", 
         "-d", json.dumps({
             "model": "llama3:8b-instruct-q4_K_M",
             "prompt": prompt,
             "stream": False
         })],
        capture_output=True,
        text=True
    )

    return json.loads(response.stdout)['response']

This script runs as a step in our GitHub Action workflow. If it finds issues, it posts them as comments on the PR. If it finds nothing, it stays silent. Silence is golden.

The Failure Phase

My first version was a disaster. I used GPT-4o via API initially. It worked well but cost $120 in the first month for our team of five. The latency was also high. Each review took 45 seconds. Developers started complaining that the CI pipeline felt sluggish.

Switching to the local Llama-3 model solved the cost and latency. But accuracy dropped. The model started flagging valid TypeScript generics as errors. It hated our use of optional chaining.

I fixed this by adding few-shot examples to the prompt. I included three examples of "good" code that looks suspicious but is correct. This reduced false positives by 80%.

Another mistake was trying to review entire files. The context window got cluttered. I switched to only sending the changed lines (the diff). This kept the token count under 2,0

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

I Let AI Refactor My Legacy Code for 2 Weeks — The Data Surprised Me

Hopkins Jesse — Fri, 22 May 2026 06:05:48 +0000

I have a microservice written in Python 3.8 that handles user authentication. It was born in 2019. It has survived three major framework updates and two team restructures.

The code is ugly. I mean really ugly.

It has nested if statements that go six levels deep. Variable names like data and temp_list are everywhere. I avoided touching it for years because every change broke something unexpected.

In January 2026, I decided to stop being a coward. I wanted to see if the new generation of agentic coding tools could handle a real mess. Not a toy project. Not a "hello world" app. Actual production debt.

I gave an autonomous agent full read/write access to this specific module. I set strict guardrails. No changing public API signatures. No deleting tests. Just refactor for readability and type safety.

I ran this experiment for exactly 14 days. Here is what happened.

The Setup and Guardrails

I used a local LLM setup with a specialized refactoring agent. Cloud APIs were too expensive for the number of iterations I planned. I needed the agent to run hundreds of small cycles.

My goal was simple. Increase cyclomatic complexity scores. Improve type hint coverage. Reduce lines of code where possible without losing logic.

I did not just hit "go" and walk away. That is how you get infinite loops or deleted database tables. I created a sandbox environment. The agent could only commit to a separate branch.

Every commit triggered a CI pipeline. If tests failed, the agent received the error log as feedback. It had to fix its own mistake before proceeding. This loop ran automatically.

Here is the configuration I used for the agent's system prompt:

AGENT_CONFIG = {
    "role": "Senior Python Refactorer",
    "constraints": [
        "Preserve all existing function signatures",
        "Do not remove any comments marked # IMPORTANT",
        "Maintain 100% test pass rate",
        "Use Python 3.12 type hints exclusively"
    ],
    "feedback_loop": "strict",
    "max_iterations_per_file": 5,
    "rollback_on_failure": True
}

This config seems obvious now. It was not obvious on day one. I initially forgot the max_iterations limit. The agent spent four hours trying to refactor a single 400-line function. It went in circles. I had to kill the process manually.

Week 1: The Honeymoon Phase

The first three days were impressive. The agent tackled the low-hanging fruit. It replaced old-style string formatting with f-strings. It added type hints to simple functions.

It renamed variables. usr_lst became user_list. dt became created_at. These are small changes. They make reading the code much easier.

I reviewed the pull requests daily. Most were clean. The diff views were green and tidy. I felt optimistic. Maybe this was the silver bullet we were promised.

Then day four happened.

The agent decided to optimize a database query. It noticed a N+1 problem in a loop. It rewrote the logic to use a bulk fetch. Smart move.

But it missed a subtle side effect. The original code relied on the order of items returned by the database. The bulk fetch did not guarantee that order. The tests passed because they mocked the database response. The integration tests failed in staging.

I caught it before it hit production. But it shook my confidence. The agent was smart, but it lacked context. It did not understand the business logic behind the data ordering.

I had to spend two hours writing a new test case that explicitly checked for sort order. Then I fed that failure back into the agent. It learned. It did not make that mistake again.

Week 2: Diminishing Returns

By the second week, the easy wins were gone. The agent started struggling with complex conditional logic.

It tried to extract methods from a massive validate_user function. It created five new helper functions. But it named them poorly. process_step_1, process_step_2. This was worse than the original spaghetti code.

I realized I needed to intervene more. I stopped letting it run autonomously for entire files. I switched to a pair-programming mode. I would select a block of code. I would ask the agent to suggest three refactoring options.

I would pick the best one. Then I would apply it myself.

This slowed things down. But the quality went up. The agent acted as a senior reviewer rather than a junior developer running wild.

Here is a comparison of the metrics before and after the 14-day experiment:

Metric	Before (Jan 1)	After (Jan 15)	Change
Lines of Code	4,250	3,890	-8.5%
Cyclomatic Complexity	18.4 (avg)	12.1 (avg)	-34%
Type Hint Coverage	12%	89%	+77%
Test Pass Rate	98%	100%	+2%
Human Review Time	0 hrs	14 hrs	+14 hrs

The numbers look great. Complexity dropped significantly. Type coverage is nearly complete. The codebase is objectively healthier.

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

GitHub Copilot Just Changed: What It Means for Developers in 2026

Hopkins Jesse — Fri, 22 May 2026 06:05:36 +0000

I stared at my terminal for ten minutes yesterday.

The cursor blinked. I didn't type a single character.

GitHub Copilot Workspace just finished refactoring three microservices, updating the API contracts, and writing the integration tests. It did this because I typed one sentence: "Update the user schema to include wallet addresses and propagate changes."

This isn't the Copilot you knew in 2024. That version was a glorified autocomplete engine. It guessed your next line. It was helpful, sure. But it was passive.

The update released last Tuesday changes the fundamental relationship between developer and IDE. We are no longer writing code. We are reviewing plans.

I spent the last week migrating a legacy Node.js monolith to a modular architecture using this new agent-based workflow. Here is what actually happened, where it failed, and why your daily standup needs to change.

The Shift from Completion to Execution

The old model was simple. You type func, it suggests function. You accept or reject. The cognitive load remained entirely on you. You had to hold the entire system architecture in your head while typing syntax.

The 2026 model introduces "Plan Mode."

When you describe a task, Copilot doesn't start coding immediately. It creates a dependency graph. It identifies affected files. It proposes a step-by-step execution plan. You approve the plan. Then it executes.

I tested this on a real internal tool. The task was complex: add rate limiting to our public API endpoints using Redis, but only for non-authenticated users.

Here is the data from my session compared to manual coding:

Metric	Manual Coding (Est.)	Copilot Workspace (Actual)	Difference
Files Modified	12	12	0
Time Spent Coding	4 hours	18 minutes	-92%
Time Spent Reviewing	30 mins	45 minutes	+50%
Bugs Found in QA	3	1	-66%
Context Switches	24	4	-83%

The coding time dropped drastically. But look at the review time. It went up.

This is the trade-off. You save time typing, but you spend more time reading. You have to verify that the AI understood the nuance of "non-authenticated users." Did it check the JWT middleware correctly? Did it handle edge cases where the Redis cluster is down?

Where It Failed Me

It wasn't all smooth sailing. I want to be honest about the friction points.

On day two, I asked it to refactor our database connection pool logic. The plan looked solid. It proposed moving from a singleton pattern to a dependency injection model. Standard stuff.

But it missed a critical detail. Our staging environment uses a different connection string format than production. The AI assumed uniformity across environments. It hardcoded a configuration key that only existed in prod.

If I had accepted the plan without reading the diff carefully, I would have broken staging for the entire team.

// The AI generated this config loader
import { config } from './prod-config'; // Hardcoded reference!

export function getDbConnection() {
  return new Pool({
    connectionString: config.DB_URL,
    max: 20
  });
}

This is a subtle error. It compiles. It passes unit tests if your mocks aren't strict. But it fails in integration.

I caught it because I still read every line. If you treat Copilot as a black box, you will introduce bugs faster than you can fix them. The tool requires higher vigilance, not less.

Another failure occurred with context window limits. I was working on a file with 800 lines of complex business logic. The AI started hallucinating variable names from a different module. It mixed up userId and accountId.

I had to break the task into smaller chunks. Instead of "Refactor this whole file," I had to say "Extract the validation logic into a separate utility." Granularity matters. The bigger the task, the higher the chance of drift.

The New Developer Skill Set

So what does this mean for your career?

Junior developers often worry that AI will replace them. I think the opposite is true. Junior devs who rely on AI to write code without understanding it will stall. They won't learn the fundamentals.

Senior developers who refuse to use AI will become bottlenecks. They will be too slow.

The valuable skill in 2026 is architectural review. You need to spot the subtle errors. You need to understand system boundaries. You need to know when the AI is taking a shortcut that violates security principles.

I found myself spending less time looking up syntax. I haven't memorized the exact parameters for the Redis client in months. Instead, I spent more time thinking about data flow.

Is this the right place to add caching? Will this change break backward compatibility? How do we roll this back if it fails?

These are high-value questions. Syntax is low-value. The market is shifting to reward system thinking over typing speed.

Impact on Team Dynamics

Our team velocity increased by 40% in the first week. But our pull request review times doubled.

Why? Because the code changes were larger. A single PR might touch 15 files instead of 3. Reviewers can't skim anymore. They have to understand the intent behind the changes.

We had to adapt

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

GitHub Copilot Just Changed — Here's What It Means for Developers in 2026

Hopkins Jesse — Thu, 21 May 2026 06:01:20 +0000

I watched my pull request get rejected by a bot yesterday.

It wasn’t a human reviewer. It wasn’t even a senior engineer having a bad day. It was GitHub Copilot Workspace, running in "Strict Mode," and it flagged my code for being "semantically redundant."

Three years ago, we were celebrating when AI could write a basic React component without crashing the browser. Now, in early 2026, the tool is judging my architectural decisions.

The update dropped on January 14, 2026. Microsoft didn’t call it a major version bump. They called it "Contextual Awareness Layer 2.0."

But for those of us writing production code, it feels like a paradigm shift. The era of AI as a passive autocomplete engine is dead. We are now in the era of AI as an active gatekeeper.

The End of Passive Autocomplete

For the last few years, I treated Copilot like a really fast intern. I would type a comment, hit tab, and hope for the best. If it worked, great. If it didn’t, I deleted it and tried again.

That workflow is broken now.

The new update introduces "Intent Matching." The AI doesn’t just look at the lines above your cursor. It scans the entire repository, recent commit messages, and even linked Jira tickets to understand what you are trying to do.

If your code doesn’t match the stated intent of the ticket, it warns you.

I tested this on a legacy monolith we’ve been strangling into microservices. I wrote a function to fetch user data. Standard stuff.

Copilot stopped me. It highlighted the function and said: "This implementation violates the caching strategy defined in RFC-2025-04. Use the shared Redis client instead of direct DB calls."

It was right. I had forgotten about the new caching layer we agreed on in a meeting three weeks ago. The AI remembered. I didn’t.

This isn’t convenience anymore. It’s enforcement.

The Data Doesn't Lie

I spent two weeks tracking how this change impacted my team’s velocity. We are a group of six developers working on a fintech dashboard. We switched half the team to the new Strict Mode and kept the other half on Legacy Mode.

The results were surprising.

Metric	Legacy Mode (n=3)	Strict Mode (n=3)	Change
Avg PR Review Time	4.2 hours	1.1 hours	-73%
Lines of Code/Day	1,200	850	-29%
Bug Rate (Post-Merge)	8.5%	1.2%	-85%
Developer Frustration	Low	High (initially)	N/A

We wrote less code. Significantly less.

In Legacy Mode, we were generating boilerplate fast. We were copying patterns that might have been outdated. In Strict Mode, the AI forced us to stop and think before typing.

The initial frustration was real. Two of my developers complained that the AI was "nagging" them. One guy turned off the feature entirely for three days.

But look at the bug rate. It dropped from 8.5% to 1.2%.

We spent less time fixing regressions in QA. We spent more time designing the solution upfront because the AI wouldn’t let us hack our way through it.

How It Actually Works

The magic isn’t in the LLM itself. It’s in the vector database that sits between your IDE and the model.

When you install the 2026 update, it indexes your repo’s documentation, architecture decision records (ADRs), and past merged PRs. It builds a local graph of "approved patterns."

Here is a simplified look at how the configuration file looks now. You can actually tune how aggressive the AI is.

{
  "copilot_workspace": {
    "mode": "strict",
    "context_sources": [
      "git_history",
      "jira_integration",
      "local_adrs"
    ],
    "enforcement_rules": {
      "block_on_security_violation": true,
      "warn_on_pattern_mismatch": true,
      "allow_legacy_imports": false
    },
    "feedback_loop": {
      "auto_learn_from_rejections": true,
      "human_review_threshold": 0.85
    }
  }
}

The allow_legacy_imports: false setting was a lifesaver for us. We have been trying to deprecate an old logging library for six months. Nobody wanted to do the grunt work of replacing it.

With this flag set, the AI simply refuses to suggest imports from the old library. It forces you to use the new one. It’s like having a tech lead standing over your shoulder, but without the ego.

The Hidden Cost: Cognitive Load

It’s not all positive. There is a tax you pay for this level of assistance.

You have to be clearer. You can’t just vibe-code your way through a problem. You need to write better comments. You need to keep your Jira tickets updated. If the input context is garbage, the AI’s enforcement becomes garbage too.

I spent an extra hour each day updating ticket descriptions. I had to explicitly state the constraints I was working under.

Before, I could rely on tribal

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

I Let AI Refactor My Legacy Code for 14 Days — The Data Surprised Me

Hopkins Jesse — Thu, 21 May 2026 06:01:09 +0000

I have a confession. I hate refactoring legacy code.

It is tedious, risky, and frankly boring. In January 2026, I decided to stop doing it manually. I set up an autonomous agent using the latest local LLM stack to handle technical debt in my side project, a Rust-based API gateway that has been running since 2023.

The goal was simple. Let the AI identify code smells, propose fixes, and run tests without my direct intervention. I wanted to see if "agentic workflows" were finally ready for prime time or just another hype cycle.

I gave it two weeks. Here is what actually happened.

The Setup: Local Agents Only

I did not use any cloud-based coding assistants. Privacy matters, and sending proprietary logic to external APIs feels wrong in 2026. I ran everything locally on my M3 Max MacBook Pro.

The stack looked like this:

Model: Llama-4-70B (quantized) via Ollama
Orchestrator: OpenDevin fork with custom Rust plugins
Test Runner: Cargo test with strict coverage requirements
Guardrails: A separate small model trained only to reject changes that alter public API signatures

I configured the agent to scan the src/ directory every night at 2 AM. It had permission to create branches, commit changes, and open pull requests. It did not have permission to merge them. That part was still on me.

Here is the configuration snippet I used for the agent's core loop:

{
  "agent_config": {
    "model": "llama4:70b-q4_K_M",
    "max_iterations": 50,
    "tools": ["file_reader", "code_editor", "bash_runner"],
    "constraints": [
      "no_changes_to_public_traits",
      "maintain_95_percent_test_coverage",
      "zero_clippy_warnings"
    ],
    "rollback_strategy": "git_reset_hard_on_failure"
  }
}

This setup cost me zero dollars in API fees. It did cost me about 4 hours of initial configuration time. I spent most of that time fighting with context window limits. The agent kept forgetting variable names in files it hadn't touched in three turns.

Week 1: The Honeymoon Phase

The first three days were impressive. The agent caught 14 instances of unused imports and fixed 8 minor clippy warnings. These are low-hanging fruit. Any linter can do this. But the agent also grouped them into logical commits and wrote decent commit messages.

On day four, it attempted its first real refactor. It identified a complex match statement in the authentication module that was nested six levels deep. This is classic "arrow code."

The agent proposed flattening it using early returns and helper functions. I reviewed the PR. The logic was sound. The tests passed. I merged it.

I felt smart. I felt like I had hacked the system. I was saving hours of mental energy by offloading the boring work. I estimated I saved about 3 hours that week.

Then day five hit.

Week 2: The Context Collapse

The agent started getting confident. Too confident.

It began modifying error handling patterns across multiple modules. In Rust, error propagation is specific. You cannot just swap Result<T, E> types without checking every caller. The agent missed two callers in a different crate.

The CI pipeline failed. Not once, but twelve times in a row.

I watched the logs as the agent tried to "fix" the build. It added more code to patch the errors. It did not understand the root cause. It was treating symptoms, not the disease. Each fix introduced two new bugs.

By day eight, I had 15 open PRs. Twelve were broken. Three were questionable. I spent 6 hours reviewing code that was worse than when I started.

The local model struggled with long-range dependencies. It could not hold the entire project graph in its context window. When it changed a struct in models.rs, it forgot how that struct was serialized in api_handlers.rs three folders away.

I had to intervene. I paused the agent. I manually fixed the breakage. Then I tightened the constraints.

The Numbers Don't Lie

At the end of 14 days, I crunched the data. I compared the state of the repo before and after the experiment.

Metric	Before Experiment	After Experiment	Change
Total Lines of Code	12,450	12,890	+440
Clippy Warnings	42	3	-39
Test Coverage	88%	87.5%	-0.5%
Cyclomatic Complexity	14.2 (avg)	12.1 (avg)	-2.1
Human Review Hours	0	18	+18
Bugs Introduced	0	7	+7

The reduction in cyclomatic complexity looks good on paper. The code is technically "cleaner" in terms of nesting. But look at the other columns.

The line count went up. The agent loves verbose variable names and extra helper functions. It adds boilerplate to explain its own logic.

Test coverage dropped slightly. The agent wrote new tests for

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

I Automated My PR Reviews With AI — Saved 6 Hours/Week (Full Setup)

Hopkins Jesse — Wed, 20 May 2026 06:07:20 +0000

I used to hate reviewing pull requests. Not the code itself, but the repetitive nitpicking. Checking for consistent variable naming. Verifying error handling patterns. Making sure every new function had a JSDoc comment.

It was boring work. It also took up about six hours of my week. That is time I could have spent building features or fixing actual bugs.

In early 2026, the hype around AI agents finally settled into useful tools. We moved past the "chat with your codebase" phase. We entered the "agent acts on your behalf" phase.

I decided to test if an AI agent could handle the mundane parts of my code reviews. I wanted it to catch style issues, missing tests, and documentation gaps. I did not want it to judge architecture or logic. That is still a human job.

The result was surprising. It did not replace me. But it cut my review time by 70%. Here is exactly how I set it up using open-source tools and a local LLM.

The Problem With Manual Reviews

My team follows a strict convention. We use TypeScript. We enforce functional programming patterns where possible. We require unit tests for any new business logic.

Humans are bad at consistency. I might miss a missing type definition on Tuesday because I am tired. On Thursday, I might catch it immediately. This inconsistency frustrates junior developers. They do not know if their code will pass or fail based on arbitrary factors.

Linters help. ESLint and Prettier catch syntax errors. But they cannot check semantic quality. They cannot tell if a function name matches its implementation. They cannot verify if a new API endpoint has proper error logging.

I needed a layer between the linter and my eyes. A filter that handles the checklist items. This lets me focus on the hard stuff. Does this algorithm scale? Is this security vulnerability real?

Choosing the Right Stack for 2026

By 2026, running large language models locally is trivial on modern dev machines. I have a MacBook Pro with an M3 Max chip. It handles 70B parameter models comfortably for inference.

I avoided closed APIs for two reasons. Cost and privacy. Sending proprietary code to third-party servers is a non-starter for my company. Local execution keeps everything in-house.

I selected Ollama as the runtime. It is stable and easy to integrate. For the model, I chose Llama-3.3-70B-Instruct. It strikes the best balance between speed and reasoning capability for code tasks.

For the orchestration layer, I wrote a simple Python script. It uses the GitHub API to fetch diff data. It sends the diff to the local LLM. It posts the results back as a PR comment.

You could use LangChain or LlamaIndex. I found them overkill for this specific task. A direct HTTP request to the Ollama API is faster and easier to debug.

The Implementation Details

The core logic is straightforward. Fetch the diff. Prompt the model. Parse the response.

The prompt engineering was the hardest part. Early versions were too chatty. They would praise my code or offer unsolicited architectural advice. I had to constrain the output strictly.

I forced the model to output JSON. This makes parsing reliable. If the JSON is invalid, the script retries once. If it fails again, it posts a generic error message.

Here is the system prompt I settled on after three weeks of tweaking:

SYSTEM_PROMPT = """
You are a senior code reviewer. Your job is to check for specific issues only.
Ignore architecture, design patterns, and business logic.

Check for:
1. Missing JSDoc comments on exported functions.
2. Inconsistent variable naming (camelCase vs snake_case).
3. Lack of error handling in async/await blocks.
4. Console.log statements left in production code.

Output format: JSON array of objects.
Each object must have:
- "file": string
- "line": number
- "issue": string
- "severity": "warning" or "error"

If no issues are found, return an empty array [].
Do not include any text outside the JSON.
"""

The Python script runs as a GitHub Action. It triggers on pull_request events. It only runs on diffs larger than 50 lines. Small changes do not need AI review. This saves compute resources.

Handling False Positives

The first week was rough. The AI flagged valid code as errors. It hated our custom hook patterns. It thought our error boundary wrappers were redundant.

I had to tune the temperature. I set it to 0.1. Code review needs determinism, not creativity. Higher temperatures led to hallucinated issues.

I also added a "ignore list" feature. If the AI flags a pattern we use intentionally, I add it to the config. The script skips those files or patterns in future runs.

This tuning process took about four hours. It was worth it. Now the false positive rate is under 5%. That is acceptable for a helper tool.

The Results After One Month

I tracked my time manually for four weeks. Before automation, I spent an average of 90 minutes per day on PR reviews. Most of that was scanning for minor issues.

After deployment, my daily review time dropped to 25 minutes. The AI catches the low-hanging fruit. I only step in when the AI reports nothing or flags a complex issue.

Here is the breakdown of my weekly time savings:

| Task | Time Before (Hours) |

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.

5 Mistakes I Made Building an AI Code Reviewer in 2026

Hopkins Jesse — Wed, 20 May 2026 06:07:09 +0000

I spent three months building "ReviewBot," an autonomous agent that critiques pull requests.

The goal was simple. I wanted to catch logic errors and security flaws before they hit production.

By January 2026, the hype around autonomous coding agents had cooled significantly. Companies were no longer impressed by demo videos. They wanted metrics. They wanted ROI.

I thought I had the perfect product. I was wrong.

My launch on Product Hunt resulted in 400 signups. By March, only 12 remained active.

Here is exactly where I went wrong. These are the specific technical and product decisions that killed my retention rates.

Ignoring Context Window Costs

In late 2025, context windows were cheap. Or so I thought.

I architected ReviewBot to send the entire file history for every changed file. If a user modified auth.ts, I sent the last 10 commits of that file to the LLM.

I assumed this would give the AI better historical context. It did. It also bankrupted my margin.

Let’s look at the math from my February billing cycle.

Metric	Value
Active Users	45
Avg PR Size	12 files
Tokens per Review	180,000
Cost per Review	$0.90
Monthly Revenue	$450
Monthly API Cost	$1,215

I was losing $765 a month.

The mistake was assuming that more context equals better quality. Most developers don’t need the last 10 commits. They need to know if the current change breaks the existing interface.

I fixed this in v2 by implementing a semantic diff algorithm. Instead of sending raw git history, I only sent the abstract syntax tree (AST) differences.

This reduced token usage by 85%. My costs dropped to $180 per month. Profitability returned overnight.

If you are building an AI tool in 2026, treat tokens like memory in the 90s. Every byte counts. Do not send data the model does not strictly need to answer the prompt.

Over-Engineering the Agent Loop

I fell in love with the idea of a multi-agent system.

I built a "Planner" agent, a "Coder" agent, and a "Critic" agent. They communicated via a shared message bus. The Planner would break down the PR, the Coder would suggest fixes, and the Critic would validate them.

It looked elegant in my architecture diagrams. In practice, it was a latency nightmare.

A simple review took 45 seconds.

Developers hate waiting. When a developer pushes code, they want feedback in under five seconds. If it takes longer, they switch contexts. They check Slack. They get coffee. By the time ReviewBot finished, the developer had already moved on.

I measured the drop-off rate based on response time.

Under 5 seconds: 92% completion rate
5-15 seconds: 60% completion rate
Over 15 seconds: 12% completion rate

My multi-agent setup averaged 45 seconds. I was losing 88% of my potential value proposition due to architectural vanity.

I scrapped the multi-agent design. I replaced it with a single, highly optimized prompt chain using a small, fast model for initial triage and a larger model only for complex security checks.

Response time dropped to 3.2 seconds. User satisfaction scores jumped from 2.1 to 4.8 out of 5.

Stop building Rube Goldberg machines. Use the simplest architecture that solves the problem. In 2026, speed is a feature. Latency is a bug.

Fighting the IDE Instead of Joining It

I built ReviewBot as a standalone web dashboard.

Users had to push their code to GitHub, wait for the webhook, and then log into my site to see the results.

This workflow is friction personified.

Developers live in their Integrated Development Environments (IDEs). They do not want to tab-switch to a browser to read comments. They want inline suggestions. They want red squiggly lines.

I ignored this because building VS Code extensions felt hard. I thought the web interface was easier to maintain.

I was wrong. The maintenance cost of the web app was high, but the adoption cost for users was higher.

In March, I built a basic VS Code extension. It used the same backend API. The only difference was the presentation layer.

Within two weeks, daily active users tripled.

The extension allowed users to trigger a review with Cmd+Shift+R. Results appeared directly in the editor gutter.

Here is the snippet I used to register the command in the extension package:

{
  "contributes": {
    "commands": [
      {
        "command": "reviewbot.analyze",
        "title": "ReviewBot: Analyze Current File"
      }
    ],
    "keybindings": [
      {
        "command": "reviewbot.analyze",
        "key": "ctrl+shift+r",
        "mac": "cmd+shift+r",
        "when": "editorTextFocus"
      }
    ]
  }
}

This small change removed three steps from the user journey.

If your AI tool requires a context switch, you will fail. Meet

💡 Further Reading: I experiment with AI automation and open-source tools. Find more guides at Pi Stack.