DEV Community: Tech Geeks

Why AI Writes Broken ESP32 Code (and How to Fix It for Arduino-ESP32 3.x)

Tech Geeks — Fri, 06 Nov 2026 19:10:49 +0000

You ask an AI assistant for a simple ESP32 sketch, it hands you clean, confident code, and the compiler replies with 'ledcSetup' was not declared in this scope. Or worse, it compiles, you flash it, and the board boot-loops.

This isn't bad luck. It's the most predictable failure in AI-assisted firmware, and once you know its shape it's easy to prevent.

Firmware is not a forgiving sandbox

A hallucinated function in a web app gives you a red squiggle. A hallucinated pin number on a microcontroller can drive current into the wrong pin and release the "magic smoke". Firmware talks to real hardware in real time: a blocking call trips a watchdog and reboots the board, and a wrong register writes garbage to real silicon. The AI can't see your wiring, feel heat or measure current. It only sees text.

Two failure modes dominate:

Hallucination. The model invents a function, register, pin or whole library. A USENIX Security 2025 study of 576,000 generated code samples found hallucinated packages in at least 5.2% of cases for commercial models and 21.7% for open-source ones.
Deprecated APIs. The model writes real code that used to work. An ICSE 2025 study measured deprecated API usage of 25% to 38% across eight Python libraries. On the ESP32 this shows up constantly.

A 2026 study that tested 27 LLMs on ESP32 hardware across eight embedded scenarios found that 19 to 23 models could produce compilable code for simple apps, but only 3 to 5 produced working solutions for complex ones. The most common failure was hallucinated libraries or incorrect API usage.

The version trap: Arduino-ESP32 2.x vs 3.x

The ESP32 world had two big breaking changes recently: Arduino-ESP32 went from 2.x to 3.x, and ESP-IDF went from 4.x to 5.x. Models were trained on years of 2.x examples, so they confidently generate the old API.

PWM (LEDC) is the classic example. In core 2.x you set up a channel, attached a pin to it, and wrote to the channel:

// Arduino-ESP32 2.x: does NOT compile on core 3.x
ledcSetup(0, 5000, 8);      // channel, frequency, resolution
ledcAttachPin(5, 0);        // pin, channel
ledcWrite(0, 128);          // channel, duty

In 3.x, channels are assigned automatically and you work with the pin directly:

// Arduino-ESP32 3.x
const int ledPin = 5;  // a safe output pin

void setup() {
  ledcAttach(ledPin, 5000, 8);  // pin, 5 kHz, 8-bit resolution
}

void loop() {
  for (int duty = 0; duty <= 255; duty++) {
    ledcWrite(ledPin, duty);
    delay(5);
  }
}

Hardware timers changed too. The old four-argument style, timerBegin(0, 80, true) with a timer number, prescaler and count direction, is gone. In 3.x, timerBegin takes the timer frequency in hertz, and alarms are set with timerAlarm.

I2C in ESP-IDF 5.x replaced the legacy driver (i2c_config_t, i2c_driver_install, i2c_master_write_to_device) with a bus-and-device model (i2c_new_master_bus, i2c_master_bus_add_device, i2c_master_transmit).

The fix is to make the model confront the version before it writes anything:

I'm on Arduino-ESP32 core 3.3.x (ESP-IDF 5.5). Before writing code,
list any APIs you plan to use that CHANGED between core 2.x and 3.x
(e.g., LEDC, timer, I2C). For each, show the OLD call and the NEW 3.x call.
Then write the sketch using ONLY the 3.x API.

The pin rules your AI doesn't reliably know

ESP32 pins are not interchangeable, and this is where AI most often bricks projects. For the classic ESP32-WROOM-32:

Pins	Rule
GPIO 6–11	Wired to the internal SPI flash. Using them crashes the chip. Never touch them.
GPIO 34, 35, 36, 39	Input only: no output drivers and no internal pull-ups. They can't drive an LED or relay.
GPIO 0, 2, 12, 15	Strapping pins, sampled at boot. The wrong level or an attached load can stop the board booting.
ADC2 pins (0, 2, 4, 12–15, 25–27)	Unusable for `analogRead()` while Wi-Fi is on. Use ADC1 (GPIO 32–39) in any networked project.

I2C also needs pull-up resistors (4.7 kΩ is typical). That is a hardware fact no amount of code can fix.

Put these rules in the prompt, and ask the model to justify every pin it picks:

Write an Arduino-ESP32 3.x sketch for an ESP32-WROOM-32 that:
- reads a potentiometer on an ADC1 pin (Wi-Fi is ON, so NOT ADC2),
- drives an LED with PWM using ledcAttach()/ledcWrite() (NOT ledcSetup),
- avoids GPIO 6–11 and never uses 34–39 for the LED.
State which pins you chose and why each is safe.

Make the rules permanent with an instruction file

Rather than repeating yourself, put the rules in the project instruction file your agent reads: CLAUDE.md for Claude Code, AGENTS.md for Codex and many other tools, or .cursor/rules for Cursor.

# Firmware project rules
- Board: ESP32-S3-DevKitC-1. Framework: Arduino-ESP32 core 3.3.x (ESP-IDF 5.5).
- Use ONLY the 3.x API. Never use ledcSetup/ledcAttachPin or the 4-arg timerBegin.
- Never assign GPIO 6–11 (SPI flash).
- Analog sensors must use ADC1 pins because Wi-Fi is enabled.
- After any change, run `pio run`. Paste the full error output before editing.
- Never use delay() inside a FreeRTOS task longer than 50 ms; use vTaskDelay.
- Cite the doc page for any peripheral API you use.

Pin the board and framework in platformio.ini too, so builds are reproducible and the agent can run pio run itself. Then it can write, compile, read the errors and fix them in a loop.

A five-step verification ritual

The model's confidence has nothing to do with its correctness, so check every time:

Compile. If it doesn't build, feed back the full error verbatim.
Static analysis. cppcheck --enable=all --inconclusive src/ catches null dereferences, buffer overflows and uninitialised reads that both the compiler and the AI miss.
Cross-check the docs. Verify every pin, register and function against the current reference. This is where deprecated APIs get caught.
Demand citations. Ask the model to name the documentation page for each peripheral call. Unsourced claims are where hallucinations hide.
Simulate before you flash. Wokwi simulates the ESP32 family in the browser or VS Code, so the AI can test logic without risking your board.

AI is a strong assistant and a weak authority. Used with a tight loop and these guardrails, it will write a lot of your firmware, and your board will survive the experience.

Originally published on TechBrosEbooks.

This article is adapted from the ebook Vibe-Coding Your ESP32, which also covers ESPHome, hardware-in-the-loop debugging, memory, timing and power, safety rails for mains and motors, a complete 3D-printer enclosure monitor build, and a 12-point pre-flash checklist.

How to Test an AI Agent: Start With Error Analysis, Not Metrics

Tech Geeks — Fri, 06 Nov 2026 19:08:19 +0000

Your agent passed the demo. Does it still work on the 500th request?

Agents fail differently from normal software. A normal bug throws an exception. An agent bug returns a confident, well-formatted, wrong answer with HTTP 200: the right tool with the wrong ID, an extra loop, a step quietly skipped. Nothing alerts you, and the only witness is an unhappy user.

The good news is that you already know the fix. It's the testing discipline you practise every day, adapted to a component that is non-deterministic and takes many steps. Traces replace stack traces, eval datasets replace test fixtures, and graders replace assertions.

Why agents are hard to test

Four properties make agents different:

Non-determinism. The same task can produce different tool sequences on different runs. One green run proves the agent can succeed, not that it reliably will.
Multi-step trajectories. The answer can be right for the wrong reasons. You have to test the path, not only the destination.
Compounding errors. An agent that is 95% reliable per step succeeds only about 60% of the time over ten steps (0.95 to the power of 10).
Silent failure. Fluent text with a 200 status. Monitoring has to inspect content, not just status codes and latency.

Step 1: measure flakiness before anything else

The cheapest first test needs no graders at all. Run each task several times and see how consistent the agent really is:

from collections import Counter

TASKS = ["Refund order A123", "Change my email to a@b.com", "Where is my parcel?"]
N = 5

for task in TASKS:
    outcomes = []
    for _ in range(N):
        trace = run_agent(task)  # your agent, instrumented
        tools = tuple(c.name for c in trace.tool_calls)
        outcomes.append((trace.success, tools))
    passes = sum(ok for ok, _ in outcomes)
    paths = Counter(t for _, t in outcomes)
    print(f"{task!r}: {passes}/{N} passed, {len(paths)} distinct tool paths")

If one task shows 3/5 passes and four different tool paths, you have found your first reliability problem.

Step 2: read your traces before you write a single metric

This is the step most teams skip, and the one that matters most. Teams that ship reliable agents start by reading 50 to 100 real traces and writing down what went wrong. The process is borrowed from qualitative research:

Collect traces. Pull 50 to 100 real runs from production or staging.
Open coding. For each trace, write a free-form note on the first failure you see. Later failures are often side effects of the first.
Axial coding. Group your notes into a taxonomy of 5 to 8 specific failure modes.
Count. Tally each mode. The top two or three are where you write evals.

A spreadsheet is enough. A typical result for a support agent might be: wrong customer looked up (14), refund issued without a policy check (9), answered from memory instead of the knowledge base (7). Those three lines are your eval roadmap.

Stop when new traces stop revealing new failure modes, which usually happens within about 100. An LLM can help cluster your notes, but it never replaces the reading.

Step 3: trace everything

You can't do error analysis without traces. A trace is the agent's flight recorder: every model call, tool call and retrieval, with tokens, cost and timing. It's a tree of spans, with one invoke_agent root and chat and execute_tool children.

Instrument against the OpenTelemetry GenAI conventions for portability, but pin versions, because they are still marked as in development. Tag every run with a prompt version and an agent version, so a regression can be tied to the change that caused it. Langfuse and Arize Phoenix are open source; LangSmith and Braintrust are managed; Datadog, Honeycomb and Grafana let you keep agent traces next to the rest of your telemetry. The tool matters less than the habit of reading traces every week.

Step 4: build a small eval dataset from real failures

Start with 20 to 50 tasks drawn from the failures you found, and grow the set every week. Every task needs a success criterion two engineers would agree on. Include negative cases too, such as a refund request that should be refused, or you will optimise the agent into acting when it shouldn't.

If you generate synthetic inputs, generate them along explicit dimensions (intent × persona × difficulty) rather than asking for "50 questions". Let an LLM draft the inputs, but write the expected outcomes yourself.

Step 5: grade with code wherever you can

Use code for anything checkable: final state, which tools were called, their arguments, output format, step and cost budgets, safety rules. Final-state checks are the strongest graders for agents that take actions. Did the refunds table end up with exactly the right rows?

Two rules keep code graders from becoming brittle. Assert what must and must not happen, but never hard-code one exact tool sequence. And give every trial a fresh, isolated environment, or your scores measure test pollution instead of the agent.

Step 6: use LLM judges narrowly, and calibrate them

For things code can't check, like "did every policy claim come from the knowledge base?", an LLM judge can scale human judgement to thousands of traces. But judges have known biases: position, verbosity and self-preference. Design around them:

one narrow judge per failure mode, with a binary pass or fail
reasoning before the verdict
"length is not a virtue" stated in the rubric
a judge from a different provider than the agent

Then calibrate the judge like a classifier against traces a human has labelled. Measure its true-positive and true-negative rates, not raw agreement. A judge with a 60% true-negative rate misses four in ten real failures.

Step 7: track the right numbers

Report four numbers per suite: pass@1, pass^k, cost per task and latency per task. pass@k means at least one of k attempts succeeded; pass^k means all k did. A customer-facing agent needs pass^k, because a customer gets one attempt. Run each task 3 to 5 times, and watch median and p95 cost and latency: long tails usually reveal loops.

The north star

Look at your data before you build anything. The rest — datasets, graders, judges and CI gates — only works if it targets the failures your agent actually has.

Originally published on TechBrosEbooks.

This article is adapted from the ebook Is Your AI Agent Actually Working?, which goes further with OpenTelemetry instrumentation code, a reusable grader library, judge calibration, evals in CI/CD and production, and an end-to-end example of fixing a real support-triage agent.

Why Your AI Coding Agent Bill Grows With Every Turn (and 7 Ways to Cut It)

Tech Geeks — Fri, 06 Nov 2026 19:06:25 +0000

If you use Claude Code, Cursor or Codex every day, you have probably had one of two surprises: an API bill much larger than you expected, or a "you've hit your usage limit" message halfway through the afternoon. Both have the same cause, and once you see it, most of the fixes become obvious.

An agent resends everything, every time

A chatbot answers one message. A coding agent works in a loop: it thinks, reads a file, runs a command, reads the output, and thinks again. Every step is a separate request to the model, and every request carries the whole context:

Part of the request	Typical size	Sent when
System prompt and tool definitions	Thousands of tokens	Every request
Instruction files (CLAUDE.md, AGENTS.md, rules)	Hundreds to thousands	Every request
Conversation history	Grows every turn	Every request
Tool results (file contents, command output)	Often the largest part	Every request after they appear

So a one-line question asked in a session that has been open all day still pays for the whole day's conversation.

Why cost grows with the square of session length

Say each turn adds about 5,000 tokens. Turn 1 sends 5k, turn 2 sends 10k, and turn 20 sends 100k. Add it up and a 20-turn session processes roughly 1.05 million input tokens, even though only 100k of them were new.

Here is a quick model of that at illustrative Claude Sonnet 5 list prices ($2 per million input tokens, $0.20 per million for cache hits):

PRICE_IN, PRICE_CACHE_WRITE, PRICE_CACHE_HIT = 2.00, 2.50, 0.20  # $ per million tokens

def session_cost(turns, tokens_added_per_turn, cached=True):
    total_in, new_total = 0, 0
    for t in range(1, turns + 1):
        total_in += t * tokens_added_per_turn  # whole context re-sent
        new_total += tokens_added_per_turn
    if not cached:
        return total_in / 1e6 * PRICE_IN
    reused = total_in - new_total
    return new_total / 1e6 * PRICE_CACHE_WRITE + reused / 1e6 * PRICE_CACHE_HIT

# 10 turns: $0.55 uncached, $0.17 cached
# 20 turns: $2.10 uncached, $0.44 cached
# 40 turns: $8.20 uncached, $1.28 cached

Doubling a session from 20 to 40 turns roughly quadruples the uncached input cost. And that ignores output tokens, which cost about five times as much as input across the Claude line-up.

Subscriptions are not immune. On Pro, Max or Team plans you don't see a per-request bill, but the same tokens drain your usage window.

Seven ways to cut the cost

1. Measure for a week first

You can't cut what you can't see. In Claude Code, /usage shows session tokens by model, cache reads and writes and an estimated cost, and /context shows what is filling the window right now. Track four numbers: cost per session, cost per completed task, the share of input served from cache, and how much of your spend goes to the premium model. Judge every later change by cost per completed task. A cheap model that needs three attempts isn't cheap.

2. One task, one session

This single habit fixes the most common cause of runaway spend. When you switch to unrelated work, start fresh (/clear in Claude Code, a new chat in Cursor or Codex). Clearing costs nothing; carrying yesterday's debugging session into today's feature makes every message pay for both. Use /rename first if you might want to /resume the old session.

3. Default to the mid-tier model

Most daily coding work doesn't need a flagship model. Use a mid-tier model (Sonnet-class) for features, bug fixes and refactors, a small one (Haiku-class) for renames, boilerplate and log summaries, and save the premium tier for architecture and nasty concurrency bugs. A useful pattern: plan with the premium model, then switch with /model to implement. The expensive model spends a few thousand tokens thinking instead of hundreds of thousands editing files.

Thinking is billed as output, the most expensive kind of token. Lower the effort level for routine tasks.

4. Put your instruction file on a diet

CLAUDE.md, AGENTS.md and Cursor rules are loaded on every request, so every line is a recurring cost. Anthropic recommends keeping CLAUDE.md under 200 lines. Apply a simple test to each line: if I delete this, will the agent make a mistake in most sessions? If not, move it out. Multi-step procedures like releases belong in skills or on-demand docs that load only when needed.

Cutting a 400-line file (about 5,000 tokens) to 40 lines saves around 4,500 tokens per request. Over a few hundred requests a day, that's over a million tokens.

5. Keep noisy output out of context

Test runners and build logs can produce thousands of lines, and every line is resent on every later turn. Ask for filtered commands ("run pytest -q and show only failures"), push verbose jobs to a subagent so only a summary comes back, and block generated code, lockfiles and vendored dependencies from being read.

6. Stop breaking the prompt cache

Prompt caching lets the model reuse a context prefix it has already processed at a tenth of the normal input price, and your coding tool usually turns it on for you. But the prefix has to be byte-identical. These habits break it:

connecting or disconnecting MCP servers mid-session (tool definitions sit near the start of the prefix)
switching models mid-conversation (caches are per model)
editing your instruction file during a session
coming back to a huge session after a long break, once the cache has expired

/usage reports your cache-hit share. If it drops, one of these is usually why.

7. Use cheap subagents for legwork, but not for everything

A subagent runs in its own context and returns only a summary, which is ideal for searching a codebase or digesting test output. Pin it to a small model and your main context stays small too. But each subagent loads its own instructions and tools, so spawning one for a two-line lookup can cost more than doing it inline. Agent teams multiply this: Anthropic's docs put them at about 7x the tokens of a standard session when teammates run in plan mode.

What normal looks like

For a sense of scale, Anthropic reports that Claude Code averages about $13 per developer per active day across enterprise deployments, with 90% of users under $30. The two usual causes of unexpectedly high spend are exactly the ones above: long sessions that are never cleared, and a premium model left as the default.

Prices and plan limits change often, so check your provider's pricing page before acting on any figure here. The principles don't change: measure, route, trim, cache and guard.

Originally published on TechBrosEbooks.

This article is adapted from the ebook Stop Burning Tokens, which covers each technique in depth with configs and scripts, including a hook that filters test output, local-model fallback, spend guardrails, and a 30-day rollout plan.

AI Agents for Backend Engineers: What Carries Over and What's New

Tech Geeks — Fri, 06 Nov 2026 19:02:45 +0000

If you write backend code, you're closer to building production AI agents than you think. Queues, retries, idempotency, logs: it all carries over. What's new is a component that makes decisions, gives different answers to the same input, and can be talked into things. Here's the translation layer.

An agent is an LLM with three upgrades

A plain LLM call can only talk. An agent can act, because it adds:

Tools: functions it can call, such as querying a database or sending an email.
Memory: a record of what has happened so far.
A loop: it works step by step until the job is done. It thinks, acts, checks the result, and decides what's next. This pattern is called ReAct (reason + act).

Before you build one, ask whether a single, well-prompted LLM call would do. Summarising a paragraph is one call. Finding, fixing and testing a bug across 40 files is a job for an agent.

Tools: the model asks, your code decides

Function calling is the mechanism behind tools, and the most important thing to internalise is that the model never runs anything itself. You give it a menu of tools, each with a name, description and parameters. When it wants one, it replies with a structured request, and your code decides whether to run it. That's what keeps you in control.

Two habits make tools reliable:

Write descriptions like API docs, including when not to use the tool. "Fetch a customer's recent orders. Use for order status or refunds. Not for product search." That last sentence prevents a surprising number of wrong picks.
Validate arguments like any user input. Check the schema, check that referenced IDs exist, and rate-limit.

The Model Context Protocol (MCP), released by Anthropic in late 2024, lets you wrap a tool once and reuse it across any MCP-compatible agent. Think of it as USB-C for AI tools.

Memory is plumbing you build

LLMs remember nothing between calls; every bit of memory is something you store and feed back in. Short-term memory is the current conversation, limited by the context window. Long-term memory lives outside the model and is loaded only when relevant.

RAG (retrieval-augmented generation) is the common way to bring in outside knowledge: store documents in a vector database, retrieve the closest chunks, and put them in the prompt. Retrieve broadly, then re-rank and keep only the best few, and tell the model to answer using only that context so it doesn't guess.

Checkpointing means saving the agent's state after every step, so a restart resumes instead of starting over. It's the same reason you keep sessions in a database, not an in-memory dict.

Infrastructure: normal backend work, with two twists

Production agents are mostly the backend engineering you already do. The twists are that LLMs are non-deterministic and that their tool calls have real side effects. Four things keep you safe:

Durable execution. Tools like Temporal save progress at every step, so a crash resumes where it left off.
Sensible timeouts and retries. LLM calls can take a minute or more; back off politely on rate limits.
Idempotency keys. A retried tool call must happen only once in the real world.
Sandboxes. Run agent-written code in an isolated environment, never on your own server.

The idempotency key is the one engineers most often forget, and it's one line:

payments.refund(order_id="A123", amount=500, idempotency_key="refund:A123")
# The payment service sees the same key twice but refunds only once.

Cost matters too. Prompt caching lets providers reuse the repeated parts of your prompt; Anthropic reports savings of up to 90% on long prompts.

Observability: agents fail quietly

An agent's failures don't throw errors. You get a confident wrong answer, or a strange route to a right one, and a health check won't catch either. Tracing records every step as a span: what the model decided, which tool it called, what came back, the cost and the latency. Tools like Langfuse and LangSmith show this as a timeline.

Evaluation is testing for agents: plain assertions for exact things (right tool, right ID), and an LLM judge scoring fuzzy things against a rubric. Run evals in CI before every release, sample live traffic afterwards, and turn every production failure into a new test case.

Security: assume the model can be tricked

Agents act on text, and attackers can write text. The top risk is prompt injection: instructions hidden in a page, email or document the agent reads. You can't fully prevent it, so contain it:

Least privilege, enforced in code, not in the prompt.
Human approval for anything risky or irreversible: refunds, deletions, outgoing emails.
Treat model output as untrusted. Never eval() it, and parameterise any SQL it touches.
Keep secrets out of prompts. The model gets tools, never raw API keys.

The mantra: assume the model can be tricked, and make sure a tricked model can't do much damage. For example, an issue_refund() that raises NeedsHumanApproval above a set amount can't be pushed past that limit by a fooled model, because the code enforces the limit, not the prompt.

Go multi-agent last

Multi-agent systems are expensive and fragile. Anthropic's multi-agent research system used about 15 times more tokens than a normal chat, and a 2025 study found failure rates of 41% to 87% across multi-agent frameworks. What works best is one lead agent holding the full picture, with small helpers that do one job and report back a summary.

A learning path for a few weekends

Build a tiny agent with raw API calls and two tools, no framework.
Add RAG over a folder of your own notes.
Add tracing and 50 test cases, then break things on purpose.
Rebuild it in a framework: LangGraph, or the OpenAI or Claude Agent SDK.
Add durable execution and an approval step.
Expose a tool through MCP.

By step six you'll understand what frameworks do under the hood, which makes you much better at choosing and debugging them.

Originally published on TechBrosEbooks.

This article is adapted from the quick-start guide AI Agent Orchestration & Infrastructure for Software Engineers, which covers each idea in one short chapter, with orchestration patterns, multi-agent handoffs and A2A, an end-to-end support-ticket triage agent, and a pre-ship checklist.

Workflows vs Agents: The 6 AI Agent Orchestration Patterns (and When to Go Multi-Agent)

Tech Geeks — Fri, 06 Nov 2026 18:07:33 +0000

"Agent orchestration" sounds exotic, but the question underneath it is plain: how do you arrange one or more LLM calls into a system that solves a real task, reliably? The answer most teams who have shipped agents converge on is also plain. As Anthropic put it in its December 2024 guide Building effective agents: "the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns."

Workflows versus agents

Anthropic draws a line that's worth adopting:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths. You write the control flow.
Agents are systems where the LLM dynamically directs its own process and tool use. The model decides the control flow at runtime.

Workflows give predictability and consistency for well-defined tasks. Agents are the better option when the path genuinely can't be predicted in advance. For many applications, a single well-prompted LLM call with retrieval and good examples is enough.

The six patterns

Each one maps onto something you already know from distributed systems.

Pattern	What it does	Familiar analogy	Use it when
Prompt chaining	A fixed sequence; each call's output feeds the next, with optional checks between steps	A Unix pipe or ETL pipeline	The task splits into clean, fixed subtasks (outline → draft → revise)
Routing	A classifier sends each input to a specialised handler	An API gateway routing by path	Different request types need different prompts, tools or models
Parallelization	Run subtasks concurrently and merge, by sectioning or by voting	Scatter-gather / MapReduce	Parts are independent, or you want confidence from repeated runs
Orchestrator–workers	A lead LLM decides the subtasks at runtime and delegates them	A coordinator fanning jobs out to a worker pool	You can't know the subtasks up front, such as code changes across an unknown number of files
Evaluator–optimizer	One LLM generates, another critiques against criteria, repeat	A CI suite gating a build	You have clear quality criteria and iteration measurably helps
Autonomous agent	The model runs a reason–act–observe loop freely until done	—	Maximum flexibility is worth maximum cost and unpredictability

The patterns compose: a router often feeds a prompt chain, and an orchestrator's workers often run an evaluator–optimizer loop. But each added layer adds a failure mode. Every extra LLM call adds latency, cost and a compounding error rate, so add calls deliberately, at decision points you chose.

Routing: the most underrated pattern

Routing deserves a closer look because it's cheap and it pays twice. A support inbox with billing, tech and general questions sends each message through a small classifier first:

input -> [ Router LLM ] classifies intent
               |
      +--------+---------+
      v        v         v
   billing  tech-support  general
  (prompt+   (prompt+     (cheap
   tools)     tools)      model)

The router does one narrow job: output a category. Keeping it tool-free and single-purpose makes it fast, cheap, and easy to test with a plain classification eval. And it lets you send easy questions to a cheap model and hard ones to a capable one.

The agent loop, and why it needs a budget

Underneath every autonomous agent is the ReAct loop (Yao et al., 2022): the model writes a thought, takes an action, observes the result, and repeats. To a backend engineer, it's a while loop around a stateless handler:

for step in range(MAX_STEPS):  # a budget cap prevents infinite loops
    response = llm.chat(messages, tools=tool_schemas)
    if response.tool_call:
        result = tools[response.tool_call.name](**response.tool_call.args)
        messages.append(response.raw)
        messages.append({"role": "tool", "content": str(result)})
        continue
    print(response.content)  # no tool call -> final answer
    break

That step budget isn't decoration. An unbounded agent loop is the agent equivalent of runaway recursion.

Single agent or multi-agent?

This was the field's biggest architecture debate of 2025, and the evidence is worth knowing before you build a team of agents.

Against: in June 2025, Cognition (the makers of Devin) published Don't Build Multi-Agents. Their argument: parallel subagents that don't share full context make conflicting implicit decisions, and "actions carry implicit decisions, and conflicting decisions carry bad results."
For, with a price: a day later, Anthropic described its multi-agent research system, a Claude Opus 4 lead with Claude Sonnet 4 subagents. It beat a single Claude Opus 4 agent by 90.2% on their internal research eval. It also used about 15 times more tokens than a chat interaction, against roughly 4x for a single agent.
The failure data: the MAST study (Cemri et al., NeurIPS 2025) annotated 1,642 execution traces across seven multi-agent frameworks and found failure rates from 41% to 86.7%. About 42% of failures came from specification and system design, 37% from misalignment between agents, and 21% from weak task verification.

The consensus that emerged by 2026: multi-agent helps for genuinely parallel, breadth-first work such as research. The reliable shape is one stateful orchestrator that holds the full context, with narrow, stateless subagents that return summaries, not a chatty peer-to-peer mesh. Even Cognition's author later acknowledged that setups following exactly this shape "actually work."

How agents talk to each other

When you do need more than one agent, there are three coordination mechanisms, from tightest to loosest coupling:

Shared state: every agent reads and writes one common state object. Best for agents inside one process.
Handoffs: one agent transfers control, usually with the conversation history, to another. A triage agent handing a billing issue to a billing specialist means the user never repeats themselves.
Agent-to-agent protocols: for agents across processes or organisations. A2A (Agent2Agent), announced by Google in April 2025 and donated to the Linux Foundation that June, has each agent publish an Agent Card describing its skills and endpoints.

The tidy distinction: MCP is vertical (agent to tools) and A2A is horizontal (agent to agent). Production systems often use both: A2A routes a task to the right specialist, and MCP gives that specialist its tools.

The rule to remember

Start at the lowest rung that works: a single call, then a workflow, then an agent, and only then a team of agents. Climb only when the simpler design demonstrably fails.

Originally published on TechBrosEbooks.

This article is adapted from the ebook The AI Agent Field Manual, which goes on to memory and RAG, checkpointing, durable execution, queues and retries, observability and evals, security, and a complete support-ticket triage system built end to end.

A Vibe Coding Workflow That Ships: Plan, Code, Test, Deploy

Tech Geeks — Fri, 06 Nov 2026 18:01:02 +0000

You've used an AI coding assistant a few times. Sometimes you got something great; sometimes you got something terrible; and you're not sure which to expect next. The difference is rarely the model. It's the workflow around it.

Let's get the honest version out of the way: AI won't turn you into a 10x engineer. It makes typing about 10x faster and thinking maybe 1.2x faster. Most of this job is thinking, so the gain is smaller than the demos promise, but it's real. The actual shift is that you spend less time producing code and more time specifying and verifying it.

The loop: plan, code, test, deploy, with a verify after each

This isn't a new lifecycle. It's the four stages you already know, with an assistant plugged into each, plus one step that matters more than anything else: verify after every stage.

Skip it and errors compound forward. A made-up requirement in planning becomes a wrong function, then a test that certifies the wrong function is correct, and now it's in production. Catching it in planning costs a minute. Catching it after release costs an afternoon.

The other thing that decides whether this works is how much you ask for at once. Ask for "a user management system" and you'll get 800 lines you'll skim and merge without understanding. Ask for "the POST /users handler, validation only, no database calls yet" and you get 40 lines you can read properly in two minutes.

If you can't verify the output in five minutes, you asked for too much.

Context is the whole game

Your assistant isn't stupid; it's working blind. It can't see your repo conventions, your team's decisions or your production constraints. Almost everything it gets wrong for apparently silly reasons traces back to something you knew and it didn't.

Write the project rules file once. Most AI coding tools read one from the repo root: stack, conventions, and things not to do. Ten minutes writing it saves correcting the same three mistakes forty times.
Paste the real thing. The actual schema, real sample data, the exact error message, not a paraphrase from memory.
Say "follow the patterns in this file". Paste an existing file from your codebase. A real file is unambiguous where a prose style guide is vague, and this one habit kills most convention drift.

Plan: ask for questions before you ask for a plan

Planning doesn't feel like progress, which is why engineers skip it. It's also the stage that pays best. Don't ask for a plan; ask to be interrogated:

I need to add recurring expenses to our tracker. Before proposing
anything, ask me the 7 questions that most affect the design. Number them.

You'll get questions like: what happens to future instances when someone deletes the series? What does a monthly rule starting on the 31st do in February? Some you'd have thought of. Some you'd have discovered three days later with the feature half-built.

Then harvest edge cases, grouped by data model, calendar and timezone, user actions and failure recovery, and triage them yourself into now, later and not doing. That triage is your spec. Ask for trade-offs, never for the answer: you know your team and your deadline, and the model doesn't.

On a CSV export feature, this step surfaced CSV injection: a user names an expense =HYPERLINK("http://evil.com","click"), a colleague opens the export in Excel, and it's a live formula. One prompt, forty seconds, and a vulnerability that never shipped.

Code: small batches, and read before you run

Match the mode to the task. Autocomplete is for the line you were already going to write. Chat is for self-contained units such as one function or one migration. Agents are for mechanical sweeps with a clear finish line, like a rename everywhere. The more judgement a task needs, the smaller the batch should be.

Work skeleton first: types, signatures and file layout before any function bodies. Then implement one unit at a time: implement, read, run, and commit at every green point. Scan every block that comes back with four questions:

Does it do what I asked, not what it decided would be nicer?
What did it invent? Check that every import and method actually exists.
Where does it break? Empty, null, zero, negative, enormous, concurrent.
Can I explain every line? If not, learn it or delete it.

And follow the two-strikes rule. If the second attempt at the same fix didn't work, the third won't either. The model has locked onto a wrong mental model, so reset with a fresh framing or write it yourself.

Test: from the spec, never from the code

This is where AI helps most and hurts most. The mistake almost everyone makes once: pointing it at finished code and saying "write tests for this". It reads your implementation and writes tests asserting that your code does exactly what it currently does, bugs included. You now own a suite that locks your bug in place, plus a green CI badge.

The fix is one sentence: generate tests from the rules, and keep the implementation off-limits.

Write tests for calculateRefund. Here are the business rules — do not
look at my implementation.
- Full refund if cancelled more than 7 days before the event
- 50% between 7 days and 24 hours
- Nothing within 24 hours
- The 2% processing fee is never refunded
- Refunding an already-refunded booking throws AlreadyRefundedError
Include boundary cases at exactly 7 days and exactly 24 hours,
with hardcoded expected values.

"Hardcoded expected values" matters more than it looks. If a test computes the expected number with the same formula as your code, it's running your bug twice and checking the results match.

Before you trust a suite, sabotage it: flip a comparison operator on purpose. If nothing goes red, your tests are decoration. And read the assertions: expect(save).toHaveBeenCalled() tells you far less than expect(saved.amount).toBe(4750).

Deploy: self-review first, and say what to skip

Run the AI over your diff before you ask a human, and tell it what to ignore:

Review this diff as a senior engineer on this codebase. Focus on unhandled
error paths, race conditions, N+1 queries and missing validation. Cite line
numbers. Skip style nits — the linter handles those.

Leave out the last line and you'll get twelve comments about variable names and nothing about the transaction you forgot to roll back. The AI is good at null paths, off-by-one errors and unclosed resources. It's blind to whether the feature is a good idea, which is why humans stay in the loop.

PR descriptions and changelogs are free money. In incidents, ask for ranked hypotheses plus a command that would confirm or rule out the top one, rather than an answer. And scrub emails, tokens and account IDs from logs before pasting them anywhere.

Originally published on TechBrosEbooks.

This article is adapted from the ebook The Vibe Coding Playbook, which includes a complete 90-minute worked example building a CSV export end to end, guardrails for security, licensing and privacy, the anti-patterns that waste the most time, a 30-day plan, a prompt library and a pre-merge checklist.
~

~

~

Vibe-Coded App Security: A 30-Minute Pre-Launch Checklist

Tech Geeks — Fri, 06 Nov 2026 17:50:41 +0000

You built an app with Lovable, Bolt, Cursor or Claude Code. It works, people are signing up, and it might be leaving the doors wide open.

AI writes code that works, not code that's safe, unless you specifically ask. Security is the invisible part of the spec, so the AI skips it, and so do you, because you can't see what's missing.

How often this goes wrong

Veracode's 2025 GenAI Code Security Report found that 45% of AI-generated code samples contained an OWASP Top 10 flaw, and the number barely moved between older and newer models. (Veracode sells security products, so treat it as directional.) A scan of 5,600 public vibe-coded apps found 2,038 high-impact vulnerabilities and 175 instances of exposed personal data, all live in production.

The incidents are concrete. In CVE-2025-48757, more than 170 Lovable apps had Supabase databases anyone could read, because Row Level Security was never turned on. In July 2025 the Tea app left about 72,000 images in an unsecured storage bucket. The same month, an AI agent on Replit deleted a live production database during a code freeze.

None of these were exotic attacks. They were basic, preventable mistakes. Here are the ones that matter most, and how to check each in minutes.

1. Secrets in the browser

An API key is a password. If it ends up in code that runs in someone's browser, it's public.

The trap: NEXT_PUBLIC_ (Next.js) and VITE_ (Vite) prefixes deliberately bake a variable into the public JavaScript bundle. AI tools often add that prefix to a secret to fix an "undefined" error, and now your Stripe secret key ships to every visitor. The Supabase service_role key deserves special fear: it bypasses every Row Level Security rule.

Check it: open your live site, press F12, go to Sources, and search the loaded scripts for sk_live, service_role, secret and AKIA. Any match is exposed. Move it to a server-only variable used from a server route, then rotate the key. Deleting it from the file doesn't remove it from git history, and bots scrape public bundles within minutes.

2. Database rules that were never turned on

With Supabase and Firebase, your app talks to the database straight from the browser using a public key. The only thing stopping a stranger from reading everyone's data is access rules you must enable yourself. This is the single most common vibe-coding breach.

Supabase: security comes entirely from Row Level Security (RLS), and tables have it off by default. Dashboard → Database → Security Advisor flags every table without it.
Firebase: "test mode" (allow read, write: if true) lets anyone read and write everything. Replace it before launch.

A subtle trap on both: "logged in" is not "owns this data". A policy of USING (true), or a rule that only checks request.auth != null, lets any signed-up user (including an attacker who just registered) read everyone's rows. Scope every rule to the user, for example (select auth.uid()) = user_id.

Test it: create two accounts, sign in as A, and try to fetch B's row. If data comes back, the policy is broken.

3. "Change the ID in the URL" bugs

Authentication is who you are; authorization is what you may touch. AI apps usually get login roughly right and authorization badly wrong.

The most common flaw is IDOR (also called BOLA), number one on the OWASP API Security list. Your app has /api/invoices/1042; change it to 1043 and you see someone else's invoice, because the server checked you were logged in but never that the invoice was yours. Hidden buttons and random UUIDs don't fix this. Anyone can call an endpoint directly, and IDs leak in emails and shared links.

Check it: log in as user A, find a request like /api/orders/1001 in the Network tab, copy it as cURL, change the ID, and re-run it. Fix it by putting ownership inside the query (WHERE id = :id AND user_id = :currentUser) and returning 404 when the record isn't theirs.

4. Payments the browser controls

Two money mistakes dominate AI-built apps:

Client-set prices. If checkout sends an amount from the browser, an attacker changes it to 1. Decide prices on the server from a fixed price ID.
Unverified webhooks. Your payment provider tells your server "payment succeeded" by calling a public URL, so anyone can POST a fake event to it. Verify the signature on every webhook, using the raw, unparsed request body, and reject anything invalid.

Then check subscription status server-side before unlocking paid features, not a flag the browser set once.

5. Unescaped input

React escapes text safely, but dangerouslySetInnerHTML turns that protection off. Veracode found AI tools failed to defend against cross-site scripting in 86% of relevant cases. Sanitise with DOMPurify, or avoid raw HTML entirely. Use parameterised queries, never SQL glued together from strings. And if your app passes user text to an LLM, never let the model's output trigger a sensitive action without a separate check.

6. Endpoints with no limits

If an endpoint that costs you money (an AI call, an email, a signup) has no limit, someone will hit it thousands of times. AI tools almost never add limits unless asked. Layer your defences:

Set a hard monthly spend cap in your AI provider's dashboard. Do it today.
Rate-limit per user ID, not just per IP.
Require login before any expensive action.
Cap input size. A rate limit controls how often, not how expensive one request is.

7. Packages that don't exist

AI sometimes recommends packages that don't exist, and attackers register those exact names with malware ("slopsquatting"). A USENIX Security 2025 study found 19.7% of packages recommended across 16 models didn't exist. Look up every AI-suggested package on npmjs.com before installing it, and run npm audit on what you already have.

The 30-minute pre-launch routine

You don't need a pentest to catch the big stuff:

Search your bundle for sk_live and service_role (5 min)
Enable RLS everywhere; kill Firebase test mode (5 min)
Run the two-account IDOR test (5 min)
Confirm webhook verification and server-side prices (3 min)
npm audit, and check for hallucinated packages (3 min)
Check your grade on securityheaders.com, and make sure errors hide stack traces (3 min)
Set a hard AI spend cap (3 min)
Confirm backups work and dev and prod are separate (3 min)

Run it before every launch and after any big change. If you handle health, financial or children's data, or a breach would sink the business, pay a professional for a review as well. And if you're in India, note that the DPDP Act requires reporting personal-data breaches to the Data Protection Board and affected users.

Originally published on TechBrosEbooks.

This article is adapted from the ebook Ship Your Vibe-Coded App Without Getting Hacked, which gives a detection step and a copy-paste fix-it prompt for every risk above, plus security headers, CORS, supply-chain tooling, and a full walkthrough auditing a sample Lovable + Supabase + Stripe app.