DEV Community: James Anderson

Most 'AI Agents' Are Just If-Statements in a Trench Coat

James Anderson — Tue, 08 Sep 2026 06:12:03 +0000

I built an agent last year, and I was proud of it.

It had a planner. It had tools. It had a reasoning loop that decided what to do next, reflected on its own output, and chained steps together to get real work done. In the demo, it was genuinely impressive — the kind of thing that makes a room go "ooh."

Then it went to production, and it was slow, expensive, and failed in ways I couldn't reproduce. Same input on Tuesday, different behavior on Wednesday. When it broke, the cause was three "autonomous decisions" upstream that I didn't control and couldn't see.

So I did the unglamorous thing: I rewrote it as a boring, linear pipeline. Fixed steps. No reasoning loop. And it was better on every axis that mattered — faster, cheaper, testable, debuggable.

Then I looked at the logs from the old "agent" and felt slightly sick. It did the same three steps every single time. Extract, transform, respond. Every run. It never once used its precious autonomy to do anything different. I had built a for-loop, given it a system prompt, and called it an agent.

I don't think I'm alone. I think most of what's being called an "agent" in 2026 is a pipeline in a trench coat — and I want to make the case that this is not an insult. It's a relief.

What an "agent" actually is (the definition nobody pins down)

"Agent" has become one of those words that means everything and therefore nothing. So let me pin the one distinction that actually matters, because the whole argument rests on it.

An agent decides its own control flow at runtime. Which tool to call, which step comes next, whether to loop again, when to stop — the model chooses the path, dynamically, based on what it sees.

A pipeline has that control flow fixed by you, at design time. Step one, then step two, then step three. Same path every time. The LLM does work inside the steps, but it doesn't get to choose the steps.

That's the entire difference. And here's the part people skip: an LLM doing something smart inside a fixed step is not agency. Extracting fields, classifying a ticket, generating a summary — that's just using an LLM. It's a smart function call. Agency is specifically when the model is handed the steering wheel and gets to pick the route.

Most "agents" never actually hand over the wheel. They just narrate the fixed route in fluent natural language and call the narration "reasoning."

The test: can you draw the flowchart in advance?

Here's the one-line litmus that does all the work:

If you can draw the flowchart of what your system does before it runs, you don't have an agent. You have a pipeline.

Sit with your "agent" for a second. Step 1: it retrieves some context. Step 2: it calls a tool. Step 3: it formats a response. Could you have drawn that on a whiteboard before writing a line of code? Then it's a pipeline. The model isn't deciding the path — you already decided it. The model is just doing the work at each node while sounding like it's deciding.

You only need real agency when the flowchart genuinely cannot be drawn ahead of time — when the next step depends on discovering something you couldn't have known in advance. That's rare. Most business tasks have a shape you already understand. You know the steps. You're just letting the model improvise them, at great expense, for no benefit.

Why the costume is expensive

"Fine," you might say, "so it's technically a pipeline. But it works, so who cares?" Here's who cares: everyone who has to run it, pay for it, or debug it at 2 a.m. Pretending a pipeline is an agent has a real bill, and it's itemized.

Nondeterminism. When the model chooses the path, the same input can take different paths on different runs. Great for a demo, miserable in production, because now your bugs don't reproduce. "It worked when I tried it" becomes a permanent state.

Debuggability collapse. When a fixed pipeline breaks, you know exactly which step failed. When an agent breaks, it failed at step 12 because of a decision it made at step 4 that you didn't control and can't easily replay. You're not debugging code anymore; you're doing forensics on a choice.

Multiplied failure surface. Every autonomous decision is another place to go wrong, and the failures compound across steps. A pipeline with five fixed steps has five things to check. An agent that makes five decisions has five decisions each of which can be wrong, in combination, in an order that changes each run.

Cost and latency. A reasoning loop makes many more model calls than a fixed sequence — it thinks, it re-thinks, it reflects, it decides to loop again. You're paying per token to have the model deliberate about a route you already knew.

You can't test it. Regression testing needs a fixed set of paths to test against. An agent, by definition, doesn't have one. So the thing making autonomous decisions in production is also the thing you can't write reliable tests for. Fantastic.

Add it up and the punchline is brutal: you paid all of that — the nondeterminism, the debugging nightmares, the token bill — to let the model decide something you already knew the answer to.

What you actually wanted was a pipeline

Here's the boring thing that wins.

A pipeline is a fixed sequence of steps, with LLM calls at the specific points where a model genuinely adds value, and deterministic control flow that you own. It's reproducible: same input, same path. It's testable: fixed paths mean real regression tests. It's cheap: no reasoning loop burning tokens to re-decide the obvious. It's debuggable: when step 3 fails, you look at step 3.

And here's the thing people miss — the LLM still does all the smart parts. It still extracts, classifies, reasons about content, generates language. You haven't dumbed anything down. You've just stopped letting it improvise the structure of the work, because the structure was never the part that needed intelligence. The structure was the part you already understood.

Look closely at the "agentic" systems that actually work in production and you'll usually find this: a mostly-fixed pipeline with one or two carefully-constrained decision points, not a free-roaming reasoning loop. The good ones minimized the autonomy to the smallest possible surface. They're pipelines that occasionally, deliberately, ask the model to make one bounded choice — not agents that were trusted to run the whole show.

When you do need a real agent

Now let me argue against myself, because "agents are always bad" would be as dumb as "everything must be an agent." Real agency earns its cost — genuinely — in specific cases:

The steps truly can't be known in advance. Open-ended research, exploration, debugging an unknown problem — tasks where the path genuinely emerges from what you find. You can't draw that flowchart because the flowchart is the thing being discovered.
Each step depends on discovering the last. Real multi-hop work: "find the thing, then based on what the thing is, figure out the next thing." If step 2 is genuinely unknowable until step 1 runs, you need something that can decide at runtime.
Branching is unbounded and real — not "an if-statement with three cases," which is just a pipeline with a switch, but a space of possibilities too large to enumerate ahead of time.

If one of those describes your task, build the agent — you've earned it. And even then, the move is to minimize the agency: hard-code everything you can, and reserve the model's runtime decision-making for the one place that genuinely needs it. Autonomy is a cost. Spend it only where it buys something.

The point was never "agents are bad." It's that agency is a cost you should have to justify, and most systems calling themselves agents never justified it — they just liked the word.

Why everyone builds the agent anyway

So if pipelines are cheaper, safer, and more debuggable, why is everyone building agents? Here's the uncomfortable answer: agents are built for the builder, not the task.

An agent demos better. "Watch it reason through the problem autonomously" makes a room lean in; "I wrote a function that calls the model three times" does not. An agent feels like real AI, like the future, like the thing you got into this for. And "agentic" is a resume word and a fundraising word — it signals sophistication in a standup and in a pitch deck in a way "deterministic pipeline" never will.

None of those reasons have anything to do with whether your task needs an agent. They're about how the architecture makes you feel and look. And that's exactly why the boring pipeline is the senior move — because choosing the less impressive thing that actually works, when the flashier thing would've gotten more claps, is the discipline the hype actively punishes. Nobody screenshots your while-loop. Your while-loop just quietly stays up.

The takeaway

"Agent" should be the thing you escalate to when a pipeline provably can't do the job — not the default you reach for because the word sounds advanced.

Start boring. Draw the flowchart. If you can draw it, build the pipeline — fixed steps, LLM calls where they earn their place, control flow you own. Add real agency only at the specific point where a fixed path demonstrably fails, and no further. The system that ships and stays up in production is almost always more boring than the one that wins the demo.

Most of what's being called an agent right now is a pipeline in a trench coat. And I'll say again what I said at the top: that's not an insult. It's a relief. Because a pipeline is the thing you can actually run, test, afford, and debug — and "impressive in a demo" was never the goal. "Still working on Wednesday" was.

Take the coat off. You'll like what's underneath better.

Two questions, and I want both in the comments. First, the fun one: what did you build as an "agent" that turned out to be a pipeline in disguise? And the real argument — where's the line for you? What's the smallest task where you think a genuine agent actually earns its complexity? I suspect we'll all draw that line in a different place, which is exactly why it's worth arguing about.

How to Add AI to an Existing App Without Breaking It

James Anderson — Mon, 07 Sep 2026 05:41:58 +0000

Here's the situation almost every guide ignores.

You have an app. It's in production. It has real users, real uptime, and real tech debt. And now someone — your boss, your users, your own curiosity — wants AI in it.

So you go looking for how to do it, and every tutorial starts the same way: npx create-something, a fresh repo, a blank canvas. None of them tell you how to add AI to the thing you already have — the codebase with paying customers and a pager that goes off when it's down.

That's a completely different problem, and it's the one that actually matters. Adding AI to a greenfield project is easy because there's nothing to break. Adding it to a real app is an exercise in not breaking the thing that already works.

This is that guide. How to bolt AI onto a live codebase as a removable enhancement — carefully, reversibly, without regret.

The one idea everything else follows from: enhancement, not rewrite

Before any steps, internalize this, because it prevents 90% of the mistakes:

You are not rebuilding your app around AI. You are adding AI as a layer on top of the app you already have.

Your existing application is the asset. It works. People rely on it. AI is an addition — reachable through the interfaces you already have, removable if it fails, invisible when it's off. If adding an AI feature requires you to touch your core domain logic, restructure your data model, or make your critical path depend on a model call, stop. You've stopped enhancing and started endangering.

The mental test for every decision below: if this AI feature broke completely right now, would the rest of my app still work? If the answer is ever "no," you've wired it in too deep.

Here's how to keep the answer "yes."

1. Start with one specific pain point — not "add AI"

"We want to add AI" is not a feature. It's a mood. And it's the fastest way to a sprawling, half-finished integration tangled through your whole codebase.

Pick one narrow, real, high-value workflow. Not "make the app intelligent" — something like "auto-summarize support tickets so triage is faster" or "suggest tags for uploaded documents." Specific enough that you could describe it in one sentence, and small enough that you could rip it out in an afternoon if it doesn't pan out.

The first AI feature you add isn't really about the feature. It's about learning how AI behaves in your system — your data, your traffic, your users — on a small surface where a mistake is cheap. Earn the right to expand by shipping one thing well.

Do: choose the smallest workflow with a clear win.
Don't: try to AI-ify the whole product in one release.

2. Put it behind an abstraction layer from day one

The single most common mistake: scattering raw openai.chat(...) calls throughout your codebase. Do that, and the vendor's SDK is now welded into twenty files, model changes ripple everywhere, and there's no single place to add caching, logging, or a kill switch.

Wrap it. The rest of your app should talk to your interface, never the vendor's SDK directly.

// lib/ai.ts — the ONLY place your app touches an AI provider
interface AIProvider {
  complete(prompt: string, opts?: AIOptions): Promise<string>
}

// swap this implementation without touching a single caller
class OpenAIProvider implements AIProvider {
  async complete(prompt: string, opts?: AIOptions) {
    // provider-specific details live here and nowhere else
  }
}

export const ai: AIProvider = new OpenAIProvider()

Now your features call ai.complete(...), not a vendor. That one boundary buys you: swapping providers or models without a refactor, adding caching and rate limiting in one place, mocking AI in tests, and a single choke point where you can log everything or turn it all off. AI touches your app in exactly one file.

3. Isolate it so it can fail without taking the app down

This is the reliability move that separates a safe integration from a time bomb.

An AI call is slow, occasionally fails, and depends on a third party you don't control. So it must never sit in the critical path of something that has to succeed. If your checkout, your login, or your page load waits on a model call, then when the model is slow or down — and it will be — your core app is slow or down too. You've made your most reliable feature depend on your least reliable one.

Isolate every AI call with a timeout and a fallback, and let the feature degrade gracefully to "off":

async function getSummary(ticket: string): Promise<string | null> {
  try {
    return await withTimeout(ai.complete(summaryPrompt(ticket)), 3000)
  } catch {
    return null   // AI is down/slow → feature quietly disappears, app is fine
  }
}

Then the UI treats the summary as optional: if it's there, show it; if it's null, show the normal ticket. The AI feature is a bonus that can vanish, not a dependency that can crash you. Your uptime should never be hostage to a model endpoint.

4. Ship it opt-in, not opt-out

Don't flip AI on for everyone by default. Surprising users with AI behavior they didn't ask for is how you earn a backlash instead of adoption.

Put it behind a feature flag, make it a clearly-labeled option users choose, and release it to a small cohort first — ideally the early-adopter types who want to try it. This does three things at once: it respects users' choice, it limits your blast radius if something's wrong, and it gives you an instant kill switch (turn the flag off, the feature's gone, no deploy needed).

Opt-in also makes the AI identifiable. Users should always know when they're interacting with AI, with clear cues in the UI. "Quietly replaced the thing you trusted with an AI version" is the pattern that generates angry threads. "Here's a new optional AI helper, try it if you like" is the one that generates signups.

5. Keep a human in the loop early

For anything consequential — anything that sends, charges, deletes, or is shown to a customer as fact — start with the AI proposing and a human confirming. Draft-and-approve, not act-directly.

You relax this as your confidence grows and the data backs it up. But you start with the brake on, because early on you don't yet know how the model behaves on your real inputs, and the cost of finding out in production — on a live customer — is high. Human-in-the-loop early is how you gather that evidence safely. Reduce the hand-holding when the metrics earn it, not before.

6. Instrument cost and behavior before you scale

AI has a failure mode ordinary features don't: it can be quietly expensive and quietly drifting, and you won't see either without instrumentation. Add it up front:

Cost per feature, not just total spend. Track tokens/cost tagged to this feature, so you know what it actually costs to run — and catch a runaway before it's a bill.
Latency. Model calls are slow and variable; watch p95, not just averages.
Input/output logging. So when it does something weird, you can see what it was given and what it returned. (Log shape and metadata, not raw sensitive data.)
Pin the model version. Don't let "latest" silently change your feature's behavior overnight. Pin a version and upgrade deliberately.

You want to know what this costs and how it behaves on real traffic while it's still small — because those numbers are your evidence for whether it's safe to widen.

7. Canary the rollout — widen only when the metrics hold

You already have the flag from step 4. Now use it as a dial, not a switch.

Small cohort first. Watch the numbers that matter: output quality, cost per request, latency, complaint rate. If they hold, widen — 5%, 25%, 50%, everyone — in stages, pausing to check at each step. If a metric goes bad, you roll back one feature flag, not your whole app. That's the entire point of everything above: when something goes wrong, the blast radius is one optional feature, contained behind one flag, not a core system you now have to hotfix under pressure.

The teams who get burned are the ones who went from "works on my machine" to "on for 100% of users" in one step. Don't. Let real traffic earn each expansion.

8. Treat the AI's output as untrusted input

Here's the through-line that ties this to everything: "it returned something" is not "it returned something correct, safe, or well-formed." The AI is the least trustworthy component in your new pipeline, and you should treat its output exactly like user input from a stranger — because in terms of trust, that's what it is.

Before an AI response reaches a user or your database:

Validate the structure. If you asked for JSON, parse and schema-check it before using it — don't assume it came back well-formed.
Check it against your constraints. Is it in range? Does it reference real entities? Does it violate a rule your app enforces?
Sanitize before display or storage. AI output can contain injection payloads, broken markup, or content you don't want rendered raw. Escape and clean it like any untrusted string.

The insecure and the buggy versions of AI integration have the same tell: the output looked fine and got used without a check. A validation step between "the model responded" and "we acted on it" is the cheapest insurance you'll buy.

The pattern underneath all eight

Look back and every step is really the same instinct: contain the risk.

One pain point (small surface). An abstraction layer (one touch point). Isolation with fallback (can't crash you). Opt-in behind a flag (limited blast radius, instant off). Human-in-the-loop (a brake while you learn). Instrumentation (see it before it hurts). Canary rollout (widen only on evidence). Output validation (don't trust it blindly).

The teams who successfully add AI to an existing app aren't the ones who rebuilt around it. They're the ones who treated it as a removable enhancement — bolted on behind an interface, behind a flag, with a fallback and a kill switch, verified on the way out. Your existing app is the thing that pays the bills. AI is an addition, and no addition should ever be allowed to take down the foundation.

Add it the way you'd add any risky third-party dependency: behind a boundary, behind a flag, with a fallback, and with your finger on the switch that turns it off. Do that, and the worst case isn't an outage — it's a feature you quietly disable while you figure out what went wrong. That's the whole goal: make AI something that can fail safely, so you can add it confidently.

What broke the first time you added AI to a real, existing app — and what do you wish you'd isolated before you shipped it? The "it was fine until traffic hit it" stories are the ones worth trading. Drop yours below.

The Next.js Security Checklist for AI-Generated Code

James Anderson — Sun, 06 Sep 2026 05:10:45 +0000

Here's a number that should make you check your own repo: an audit of over 200 vibe-coded applications in early 2026 found that 91.5% contained at least one security vulnerability traceable to AI-generated code. And AI-assisted commits leak secrets at more than double the rate of human-written code.

One security researcher summed up the problem better than any statistic. In a 67-line demo app, the AI he prompted produced: hardcoded JWT secrets, MD5 password hashing, tokens that never expire, an XSS hole, and zero rate limiting — "all in a working application that looks completely normal to a non-security person."

That last phrase is the whole problem. The code works. The demo runs. Nothing errors. And it's full of holes, because the dangerous security mistakes don't announce themselves — they look exactly like success until someone finds them.

Why AI generates insecure Next.js code specifically

Before the checklist, understand why this keeps happening, because it's not random.

An AI coding assistant learned from years of public code — tutorials, Stack Overflow, starter repos. And in that training data, the insecure pattern is often the most common pattern. The most-upvoted JWT tutorial stores the token in localStorage. The quickest auth example checks the session only in middleware. The fastest way to pass data to a component is to hand it the whole database row.

So when you ask AI to "add authentication," it doesn't give you the secure approach — it gives you the popular one, because popular is what it saw most. And Next.js makes this especially dangerous, because server and client code live in the same files, so a single wrong line can quietly ship a secret to the browser or leave a route completely unguarded.

Here's the checklist. Keep it open during your next AI-code review.

☐ 1. Don't trust middleware for auth — it's a UX layer, not a security boundary

This is the biggest one, and it's backed by a string of 2026 CVEs. Attackers found repeated ways to bypass Next.js middleware entirely — with a crafted x-middleware-subrequest header (CVE-2025-29927), with dynamic route parameter injection (CVE-2026-44574), with Turbopack's separate request pipeline (CVE-2026-45109). Next.js 16 even renamed middleware.ts to proxy.ts specifically to signal: this is a routing/UX layer, not a security gate.

AI generates middleware-only auth constantly, because that's the tutorial pattern. It looks airtight and is trivially bypassable.

// ❌ BAD — AI's favorite pattern: middleware is the ONLY gate
// proxy.ts (or middleware.ts)
export function proxy(request: NextRequest) {
  const token = request.cookies.get('session')?.value
  if (!token) return NextResponse.redirect(new URL('/login', request.url))
  return NextResponse.next()
}
// The Route Handler below assumes it's protected. It isn't.
// An attacker who bypasses the proxy layer hits it directly, unauthenticated.

The fix: middleware/proxy can do the fast UX redirect, but every Route Handler and Server Action must independently verify identity at the data layer — the one place that can't be bypassed at the network level.

// ✅ GOOD — verify auth where the data actually lives
// app/api/orders/route.ts
import { verifySession } from '@/lib/auth'

export async function GET() {
  const session = await verifySession()      // checks + cryptographically verifies
  if (!session) return new Response('Unauthorized', { status: 401 })

  // now safe to use session.userId
  const orders = await getOrdersForUser(session.userId)
  return Response.json(orders)
}

Verify: for every protected route, ask "if someone hits this directly, bypassing the proxy, are they still stopped?" If the only check is in middleware, the answer is no.

☐ 2. Every `"use server"` Server Action is a public POST endpoint

AI treats Server Actions like internal functions you call from a component. They aren't. Every Server Action is a publicly callable POST endpoint — anyone can invoke it directly, with any arguments, regardless of what your UI does. This is the exact class of bug that hit a major vibe-coding platform in 2026, where a handful of API calls from a free account reached any other user's data.

// ❌ BAD — no auth, no ownership check. Anyone can call this with any id.
'use server'
export async function deleteProject(projectId: string) {
  await db.project.delete({ where: { id: projectId } })
}

// ✅ GOOD — auth + ownership + validation, inside the action
'use server'
import { verifySession } from '@/lib/auth'
import { z } from 'zod'

export async function deleteProject(projectId: string) {
  const session = await verifySession()
  if (!session) throw new Error('Unauthorized')

  const id = z.string().uuid().parse(projectId)          // validate input

  const project = await db.project.findUnique({ where: { id } })
  if (project?.ownerId !== session.userId) throw new Error('Forbidden') // ownership

  await db.project.delete({ where: { id } })
}

Verify: treat every "use server" function as a public API endpoint. Does it check who is calling, whether they own the thing, and whether the input is valid — every time?

☐ 3. Never store tokens in localStorage — use httpOnly cookies

The single most common Next.js auth mistake, and AI's default because it's the classic tutorial pattern. Anything in localStorage is readable by any JavaScript on the page — including a script injected by an XSS attack, which can then exfiltrate every user's session.

// ❌ BAD — readable by any script, including an XSS payload
localStorage.setItem('token', userToken)

// ✅ GOOD — httpOnly cookie, set server-side, invisible to JavaScript
// in a Server Action or Route Handler
import { cookies } from 'next/headers'

export async function setSession(token: string) {
  const cookieStore = await cookies()
  cookieStore.set('session', token, {
    httpOnly: true,   // JavaScript cannot read it → survives XSS
    secure: true,     // HTTPS only
    sameSite: 'lax',  // CSRF protection
    path: '/',
    maxAge: 60 * 60 * 24, // keep sessions short
  })
}

"But my Client Component needs the user's name!" — it doesn't need the token for that. Expose a small /api/me Route Handler that reads the cookie server-side and returns only the fields the client needs.

Verify: search your codebase for localStorage.setItem near anything auth-related. If a token lives there, move it to an httpOnly cookie.

☐ 4. Don't leak secrets to the client

Two ways AI ships your secrets to the browser, both silent. First, the NEXT_PUBLIC_ prefix — it publishes a variable to the client bundle. AI sometimes slaps it on things to "make them work," including secrets. Second, using a server-only secret inside a component that's actually a Client Component.

// ❌ BAD — NEXT_PUBLIC_ ships this straight to the browser bundle. Forever.
const stripeSecret = process.env.NEXT_PUBLIC_STRIPE_SECRET_KEY

// ❌ BAD — a secret referenced in a "use client" component leaks too

// ✅ GOOD — server-only secret, never prefixed, guarded by the server-only package
// lib/payments.ts
import 'server-only'   // build-time error if this is ever imported client-side

const stripeSecret = process.env.STRIPE_SECRET_KEY   // no NEXT_PUBLIC_

Add a build-time guard so a secret can never accidentally go public:

// lib/env.ts — fail the build if a server secret got a NEXT_PUBLIC_ prefix
const serverOnly = ['STRIPE_SECRET_KEY', 'JWT_SECRET', 'DATABASE_URL']
for (const key of serverOnly) {
  if (process.env[`NEXT_PUBLIC_${key}`]) {
    throw new Error(`Security error: ${key} must not be NEXT_PUBLIC_`)
  }
}

And run a secret scanner — gitleaks or trufflehog — against your repo and your built client bundle. Anything in client-side JavaScript is public. (29 million hardcoded secrets were found on GitHub in 2025, and most never get rotated.)

Verify: grep for NEXT_PUBLIC_ and confirm not one of them is a secret. Scan the built bundle for anything that looks like a key.

☐ 5. Watch the `"use client"` boundary — it's contagious

Here's the architectural root cause behind mistakes 3 and 4. The moment you put "use client" at the top of a component, every component it imports becomes client code too — it all ships to the browser. AI sprinkles "use client" around liberally to make hooks work, and can accidentally drag secret-touching or data-fetching logic across the line into the browser.

// ❌ BAD — "use client" at the top, then secret-touching code below it
'use client'
import { db } from '@/lib/db'   // this now tries to ship to the browser

export function Dashboard() {
  const data = db.query(...)     // server logic stranded on the client
  // ...
}

The fix is a mental model: server by default, client only at the leaves. Keep "use client" as far down the tree as possible — only the tiny interactive bit (a button, a dropdown) needs it. Fetch data and touch secrets on the server; pass down only what the UI needs.

// ✅ GOOD — server component fetches; a small client leaf handles interaction
// Dashboard.tsx (server component, no "use client")
import { verifySession } from '@/lib/auth'
import { LikeButton } from './LikeButton'   // the only client piece

export default async function Dashboard() {
  const session = await verifySession()
  const data = await getDashboardData(session.userId)   // safe, server-side
  return <LikeButton count={data.likes} />               // pass only what's needed
}

Verify: how far down your tree does your first "use client" sit? If it's on a layout or a big parent, you're shipping more to the browser than you think.

☐ 6. Don't over-pass data from server to client

A subtle leak AI causes constantly: a Server Component fetches a full record and passes the whole object as props to a Client Component — shipping every field to the browser, even ones the UI never displays.

// ❌ BAD — SELECT * then hand the whole row to the client
const user = await db.user.findUnique({ where: { id } })  // includes passwordHash, role, internal flags
return <Profile user={user} />   // all of it is now in the browser's payload

// ✅ GOOD — select only what's needed, pass only what's shown
const user = await db.user.findUnique({
  where: { id },
  select: { displayName: true, avatarUrl: true },   // nothing sensitive
})
return <Profile user={user} />

Treat the server→client props boundary as a trust boundary. If a field crosses it, assume it's public.

Verify: for every object passed from a Server Component to a Client Component, check what's actually in it. SELECT * into props is a leak.

☐ 7. Validate input on the server — always

AI loves to validate on the client ("the form checks it!") and treat that as done. Client validation is a UX nicety; it's trivially bypassed by anyone hitting your endpoint directly. Every Server Action and Route Handler must validate its input server-side, or you're open to injection, SSRF (a real 2026 Next.js CVE class), and malformed-data bugs.

// ❌ BAD — trusts the client to have sent well-formed, safe data
'use server'
export async function updateEmail(email: string) {
  await db.user.update({ where: { id: currentUser() }, data: { email } })
}

// ✅ GOOD — validate on the server with a schema, treat all input as hostile
'use server'
import { z } from 'zod'

const schema = z.object({ email: z.string().email() })

export async function updateEmail(raw: unknown) {
  const { email } = schema.parse(raw)   // throws on anything unexpected
  const session = await verifySession()
  if (!session) throw new Error('Unauthorized')
  await db.user.update({ where: { id: session.userId }, data: { email } })
}

Verify: for every server entry point, is the input parsed and validated on the server, or is it trusting the client?

☐ 8. Get the JWT details right (AI gets them wrong)

If you're hand-rolling JWTs — which AI will happily do — here are the specific things it botches:

Hardcoded or weak secret. AI writes const SECRET = 'mysecret'. Generate a real one: openssl rand -base64 32, and keep it in env, not source.
alg: none or accepting the token's own alg. Always pin the algorithm on verification.
Tokens that never expire. Always set exp. Short-lived access tokens, longer refresh tokens.
Weak password hashing. AI still reaches for MD5/SHA-1. Use argon2id or bcrypt. MD5/SHA-1 for passwords is disqualifying.
jsonwebtoken in middleware. It relies on Node crypto, which isn't available in the Edge runtime — it can crash or silently misbehave. Use jose, which works on the Edge.

// ✅ GOOD — jose, works on the Edge, algorithm pinned, expiry enforced
import { jwtVerify } from 'jose'

const secret = new TextEncoder().encode(process.env.JWT_SECRET)

export async function verifyToken(token: string) {
  const { payload } = await jwtVerify(token, secret, {
    algorithms: ['HS256'],   // pin it — never trust the token's own alg
  })
  return payload
}

Also: checking that a cookie exists is not verifying it. A user can set a fake cookie. You must cryptographically verify the token, not just read it.

Verify: is the secret strong and in env? Is the algorithm pinned? Do tokens expire? Are passwords hashed with argon2/bcrypt, not MD5?

☐ 9. Rate-limit your endpoints

AI almost never adds rate limiting. Without it, login endpoints are open to credential stuffing and brute force, and your RSC/Server Function endpoints are open to denial-of-service (two high-severity DoS CVEs shipped for Next.js in 2026, neither requiring auth). Even a simple per-IP limit at the edge dramatically shrinks the attack surface.

Verify: do your login, password-reset, and expensive endpoints have a rate limit? If AI built them, they almost certainly don't.

☐ 10. Patch Next.js — you're probably on a vulnerable version

AI generates code for whatever patterns dominated its training data, and it will not tell you your framework version has known auth-bypass holes. The May 2026 coordinated release alone fixed 13 CVEs, including three auth-bypass vulnerabilities exploitable without any credentials.

Upgrade to a patched version (15.5.18 / 16.2.6 or later).
If you're on Next.js 16, run the official codemod to migrate middleware.ts → proxy.ts: npx @next/codemod@latest middleware-to-proxy. If you haven't, your route protection may be silently inactive.

Verify: what version are you actually on? Check it against the latest advisories before you ship.

The pattern underneath all ten

Look back at the list and notice what every item has in common: the insecure version works. The localStorage auth logs you in. The middleware-only gate passes your tests. The SELECT * returns data. The NEXT_PUBLIC_ secret makes the feature run. Nothing errors. Nothing goes red.

That's the trap of AI-generated security code, and it's the same trap in everything AI builds: "it works" and "it's secure" are different claims, and only one of them shows up in the demo. The AI is confident, the code is fluent, and the hole is invisible until someone finds it.

So the mindset that actually protects you isn't "trust the AI less." It's: you are the verification layer the AI doesn't have. It writes the code; you own the boundary — where auth is checked, what crosses to the client, what a secret is, what gets validated. Those are decisions about your app's trust model, and no model can make them for you, because it doesn't know your threat model. It only knows what code usually looks like.

Run this today

Don't wait for a rewrite. Right now:

Run gitleaks or trufflehog against your repo and your built client bundle.
Grep for localStorage.setItem and NEXT_PUBLIC_ and audit every hit.
Pick your three most sensitive Server Actions and confirm each checks auth, ownership, and input.
Check your Next.js version against the latest advisories.
Keep this checklist open the next time you accept AI-generated code.

The barrier to writing Next.js code has collapsed. The barrier to securing it hasn't moved an inch — and that part is entirely yours.

What's the scariest thing you've caught in AI-generated code — or, worse, shipped and found later? I collect these, and the "it looked completely fine" stories are the ones worth learning from. Drop yours in the comments.

Generating code with AI is easy. Building something that survives its own growth is not — and those aren't the same skill. This article wrote about the workflow that actually holds up for real apps: give AI structure, rules, and context before it writes a line — and

James Anderson — Sat, 05 Sep 2026 08:03:35 +0000

Robert Adamson

Aug 24

Vibe Coding in the New Era: How to Build Real Applications with AI

#ai #programming #vibecoding #softwareengineering

3 min read

AI Is Getting Dramatically More Capable, Fast. Where Is This Heading?

James Anderson — Sat, 05 Sep 2026 07:49:50 +0000

On September 3, 2026, OpenAI released a model called GPT-6 Astra, and one of its senior figures told reporters it's "not unreasonable to feel that we are now in the AGI era."

You can read that as marketing. A lot of it is. But I'd suggest not scrolling past it too quickly, because underneath the swagger, something genuinely worth understanding happened — and the pace at which these moments are arriving has gotten strange enough that it's worth stopping to make sense of it.

So let's do that. Not with hype, and not with the reflexive eye-roll either. Let's actually understand what AGI means, what it means to "measure" an AI's intelligence, what this particular model did, and then think honestly about the only question that really matters: what does any of this mean for us?

First — what does "AGI" even mean?

The term gets thrown around so much it's nearly lost its shape, so let's put it back.

The AI you use today is what researchers call narrow intelligence. It's extraordinary at specific things — writing, coding, summarizing, answering questions — but each capability was trained in, and it operates within the shape of what it learned. It's a spectacularly wide, spectacularly shallow pool.

AGI — artificial general intelligence — is the idea of a system that can handle any intellectual task a human can. Not "better at one thing," but general: able to walk up to a problem it has never seen, in a domain it wasn't built for, and figure it out — the way a capable person can move from cooking to taxes to consoling a friend without being "retrained" for each.

The important, honest catch: there is no agreed definition of AGI, and no official finish line. Ask ten researchers and you'll get ten thresholds. This is exactly why "are we there yet?" produces such heated, unresolvable arguments — people aren't disagreeing about the facts so much as about where the line even is. Keep that in your pocket; it matters for everything that follows.

How do you even measure intelligence in a machine?

Here's a concept most people outside the field don't have, and it's the key to reading every AI headline you'll ever see: we measure these systems with benchmarks — standardized tests.

A benchmark is a big set of problems with known answers — math problems, coding challenges, reasoning puzzles, science questions — that you run the model through to get a score. It's the SAT for AI, basically. When you see "the new model scored 90% on such-and-such," that's a benchmark.

Benchmarks are genuinely useful. They let you compare models and track progress. But here's the thing to hold onto, because it's the honest heart of this whole piece:

A benchmark measures what the AI did on a test. It does not measure what the AI is.

A high score tells you the system produced the right answers under specific conditions. It does not, on its own, tell you the system understands anything, or that it will behave the same way in the mess of the real world, or that it has the thing we mean when we say a person is "intelligent." The map is not the territory, and the score is not the mind. Remember that as we look at what just happened.

What Astra actually achieved (told straight)

Now the news, honestly framed — because the real story is more interesting than either the hype or the dismissal.

Astra didn't just win another leaderboard. It saturated a benchmark called ARC-AGI-3, scoring 99.9%. What makes that notable isn't the number — it's the test. ARC-AGI-3 was designed specifically to be hard for AI: to measure whether a system can reason about genuinely novel problems it couldn't have memorized, the exact kind of flexible, general reasoning that's supposed to separate narrow AI from something more. It was built to stay ahead of the machines. And it got beaten. Astra also saturated a frontier math benchmark and hit 100% on a hard cybersecurity challenge. On paper, these are the strongest results the field has published.

And now the asterisk, because it matters as much as the score. That 99.9% was achieved using a special, expensive setup — a "souped-up harness," extra tooling wrapped around the model to help it work through the task. On the standard setup, the same model scored around 66%. So the headline number reflects "the model plus an elaborate system built around it," not the model alone. And every one of these figures is reported by OpenAI, run at maximum effort, and not yet independently verified by outside labs.

Hold both of these in your head at once, because both are true:

Something real happened. A test built to resist AI got saturated. That's not nothing.
The headline needs reading carefully. The conditions matter, the tooling matters, and "vendor-reported, two days old" is not the same as "confirmed."

The temptation is to collapse into one or the other — "AGI is here!" or "it's all hype." The honest position is the uncomfortable middle: a genuine jump, wrapped in a number you should read with your eyes open.

The thing that actually matters isn't the score. It's the pace.

Step back from any single benchmark, because the truly striking part is the rate.

Look at the trail: over roughly a single year, the field moved through a rapid series of releases, each meaningfully more capable than the last — and benchmarks that were designed to last, to stay ahead of AI for years, are being saturated within months of coming out. The people building the tests to measure the frontier can barely keep the frontier in frame.

That's the part that should make you sit up — not "the AI is smart," but how fast the ceiling is moving. We have gotten used to a cadence where each new model makes the previous one look quaint within a season. Whatever you personally believe about whether this is "real intelligence," the derivative — the speed of change — is the actual headline. Capability is compounding faster than almost anyone's intuitions are updating.

So what does this mean for regular people?

Not robots marching down the street. Something quieter, and realer.

Tasks that used to sit firmly on the "only a human can do this" side of the line are steadily crossing over. Not all at once, not perfectly — but the boundary of what is exclusively ours is being redrawn faster than our jobs, our institutions, and our habits can comfortably adjust to. That's the actual disruption: not a dramatic event, but a boundary quietly moving, month after month.

For how you work, it points at a real shift: the value moves from doing tasks to judging, directing, and verifying what an AI does. When a machine can produce the work, the scarce human skill becomes knowing what good looks like, deciding what's worth doing, and catching it when it's confidently wrong. The doing gets cheap; the judgment gets precious.

And that leads to the most important thing to keep clear, especially in a moment of big scary numbers: capability is not wisdom. A model that saturates a reasoning test still has no stake in the outcome, no skin in your life, no care about whether it's right. It can be brilliant and have nothing at risk. Which is precisely why human judgment gets more important as the tools get more capable, not less — someone still has to be the one who actually cares how it turns out.

Should we think about AI differently now?

Yes — but probably not in the direction the headlines push you.

The shift isn't "start fearing it" or "start worshipping it." It's more mundane and more useful: stop treating each AI release as a gadget update, and start treating AI as a general-purpose capability that will keep expanding into whatever you do.

Most people file each new model under "cool, a better version of the app." The more accurate frame is: this is a capability that has been getting dramatically better on a steep curve, and the right question is no longer "what can it do today?" but "what happens to what I do when this is meaningfully more capable next year — because it probably will be?" You prepare differently for a moving target than for a fixed tool. That's the mental adjustment worth making, and almost nobody has made it yet.

Where is this actually heading?

I'm not going to hand you a fake prediction, because anyone who's certain here is selling something. But I can lay out the honest range.

Maybe we really are near something genuinely general, and the last few years will look like the steep part of the curve right before everything changed. Maybe we're watching benchmarks get saturated while real-world reliability quietly lags behind — the asterisk on that 99.9% is a hint that the demo and the deployment aren't the same thing. Or maybe "AGI" turns out to be less a moment and more a fog we walk into gradually, crossing the line without ever agreeing on where it was.

What's not uncertain is the shape of the curve. It's steep, and it hasn't bent yet. Tests meant to last years are lasting months. Whatever the destination, the travel speed is real.

So the honest stance is neither the hype nor the eye-roll. It's attention. This is one of the few technologies where the future is genuinely arriving faster than the conversation about it — where the thing outruns our ability to make sense of the thing. Paying attention, clearly and without panic, is not a small act right now. It might be the whole job.

What this means for us

Here's where I land.

The question that gets all the airtime — "is this AGI or not?" — is mostly a definitional argument, and it'll never be settled, because we never agreed on the line. It's the wrong thing to fixate on.

The real question is quieter and harder: machines are becoming capable of more and more of what we thought was uniquely, permanently ours — and doing it faster than we're adjusting. What do we do with that? How do we keep human judgment, meaning, and agency at the center while the tools race ahead? How do we use this well instead of just being used by it?

And the thing to hold onto is that we are not passengers watching this happen from the window. The choices about how these systems get built, how they get used, where the guardrails go, and what stays human — those are being made right now, by people. Including, in whatever corner of it you touch, you.

The capability is going to keep coming. Fast. The open question was never really about the machines. It's about what we decide to do while they get more capable — and whether we stay awake enough to decide it on purpose.

When did you first feel the ground shift with AI — the moment it stopped being a novelty and became something you had to take seriously? And honestly: where do you think this is heading? I'd rather hear a room full of thoughtful guesses than one confident prediction.

Agents That Act Need Brakes, Not Just Brains

James Anderson — Wed, 02 Sep 2026 06:05:06 +0000

Here's the moment a lot of us had this year.

You built an agent. It was genuinely impressive — it reasoned, it planned, it called tools, it chained steps together to get real work done. The demo was so good that you gave it real permissions. Let it send the emails. Let it update the records. Let it hit the production API.

And then, one day, it did something you never sanctioned. Not maliciously — confidently. Correctly-looking. It completed the action, reported success, and moved on. You only found out later, when the consequence surfaced somewhere downstream. That's the moment you realize what you actually shipped: a system with a powerful engine and no brake pedal.

The entire industry has spent two years racing to make agents smarter. Better reasoning, longer context, more tools, more autonomy. Almost nobody spent that time making them safe to let act. And those are completely different problems — because the instant an agent stops just talking and starts doing, a more capable agent isn't more useful. It's more dangerous.

Let me make the case, and then talk about the brakes.

Brains and brakes are different problems

This is the distinction the hype cycle blurred, so it's worth stating plainly.

Making an agent capable — able to reason about a task, plan a sequence of steps, pick the right tool, recover from an error — is one problem, and it's the one all the model progress went into. Every new release is a better brain.

Making an agent safe to let act — ensuring that when it's wrong, the wrongness is caught, bounded, reversible, or stopped before it does damage — is a completely different problem. And here's the uncomfortable part: no smarter model solves it for you. A more capable agent doesn't act wrongly less; it acts wrongly faster and more convincingly. Upgrading the brain does nothing for the brakes, because the brakes were never in the model. They're in the architecture around it — the part you have to build yourself.

You can have the best engine in the world. Without brakes, that just means you hit the wall sooner.

Why "acting" changes everything

A chat that only talks has a wonderful property: its mistakes are free. It says something wrong, you read it, you move on. Nothing happened in the world.

The moment an agent can act, three things change, and each one is a reason you need control that a chatbot never did.

Irreversibility. A wrong sentence costs nothing. A wrong DELETE, a sent email, a processed refund, a pushed commit — those don't have an undo button. The cost of an agent's mistake stops being "re-read it" and becomes "clean up the damage," and some damage doesn't clean up.

Confidence is not correctness. This is the through-line of everything I write, and it's never more dangerous than when an agent can act. The agent takes the wrong action with exactly the same confidence it takes the right one. There is no tremor in its voice, no hedge, no tell. It is equally sure when it's about to help you and when it's about to hurt you.

Silent success. The nastiest one. The agent completes the wrong action and reports success — because from its point of view, it did complete the task, it just completed the wrong one. Your monitoring goes green. Nobody sees the wrong customer getting the refund until the wrong customer gets the refund. "It worked" and "it did the right thing" are different claims, and only one of them shows up on the dashboard.

Put those together and you get the core risk: an agent that can act is a system that can cause irreversible harm, confidently, without anything flagging it. That is precisely the situation brakes exist for.

Brake #1: Approval gates — ask before acting

The most basic brake, and the one most systems skip: for anything consequential, the agent doesn't do the thing. It proposes the thing, and waits for a human to say yes.

The skill here isn't gating everything — that would make the agent useless, and you'd click "approve" on autopilot until the gate meant nothing. The skill is gating by blast radius:

Auto-run the trivial and reversible — read something, format something, draft something. No gate needed.
Ask first for the consequential — sending, paying, writing to production, anything a user would want to see before it happens.
Draft-only for the dangerous — the agent prepares the action but cannot execute it at all; a human does.

The mistake to avoid is a uniform gate on everything, which trains people to rubber-stamp. A good gate is rare enough that when it fires, the human actually looks.

Brake #2: A reviewer that can actually say no

The popular pattern: a second agent reviews the first agent's work before it's allowed to proceed. A judge. A critic. It's a good pattern — and it has a failure mode that will fool you completely if you're not watching for it.

A reviewer that has never been seen to fail is indistinguishable from a reviewer that approves everything.

If your judge-agent has been green for months, that is not evidence it's working. It might be catching problems. It might also be rubber-stamping, silently broken, or checking the wrong thing entirely — and you would not be able to tell the difference from the outside, because both produce the same clean logs right up until the day the rubber stamp waves through the thing that hurts you.

A brake you have never tested is not a brake. It's a brake-shaped object. So if you build a reviewer, you have to build the thing that proves it can still say no: wire a known-bad action through the live review path, on a schedule, and confirm the reviewer rejects it. Surface when it last refused something, the way you'd surface uptime. If that "last refusal" date goes stale, your brake may already be dead and you just can't see it yet. Prove the "no" is reachable, continuously, or you're trusting a checker you've never watched work.

Brake #3: An audit trail — know what it did

You cannot control what you cannot see. If an agent takes actions in the world and you have no durable record of what it did, when, and why, then a wrong action is invisible until its damage surfaces on its own schedule — usually the worst possible one.

Every consequential action should leave a trail: what the agent did, what triggered it, what it was acting on, and ideally a path to reverse it. This isn't bureaucracy. It's the difference between "we caught the bad action in the log an hour later and rolled it back" and "a customer told us three weeks later, and we have no idea how many others it happened to."

An audit trail is also what makes every other brake improvable — you can't tune a gate or a reviewer if you can't see what got through.

Brake #4: Blast-radius limits — caps and scopes

The final brake is the one that works even when every other brake fails: hard limits the agent physically cannot exceed, enforced by the system rather than by the agent's good behavior.

Rate caps (no more than N actions per minute). Spend caps (cannot exceed $X without escalation). Scoped permissions (can read these tables, can write only that one). Iteration limits (stop after K steps instead of looping forever). These don't depend on the agent reasoning correctly — they're walls, and walls hold whether or not the thing inside them is behaving.

This is where these controls show up in practice, and a few platforms now build them in as first-class features rather than something you bolt on — for instance, Xenition ships approval gates, an audit log, and a second agent that reviews the first's work before it reaches you (disclosure: it's a product I've been exploring). But whether you adopt a workspace that includes them or wire them yourself, the principle is the same and it's not optional: the agent's freedom should be bounded by design, not by hope. You do not want "it usually behaves" to be the only thing standing between your agent and your production database.

The uncomfortable truth: brakes are unsexy

Here's why, despite all of the above being fairly obvious, most agents ship without any of it.

Nobody demos the approval gate. Nobody opens a pitch with their audit log. Brains are exciting — they make the standup go "wow." Brakes are boring — they only ever matter on the day something goes wrong, and if they're doing their job, that day never visibly arrives, so it looks like you built them for nothing.

Choosing to build the unglamorous safety layer before you need it, when there's no incident yet to justify it, is exactly the kind of discipline that doesn't get applause. It's the senior move precisely because it's invisible when it works. The engineer who spent a day on blast-radius limits instead of adding a fifth capability looks slower — right up until the week their agent doesn't drain the account.

The takeaway

Capability has become table stakes. Every model is smart enough now; smartness is no longer where systems differ. Controllability is the differentiator — whether you can let the thing act without lying awake about what it might do.

So the question to ask about your agent isn't "how smart can I make it?" It's "what happens when it's confidently wrong — and can I stop it in time?" If the answer is "I'm trusting it not to be," you don't have an agent you can deploy. You have a liability with a good demo.

Build the brakes. Not because your agent is dumb, but because someday it will be confidently, fluently, irreversibly wrong — and on that day, the only thing that matters is whether you built a way to stop it before you needed one.

What's the one action you'd never let an agent take without a human in the loop? And the more honest question: have you actually wired that gate — and tested that it fires — or are you quietly trusting the agent to behave? I want to hear where people drew the line.
Disclaimer: This article was written with AI assistance and reviewed and edited by me before publishing.

Building With AI When You Don't Know Architecture: A Survival Guide

James Anderson — Tue, 01 Sep 2026 05:53:57 +0000

Let me describe a moment you might recognize.

You had an idea for an app. You didn't know how to build it — not really — but you opened an AI chat, described what you wanted, and it gave you working code. You ran it. It worked. It felt like magic, because it was: something you couldn't have written a month ago was running on your screen.

So you added another feature. And another. And somewhere around the fifth one, things started breaking. You'd fix one thing and two others would stop working. You'd ask the AI to help, but you couldn't even explain what was wrong, because you didn't understand your own project anymore. Every change felt like defusing a bomb with the lights off.

That wall — not the coding — is where most people get stuck. And here's the thing nobody tells you: AI removed the barrier to writing code. It did not remove the barrier to structuring it. Those are two different skills, and the second one is now the one that decides whether you can keep building or watch your project collapse.

This is a survival guide for that exact wall. No computer science degree required. Just the handful of habits that keep an AI-built project from turning into spaghetti — explained for someone starting from zero.

The real problem: you can generate code faster than you can understand it

Before the how-to, understand what's actually going wrong, because it's not what beginners think.

The problem is almost never that the AI wrote bad code. The problem is that AI lets you produce a working app before you understand how it's held together. You're flying a plane you can't see the controls of. As long as the autopilot holds, you're fine. The moment something needs adjusting, you're lost — because you never learned where anything is.

"Architecture" sounds like a scary, advanced, computer-science thing. It isn't. In plain terms, architecture is just the shape of your project — what the pieces are, what each one does, and how they connect. And structure is what makes a project understandable. Understanding is what lets you keep building instead of getting buried.

So this whole guide is really about one goal: stay able to understand your own project as it grows. Everything below serves that.

Survival skill #1: Separate the pieces

This is the single most important habit, so give it the most attention.

Left alone, AI will often cram everything into one giant file — the part that draws the screen, the part that does the actual work, and the part that talks to the database, all tangled together. It runs fine at first. But when everything is in one place, every change risks breaking everything, because nothing is separated from anything else.

The fix is to keep three kinds of things apart:

What the user sees — the interface, the buttons, the layout (often called the "UI" or "frontend").
What the app does — the actual logic, the rules, the calculations (the "business logic").
Where the data comes from — talking to the database or external services (the "data layer").

When these are separated, you can change how something looks without touching what it does, and change what it does without breaking how it talks to your data. Problems stay contained instead of spreading.

The instruction to paste to your AI: "Keep the interface, the business logic, and the data access in separate files. Don't mix them together."

That one sentence, used consistently, prevents more collapses than anything else in this guide.

Survival skill #2: One job per thing

Here's a rule simple enough to hold in your head: a file or a function should do one thing you can name in a single sentence.

If you try to describe what a file does and you have to say "it does this and this and also that" — it's doing too much, and it's going to become a place where bugs hide and changes go wrong.

Small, single-purpose pieces are survivable. You can find them, understand them, and change one without fear. Giant do-everything files are quicksand: the more they hold, the more every edit becomes a gamble.

The instruction to paste to your AI: "Each file and function should have a single, clear responsibility. Split anything that's doing more than one job."

Survival skill #3: One home for each piece of data

This is the number-one way beginner projects break, so watch for it carefully.

As your app grows, the same piece of information — the logged-in user, the items in a cart, whatever — can end up copied and tracked in several different places. Then those copies drift out of sync. One part of your app thinks the cart has three items, another thinks it has one, and your app starts behaving in ways that make no sense and are almost impossible to debug.

The survival rule: every piece of data lives in exactly one place, and everything else reads from that one place. One source of truth. Don't let the AI casually make copies of your state scattered around the project.

The instruction to paste to your AI: "This data should have a single source of truth. Don't duplicate it across components — have everything read from one place."

This one prevents the most maddening category of bug there is: the kind where nothing is technically broken, but nothing agrees.

Survival skill #4: Decide the shape before you generate

Most beginners build like this: think of a feature, ask the AI for it, repeat. The AI improvises structure each time, and the pieces never fit together — because nobody decided how they should.

The shift that changes everything: name the parts of your app before you build them.

Before generating code, write down the main pieces and what each is responsible for. For a simple app that might be: login, user profile, the main dashboard, payments. Four pieces, each with a clear job. That list — knowing your pieces and their responsibilities in advance — is architecture, in plain language. You just did it. It's not scary.

Then build one piece at a time, fully, before moving to the next. Don't ask the AI to generate the whole app in one go — you'll get a tangle you can't understand. Build the login. Get it working. Understand it. Then the next piece.

The instruction to paste to your AI: "Here are the main components of my app and what each does: [your list]. Let's build them one at a time, starting with [X]."

Survival skill #5: Make the AI explain itself

This is the habit that turns building into learning — and it's what slowly turns a beginner into someone who actually understands structure.

Don't just accept the code the AI gives you and move on. Ask it:

"Why did you structure it this way?"
"What does each part do, in plain English?"
"If I change this, what else will be affected?"

You're using the AI as a tutor, not a vending machine. Every answer teaches you a little more about how your project fits together — which is exactly the knowledge you started without. Do this consistently and, project by project, you stop being someone who generates code and become someone who understands it. That understanding is the whole game.

The instruction to paste to your AI: "Before I use this, explain how it's structured and why, in simple terms — I want to understand it, not just run it."

Survival skill #6: Keep it boring and consistent

Two habits bundled together, because they work as a pair.

Boring beats clever. When the AI offers you a slick, complex, impressive-looking solution, ask if there's a simpler one. As a non-architect, the fancy version isn't a win — it's a liability, because complexity you don't understand is complexity you can't maintain or fix. The simple, obvious version is the one you'll still be able to work with next month. Always prefer it.

Consistency beats variety. If you build feature A one way and feature B a completely different way (easy to do across separate AI sessions), your project becomes a patchwork where nothing follows the same rules and everything is confusing. Make new features match the pattern of existing ones.

The instruction to paste to your AI: "Give me the simplest version that works, not the cleverest. And match the structure and patterns already in my project."

Know the warning signs before it's too late

Here's your smoke alarm. When you start feeling any of these, stop adding features and clean up the structure first:

You're afraid to change things because you don't know what will break.
One fix keeps breaking something else, over and over.
You can't find where anything is anymore.
Files keep growing and growing and doing more and more.
You don't understand your own project when you look at it.

None of these mean you failed. They mean your project outgrew its structure — which happens to everyone, including professionals. The difference is that now you know what the feeling means, and what to do: pause, separate the pieces, and get your understanding back before you build more on top.

The takeaway

You do not need a computer science degree to build real software with AI. That barrier is genuinely gone, and it's not coming back.

But "I can generate code" and "I can build something that survives its own growth" are different things, and the gap between them is structure — the one part the AI can't supply for you, because it depends on decisions about your project that only you can make. AI gives you the code. You have to give it the shape.

The good news is that the shape isn't hard. It's a handful of habits: keep the pieces separate, give each thing one job, one home for each piece of data, plan before you generate, make the AI teach you, and keep it boring. Do those, understand what you're building as you go, and you can build far more than you'd think — without watching it collapse.

Start simple. Stay able to understand it. That's survival.
Disclaimer: This article was written with AI assistance and reviewed and edited by me before publishing.

9 Ways Your AI Agent Silently Fails (and How to Catch Each)

James Anderson — Mon, 31 Aug 2026 04:36:46 +0000

Your agent passed its tests. It ran clean in the demo. You shipped it.

Two days later it's confidently returning wrong answers to real users — and your dashboard is still green. No error code. No stack trace. No 500. Nothing to grep for. The workflow completed. The response looks great. It's just wrong, and it has probably been wrong the same way a few dozen times already.

This is the thing nobody warns you about when you move an agent to production: agents don't fail like normal software. A database fails with an error code. An API fails with a 500. A bad LLM call fails with an obviously low-quality response you can spot. An agent fails by completing the task and handing you a confident, well-formatted, plausible answer that happens to be false — and by the time a downstream consequence makes it visible, hours have passed and the root cause is buried.

There's a name for this from classic reliability work: differential observability. The application suffers, but the observer designed to notice reports health. Gray failures. Fail-slow. The monitor stays green while the thing it's watching quietly rots.

Every failure below is a version of that same gap — a place where "success" and "correct" quietly come apart, and your monitoring only sees the first one. Here are nine of them, each with the symptom, why it stays invisible, and how to actually catch it.

1. The HTTP 200 empty payload

Tool calls fail 3–15% of the time in production. The loud failures are fine — a timeout, a 500, an exception you can catch. The silent ones are the killers: a tool returns HTTP 200 with an empty body, a null, or a garbage payload, and the agent treats it as a successful result and proceeds to reason over nothing.

Why it's invisible: everything downstream is technically "working." Status code says success. The agent got a response. Nothing threw.

How to catch it: validate the shape and content of every tool result, not just the status code. Assert the payload is non-empty, matches the expected schema, and contains the fields you're about to use. Treat a 200 with an empty body as a failure, loudly — because to your agent, it's worse than an error, it's a confident lie.

2. The poisoned step (error propagation)

A single wrong tool argument at step 2 silently corrupts step 3, which feeds step 4, and so on. By the time you see a bad final answer at step 20, the actual defect is nineteen steps upstream and nearly impossible to trace. This is repeatedly cited as the single most common and most insidious agent failure mode.

Why it's invisible: each individual step looks locally fine. The corruption is in the hand-off, not any one action, and you're only inspecting the final output.

How to catch it: check intermediate outputs at each reasoning step, not just the end. A guard that validates the state between steps flags the corruption at its origin — where it's cheap to fix — instead of at the final output, where the root cause is hardest to find. Bonus: each intercepted failure becomes a structured record (input state, tool call, what blocked it, which step), which is your audit trail.

3. Goal drift

Over a long, multi-step run, the agent slowly wanders away from the objective you actually gave it. No single step is wrong. Step 8 is a reasonable follow-on from step 7. But the sum of forty locally-reasonable steps ends up solving a subtly different problem than the one you asked for.

Why it's invisible: you're evaluating individual actions, and each one passes. Drift only shows up in the trajectory, which nobody's measuring.

How to catch it: periodically re-anchor to the original goal — literally re-inject the objective and ask "is what I'm doing now still serving this?" Evaluate the whole trajectory against the stated intent, not just the final answer's surface plausibility. Goal drift is a trajectory bug; you can't catch it one step at a time.

4. The amnesiac mid-task (context loss)

The context window fills up during a long run. Tool definitions, key constraints, or facts established early get pushed out of scope. The agent keeps going — now quietly acting without the information it had ten steps ago, and with no idea it lost anything.

Why it's invisible: there's no error when context gets evicted. The agent doesn't know what it can no longer see; it just proceeds on a smaller picture and sounds equally confident doing it.

How to catch it: monitor context budget as a first-class metric, and assert that critical facts (the constraints, the tool schemas, the original ask) are still present at the moment a decision is made. If the thing the agent needs to decide correctly has been evicted, that's a failure state — treat it like one instead of letting the agent guess.

5. The retry loop / runaway cost

A tool call fails, so the agent retries. It fails again, so it retries again. With no hard limit, this runs indefinitely — and it doesn't produce a wrong answer, it produces exploding latency and a cloud bill that can burn through your budget in minutes, before any human notices.

Why it's invisible: there's no incorrect output to catch — the agent is "still working." The damage is in cost and latency, which your correctness monitoring isn't watching.

How to catch it: hard iteration caps and explicit loop detection are non-negotiable. Cap total steps, cap retries per tool, and wire a cost/latency alarm that fires on runtime, not just on errors. An agent that never terminates is a failure even though it never returns anything wrong.

6. Plausible-speech hallucination

When an upstream error leaks into the agent's context, its failure mode is not silence — it's fluent, confident narrative built around the bad data. The agent takes the corrupted input and writes you a beautifully reasoned, completely wrong explanation. This is the failure that's unique to language-generating systems: the output's polish is inversely related to how much you should trust it.

Why it's invisible: fluency reads as competence. A confident, well-structured answer feels correct, and there's no signal in the prose itself that the premise underneath it was garbage.

How to catch it: never treat fluency as a health signal — it carries zero information about accuracy. Add a verification pass that checks the final claims against the actual source data or system state. If the answer asserts something, confirm the something is true before you ship it, no matter how good it sounds.

7. The unsanctioned action

The agent retrieves exactly the right context — and then does something with it that nobody authorized. High retrieval accuracy makes this worse, not better, because the unpermitted action arrives wrapped in real, correct evidence. It looks justified. The chunk was right, the reasoning was clean, the action was never allowed.

Why it's invisible: every quality check passes. Retrieval was accurate, the citation is real, the logic is sound. Nothing in your retrieval or faithfulness evals asks "was the agent permitted to do this?"

How to catch it: enforcement is a separate failure surface from retrieval, and it needs its own gate. "Found the right context" and "allowed to act on it this way" are different claims — and only one of them is what your accuracy metrics measure. Put a deterministic policy check between decision and action, so a perfectly-grounded-but-unauthorized action gets stopped regardless of how good the evidence looks.

8. Cross-surface / cross-model inconsistency

A prompt that worked perfectly on one model behaves differently on another. Your agent gives one answer on the web surface and a different one on mobile, or drifts when you swap the underlying model, and nothing flags the divergence. Users hit different behavior depending on where and when they showed up.

Why it's invisible: each individual response looks fine in isolation. The bug is the inconsistency between responses, which no single-run check can see.

How to catch it: run consistency checks across models and across repeated runs of the same input — if two "equivalent" paths disagree, that disagreement is signal. And pin your deployment: test against the exact model version you actually ship, because "it worked on the old model" is not evidence it works now.

9. The green checkmark that can't fail

The most insidious one, because it's the failure in your safety net itself. Your guard, your eval, your reviewer-agent, your monitor — it's been green for months. Reassuring, right? Except: have you ever seen it go red? A check nobody has ever seen fail is indistinguishable from a check that approves everything. The two produce identical logs right up until the day the rubber stamp waves through the thing that hurts you.

Why it's invisible: silence and health look identical. Most systems model pass and fail and nothing else — so a guard that has quietly lost the ability to fail reads exactly like a guard that simply had nothing to catch.

How to catch it: wire a known-bad case through the live path (not a side test), on a schedule, and surface the timestamp of its last refusal like an uptime number — "last time this guard said no: 2 days ago." If that date ages past the schedule, staleness itself is the alarm. Prove your checks can fail, continuously, or you don't have guards — you have decorations.

One bug wearing nine costumes

Step back and all nine are the same disease.

Traditional systems model two states: pass and fail. Green and red. But every failure above lives in a third state your monitoring doesn't have a name for — unverified, unsanctioned, not-checked-lately, observed-but-not-validated. And because there's no state for it, the absence of a red signal gets read as health. Silence gets read as success.

The fix pattern recurs across all nine, and it comes down to two disciplines:

Existence checks, not just quality checks. Don't only ask "is this output good?" Ask "did I actually observe what I claim to have observed?" A tool returned 200 — did it return data? Retrieval succeeded — was the action permitted? The guard is green — has it ever gone red?
Verify at every boundary, and build a state for "not verified." Check between steps, not just at the end. Confirm claims against source, not against fluency. Make "we haven't validated this lately" a visible, alarming state instead of an invisible assumption.

Agents fail differently from everything you've monitored before, and the tools most teams reach for — log dashboards, error-rate monitors, single-turn observability — are built to catch crashes. Agents don't crash. They succeed, confidently, at the wrong thing.

The 60-second self-audit

Run these against your own agent right now:

When a tool returns 200, do you validate the payload, or just the status?
Do you check intermediate step outputs, or only the final answer?
Is there a hard cap on iterations and retries, with a cost alarm?
Do you verify final claims against source/state, or trust the prose?
Is there a policy gate between "decided" and "acted"?
Can your monitoring actually go red — and when did it last?

If any answer is "uh… I'd have to check," you've found a silent failure waiting to happen.

Which of these has bitten you in production — and, the more useful question, what's the silent failure mode I left off the list? The agents that hold up aren't the ones with the best models; they're the ones instrumented for the failure that doesn't announce itself. I want to hear the ones that fooled you.

Prompt Fluency Is the New Literacy — and Most Devs Are Winging It

James Anderson — Sun, 30 Aug 2026 04:58:39 +0000

I need to confess something, because I suspect you've done it too.

A few months ago I caught myself saying, out loud, "Claude's gotten worse at this." I'd asked it to do something, gotten back something mediocre, sighed, and filed it under the model is slipping. Very satisfying. Nobody's fault but the AI's.

Then I watched a colleague hand the same model a better-formed version of the same request and get back exactly what I'd failed to get. Same model. Same day. Same task. Wildly different output.

It wasn't the model. It was me. I just really, really didn't want it to be me.

That's the whole article, honestly. But let me make the uncomfortable case in full, because I think this is one of the biggest quiet skill gaps in software right now — and almost nobody is treating it like a skill at all.

The model isn't the variable anymore. You are.

For a couple of years, "better results" had an easy explanation: better model. If your output was weak, you waited for the next release, upgraded, and things improved. The tool was the variable, so the tool was the excuse.

That excuse is dead, and it's not coming back.

Everyone has the same frontier models now. You, your teammate, your competitor, the intern — same GPT, same Claude, same Gemini, one click apart. Raw access stopped being an edge the moment it became universal.

So here's the part that stings: when two developers get dramatically different results out of the identical model, the difference has nowhere left to hide. It isn't the model — you both have the same one. It's the operator. It's the ask. It's the fluency.

We're standing in front of the same instrument getting different music out of it, and most of us are still blaming the instrument.

Prompt fluency is the last remaining variable in the equation, and the majority of developers are pretending it's still about the tool — because "the model isn't good enough yet" is a much more comfortable sentence than "I'm not good at this yet."

A vague prompt is a vague thought wearing a question mark

Here's the reframe that changed how I see this, and it's the part most "prompt tips" articles completely miss.

When you get a bad answer from a capable model, your instinct is that you phrased it wrong — that there's some magic wording you're missing, some incantation that unlocks the good output. That's why prompt-hack listicles do numbers: they promise the secret phrase.

But most bad prompts aren't a wording failure. They're a thinking failure.

You didn't fail to express what you wanted. You hadn't actually decided what you wanted — and the model, doing exactly its job, reflected your fuzziness right back at you. A vague request produced a vague result because the request was the vagueness, made visible.

The model is a mirror for the clarity of your intent. If the reflection is blurry, check the thing standing in front of it before you blame the glass.

This is also why "prompt hacks" are mostly a dead end. You cannot trick your way out of not knowing what you're asking for. There's no magic phrase that compensates for an undecided goal, unspecified constraints, and no definition of what "good" looks like. The secret isn't a better spell. It's actually knowing what you want before you ask — which is harder, less fun to tweet, and the entire game.

Winging it has a bill — and you're paying it without seeing the invoice

"Okay," you might say, "but I get there eventually. A few tries and it's fine." Sure. But look at what those few tries actually cost, because the invoice is itemized and most people never read it.

Charge one: the correction spiral. You fire off a fuzzy prompt, get something wrong, and start the "no, not like that" loop — fifteen rounds of nudging the model toward the thing a thirty-second upfront spec would have landed in one. You feel productive because you're actively working. You're just paying in small installments for a bill you could have settled once.

Charge two: the un-reusable result. Even when you finally get something okay, you got it through improvised back-and-forth you couldn't reproduce if you tried. So it doesn't compound. Tomorrow's similar task starts from zero, because you never captured how you got there — only the messy output, not the method.

Charge three — and this is the expensive one: the false verdict. You conclude "the AI just isn't good at this," and you stop. Except it wasn't the AI — it was the spec you handed it. So now you've done real damage: you've permanently underrated a capable tool based on your own unclear ask, and you'll avoid using it for exactly the things it could've handled.

That third charge is the real cost of winging it. It doesn't only waste your afternoon. It teaches you wrong lessons about what these tools can do — and wrong lessons are stickier and more expensive than wasted time.

What fluency actually looks like

None of this is mystical, and that's the good news. Prompt fluency is a set of learnable, boring, repeatable habits — closer to writing a good bug report than casting a spell. Here's the core of it:

State the constraints, not just the goal. "Write a function to parse this" is a wish. "Write a Python function that parses this CSV, handles missing fields by skipping the row, returns a list of dicts, and raises on malformed headers" is a spec. The model can only hit a target you actually drew.

Give it the shape of the answer you want. Format, length, structure, tone. If you know you want a table, or bullet points, or a single function with no explanation — say so. Leaving it implicit means you're gambling, then correcting.

Show it good and bad. One example of what you want and one of what you don't teaches the model more than a paragraph of description. Concrete beats abstract, every time.

Tell it what to do when unsure. "If you don't have enough information, ask instead of guessing" or "say you're not certain rather than inventing" — this alone kills a whole category of confident-wrong output.

Decompose instead of mega-prompting. A giant vague ask produces a giant vague answer. Break it into steps you can verify. You'll debug the pieces instead of the blob.

Capture what worked. This is the one nobody does and everybody should — more on it next, because it's the difference between a skill and a leverage machine.

Notice what's not on this list: clever phrasing, secret keywords, jailbreak tricks. It's all just precise specification. Clear thinking, written down.

The real unlock: fluency is reusable, not per-attempt

Here's where prompt fluency stops being a nice habit and becomes actual leverage — and it's the insight that reframes the whole thing.

Getting marginally faster at improvising prompts is fine. But the real move isn't improvising faster. It's not improvising at all.

A good prompt — a real, precise, constraint-loaded spec that reliably gets you what you want — is worth writing once and keeping forever. Save it. Template it. Turn it into a reusable form. After that, the marginal cost of getting a great result on that kind of task drops toward zero, because you're no longer re-deriving the spec every single time. You're invoking it.

This is the part that turns a 20%-of-the-time-it-works habit into something categorically different. The expensive part of prompting was never the typing — it was the deciding: what do I want, what are the constraints, what does good look like. Do that thinking once, capture it, and every future use is nearly free.

That's the difference between a developer who's a fast typist at the AI and a developer who's built a personal library of specifications that consistently produce reusable output. One is winging it quickly. The other has stopped winging it entirely. Only one of those compounds.

The part that'll annoy some of you

I know how a chunk of readers are reacting right now: prompting isn't real engineering. It's typing English at a chatbot. It'll be obsolete when the models get good enough to read my mind.

I want to gently point out that this exact dismissiveness is the winging-it problem, wearing a lab coat.

"It's beneath me to get good at this" is precisely the attitude that keeps someone improvising, blaming the model, and getting mid results while a colleague who took it seriously runs circles around them with the same tools. The disdain isn't protecting you from a fake skill. It's protecting you from noticing a real one you haven't developed.

And "it'll be obsolete when models improve" gets it backwards. Better models don't make specification less important — they make it more valuable, because a more capable model can execute a precise spec far more completely than a weak one could. The ceiling on what you can get out of these tools is rising, and the thing that determines how close you get to that ceiling is how clearly you can say what you want. That skill isn't going away. It's the one thing that's been appreciating this whole time.

The takeaway

Prompt fluency is quietly becoming table stakes — the way knowing how to search well, or write a clear commit message, or read documentation became table stakes. It's not glamorous. It's not a trick. It's clear specification, made into a habit, and then captured so you don't have to repeat the thinking.

The developers who treat it as a craft are getting consistent, reusable, compounding leverage out of the exact same models everyone else has. The ones winging it are getting inconsistent results and a growing list of wrong conclusions about what AI "can't do."

Same instrument. Very different music. The gap isn't the model anymore. It hasn't been for a while.

What's the one prompt habit that actually moved your hit rate — and, real question: do you save your good prompts, or retype them from scratch every single time? I'm trying to figure out if prompt-capturing is a rare discipline or if I've just been the last one to it.

How a 6B-Active Model Beats 17B-Active Ones: What Qwen3.8-Flash-Next Actually Changed

James Anderson — Sat, 29 Aug 2026 06:07:05 +0000

Here's a number that shouldn't make sense on first read.

Qwen just released Qwen3.8-Flash-Next, and it activates 6 billion parameters per token — while matching or beating models that activate 13B (DeepSeek-V4-Flash) and 17B (Qwen3.7-Plus) per token. On its own base benchmarks, the 6B-active model reportedly tops 8 of 14 tests against much heavier-activating competition.

Your instinct is probably: smaller model, worse results — that's how this works. And for a long time, roughly, it did. So how does a model doing less compute per token beat models doing two to three times more?

The answer isn't a magic benchmark trick. It's a concept most people skip past when they read a model's spec sheet — and once you understand it, a lot of the modern AI efficiency race suddenly makes sense. Let's break it down.

First: the number on the spec sheet that actually matters

When people size up a model, they quote total parameters — "it's a 125B model," "it's a 400B model." That's the number in the headline, and it's almost the wrong one to care about for cost and speed.

There are two numbers, and they mean very different things:

Total parameters — how big the model is on disk. Everything it knows, stored in its weights. This governs how much memory you need to hold the model.
Active (activated) parameters — how many of those parameters actually get used to process each individual token. This governs how much computation happens per token — which drives inference cost, speed, and latency.

For a traditional ("dense") model, these two numbers are identical: every parameter fires for every token. A 27B dense model does 27B parameters' worth of math on every single token, no exceptions. That's simple and predictable — and expensive, because you're paying full price on every token whether the token needs it or not.

The whole trick of the last few years has been breaking that link — building models where total is huge (so the model knows a lot) but active is small (so each token is cheap to compute). That's what lets Qwen3.8-Flash-Next store ~180B parameters on disk yet route each token through only 6B of them.

The one-line version: total parameters tell you how much the model knows; active parameters tell you what it costs to run. For price and speed, active is the number to watch.

What a Mixture-of-Experts actually is

The technique that decouples those two numbers is Mixture-of-Experts (MoE), and it's simpler than the name suggests.

Instead of one giant network where everything fires every time, an MoE splits much of the model into many smaller sub-networks called experts — Qwen3.8-Flash-Next has a pool of 512 of them. For each token, a small component called the router picks just a few experts to actually run. The rest sit idle for that token.

An analogy: imagine a hospital with 512 specialists on staff. When a patient walks in, you don't wake up all 512 — you route them to the two or three relevant specialists. The hospital contains an enormous amount of expertise (that's the total parameter count), but any single patient only consumes a tiny slice of it (that's the active count). You get the knowledge of a huge institution at the cost of a small clinic visit.

That's the core move. The model can be enormous in what it collectively knows, while any given token only pays for the handful of experts it actually needs. Total stays high; active stays low.

The Qwen3.8-Flash-Next numbers, decoded

Source: Qwen team blog (qwen.ai) — vendor-reported figures.

Now the spec sheet reads very differently. Here's the shape of it:

~180B parameters total on disk — a 125B MoE backbone, a 51B N-gram embedding table, and a small (~4B) multi-token-prediction module.
6B parameters active per token — a sparsity of roughly 95%. Nineteen of every twenty parameters sit idle on any given token.
262,144-token native context, extensible to 1M.
Trained, Qwen says, at about one-ninth the cost of Qwen3.7-Plus.

Set that 6B-active figure next to the competition in Qwen's own comparison table: Qwen3.7-Plus activates 17B per token, DeepSeek-V4-Flash activates 13B. Qwen3.8-Flash-Next activates 6B — and still posts wins across a chunk of the coding, agent, and general benchmarks. Reported highlights include 62.5 on SWE-bench Pro, 73.9 on CoWorkBench, and strong multimodal numbers.

The reason a 6B-active model can even compete here is that its 6B is punching above a naive 6B: it's drawing on a 125B pool of specialized experts plus that 51B embedding table. It knows like a big model and computes like a small one. (More on how they pulled the active count down so far in a moment — and more on trusting these numbers near the end.)

Why sparsity is the real story — not the benchmark scores

Source: Qwen team blog (qwen.ai) — vendor-reported figures.

It's tempting to read a launch like this as "Qwen won some benchmarks." That's the boring, forgettable version. The actually-interesting story is the direction:

The industry is proving you can keep pushing the active-parameter count down while capability stays roughly flat.

Think about what that means economically. Inference cost scales with active parameters, not total. So if you can hold quality steady while cutting active params from 17B to 6B, you've made the model dramatically cheaper to run without making it dumber. That's not a better score on a leaderboard — that's a structural shift in the cost of serving AI.

You can see it in the price. The production version is served at $0.15 per million input tokens and $0.47 per million output — the kind of number that's only possible because each token touches so little of the model. Cheap inference isn't a discount here; it's a direct consequence of the architecture.

This is the same theme underneath a lot of recent model releases: the frontier isn't only "make it bigger and smarter," it's increasingly "make it this smart for a fraction of the compute per token." Sparsity is where that race is being run.

The four tricks that make ultra-low activation work

Getting active params down to 6B without tanking quality took more than just "add more experts." Qwen stacked four changes, and you don't need the math to get the idea:

GDN + QSA hybrid attention. Three of every four layers use Gated DeltaNet (GDN), which compresses history into a small fixed-size state — cheap "memory." The fourth layer uses Qwen Sparse Attention (QSA), which selectively pulls the important context back out — precise "retrieval." In Qwen's phrasing: GDN remembers, QSA retrieves. Together they slash the cost of long context, which is normally where attention gets brutally expensive.

N-gram embeddings. A 51B lookup table that adds capacity based on local word patterns — and crucially, it's addressed by deterministic lookup, so it adds almost zero per-token computation and can be offloaded to host memory rather than occupying the GPU. Capacity that's nearly free at inference time.

Gated Residual. The residual stream (how information flows between layers) is widened into parallel branches with dynamic gates, so information moves through the network more effectively — meaning each active parameter does more useful work.

Ultra-sparse MoE. A big expert pool (512) with only a few routed per token, plus one shared expert. More total knowledge, same small active slice.

The common thread: every one of these adds capability without adding much per-token compute. That's the whole game — grow what the model knows, not what each token costs.

What this means for you

A few practical takeaways, whether you're choosing a model or just trying to read the field clearly:

Check active params before total. When you're estimating cost or speed, "125B model" tells you almost nothing useful. "6B active" tells you what a token actually costs to compute. Total size predicts memory footprint; active size predicts your bill.
Cheap-to-serve is becoming a design goal, not an afterthought. Models are now being architected around low activation. Expect more "huge total, tiny active" releases, and expect prices to reflect it.
Small-active doesn't mean small-footprint. Here's the catch worth internalizing (see caveats below): 6B active is cheap to compute, but you still have to hold ~180B parameters somewhere. This is not a model you run on your laptop.

If your mental model was "parameter count = capability = cost," this release is a clean example of why that shortcut breaks. Those three things have come apart, and active params is the thread that connects capability to cost now.

The honest caveats

Every number above comes from Qwen's own evaluations, published at launch and — at the time of writing — not independently reproduced by any outside lab. Treat them as directional claims, not settled facts. Vendor launch benchmarks are run on the vendor's chosen harness against the vendor's chosen baselines; that's not necessarily dishonest, but it's not neutral either. Wait for third-party reproductions before betting anything important on the exact figures.

A few more things worth keeping straight:

This is not "Qwen4." Qwen positions it as an early preview of the architecture that Qwen4 will build on — the same role Qwen3-Next played before Qwen3.5. Some aggregators mislabeled it as Qwen4 itself; it isn't. (Its config file even tags the architecture as experimental.)
Low active ≠ easy to run. The FP8 checkpoint is ~172 GB and the BF16 is ~335 GB. You need serious multi-GPU hardware to hold it — the "6B active" cheapness is about throughput and cost per token, not about fitting it on a workstation.
Sparsity has real tradeoffs. Very sparse MoEs carry the memory cost of all those total parameters, and routing can introduce its own instabilities. "6B active" is a genuine efficiency win, not a free lunch.

None of this undercuts the core point — it sharpens it. The achievement is specifically compute-per-token efficiency, and that's exactly the axis the spec-sheet headline number hides.

The number to watch

If there's one habit to take from this: stop reading total parameter count as the headline, and start reading active parameters.

Total tells you how much a model knows and how much memory you need to hold it. Active tells you what it costs and how fast it runs. For years those numbers moved together, so the shortcut worked. They don't move together anymore — MoE and aggressive sparsity have pried them apart, and Qwen3.8-Flash-Next is a vivid example: ~180B on disk, 6B in motion, competitive with models spending far more per token.

The efficiency race used to be "who has the biggest model." Increasingly it's "who can do the most with the fewest active parameters." That's a quieter number, buried below the headline — and it's the one that's actually going to decide what AI costs to run.

Do you check active params before total when picking a model — or has the headline number been doing your thinking for you? And what's the lowest-active model you've gotten real production work out of? Curious where people have found the floor.

Are AI Tools Actually Making Us Productive — or Just Giving Us Something New to Play With?

James Anderson — Wed, 26 Aug 2026 05:36:22 +0000

I want to describe a completely ordinary hour from my week, because I suspect it's your week too.

A scene from my actual workday

I'm building a small feature. I open my AI chat — Claude on this particular day, but it rotates — and ask it to draft the data model. Great answer. I copy it into my editor. I tab back and ask for the API route. Good answer. Copy, paste, tweak the parts that don't fit my setup. Now I need a quick migration script, so I ask again, copy again, switch again.

Then the feature needs a short doc. Different tab. I paste the AI's summary into the doc, reformat it, fix the parts that assumed things that aren't true. Then someone wants the numbers in a spreadsheet, so I ask the AI to compute them, and I copy the results into the sheet by hand. Then a couple of slides. Another tool. Another paste.

At some point I looked at my screen and counted the tabs. Fourteen. The AI was living in exactly one of them, narrating instructions to me while I ran around being its hands.

The moment it clicked that something was wrong

Here's the thing that nagged at me: none of the individual answers were bad. The AI was smart the whole time. The friction wasn't intelligence — it was that I had become a courier.

I was ferrying the AI's output from the one window it lived in, out to a dozen windows it couldn't touch. It could tell me what the spreadsheet formula should be, but it couldn't put it in the spreadsheet. It could write the doc, but it couldn't be in the doc. Every useful thing it produced had to be carried, by me, across a border it couldn't cross.

And every crossing cost something. Not just the seconds of copy-paste — the thread. Each time I left the chat to go do the thing, I dropped the context. When I came back, I had to re-explain where we were. The AI didn't know I'd tweaked its code, didn't know the doc had changed, didn't know what the last three tools had done. It was brilliant and amnesiac and boxed in, all at once.

That's when the real question landed for me: is this actually making me productive, or is it just a very impressive new thing to play with?

What I was actually missing

It took me a while to name it, but here it is: the AI I use every day is an advisor, not a workspace.

It's phenomenal at telling me things. It is structurally incapable of doing things where I actually work. The whole interaction model is "ask a smart oracle, receive text, go apply the text yourself." The intelligence is real. The hands are missing. And once I saw the gap that way, I couldn't unsee it — because so much of my day is spent being the bridge between an AI that knows and a set of tools it can't reach.

We've spent three years making the oracle smarter. We've spent almost none making it able to act.

The hidden costs I hadn't noticed I was paying

Once I started paying attention, the taxes were everywhere:

Context loss. Every tool switch is a little amnesia event. The chat doesn't follow me into the editor, the doc, or the sheet, so the shared understanding I built up in the conversation evaporates the moment I leave it — and I rebuild it, from scratch, on the way back.

Fragmentation. My actual work — the code, the doc, the data, the slides — ends up scattered across a dozen apps, none of which know about each other, and none of which the AI can see once I've moved things there. The "project" exists only in my head, as the thing holding all those tabs together.

A blind assistant. This is the one that finally got me. The instant I act on the AI's advice, the AI goes blind. It can't watch me apply its suggestion, can't see the result, can't catch that I changed one line and broke the thing three files over. It gave me a map and then closed its eyes while I walked.

None of these are exotic. They're the quiet, everyday texture of using AI tools in 2026 — and I'd stopped noticing them the way you stop noticing a sound that's always on.

"But wait — don't Claude, GPT, and Gemini already do this?"

I want to be fair, because I asked myself exactly this before I let myself complain.

They've all added surfaces. ChatGPT has Canvas (now folded into inline writing and code blocks); Claude has Artifacts; Gemini has Canvas too. These are genuinely nice — instead of dumping a wall of text in the chat, the AI opens a side panel you can iterate on, and Claude's artifacts will even run an interactive app right there. The industry converged on this idea over the last couple of years, and for good reason.

But sit with what these actually are, and the gap becomes clear. Each is one surface, usually one at a time, mostly for writing or code, and largely walled off from everything else. Canvas is a nice document editor. Artifacts is a nice code-and-preview pane. They're separate modes you trigger, siloed from one another, and when you're done the dominant verb is still export: send it to Google Docs, copy the source out, download the HTML, paste it into your real environment. The side panel made the copy-paste shorter. It didn't remove it.

What none of them is, is a single adaptive workspace where the right surface — editor, document, spreadsheet, slides, image canvas, PDF — shows up on its own for whatever you're doing right now, all in one continuous context, with the AI able to act inside each of them rather than hand you a result to carry away. A bolted-on canvas is not that. It's a better waiting room, not a place to actually do all the work.

What I started wishing existed

So I did the thing you do when a frustration finally has a name: I started imagining the tool I actually wanted.

I wanted one conversation that didn't end at the edge of the chat box. I wanted to ask for code and have an editor appear — not a preview I copy out, an editor I work in. I wanted to draft a doc and have the document be right there, part of the same session, not a separate app I paste into. I wanted the numbers to land in a spreadsheet the AI could actually fill, the slides to build in a deck I could refine by talking, the image to open on a canvas — each surface summoning itself based on what the task needed, none of them requiring me to go find a tool or carry anything across a border.

In other words: I didn't want a smarter oracle. I wanted a workspace — one place where thinking and doing happen together, where the AI has hands, and where switching tasks doesn't mean switching apps and losing the plot.

That's the concept I keep coming back to: a multimodal AI workspace. One chat, many surfaces, summoned automatically, all sharing the same context.

Why this felt like the obvious next step

The more I thought about it, the less it felt like a wishlist and the more it felt inevitable.

There's a principle a lot of us already believe about building good AI systems: use the model for what it's good at — understanding intent, reasoning about the task — and route the actual work to the tool that's actually good at that work. Model reasons; the right tool does the job. I've argued this about calculators and code execution before: you don't make the language model be the spreadsheet, you let it drive the spreadsheet.

A multimodal workspace is just that same principle raised to the level of the whole product. Instead of one chat pretending to be everything through a straw of text, the system reads what you're trying to do and opens the surface built for it — and keeps them all in one shared context so nothing gets lost between them. Once you see it that way, the current model — brilliant chat, everything else left to you — starts to look like an obvious transitional stage, not the destination.

Then I came across Xenition

While I was chewing on all this, I recently came across a tool built around exactly this idea, called Xenition — and I'll be upfront that it's the thing that turned this from a vague wish into "oh, someone's actually doing this."

The premise is the one I'd been describing to myself: one persistent chat where the right surface opens itself — a code editor, a document, a slide deck, a spreadsheet, an image canvas, a PDF reader — based on what you're actually doing, so you're not picking a "mode" or hopping between apps. On top of that it leans into the acting part I kept missing: agents that don't just answer but take multi-step actions, with the kind of guardrails that matter once an AI can actually do things (approval gates, an audit trail, even a second agent that reviews the first one's work before it reaches you).

I'm not going to pretend I've stress-tested every corner of it, and this isn't a review. But as a concrete answer to the frustration I opened this post with — the courier problem, the fourteen tabs, the blind assistant — it's the closest thing I've found to the workspace I actually wanted, and it made the whole idea feel a lot less hypothetical.

Where I think this is heading

Here's my honest prediction. The next real leap in AI tools won't be a smarter chat. We're already deep into diminishing returns on "the oracle knows more." The leap will be collapsing thinking and doing into one place — so the intelligence we already have can actually reach the work.

Chat was the demo. It was the thing that proved these models were astonishing. But a demo that makes you copy its output into fourteen other apps isn't a finished product; it's a preview of one. The tools that win the next round won't be the ones that answer best. They'll be the ones where the answer and the doing happen in the same breath.

So, back to the question in the title: are today's AI tools making us productive, or just giving us something new to play with? My honest answer is both, and that's the problem. They're genuinely useful and genuinely a toy, at the same time — and the reason it still feels more like play than leverage is that the intelligence is trapped in a chat box while all our actual work happens somewhere it can't go. Close that gap, and "play" quietly turns into "productive."

What's the tool-switch that breaks your flow the most? The one copy-paste hop you make ten times a day and quietly resent? I'd love to hear where the courier problem hits you hardest — drop it in the comments.

The Retrieval Checklist I Wish I'd Had Before Shipping RAG

James Anderson — Tue, 25 Aug 2026 04:02:30 +0000

The first time my RAG system gave a confidently wrong answer, I did what everyone does: I blamed the model. I swapped in a bigger one. I tuned the prompt. I added "only answer from the context provided" in bold. The answer got no better.

The problem was never the model. The model was faithfully summarizing the context it was handed — the context was just wrong. It had retrieved the wrong chunks, so it answered the wrong question, fluently.

This turns out to be the norm, not the exception. Industry analysis in 2026 keeps landing on the same number: when RAG fails, the failure is in retrieval roughly 73% of the time, not generation. The LLM gets blamed for a mistake that happened several steps upstream, before it ever saw a token.

So here's the checklist I wish someone had handed me before I shipped — organized as a walk through the whole pipeline, because retrieval isn't one step, it's a chain, and it can break at any link. Naive RAG ("chunk, embed, cosine similarity, stuff into prompt") was always a prototype. This is the gap between that and production.

Let's go link by link.

First, the mental model: two paths, not one pipeline

The mistake underneath a lot of RAG pain is treating RAG as a single flow. It's actually two separate paths that most people accidentally couple together.

The indexing path (offline). Runs when documents are added or changed: parse the source → clean the text → chunk → (optionally) enrich each chunk with context → embed → write to the vector store and a keyword index. This can take minutes per document and runs in the background.

The query path (online). Runs on every user request, in real time, under a latency budget (aim for under ~3 seconds end to end): take the query → optionally rewrite it → retrieve candidates → rerank → assemble the prompt with citations → generate → log the trace.

The most common architectural mistake is coupling these. If re-indexing forces the query path offline, you can't iterate on chunking or swap embedding models without downtime — so you stop iterating, and a frozen pipeline is a stale pipeline. Keep them independent from day one.

Check: Can you re-chunk and re-embed your whole corpus without taking live search down? If not, decouple the paths before anything else.

☐ 1. Is your chunking splitting ideas in half?

Chunking is where pipelines silently fail, because bad chunks don't throw errors — they just quietly return technically-relevant, practically-useless context.

The naive default — "split every 1,000 characters with 100 overlap" — is a fast start and a slow ceiling. Fixed-size splitting cuts sentences mid-thought, tables mid-row, and code mid-function. The retrieved chunk looks relevant and is missing the half that mattered.

Better options, roughly in order of effort:

Structure-aware splitting — split on the document's own boundaries: ## headings for docs, per-function or per-class for code, per-row for tables. Low effort, big payoff, respects how the content is actually organized.
Semantic chunking — compute similarity sentence-to-sentence and start a new chunk where the meaning shifts, so each chunk is one complete thought. More compute, but a published comparison reported it lifting accuracy meaningfully over fixed-size on the same dataset.

The rule to hold onto: each chunk should be able to answer a question on its own. If a chunk only makes sense next to its neighbor, your splitting is too aggressive. Also mind chunk size — too small and you fragment ideas; too large and you dilute the signal, forcing the model to average across a wall of mostly-irrelevant text.

Check: Pull ten random chunks and read them cold. Does each stand on its own, or are half of them sentence fragments and orphaned table rows?

☐ 2. Are you embedding the chunk — or the chunk in context?

A subtle, high-impact one. If you embed only the raw body text of a chunk, you strip away the context that told a human what it meant — which section it's under, which product it's about, what came before it.

Two fixes, both cheap relative to their payoff:

Embed context, not just body. Prepend the heading, a short document summary, or a one-line description of what the chunk is about before embedding. This aligns the chunk's vector with how people actually phrase questions. (This is the core idea behind "contextual retrieval" — giving each chunk a little situating context before it's indexed measurably improves recall.)
Keep metadata attached. Every document arrives with structure — author, date, source, section, product version, document type, access level. Store it alongside the chunk. You'll use it in the next step.

Check: Does an isolated chunk in your index carry any signal about where it came from, or is it a naked paragraph with no situating context?

☐ 3. Are you using hybrid search — or just vector search?

This is the single most common retrieval mistake, and it hides in plain sight because vector search usually works.

Pure vector (semantic) search is great at meaning. Ask "how do I fix login problems?" and it'll surface chunks about authentication, OAuth, and session timeouts even if none use the word "login." That's the magic.

But it falls on its face the moment a query contains something exact. A user searches for the error code ERR_SSL_PROTOCOL_ERROR, a SKU like WX-4200, or a specific function name — and vector search has no idea what to do, because semantic similarity is meaningless for a serial number. It returns things "sort of about errors" and misses the exact match sitting right there in your corpus.

The fix is hybrid search: run keyword search (BM25/full-text) and vector search on the same query, then fuse the results — Reciprocal Rank Fusion (RRF) is the standard merge. Keyword catches exact matches; vector catches meaning. The consensus across 2024–2026 benchmarks (BEIR, MTEB, and others) is blunt: BM25 + dense embeddings fused with RRF beats either one alone, on basically every public benchmark. Dense-only retrieval lost that argument.

Check: Does your retrieval handle a literal error code and a vague conceptual question equally well? If not, you're probably vector-only, and adding a keyword index is your highest-leverage change.

☐ 4. Are you transforming the query — or retrieving the user's raw words?

Here's a link most people skip entirely: the user's literal question is often not what the retriever wants. People ask vague, compound, context-dependent questions; your index holds precise, standalone statements. Bridging that gap is query transformation, and it's one of the biggest quiet wins available.

The main patterns, each solving a different problem:

Query rewriting / expansion — clean up and enrich the raw query before retrieval so it matches the corpus better. Especially important in multi-turn chat, where "what about the second one?" is meaningless without rewriting it into a standalone query.
HyDE (Hypothetical Document Embeddings) — have the LLM generate a hypothetical answer to the question, then embed that and retrieve against it. A fake answer is often shaped much more like the real documents than the question is, which boosts precision.
Step-back prompting — rewrite a narrow question into a more general one, retrieve the background, then specialize. Good for ambiguous queries where the literal phrasing misses the corpus.
Decomposition — split a multi-part question into independent sub-queries, retrieve each separately, then synthesize. "How does our refund policy differ between B2B and B2C, and what are the exceptions?" is really three retrievals, not one.

You don't need all of these. But if you're feeding raw user text straight into the retriever, you're leaving a lot of recall on the table — especially for compound and conversational questions.

Check: Take your ten hardest real user questions. How many would retrieve better if they were rephrased, split, or expanded first? If it's most of them, add a transformation step.

☐ 5. Are you reranking — or trusting first-pass retrieval order?

The mistake that cost me the most quality for the least obvious reason: I assumed that if the right chunk was retrieved, the model would use it. But where it lands in the list matters enormously.

Vector search uses a bi-encoder — it encodes the query and each chunk separately and compares vectors. Fast, but it trades away fine-grained relevance. So the genuinely best chunk often gets retrieved... at position 8, buried under seven "pretty relevant" ones. And models demonstrably ignore information stranded in the middle of a long list — the "lost in the middle" problem. The right answer is in the context and the model still misses it.

Reranking fixes this. Retrieve a broad candidate set with hybrid search (top 20–50), then run a cross-encoder reranker that scores each (query, chunk) pair jointly — seeing query and chunk together, which makes it far better at fine-grained relevance than first-pass retrieval. Keep the top 3–8.

The impact is large, not marginal: a cross-encoder reranker commonly adds 5–15 points of MRR on hard sets, and on some reasoning-heavy benchmarks reranking pushed nDCG@10 from ~0.13 to ~0.40 — roughly 3x, just from reordering the same candidates you already retrieved.

The recipe that beats ~80% of production deployments: retrieve ~20 via hybrid search → rerank to ~5 → send 3–5 to the LLM. Reranking 100+ candidates rarely pays; the signal lives at the head.

Check: Is there a reranking step between retrieval and the prompt? If chunks go straight from vector similarity into the context window, add one — it's often the highest-ROI change in the whole pipeline.

☐ 6. Is your context assembly helping the model — or dumping on it?

You've retrieved and reranked the right chunks. You can still lose here, at the stage nobody talks about: how you actually assemble the prompt.

Things that quietly hurt:

Order. Because of "lost in the middle," put the strongest chunks at the very start and end of the context, not buried in the center.
Volume. More chunks is not better. Stuffing 30 chunks in "to be safe" dilutes the signal and invites the model to average across noise. Send the few that earned their place.
No citations. Ask the model to cite which chunk supports each claim. This both discourages free-floating fabrication and gives you a way to verify the answer against its sources.
Long-context ≠ skip retrieval. Frontier models have million-token windows now, and the reflex is "just dump everything in, who needs retrieval." Resist it. Dumping the whole corpus is slower, more expensive, and less accurate than sending a few well-chosen chunks, because the model still has to find the needle. Use the big window for genuine synthesis (long reports, whole codebases), not as a substitute for retrieval.

Check: How many chunks do you send, and in what order? If the answer is "as many as fit, in retrieval order," you're leaving quality on the floor.

☐ 7. Do you need agentic RAG — or are you bolting complexity onto a broken pipeline?

Everything above describes a single retrieve-then-generate pass. That has a ceiling: it works on simple questions and falls apart on nuanced, multi-hop ones where the answer isn't in any single chunk. Enter agentic RAG.

The shift is structural. Classic RAG does one retrieval call, up front, stateless — if it misses, there's no recovery. Agentic RAG moves retrieval inside a reasoning loop (the ReAct pattern: the model alternates between thinking and calling tools). Now the model can retrieve, look at what it got, decide it's not enough, rewrite its query, retrieve again, and stop when it has what it needs. Retrieval becomes a tool the agent uses repeatedly, not a fixed step in front of it.

This unlocks multi-hop questions — "what's the GDP of the country that hosted the 2024 Olympics?" needs hop one (Olympics → France) before hop two (France → GDP). A single retrieval can't do that; an agent that retrieves, reasons, and retrieves again can. Related advanced patterns include GraphRAG (build a knowledge graph from your docs to answer questions that require connecting entities across many documents) and giving the agent explicit tools: search, fetch-full-document-by-id, exact-match/regex, and even prune-context-to-discard-junk.

The honest caveat — and this ties straight back to over-engineering: agentic RAG costs more (more calls, more latency, more nondeterminism) and is worth it for genuinely complex or high-stakes retrieval (legal, medical, financial, multi-hop). It is not a fix for a broken basic pipeline. If your chunking is bad and you have no reranking, an agent will just make bad retrieval calls, repeatedly, more expensively. Get single-pass retrieval solid first. Add the agent loop only when the questions genuinely need multiple hops.

Check: Do your failing questions actually require chaining facts across documents — or would they be answered fine by hybrid search + reranking you haven't implemented yet?

☐ 8. Can you measure retrieval in isolation — and catch a fabrication?

This is the meta-mistake that hides all the others. Most teams evaluate RAG end-to-end: read the final answer, decide it "seems good," ship. But an end-to-end answer blends retrieval and generation, so when it's wrong you can't tell which half failed. You'll spend a week tuning prompts to fix a chunking bug.

Two things you need to measure separately:

Retrieval quality on its own. Given a query, did the right chunk make it into the retrieved set (recall), and how high did it rank (rank / nDCG / MRR)? For multi-hop and agentic setups, recall has to be measured across the whole chain, not one call. Frameworks like RAGAS exist, but even a hand-built set of real queries mapped to their correct source chunks beats vibes.

Faithfulness / groundedness of the answer. Is every claim in the final answer actually supported by the retrieved context? This is the check that catches the scariest failure: the agent that retrieves 8 chunks, uses 6, and invents the 7th fact entirely. Without a faithfulness score or a judge gating the output, that fabrication ships and a customer finds it two days later.

One hard-won caution: a tiny eval set will lie to you. If your test set is small and easy, every method scores near-perfect and they all look equally good — the differences that matter on real data are invisible on a toy set. A retrieval eval is only an eval if methods can actually fail on it. If everything scores 95%, you've built a smoke test, and a smoke test will happily bless the broken layer you were hoping to justify.

Check: If retrieval regressed tomorrow, would a number tell you — or would a user? If it's the user, you can't measure retrieval yet, and everything above this line is guesswork.

The whole checklist, in one screen

Decouple the indexing path from the query path so you can iterate without downtime.
Chunk so each piece stands alone — structure-aware or semantic, never blind fixed-size.
Embed in context — prepend headings/summaries, keep metadata attached.
Hybrid search — BM25 + vector, fused with RRF. Never vector-only.
Transform the query — rewrite, HyDE, step-back, or decompose before retrieving.
Rerank with a cross-encoder — retrieve ~20, rerank to ~5, send 3–5.
Assemble context deliberately — best chunks first and last, few not many, with citations.
Go agentic only when needed — multi-hop and high-stakes, on top of a solid base.
Evaluate both retrieval recall and answer faithfulness — on a set hard enough to fail.

The One Line to Remember

When RAG gives a bad answer, suspect retrieval first — it's the culprit far more often than the model.

The instinct to reach for a bigger model is almost always wrong. The bigger model will summarize the wrong chunks just as fluently as the small one did. The leverage is upstream — in finding the right chunk, making it usable, ranking it where the model will see it, and being able to tell when any link in the chain breaks.

I learned this checklist one confidently-wrong answer at a time. You don't have to.

What's the retrieval bug that fooled you the longest? Mine was a chunking issue I spent two weeks blaming the model for. Share yours — and any checklist items I missed — in the comments.

DEV Community: James Anderson

Most 'AI Agents' Are Just If-Statements in a Trench Coat

What an "agent" actually is (the definition nobody pins down)

The test: can you draw the flowchart in advance?

Why the costume is expensive

What you actually wanted was a pipeline

When you do need a real agent

Why everyone builds the agent anyway

The takeaway

How to Add AI to an Existing App Without Breaking It

The one idea everything else follows from: enhancement, not rewrite

1. Start with one specific pain point — not "add AI"

2. Put it behind an abstraction layer from day one

3. Isolate it so it can fail without taking the app down

4. Ship it opt-in, not opt-out

5. Keep a human in the loop early

6. Instrument cost and behavior before you scale

7. Canary the rollout — widen only when the metrics hold

8. Treat the AI's output as untrusted input

The pattern underneath all eight

The Next.js Security Checklist for AI-Generated Code

Why AI generates insecure Next.js code specifically

☐ 1. Don't trust middleware for auth — it's a UX layer, not a security boundary

☐ 2. Every "use server" Server Action is a public POST endpoint

☐ 3. Never store tokens in localStorage — use httpOnly cookies

☐ 4. Don't leak secrets to the client

☐ 5. Watch the "use client" boundary — it's contagious

☐ 6. Don't over-pass data from server to client

☐ 7. Validate input on the server — always

☐ 8. Get the JWT details right (AI gets them wrong)

☐ 9. Rate-limit your endpoints

☐ 10. Patch Next.js — you're probably on a vulnerable version

The pattern underneath all ten

Run this today

Generating code with AI is easy. Building something that survives its own growth is not — and those aren't the same skill. This article wrote about the workflow that actually holds up for real apps: give AI structure, rules, and context before it writes a line — and

Vibe Coding in the New Era: How to Build Real Applications with AI

AI Is Getting Dramatically More Capable, Fast. Where Is This Heading?

First — what does "AGI" even mean?

How do you even measure intelligence in a machine?

What Astra actually achieved (told straight)

The thing that actually matters isn't the score. It's the pace.

So what does this mean for regular people?

Should we think about AI differently now?

Where is this actually heading?

What this means for us

Agents That Act Need Brakes, Not Just Brains

Brains and brakes are different problems

Why "acting" changes everything

Brake #1: Approval gates — ask before acting

Brake #2: A reviewer that can actually say no

Brake #3: An audit trail — know what it did

Brake #4: Blast-radius limits — caps and scopes

The uncomfortable truth: brakes are unsexy

The takeaway

Building With AI When You Don't Know Architecture: A Survival Guide

The real problem: you can generate code faster than you can understand it

Survival skill #1: Separate the pieces

Survival skill #2: One job per thing

Survival skill #3: One home for each piece of data

Survival skill #4: Decide the shape before you generate

Survival skill #5: Make the AI explain itself

Survival skill #6: Keep it boring and consistent

Know the warning signs before it's too late

The takeaway

9 Ways Your AI Agent Silently Fails (and How to Catch Each)

1. The HTTP 200 empty payload

2. The poisoned step (error propagation)

3. Goal drift

4. The amnesiac mid-task (context loss)

5. The retry loop / runaway cost

6. Plausible-speech hallucination

7. The unsanctioned action

8. Cross-surface / cross-model inconsistency

9. The green checkmark that can't fail

One bug wearing nine costumes

The 60-second self-audit

Prompt Fluency Is the New Literacy — and Most Devs Are Winging It

The model isn't the variable anymore. You are.

A vague prompt is a vague thought wearing a question mark

Winging it has a bill — and you're paying it without seeing the invoice

☐ 2. Every `"use server"` Server Action is a public POST endpoint

☐ 5. Watch the `"use client"` boundary — it's contagious