DEV Community: AI Bug Slayer 🐞

The Benchmarks Are Lying to You. Here's How to Actually Evaluate LLMs.

AI Bug Slayer 🐞 — Mon, 14 Sep 2026 03:33:50 +0000

I spend a lot of time in the AI space -- reading papers, building things, talking to engineers who are actually shipping. And there is a gap between what the demos show and what production systems actually look like that nobody is being fully honest about.

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

When you do not have a precise definition for what you are building, you end up over-engineering simple pipelines and under-engineering genuinely complex ones. I have seen teams spend weeks adding "agentic" orchestration to workflows that would have been fine as a single well-structured prompt.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

Most real agent deployments are narrow. They do one thing well. Customer support triage. Document extraction. Code review on a specific codebase. They are not general-purpose reasoning engines. They are purpose-built pipelines with some intelligence in the decision layer.

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: What’s behind the AI industry’s latest warnings of doom? (TechCrunch AI). On Equity, we discussed the AI industry's latest debate about whether it poses an existential threat to humanity....

Worth reading: https://techcrunch.com/2026/09/13/whats-behind-the-ai-industrys-latest-warnings-of-doom/

Something I kept seeing pop up recently: Obama urges Democrats to have a ‘clear plan’ for AI safeguards (TechCrunch AI). Obama recently said that Democrats need to make artificial intelligence one of their “central agendas” and “have a very clear plan” to address concerns around the technology’s econ...

Worth reading: https://techcrunch.com/2026/09/13/obama-urges-democrats-to-have-a-clear-plan-for-ai-safeguards/

Something I kept seeing pop up recently: OpenAI’s Sam Altman says it would be ‘ill-advised’ to go public in 2026 (TechCrunch AI). While OpenAI has filed confidentially for an IPO, the company will not be going public this year, according to CEO Sam Altman....

Worth reading: https://techcrunch.com/2026/09/12/openais-sam-altman-says-it-would-be-ill-advised-to-go-public-in-2026/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

When you split a document into chunks and embed them, you are making assumptions about what pieces of context belong together. Those assumptions are often wrong. A paragraph that only makes sense in light of the paragraph before it gets retrieved in isolation and the model hallucinates the missing context.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

The engineers who are going to matter in two years are the ones who can build AI systems that other engineers can maintain and trust. That is a different skill set than fine-tuning or prompt engineering.

It is closer to systems design than it is to model research.

If any of this resonates with what you are building, or if you have a completely different take, I want to hear it. Drop your experience in the comments. The interesting conversations in this space are not in the keynotes -- they are in the threads where people are actually honest about what works.

The Model Is Not the Product. Here's What Actually Is.

AI Bug Slayer 🐞 — Mon, 14 Sep 2026 03:31:18 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://techcrunch.com/2026/09/13/whats-behind-the-ai-industrys-latest-warnings-of-doom/

Worth reading: https://techcrunch.com/2026/09/13/obama-urges-democrats-to-have-a-clear-plan-for-ai-safeguards/

Worth reading: https://techcrunch.com/2026/09/12/openais-sam-altman-says-it-would-be-ill-advised-to-go-public-in-2026/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

The AI Engineer Job Description Is a Lie. Here's What the Role Actually Is.

AI Bug Slayer 🐞 — Fri, 11 Sep 2026 03:33:16 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: Jensen Huang explains why Nvidia will grow an astounding 70% next year (TechCrunch AI). Nvidia has its finger in every pie, and sees another year of plenty in its future, Jensen Huang says. But, he insists, its deals are not circular....

Worth reading: https://techcrunch.com/2026/09/10/jensen-huang-explains-why-nvidia-will-grow-an-astounding-70-next-year/

Something I kept seeing pop up recently: OpenAI puts Pro subscriptions on hold due to Astra demand (TechCrunch AI). The company said Pro subscriptions put the most strain on its systems, so it's pausing sign-ups while adding more capacity....

Worth reading: https://techcrunch.com/2026/09/10/openai-puts-pro-subscriptions-on-hold-due-to-astra-demand/

Something I kept seeing pop up recently: Anthropic details distillation campaigns from Alibaba, Moonshot AI, and DeepSeek (TechCrunch AI). A new report released Thursday by Anthropic alleges persistent distillation attacks by China-based AI companies, which have escalated in recent months as competition in the space h...

Worth reading: https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

What I Learned After Running AI Agents in Production for a Year

AI Bug Slayer 🐞 — Fri, 11 Sep 2026 03:32:45 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://techcrunch.com/2026/09/10/jensen-huang-explains-why-nvidia-will-grow-an-astounding-70-next-year/

Worth reading: https://techcrunch.com/2026/09/10/openai-puts-pro-subscriptions-on-hold-due-to-astra-demand/

Worth reading: https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

What Nobody Tells You About Deploying LLMs at Scale

AI Bug Slayer 🐞 — Wed, 09 Sep 2026 03:34:44 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: Hackers are stealing Claude tokens from subscribers (TechCrunch AI). Last month, a Claude user noticed his account was consuming tokens even though he wasn't working. Anthropic has since warned users about hackers....

Worth reading: https://techcrunch.com/2026/09/08/hackers-are-stealing-claude-tokens-from-subscribers/

Something I kept seeing pop up recently: Cognition hits $48B valuation, signaling investors believe AI coding is far from a winner-take-all market (TechCrunch AI). Cognition's valuation multiple is higher than Cursor's was before selling to SpaceX....

Worth reading: https://techcrunch.com/2026/09/08/cognition-hits-48b-valuation-signaling-investors-believe-ai-coding-is-far-from-a-winner-take-all-market/

Something I kept seeing pop up recently: Meta debuts its Muse AI agent. Will consumers trust it? (TechCrunch AI). Meta's new personal AI agent Muse wants access to users' email, calendars, payments, health services, and more — making the company's biggest consumer AI bet yet a major test of wh...

Worth reading: https://techcrunch.com/2026/09/08/meta-debuts-its-muse-ai-agent-will-consumers-trust-it/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

How a Single AI Agent Replaced a 5-Person Data Team at a Fintech Startup

AI Bug Slayer 🐞 — Wed, 09 Sep 2026 03:32:37 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://techcrunch.com/2026/09/08/hackers-are-stealing-claude-tokens-from-subscribers/

Worth reading: https://techcrunch.com/2026/09/08/cognition-hits-48b-valuation-signaling-investors-believe-ai-coding-is-far-from-a-winner-take-all-market/

Worth reading: https://techcrunch.com/2026/09/08/meta-debuts-its-muse-ai-agent-will-consumers-trust-it/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

The LLM Nobody Talks About That Keeps Showing Up in Production Stacks

AI Bug Slayer 🐞 — Mon, 07 Sep 2026 03:33:54 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: Authors push back as publishers and agents make claims on Anthropic settlement (TechCrunch AI). Authors say publishers seem to be claiming more than their fair share of settlement payments....

Worth reading: https://techcrunch.com/2026/09/06/authors-push-back-as-publishers-and-agents-seek-share-of-anthropic-settlement/

Something I kept seeing pop up recently: Travis Kalanick’s Atoms might be getting into the robotaxi business (TechCrunch AI). The Uber founder has said that Atoms will allow him to complete "unfinished business."...

Worth reading: https://techcrunch.com/2026/09/06/travis-kalanicks-atoms-might-be-getting-into-the-robotaxi-business/

Something I kept seeing pop up recently: Seattle Times and Newsday are the latest publications to sue OpenAI and Microsoft (TechCrunch AI). Two more news organizations are suing OpenAI and Microsoft over the supposed use of their journalism to train AI....

Worth reading: https://techcrunch.com/2026/09/05/seattle-times-and-newsday-are-the-latest-publications-to-sue-openai-and-microsoft/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

Two Years From Now, This Will Be the Only Skill That Matters in AI

AI Bug Slayer 🐞 — Mon, 07 Sep 2026 03:32:22 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://techcrunch.com/2026/09/06/authors-push-back-as-publishers-and-agents-seek-share-of-anthropic-settlement/

Worth reading: https://techcrunch.com/2026/09/06/travis-kalanicks-atoms-might-be-getting-into-the-robotaxi-business/

Worth reading: https://techcrunch.com/2026/09/05/seattle-times-and-newsday-are-the-latest-publications-to-sue-openai-and-microsoft/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

The Exact Stack I Use to Build Production AI Agents (No Fluff)

AI Bug Slayer 🐞 — Fri, 04 Sep 2026 03:31:59 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: Crusoe reportedly raises $3B at a $30B valuation (TechCrunch AI). The round came together after the data center developer reportedly secured a $13 billion contract with Jane Street....

Worth reading: https://techcrunch.com/2026/09/03/crusoe-reportedly-raises-3b-at-a-30b-valuation/

Something I kept seeing pop up recently: Abliteration.ai is making a business out of removing AI guardrails (TechCrunch AI). Abliteration.AI is making powerful AI models without guardrails easier to access, arguing that giving defenders the same tools as bad actors could ultimately improve cybersecurity....

Worth reading: https://techcrunch.com/2026/09/03/abliteration-ai-is-making-a-business-out-of-removing-ai-guardrails/

Something I kept seeing pop up recently: Meta is paying to peek at how you use their latest AI model (TechCrunch AI). For its new Muse Spark model, intended for operating coding and other agents, Meta is offering an explicit discount averaging out to about 95% for users who "contribute" to the dev...

Worth reading: https://techcrunch.com/2026/09/03/meta-is-paying-to-peek-at-how-you-use-their-latest-ai-model/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

What Happens When You Run 10 AI Agents at Once in a Real Codebase

AI Bug Slayer 🐞 — Fri, 04 Sep 2026 03:31:20 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://techcrunch.com/2026/09/03/crusoe-reportedly-raises-3b-at-a-30b-valuation/

Worth reading: https://techcrunch.com/2026/09/03/abliteration-ai-is-making-a-business-out-of-removing-ai-guardrails/

Worth reading: https://techcrunch.com/2026/09/03/meta-is-paying-to-peek-at-how-you-use-their-latest-ai-model/

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

RAG Is Not the Answer. Here's What Developers Are Actually Switching To

AI Bug Slayer 🐞 — Wed, 02 Sep 2026 03:31:27 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Something I kept seeing pop up recently: Enterprise AI's real risk isn't autonomous agents. It's the complexity between them. (VentureBeat AI). Presented by Gravitee Agent complexity is the insidious shadow lurking inside enterprises right now that needs a light shone on it.That’s because enterprises don't deploy a si...

Worth reading: https://venturebeat.com/ai/enterprise-ais-real-risk-isnt-autonomous-agents-its-the-complexity-between-them

Something I kept seeing pop up recently: When agents act on their own, governance has to live in the data layer (VentureBeat AI). Presented by EDB As enterprises give AI agents more autonomy — the ability to plan, decide, and act across systems without a human approving each step — a hard question moves to th...

Worth reading: https://venturebeat.com/security/when-agents-act-on-their-own-governance-has-to-live-in-the-data-layer

Something I kept seeing pop up recently: Orchestration is the new challenge for CX in the age of AI agents (VentureBeat AI). Presented by Tata Communications Enterprises are deploying AI agents, voice AI, and automation across messaging, voice, and digital channels faster than the architecture meant to s...

Worth reading: https://venturebeat.com/orchestration/orchestration-is-the-new-challenge-for-cx-in-the-age-of-ai-agents

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.

The Production AI Checklist That Nobody Publishes.

AI Bug Slayer 🐞 — Wed, 02 Sep 2026 03:30:57 +0000

So here is my honest take on where things actually are.

The Problem With How We Talk About AI Agents

Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent.

This dilution is not just semantic. It is causing real engineering mistakes.

Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done.

Everything else is just a fancy function call.

🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface.

🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere.

✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing.

What Is Actually Happening in Production Right Now

The honest picture from teams I follow and talk to:

The teams getting good results are not chasing the latest model release. They are obsessing over:

☑️ Tool design -- what can the agent actually call, and how clean is the interface

☑️ Failure handling -- what happens when a tool returns nothing useful

☑️ Observability -- can you trace exactly why the agent made the decision it made

The teams getting bad results are the ones that swapped out GPT-4 for the latest frontier model and expected different behavior without changing anything else.

Worth reading: https://venturebeat.com/ai/enterprise-ais-real-risk-isnt-autonomous-agents-its-the-complexity-between-them

Worth reading: https://venturebeat.com/security/when-agents-act-on-their-own-governance-has-to-live-in-the-data-layer

Worth reading: https://venturebeat.com/orchestration/orchestration-is-the-new-challenge-for-cx-in-the-age-of-ai-agents

The Framework Wars Are a Distraction

LangChain. LangGraph. CrewAI. AutoGen. Semantic Kernel. Every month there is a new one and someone is writing a post about why the old one is dead.

Here is what I actually think: the framework matters less than the patterns.

The patterns that keep working regardless of what framework you use:

✔️ Plan-then-execute. Have one reasoning step that produces a plan, and a separate execution step that follows it. Do not mix them.

✔️ Separate retrieval from reasoning. Fetching context and using context are different jobs. Systems that conflate them get confused.

✔️ Explicit handoffs. When one agent passes work to another, the handoff should be structured and logged. Not a string passed through a prompt.

I have rebuilt the same architecture in three different frameworks and the results were similar each time. The framework is scaffolding. The architecture is the building.

The Retrieval Problem Nobody Has Solved

RAG is standard now. Almost every production AI system that touches proprietary data uses some form of it. But there is a problem that the tutorials do not cover well.

The chunk boundaries are wrong.

🟢 Better chunking strategies help. Overlapping windows, semantic chunking, parent-document retrieval.

🔵 But the real fix is rethinking what you are storing. Sometimes the right thing to store is not the raw text but a structured representation of the information.

✅ If your RAG pipeline is returning technically correct but contextually useless results, the problem is almost certainly in the chunking or the metadata, not the embedding model.

Where I Think This Is All Going

The models are going to keep getting better. Context windows are going to keep expanding. The cost per token is going to keep dropping.

None of that changes the fundamental engineering challenge: building systems you can trust to behave correctly when you are not watching.

That is the problem worth solving. Governance, observability, and reliable tool use. Not chasing benchmarks.

It is closer to systems design than it is to model research.