DEV Community: Hakeem Abbas

Server Components Don't Automatically Make Your Next.js Application Faster

Hakeem Abbas — Thu, 17 Sep 2026 17:47:30 +0000

Imagine you have a Next.js application with a dashboard that feels slow. You move several components to the server, remove some client-side JavaScript, and expect the page to become noticeably faster. It doesn't.
The database still takes 200ms to respond. Two upstream services are called sequentially. The server waits for all of them before producing the page. The browser then receives a large payload, processes it, hydrates interactive components, calculates layout, and finally paints the UI.
You changed where rendering happens. You didn't necessarily fix where the time was being spent. That's why I don't think “use Server Components” is a performance strategy. The more useful question is: Where should this computation happen?

Server Components Solve a Specific Problem

Server Components are valuable because they allow certain components to render on the server without shipping their component code to the browser as client-side JavaScript.
For a content-heavy page, that can be a significant advantage. For example, a product page might contain:
Product Page
├── Product Information
├── Reviews
├── Related Products
└── Add to Cart
The product information and reviews may not need browser-side interactivity. The cart button does. That could lead to a reasonable boundary:
Server
├── Product Information
├── Reviews
└── Related Products

Client
└── Add to Cart

The server handles the parts that don't require browser state, while the client handles the interactive portion. That's an architectural decision. It isn't automatically a performance optimization for every component.

Moving Work to the Server Doesn't Remove the Work

This is where I think the distinction gets lost. Suppose your Server Component needs data from three services:

Next.js Server
↓
User Service
↓
Billing Service
↓
Analytics Service

If those requests happen sequentially, your server might spend: 100ms + 200ms + 150ms = 450ms waiting for upstream services.
Rendering the component on the server didn't remove that latency. It simply moved the waiting from the browser to the server. If those requests are independent, the architecture might instead allow them to happen concurrently:

Now the waiting time is closer to the slowest dependency rather than the sum of all three.
The performance improvement came from changing the execution pattern, not simply from using a Server Component.

The Browser Still Has Work to Do

There's another misconception worth separating. Server Components can reduce the amount of JavaScript that needs to be sent to the browser, but they don't make browser work disappear. The browser still has to process whatever you send it. Depending on the architecture, that can include:
Download
↓
Parse
↓
Execute JavaScript
↓
Hydrate Client Components
↓
Calculate Layout
↓
Paint
If a page contains a large interactive application, moving a few components to the server doesn't automatically eliminate the client-side work that remains.
And hydration can still become expensive when a large amount of interactive UI has to become usable in the browser.
That's why I'd look at the complete browser timeline rather than assuming that fewer Client Components automatically means a fast application.

Serialization Is Another Boundary

There's also a boundary between the server and browser that is easy to overlook: data transfer. Suppose your server fetches a large object:

But the UI only needs:

Sending unnecessary data across the boundary creates additional work. The server may have fetched it quickly, but the application still has to serialize, transfer, parse, and process that data.
The question isn't only: “Where is this component rendered?” It's also: “What information actually needs to cross the boundary?” That can have a much larger performance impact.

Client Components Aren't Automatically Bad

I also wouldn't turn this into: Server Components good. Client Components bad. Client Components exist for a reason.
If a component needs browser APIs, local state, event handlers, real-time interaction, or other client-side behavior, pushing everything to the server can make the architecture awkward rather than better.
Imagine a search interface with:

Instant filtering
Keyboard navigation
Input state
Dropdown interactions
Client-side transitions

That UI needs to respond to the user. Making every part of it server-driven doesn't automatically improve the experience. The better question is where the computation and state actually belong.

Network Boundaries Matter More Than Component Labels

One of the first things I'd look at when diagnosing a slow Next.js application is the network and data-fetching graph. Something like:
Page
├── API A
│ └── Database
├── API B
│ └── Database
└── API C
└── External API
If every dependency has to be reached before useful content can appear, the architecture may have a latency problem regardless of whether the UI uses Server Components.
Sometimes the answer is parallelization. Sometimes it's caching. Sometimes it's moving a computation closer to the data. Sometimes it's streaming partial UI. Sometimes it's eliminating an unnecessary network hop entirely. The framework doesn't make that decision for you.

Start With the Workload

When deciding whether something should run on the server or client, I'd ask a different set of questions.

Does this computation need browser state?
Does it require interaction?
Does it depend on sensitive server-side data?
How expensive is the computation?
How much JavaScript does it require?
How much data needs to cross the network boundary?
What upstream services does it depend on?
Can those dependencies run concurrently?
What does the user need to see immediately?

These questions lead to much better architecture decisions than simply asking whether something should be a Server Component. The goal isn't to maximize Server Components. The goal is to put each piece of work where it makes the most sense.

Performance Is a System Property

A Next.js application can use Server Components extensively and still be slow. It can also contain Client Components and feel extremely responsive. The difference comes from the architecture around those components.
If your server waits on slow dependencies, Server Components won't magically make those dependencies faster. If your browser executes too much JavaScript, moving one component to the server won't solve the entire problem. If you're transferring unnecessary data, rendering location isn't the only thing that matters. And if your requests form a long sequential waterfall, changing component types may barely affect the real bottleneck.
That's why I think the better mental model is:

Where is the work?
↓
Where is the data?
↓
Where is the state?
↓
Where are the network boundaries?
↓
Where is the actual bottleneck?

Server Components are a tool, not a performance strategy. The workload should decide where the boundary belongs.

Most AI Agents Don't Fail Because the Model Is Bad. The Loop Is Bad.

Hakeem Abbas — Wed, 16 Sep 2026 14:31:51 +0000

Imagine an AI agent handling customer support. A customer asks: “Can you refund my order?”
The agent checks the order, confirms that the purchase is eligible, calls the refund API, and gets a timeout. What does the agent know at this point? Not necessarily that the refund failed.
The request may have reached the payment provider, the refund may have succeeded, and only the response was lost. If the agent simply retries:
Agent → Refund API
↓
Timeout
↓
Agent → Refund API
you could end up issuing the same refund twice. The model didn't necessarily make a bad decision. The system gave it an unsafe loop.
This is why I think many agent failures become easier to understand when you stop looking at the model in isolation. At a high level, an agent is running a loop:
Observe → Think → Act → Observe → Think → Act
That loop sounds simple. Production systems are not.

The Loop Needs Boundaries

An agent needs to know more than what action to take next. It needs to know when to stop, what has already happened, what failed, and whether an action can safely be repeated.
Without those controls, relatively small problems can turn into serious failures. Consider an agent trying to update a customer's address. It calls the tool:

The operation succeeds, but the response is lost because of a network failure. The agent doesn't know whether the action succeeded.
If the system allows it to retry indefinitely, the agent could keep attempting the same operation. This isn't really a reasoning problem. It's a state and execution problem.

Infinite Loops Are Easier Than They Look

One of the simplest agent failures is also one of the most obvious:
Observe
↓
Think
↓
Act
↓
Tool fails
↓
Think
↓
Retry
↓
Tool fails
↓
Retry
↓
...
If nothing tells the agent that it has reached a failure condition, there's no reason for the loop to stop. That's why I'd put an explicit iteration limit around agent execution.

The exact number isn't important. The boundary is. An agent should never have unlimited authority to keep reasoning and acting simply because it hasn't reached the desired outcome.

Every Action Needs a State

Another common problem is losing state between iterations. Imagine an agent performing a deployment:
Build
→ Test
→ Deploy
→ Verify
The deployment succeeds, but the process crashes before recording that state. When the agent resumes, it sees: “Deployment not completed.” So it deploys again.
Now you have duplicated work or, depending on the operation, potentially a much bigger problem. This is why durable checkpoints matter. Instead of relying entirely on the model's conversational context, the system should persist important state:

If the process restarts, the system knows where it actually is. The model can reason from that state instead of trying to reconstruct history.

Tools Need Structured Responses

Tool design also has a huge impact on agent reliability. Compare these two responses: "Done."
and:

The second response gives the next iteration something concrete to reason about. This becomes particularly important when tools can return partial success, warnings, or ambiguous outcomes.
For example:

That's much more useful than simply returning: "Request failed."
The agent now knows that it should not automatically assume the operation didn't happen. Structured tool contracts make the loop easier to control.

Idempotency Is Critical

If an agent can retry actions, those actions need to be designed with retries in mind. For operations such as payments, refunds, emails, account changes, or provisioning, the system should be able to recognize duplicate requests.
For example:

If the same request is sent twice because of a timeout or retry, the backend can recognize that it's the same operation rather than treating it as a completely new one. This is especially important because agents operate in environments where failures are normal.
Networks fail. Services timeout. Processes restart. Responses get lost. The system has to be designed around those realities.

Success and Failure Should Be Explicit

Another pattern I like is explicit terminal states. Instead of letting the agent keep running until it "feels" finished, define states such as:

RUNNING
WAITING
SUCCEEDED
FAILED
ESCALATED

Now the control system has something deterministic to enforce. For example, an agent might discover that it doesn't have enough information to complete a request. Instead of continuing to guess:
Missing information
↓
Ask user / escalate
↓
WAITING

Similarly, if a sensitive operation fails repeatedly:

Tool failure
↓
Retry
↓
Retry limit reached
↓
FAILED

The agent doesn't get to decide that it should retry forever. The system decides what happens when the boundary is reached.
The Model Decides. The System Controls.
This is the distinction I think is important when designing production agents. The model can decide: “I should call the refund tool.” But the system should decide:

Is this tool allowed?
Is the user authorized?
Has this operation already happened?
Can it safely be retried?
How many attempts are allowed?
What happens if the result is ambiguous?
When should execution stop?
When should a human take over?

That separation makes the architecture much safer and easier to debug. A useful mental model is:

The model provides reasoning. The surrounding system provides control. That's why I wouldn't start debugging an unreliable agent by immediately asking whether I need a better model. I'd first inspect the loop. Does it have iteration limits? Are actions idempotent? Is state durable? Are tool responses structured? Can the system distinguish failure from unknown outcome? Are checkpoints persisted? Are terminal states explicit? Can the agent escalate instead of endlessly retrying?
If those pieces are missing, replacing the model may simply give you a more capable system running the same broken loop. Agent reliability isn't just a model problem. It's a loop-design problem.

Stop Choosing RAG Chunk Sizes by Token Count

Hakeem Abbas — Tue, 15 Sep 2026 14:27:22 +0000

Imagine you're building a RAG system for a developer documentation platform. You decide every document should be split into 500-token chunks. It sounds reasonable. The chunks are small enough for retrieval, large enough to contain some context, and easy to process consistently.
Then a developer asks: “How do I configure authentication for the API?” Your retriever finds a chunk containing the configuration parameter, but the previous chunk contains the authentication prerequisites and the next chunk contains the actual example.
The system retrieved technically relevant text. It just didn't retrieve enough of the right context. This is one of the reasons I don't think chunk size should start with a number. 500 tokens isn't a chunking strategy. It's a constraint. A good chunk should represent a meaningful unit of information that can stand on its own when retrieved.

Different Documents Have Different Boundaries

Consider source code. If you split a Python file every 500 tokens, you might end up with half a class in one chunk and the rest in another.
That's usually a terrible retrieval boundary. A better approach is to preserve natural code structures:
File
├── Class
│ ├── Method
│ ├── Method
│ └── Method
└── Function
A function or class often represents a much better semantic unit than an arbitrary token range. The same principle applies to documentation. A product guide might have:
Installation
↓
Configuration
↓
Authentication
↓
Usage
↓
Troubleshooting
If someone asks how to configure authentication, I want the chunk to preserve the relevant section rather than splitting it because an arbitrary token limit was reached.
Legal documents make this even more obvious. A clause can depend heavily on its surrounding definitions, exceptions, and conditions. Splitting a clause in the middle can produce chunks that are individually readable but legally incomplete.
And structured business documents have their own natural units: tables, policies, sections, procedures, and records. There isn't one universal chunk size that works well for all of them.

Start With the Questions, Not the Tokens

I'd first ask: What kinds of questions will users ask? Suppose users are searching a technical knowledge base. They might ask: “How do I reset my API key?”
That's likely a procedure-oriented query. Another user might ask: “Which plans support SSO?” That's a product-policy query. Another might ask: “Why am I getting error 4017?” That's an error-diagnosis query.
These questions may require very different pieces of context. A chunking strategy that works for one may perform poorly for another. The objective isn't to create chunks that are mathematically uniform. It's to create chunks that are retrievable units of meaning.

Semantic Boundaries Matter More

Think about a simple document:

Refund Policy
Customers can request a refund within 30 days of purchase.
Enterprise customers must contact their account manager.
Refunds are not available for annual contracts after renewal.

If the system splits this into arbitrary pieces, a query about enterprise refunds could retrieve the first sentence without the exception that follows it. The retrieved text is technically relevant. But it's missing the condition that changes the answer.
That's a dangerous retrieval failure. A better chunking strategy recognizes that these statements belong to the same semantic section. This is why I prefer thinking in terms of information boundaries rather than token boundaries.

Metadata Is Part of the Chunk

The content itself isn't always enough. I'd also carry metadata that helps the retrieval system understand where the chunk came from.
For example:

Now retrieval can use more than the text. A query such as: “How does authentication work in Payments API v3?” can benefit from product, version, document type, and section metadata.
Metadata can also help prevent an older version of a document from competing equally with the current one. That's particularly important for technical documentation where APIs and configuration requirements change frequently.

Overlap Isn't Automatically Good

Chunk overlap is another setting that often gets treated as a default.
For example:

Chunk 1: tokens 1-500
Chunk 2: tokens 400-900
Chunk 3: tokens 800-1300

Overlap can help preserve information that sits across boundaries. But more overlap also means more duplicated content, more storage, and potentially more noisy retrieval results.
I'd use overlap when the structure of the document actually requires it. If you're splitting a long narrative where meaning crosses boundaries, some overlap can be useful. If you're splitting well-defined functions, headings, or independent records, aggressive overlap may add little value.
The question shouldn't be: “How much overlap should every chunk have?” It should be: “Where does context actually cross the boundary?”

Token Count Still Matters

This doesn't mean token count is irrelevant. It matters for context limits, latency, embedding costs, retrieval efficiency, and how much information you eventually send to the model. But I'd treat token count as a constraint, not the definition of a good chunk.
You might end up with a 200-token chunk because that's the natural semantic unit. You might have a 1,200-token chunk because a complete procedure or code class needs that much context. The important question is whether the chunk can meaningfully answer the kinds of questions you're expecting.
If a 200-token chunk contains the exact answer, making it 500 tokens doesn't automatically make it better. And if a 500-token chunk cuts a critical explanation in half, forcing it into that limit doesn't make it useful.

Evaluate Chunking Through Retrieval

The best way to know whether your chunking strategy works isn't to look at the chunks and decide they "feel right." Test them. Build representative questions and inspect what gets retrieved.
For each query, ask:

Did the correct document appear?
Did the relevant section appear?
Does the chunk contain enough context?
Was an important condition separated from the answer?
Are irrelevant chunks ranking above useful ones?
Does changing the chunking strategy improve retrieval?

This is where RAG evaluation becomes important. You can experiment with different chunking strategies and measure retrieval quality rather than relying on intuition.
For example:

Strategy A → Fixed 500 tokens
Strategy B → Fixed 1,000 tokens
Strategy C → Section-based
Strategy D → Structure-aware

The winner shouldn't be the one with the nicest-looking chunks. It should be the one that consistently retrieves the evidence required to answer real questions.

Good Chunking Is About Context

I don't think there's a universal "best" chunk size for RAG. There are good chunking strategies for particular data, query patterns, and retrieval systems.
Source code should respect code structure. Documentation should respect headings and procedures. Legal content should preserve clauses and their conditions. Business documents should preserve meaningful sections and records. Metadata should travel with the content. Overlap should be intentional. And token count should constrain the system without dictating its structure.
Because the goal of chunking isn't to create pieces of a certain size. It's to create pieces that contain the right information in the right context. A chunk is good when retrieving it gives the model enough evidence to answer the question correctly. Good chunking isn't about making chunks the right size. It's about making them contain the right context.

A 100ms API Can Still Produce a Terrible User Experience

Hakeem Abbas — Mon, 14 Sep 2026 13:14:33 +0000

Your backend team tells you the API is fast. The endpoint responds in 100ms. Database queries are optimized. Server CPU looks healthy. P95 latency is under 200ms.
So why does the application still feel slow? Because the user doesn't experience your API latency. They experience the time between doing something and seeing the result. Those are very different measurements.
Imagine a user clicks “Load Dashboard.” The API takes 100ms to respond, but the browser has to establish connections, download resources, execute JavaScript, hydrate the application, make another request, process the response, and finally render the UI. The user might wait 600ms before anything meaningful appears. Your API is still 100ms. The experience isn't.

The Request Doesn't Start at Your Server

A simplified request might look like this:
DNS
↓
TCP
↓
TLS
↓
Request
↓
Server
↓
Database
↓
Serialization
↓
Network
↓
Browser
↓
JavaScript
↓
Rendering
When engineers talk about API performance, they're usually measuring only one section:
Request → Server → Database → Response
That's useful. But it's not the complete interaction. Even before your server sees the request, there can be DNS resolution, connection setup, TLS negotiation, and network latency. And after the server responds, the work isn't finished.
The browser still has to receive the response, parse it, execute JavaScript, update application state, calculate layout, and paint pixels. That's what the user actually experiences.

A 100ms API Can Become a 600ms Interaction

Consider a simple example. A user clicks a button and the application performs two sequential requests:
Click
↓
Request A — 100ms
↓
Request B — 120ms
↓
Browser processing — 100ms
↓
Rendering — 80ms
The backend APIs are both fast. But because Request B cannot start until Request A finishes, the user has already spent 220ms waiting on the network.
Add browser work, connection overhead, JavaScript execution, and rendering, and the interaction can easily move toward 500–600ms or more. Nothing is individually terrible. The sequence is the problem. This is why optimizing a single endpoint doesn't necessarily make an application feel faster.

Waterfalls Are Often the Real Problem

One of the first things I'd look for when debugging a slow interaction is a request waterfall. For example:
Page
└── User API
└── Account API
└── Orders API
└── Recommendations API
If every request depends on the previous one, latency accumulates. Even if each endpoint takes only 100ms, four sequential requests already introduce roughly 400ms of server-side waiting before accounting for network and browser overhead.
If the requests are independent, they might instead run concurrently:
┌── User API
Page ────────┼── Account API
├── Orders API
└── Recommendations API
Now the total time is closer to the slowest request rather than the sum of all requests. That's an architectural optimization, not an API optimization.

The Browser Has Its Own Performance Budget

Another mistake is assuming that once the API response arrives, the UI can immediately update. Modern web applications can have significant client-side work. The browser may need to:

Parse HTML and JavaScript
Execute application code
Hydrate server-rendered components
Parse large JSON responses
Update application state
Recalculate layout
Paint the updated interface Consider a page that receives a 500KB JSON response but only needs ten fields to display the initial view. The API might return that data in 100ms. The backend is happy. The browser still has to process everything. That's why response size, JavaScript execution, hydration, and rendering can matter just as much as server latency.

Perceived Performance Is Also About What Happens First

There's another subtle part of performance: users don't necessarily care about the total time if useful feedback appears quickly. Compare two applications.
Application A:
Click
→ 600ms nothing
→ Complete UI appears
Application B:
Click
→ 100ms loading state
→ 250ms partial content
→ 500ms complete UI
Both might take roughly the same amount of time to reach completion. Application B usually feels much faster because the interface responds immediately and communicates progress. That means performance work isn't always about reducing the final number. It's also about reducing time to first meaningful feedback.

Measure the Whole Journey

This is why I wouldn't evaluate application performance using API latency alone. I'd want to understand the complete path:
User Action
↓
Browser Event
↓
Network
↓
Server
↓
Database
↓
Response
↓
JavaScript
↓
State Update
↓
Layout
↓
Paint
↓
User Sees Result

Each stage can introduce latency. And importantly, the bottleneck can move. You might spend a week reducing an API from 150ms to 80ms and discover that the browser spends 300ms processing the response. You just optimized the wrong part of the system.
For web applications, I'd use browser performance tooling alongside backend metrics. A server trace can tell you what happened inside your infrastructure. A browser performance trace tells you what happened to the user. You need both.

Optimize the Interaction, Not Just the Endpoint

A fast API is valuable. But it's only one component of application performance. If an interaction feels slow, don't immediately open the backend profiler and start optimizing SQL queries. First ask:

Where is the user actually spending time?
Is there a network waterfall?
Are requests unnecessarily sequential?
Is the payload too large?
Is JavaScript blocking the main thread?
Is hydration expensive?
Is the browser doing excessive layout work?
Is the UI waiting for data that could have been rendered earlier? The fastest API in the world can't compensate for a slow sequence of everything around it. That's the distinction I keep coming back to: Server latency measures your backend. End-to-end latency measures the experience. And ultimately, users don't care that your API took 100ms. They care about how long it took from “I clicked” to “I can see and use the result.” API performance is application performance only when the rest of the application isn't the bottleneck.

The Hardest Bugs I've Worked On Were Rarely Syntax Problems

Hakeem Abbas — Fri, 11 Sep 2026 16:00:19 +0000

One of the hardest production bugs I've worked on looked almost impossible at first. The API was responding normally. The database update was correct. The queue was processing messages. Logs didn't show an obvious exception. And yet, occasionally, a customer would end up with the wrong state.
The frustrating part was that every individual piece of the system appeared to be doing exactly what it was supposed to do. The bug wasn't inside one component. It was in the sequence of interactions between them.
That's something I've noticed repeatedly with production systems. The bugs that take the longest to find are rarely caused by a missing semicolon, a typo, or a simple incorrect condition. They're usually caused by two or more components making reasonable assumptions that don't hold when they interact.
Consider a simple order-processing system:
Client → API → Database → Queue → Worker → Payment Service
Now imagine a customer clicks "Pay" twice. Two requests arrive almost simultaneously. Both requests check the order:

Both see pending. Both continue. The code is correct if you look at either request independently. The system is wrong because both requests observed the same state before either one changed it. That's a race condition.
You could spend hours reviewing the payment logic and never find the real problem because the payment logic isn't actually broken. The problem exists between the requests, the database state, and the timing of those operations.

Production Systems Are Full of Timing Problems

Timing is one of the things that makes production bugs so difficult. A function might behave correctly 99.99% of the time. Then a particular sequence happens:

Request A starts
Request B starts
Request A updates state
Request B reads old state
Request A sends event
Request B sends retry
Worker processes both events

Now the final state depends on the exact ordering of events. That's why a bug may never appear during local testing but suddenly show up under real traffic.
Concurrency exposes assumptions that sequential execution hides. The same thing happens with retries. Suppose a client sends a payment request. The server processes the payment successfully, but the network connection times out before the response reaches the client. From the client's perspective: “The request failed.” From the server's perspective: “The payment succeeded.” The client retries.
Now you have a duplicate operation unless the system has some form of idempotency. Again, neither component necessarily failed in isolation. The problem is the assumption between them.

Distributed State Makes This Even Harder

In a distributed system, there often isn't one single version of reality at every moment. One service might know that an order is paid. Another might still have pending. A cache might contain yesterday's value. A message might still be sitting in a queue. A database replica might lag behind the primary. Each component can be behaving correctly according to what it knows. The bug appears when we assume they all know the same thing at the same time.
That's why debugging distributed systems often feels less like reading code and more like reconstructing a timeline. You have to ask: What did each component know, and when did it know it? That question can be more useful than asking which function returned the wrong value.

Then AI Adds Another Variable

This gets particularly interesting with AI systems. Traditional software already gives us a lot of complexity:

Service A → Queue → Service B → Database → Service C

Now add an LLM into the system:

User
↓
LLM
↓
Tool
↓
Service
↓
Retrieval
↓
LLM
↓
Response

We now have another component whose behavior isn't completely deterministic. The same user intent can potentially lead to different tool selections, different interpretations, or different retrieved context. For example, imagine an AI support system with three tools:

A customer says: “I don't want this anymore. Can you take care of it?” The model has to interpret what "this" means. Maybe it chooses cancel_subscription. Maybe the surrounding conversation makes issue_refund more appropriate. Maybe the request is ambiguous enough that neither should happen without clarification.
The interesting engineering problem isn't just whether the LLM is "smart enough." It's what happens at the boundary between the model and the deterministic system. If the model chooses a tool incorrectly, does the backend validate the request? If a tool call times out, does the model retry it? If the operation succeeded but the response was lost, can the system safely execute it again? If retrieval returns stale information, what prevents the model from treating it as current? These are system-design questions, not prompt-writing questions.

Debug the Assumption, Not Just the Component

When debugging a complicated production issue, I've found it more useful to ask: “What assumption is breaking?” Maybe Service A assumes Service B has already processed an event. Maybe the client assumes a timeout means the operation failed. Maybe two workers assume they're the only worker processing a record. Maybe the cache assumes the database hasn't changed.
Or, in an AI system, maybe the application assumes the model will always select the correct tool. Those assumptions are often invisible because each component works perfectly under normal conditions. The failure only appears when the components interact in an unexpected order. That's why observability matters so much in these systems. Logs shouldn't only tell you that something failed. They should help reconstruct what happened across the entire request.
For an AI workflow, I want to know things like:

Request ID
Model decision
Retrieved documents
Tool selected
Tool arguments
Tool result
State changes
Retry attempts
Final response

Without that trail, you're often debugging a story with half the pages missing.

The Hardest Bugs Live Between the Boxes

As systems become more distributed, the individual components often become easier to understand. The difficult part is everything connecting them. A database can be correct. An API can be correct. A queue can be correct. A worker can be correct. And the overall system can still be wrong. AI doesn't change that fundamental engineering problem. It adds another variable to it.
Now we have deterministic services interacting with probabilistic behavior, retrieval systems, tool calls, retries, and human-facing decisions. That makes clear boundaries even more important. When something breaks, I don't want to immediately ask: “Which component is broken?” I want to ask: “What assumption between these components stopped being true?”
Because that's where I've found the hardest bugs usually hide. The hardest engineering problems aren't always inside the boxes. They're between them.

If I Had to Build an AI Customer-Support System Today, I Wouldn't Start With an Agent

Hakeem Abbas — Wed, 09 Sep 2026 14:48:35 +0000

Imagine a customer sends this message: “I was charged twice, but one of the charges looks different from the other. Can you figure out what's happening?”
There isn't an obvious workflow here. The system needs to understand what the customer means, look at their account, identify the transactions, check relevant policies, compare the charges, and explain what happened.
Now compare that with: “I want to cancel my subscription.” That's a completely different problem. The system already knows what needs to happen:

Validate account
Check subscription
Check cancellation rules
Cancel subscription
Update status
Send confirmation

I don't need an autonomous AI agent deciding how to execute that process. I'd use normal software. That's why if I had to build an AI customer-support system today, I wouldn't start with an agent. I'd start with a control boundary. Something closer to:

User
↓
Intent Classification
↓
Deterministic Workflow
↓
RAG
↓
LLM
↓
Tool Layer
↓
Human Escalation

The goal isn't to minimize the use of AI. It's to put AI where it actually provides value.

Not Every Support Request Needs Reasoning

Customer support contains a surprising amount of predictable work. A refund request might follow a known set of rules:
Validate Order
→ Check Refund Window
→ Calculate Refund
→ Issue Refund
→ Update Order
→ Notify Customer
These are business rules. The system knows what to do and what conditions must be satisfied. There's very little value in asking an LLM: “Do you think this customer deserves a refund?” The model might interpret the request correctly, but the actual decision should come from deterministic business logic. For example:

This code is boring. That's exactly why I like it. When the rules are known, boring software is often the better software.

Where the LLM Actually Helps

Now go back to the customer who says: “I was charged twice, but one of the charges looks different from the other. Can you figure out what's happening?”
The system doesn't have a single obvious path. Maybe one transaction is a duplicate authorization. Maybe one is a completed payment. Maybe there's a currency conversion. Maybe the customer is looking at a temporary authorization hold. Maybe there really was a duplicate charge.
The system first needs to understand the problem. That's where an LLM becomes useful. It can interpret the user's language, identify the likely intent, retrieve relevant information, reason over the evidence, and explain the situation in a way the customer can understand. A support architecture could therefore look something like:

User Message
↓
Intent Classification
↓
Known Request? ── Yes ──→ Deterministic Workflow
│
No
↓
RAG + LLM Reasoning
↓
Tool Request
↓
Validation
↓
Execution

The model handles the ambiguous part. The application controls the actual operation.

RAG Gives the Model the Right Context

Customer support systems often need access to policies, product documentation, account information, previous conversations, and troubleshooting procedures. That's where RAG becomes useful.
Instead of expecting the model to know company-specific information, the system retrieves the relevant evidence and gives it to the model.
For example, a customer asks: “Can I get a refund if I cancelled yesterday?”
The system can retrieve the current refund policy, subscription details, and relevant account information before asking the LLM to explain the answer. The model isn't inventing the policy. It's reasoning over information supplied by the system. That's an important boundary.

Tools Shouldn't Trust the Model

This is where I think many agent architectures go wrong. Suppose the LLM decides:

Should the backend simply execute it? Absolutely not. The model can request an operation. It shouldn't have final authority over whether that operation is allowed. The tool layer should still enforce:

Authorization
Input Validation
Business Rules
Execution For example:

Even if the model makes a mistake, the backend still has control. That's the boundary I want in a production system.

Sometimes the Best AI Decision Is to Stop

There's another important component that doesn't get enough attention: human escalation. What happens when the system isn't confident? What happens when the request involves a large amount of money? What happens when the customer disputes a charge and the available evidence is contradictory?
My answer wouldn't be: “Give the agent more autonomy.” I'd escalate. A good support system should know when it doesn't have enough evidence or authority to continue. That could be as simple as:

Low confidence
↓
Human escalation
↓
Agent reviews context
↓
Customer gets resolution

The AI doesn't have to solve every problem. Sometimes its job is to recognize that a human should take over.

AI for Ambiguity. Software for Operations.

This is the distinction I'd use when designing an AI customer-support system:

Ambiguity → LLM
Predictable operation → Code

If the customer describes a complicated problem in natural language, let the model interpret it. If the system needs to retrieve company knowledge, use RAG. If an operation has known business rules, enforce those rules in software. If the model needs to perform an action, let it request a tool, but keep authorization and validation outside the model. And if the situation is sensitive or uncertain, escalate to a human.
You don't need an agent controlling the entire customer-support system to make it intelligent. In fact, I'd argue that giving an agent control over everything often creates unnecessary complexity: more tool-selection decisions, more failure modes, more state to manage, and more difficult debugging.
I'd rather build a system where the model handles the parts that require interpretation and deterministic software handles the parts that require consistency. That's a much healthier boundary for production AI. AI should handle ambiguity. Software should handle operations.

If I Were Building a Production RAG System Today, This Is the Architecture I’d Start With

Hakeem Abbas — Tue, 08 Sep 2026 17:43:37 +0000

Imagine you're building an AI assistant for a company with thousands of internal documents. An employee asks: “Can enterprise customers cancel their contracts before renewal?”
Your system searches a vector database, retrieves a few chunks, sends them to an LLM, and gets back a confident answer. The demo works. Then someone asks the same thing differently: “What happens if a large customer wants to leave before their contract renews?”
The system retrieves a different set of documents. One is an old pricing policy, another discusses renewals, and a third only partially answers the question.
The LLM still produces a convincing response. Nothing crashed. The answer just isn't well supported. That's the point where production RAG becomes very different from a RAG demo.
If I were building a production RAG system today, I wouldn't stop at: Vector DB → LLM
That's a good starting point for proving that RAG works. For production, I'd want something closer to:
User
↓
API
↓
Query Processing
↓
Hybrid Retrieval
↓
Reranking
↓
Context Compression
↓
LLM
↓
Citation Validation
↓
Response
The important part isn't having eight separate services. It's giving each stage a clear responsibility.

Start With Query Processing

A user's question isn't always a good search query. Consider: “What are the limits for it?”
A human understands what “it” means if the previous conversation was about an enterprise API. A retrieval system looking only at those words doesn't have enough information.
Query processing can use conversation history, rewriting, metadata extraction, filters, or even deterministic rules to turn the request into something retrieval can actually work with.
For example:

The goal is simple: give retrieval a better representation of what the user actually wants.

Don't Rely on Vector Search Alone

Once we have the query, we need evidence. Vector search is great for semantic similarity, but semantic similarity isn't always relevant.
If someone searches for: “OAuth refresh token rotation” a document containing that exact phrase may be more useful than a document that is semantically similar but discusses session expiration instead. That's why I'd generally use hybrid retrieval:
┌── Semantic Search
Query ───────┤
└── Lexical Search
↓
Candidate Set
Semantic search handles conceptual similarity. Lexical search handles exact terminology, product names, identifiers, error codes, and other cases where wording matters. Together, they give us a stronger candidate set.

Retrieval Finds Candidates. Reranking Chooses.

Suppose hybrid retrieval returns 30 chunks. I wouldn't send all 30 to the LLM.
The retrieval stage should be good at finding possible candidates. A reranker can then look at those candidates against the actual query and determine which ones are most useful.

This creates an important distinction: “This document looks relevant” isn't necessarily the same as “this document helps answer the question.”

More Context Isn't Always Better

When retrieval performs poorly, one common solution is to simply retrieve more. Five chunks become twenty. Twenty become fifty.
Eventually, the model gets a huge prompt containing everything vaguely related to the question. That can make the system worse.
Irrelevant context consumes tokens, increases latency, and can bury the evidence that actually matters. That's where context compression becomes useful.
Instead of giving the LLM every retrieved chunk, we can reduce the context to the passages that are actually useful for answering the question. The objective isn't: Give the model everything we found. It's: Give the model the evidence it needs.

Let the LLM Reason Over Evidence

Only after retrieval, reranking, and context construction would I let the LLM generate the answer. Its job should be reasoning over the evidence we've selected, not fixing a broken retrieval system.

If retrieval returns the wrong information, switching to a larger model isn't a reliable solution.
A stronger model can actually make the problem harder to notice because it can produce a more convincing answer from bad evidence.

Generation Shouldn't Be the Final Trust Boundary

This is where I'd add another layer: citation validation. Suppose the model says: “Enterprise customers can cancel 30 days before renewal.” Where did the 30 days come from?
If none of the retrieved evidence supports that claim, the answer is still wrong regardless of how confidently it was written. A validation layer can check whether important claims are actually supported by the available evidence.
The implementation could range from simple citation checks to dedicated verification models, depending on how much risk the application carries. The important architectural principle is: A generated citation isn't automatically a valid citation.

The Real Value Is Debuggability

The biggest advantage of this architecture isn't just better answers. It's that you can actually debug failures. If the system gives a bad answer, you can ask:

Did query processing misunderstand the request?
Did retrieval miss the correct document?
Did reranking choose the wrong evidence?
Did context compression remove something important?
Did the LLM misinterpret the evidence?
Did citation validation fail to catch an unsupported claim?

With a simple Vector DB → LLM architecture, all of those problems tend to collapse into one vague question: “Did the model hallucinate?”
That's not enough to operate a production system. The interesting part of production RAG isn't the vector database or the LLM individually. It's the entire path: Query → Evidence → Answer → Verification
Not every application needs every layer, but I'd want those responsibilities to be explicit. A vector database can retrieve information. An LLM can reason over information.
Neither one, by itself, gives you a reliable RAG system. The model is one component. The retrieval system is the product.

Most Teams Don't Have a Model Problem. They Have a Systems Problem.

Hakeem Abbas — Mon, 07 Sep 2026 15:39:27 +0000

A team ships an AI feature. A few weeks later, users start reporting that the answers aren't good. Some responses are inaccurate. Some questions return irrelevant information. An agent occasionally chooses the wrong tool. Latency is inconsistent.
Someone asks the question that has become almost automatic: “Should we switch models?” Maybe. But I wouldn't start there.
The model is usually the most visible component in an AI system, which makes it an easy thing to blame. It's also one of the easiest things to change. Replace Model A with Model B, run a few prompts, compare the outputs, and it feels like you're making progress.
The problem is that you're often changing the component without understanding the failure. An AI application is a system. The model is one component inside it. If the system is producing bad results, the first job isn't to replace the model. It's to find out where the system is actually failing.

Start With Retrieval

Take a RAG system designed to answer questions about internal company policies. A user asks: “Can enterprise customers cancel before renewal?” The retriever returns:

Pricing policy
Renewal documentation
Marketing FAQ
Cancellation policy
Product documentation

The model produces an incorrect answer. Is the model the problem? Not necessarily. Maybe the relevant cancellation exception was never retrieved. Maybe the retriever found the right document but ranked it below irrelevant documents. Maybe the chunk containing the actual cancellation condition was separated from the rest of the policy during ingestion. Maybe the retrieval threshold is too low.
In each case, the model is receiving inadequate evidence. Replacing the model might change the final answer, but it doesn't fix the retrieval pipeline. A stronger debugging process starts by inspecting what the model actually received.

What documents were retrieved?
What were their scores?
Was the required evidence present?
Did the correct chunk survive reranking?

If the answer to those questions is no, you have a retrieval problem.

Then Look at Context

Now assume retrieval is working correctly. The relevant policy was retrieved.
But the prompt contains 15 retrieved documents, the entire conversation history, previous tool results, system instructions, and several other pieces of context. The model now has the right information somewhere inside a large amount of information.
That's a different problem. It's a context management problem. More context isn't automatically better context.
Irrelevant documents can compete with relevant evidence. Long conversation histories can introduce stale information. Tool outputs can consume context without contributing to the current task.
The model might have everything it needs and still produce a poor answer because the useful information isn't being presented effectively. Again, changing models might improve the result. But you're treating the symptom rather than the system.
Before switching models, I'd want to know how much context is being sent, where it comes from, what gets discarded, and which pieces are actually necessary for the current task.

Tool Design Can Look Like a Model Problem

The same thing happens with AI agents. Imagine an agent has three tools:

search_web()
search_docs()
search_customer_db()

The agent keeps selecting the wrong one. The immediate reaction might be: “This model isn't good at tool calling.” But look at the tools.
Do their descriptions clearly explain their boundaries? Does search_web() explicitly say when it should be used? Does search_docs() say that it searches internal documentation? Does search_customer_db() clearly explain what data it can access? Or are the tools poorly named, overlapping, and vaguely described?
If the model has to infer the difference between three ambiguous tools, you've created a routing problem. A better tool definition can sometimes improve agent behavior without changing the model at all. The model can only make decisions based on the information and constraints provided to it. Bad tool design can create bad tool selection.

Without Evaluation, Model Comparisons Are Mostly Guesswork

There's another problem that often gets ignored: teams don't have a proper evaluation set. They switch from Model A to Model B and manually test ten prompts. Model B looks better. So they migrate. But what exactly improved? Did retrieval improve? Did tool selection improve? Did factual accuracy improve? Did refusal behavior improve? Did latency get worse? Did the model become more expensive?
Without a consistent evaluation dataset, you don't really know. For an AI system, I'd want a test set containing different types of cases:
Question → Expected answer
Question → Required source
Question → Expected tool
Question → Must refuse
Now you can run the same evaluation against different models and compare actual behavior. You can also identify where a model performs better. Maybe Model B is better at reasoning but worse at tool selection. Maybe Model A has slightly lower answer quality but significantly better latency. Maybe neither model performs well because the retrieval system is returning the wrong documents. Evaluation turns “this model feels better” into an engineering decision.

Observability Tells You Where the Failure Happened

You can't reliably debug an AI system if all you store is the final response. For a failed request, I want to know what happened along the entire path.
For example:

retrieval_latency
retrieved_documents
reranker_scores
prompt_tokens
completion_tokens
selected_tool
tool_latency
model_latency
evaluation_score

Now imagine a request that took three seconds. You discover retrieval took 40ms. The tool call took 2.4 seconds. The model generated the response in 300ms. The problem wasn't model latency. It was the tool.
Or perhaps the model generated a perfectly reasonable response, but the retrieved documents were outdated. Now the problem is the data. Without observability, all you know is: “The AI response was bad.”
With observability, you can start asking: “Which component caused it?” That's the difference between debugging a system and guessing.

Sometimes the Data Is the Problem

This one is particularly easy to miss. Suppose your RAG pipeline retrieves exactly the document it was supposed to retrieve. The model follows the instructions. The context is clean. The answer is still wrong. Then you inspect the source document. It was last updated two years ago.
No model change will fix outdated source data. The model can only reason over the information you give it. If the knowledge layer is incorrect, incomplete, contradictory, or stale, improving the model doesn't solve the underlying problem. This is why AI quality is often closely tied to the quality of the systems surrounding the model.

Sometimes the Requirement Was Never Clear

There's an even more fundamental failure mode: the team doesn't actually know what “good” means. Should the assistant answer from internal documentation only? Should it refuse when evidence is missing? Should it prioritize the latest policy? Should it cite sources? Should an agent use the CRM or the database for customer information? What should happen when two documents contradict each other?
If those rules aren't defined, there is no meaningful way to evaluate the system. You can't reliably optimize something you haven't defined. A model may produce a response that one engineer considers correct and another considers unacceptable. That's not necessarily a model problem. It's a requirements problem.

The Model May Still Be the Problem

None of this means models don't matter. They absolutely do. There are cases where a model has the right context, receives well-designed tools, follows the intended workflow, and still performs poorly. At that point, switching models may be exactly the right decision. The difference is how you arrive at that decision.
If you've established that retrieval is working, context is appropriate, tools are well designed, data is correct, requirements are clear, and evaluation consistently shows that another model performs better, then you have an engineering basis for changing models. That's very different from changing models because the output “feels wrong.”

Trace the Failure Before Replacing the Model

The model is the component everyone sees. That makes it an easy target. But an AI application is more like: Data → Retrieval → Context → Model → Tools → Application Logic → Output
A failure anywhere in that chain can appear to the user as a bad AI response. So when an AI feature underperforms, I wouldn't immediately ask: “Which model should we use?” I'd ask: “Where did the system fail?”
Was the right information retrieved? Was the context relevant? Was the data current? Did the agent select the correct tool? Were the requirements clear? Can we reproduce the failure in an evaluation set? Do the logs show where the latency or error occurred?
Once you can answer those questions, changing the model becomes much more meaningful. The model might be the problem. But don't assume it is simply because it's the easiest variable to change. Trace the system first. Then change the component that's actually failing.

I Don't Think Every Product Needs an AI Agent

Hakeem Abbas — Fri, 04 Sep 2026 14:58:48 +0000

There is a pattern I keep seeing in AI product development. A team has a workflow that already works. The steps are known. The inputs are defined. The possible states are understood. The failure cases can be handled.
Then someone asks: “Can we make this agentic?”
And suddenly a deterministic workflow becomes: LLM → Agent → Tools → Execution
The system is now technically more “AI-powered.” But is it actually better? I'm not convinced. I think we're increasingly using AI agents for problems that never required an agent in the first place.

Some Problems Already Have an Answer

Consider a subscription upgrade. The workflow is straightforward:
Validate subscription
↓
Charge payment
↓
Update plan
↓
Send confirmation
There isn't much ambiguity here. If the payment succeeds, update the plan. If the payment fails, don't update the plan. If the subscription is invalid, reject the request. If the plan changes successfully, send the confirmation. You can represent this using normal application architecture: if/else → workflow → queue → state machine → API
The behavior is predictable. The states are explicit. The transitions can be tested. The failure cases can be defined. The system can be monitored with conventional metrics. There is no reasoning problem that requires an LLM. Now imagine putting an agent in the middle:
User → LLM → Agent → Tools → Payment API → Database → Email
The system now has another component deciding what should happen next. For a deterministic workflow, that doesn't necessarily add intelligence. It adds uncertainty.

Deterministic Problems Should Usually Stay Deterministic

If the system already knows the correct sequence of operations, I want that sequence represented explicitly in code. A state machine might define:
PENDING
↓
VALIDATED
↓
PAYMENT_PROCESSING
↓
PAYMENT_CONFIRMED
↓
PLAN_UPDATED
↓
COMPLETED
Each transition can have explicit conditions. If something fails during payment, the system knows where it is. If the worker crashes after payment but before the plan update, the system can recover from a known state. If the confirmation email fails, the email operation can be retried independently.
This is boring software engineering. And that's a compliment. Boring systems are often easier to operate. You know what they're going to do. You know what they can do. You know what they cannot do.
And when something breaks at 2 a.m., you can follow the state transitions without wondering what the model decided to do.

What Does an Agent Actually Add?

Agents become interesting when the system doesn't know the exact path ahead of time.
Consider a customer asking: “My account keeps getting charged every month. I cancelled the subscription weeks ago. Can you figure out what's happening?”
That's not a simple workflow. The system may need to:

inspect the customer's account
look at subscription history
inspect invoices
compare transaction dates
check cancellation events
determine whether another subscription exists
inspect relevant policies
potentially ask the user for additional information
decide what action should happen next

The path depends on what the system discovers. Maybe there is an active subscription. Maybe the cancellation happened after the latest billing cycle. Maybe there are two accounts. Maybe the payment belongs to a different product. Maybe the transaction is actually a temporary authorization.
The system can't necessarily know the correct sequence beforehand. That's where an agent can provide real value.
The agent can reason about the current state, select the next useful action, inspect the result, and decide what information is needed next. That's fundamentally different from executing a predefined workflow.

Unknown Paths Are Where Agents Become Useful

The distinction I use is fairly simple:
Known path → deterministic software.
Unknown path → consider an agent.
An agent is valuable when the system has to make decisions based on information it discovers during execution. A deterministic workflow is valuable when the system already knows the correct sequence. This distinction can prevent a lot of unnecessary complexity.
Suppose your onboarding process is always: Create account → verify email → create workspace → assign default permissions → send welcome email.
That doesn't need an agent. Build a workflow. Now imagine the product receives: “Help me configure my workspace for our engineering team.”
There may be several possible configurations depending on team size, existing tools, security requirements, integrations, and user preferences.
The system may need to ask questions, inspect available resources, compare options, and decide what to do next. That's a much more reasonable candidate for agentic behavior. The difference isn't whether AI can perform the workflow. It can. The question is whether reasoning is actually required to determine the workflow.

Agents Introduce a New Class of Failure

Once an LLM is responsible for deciding what happens next, you inherit a different set of engineering problems. The model can choose the wrong tool. It can generate incorrect arguments. It can call tools in an inefficient order. It can retry unnecessarily. It can misunderstand the current state. It can fail to recognize that an operation has already completed. It can produce an action that wasn't intended.
Now you need additional infrastructure around that behavior. You need model monitoring. You need tool-call observability. You need evaluation datasets. You need guardrails. You need permission boundaries. You need retry strategies. You need state management. You need to think about prompt injection. You need to test behavior across different model outputs. You need recovery mechanisms for partially completed operations.
None of these are inherently bad. They're simply costs. And those costs make sense when you're getting something valuable in return: the ability to navigate an uncertain problem. They make much less sense when the workflow was deterministic from the beginning.

Don't Use an Agent to Hide Bad Architecture

There's another reason I'm cautious about agent-first design. Sometimes an agent is introduced because the underlying workflow is difficult to model.
Instead of understanding the system and defining its states and transitions, we give an LLM access to several APIs and ask it to figure things out.
That can feel faster during prototyping. But you've effectively moved architecture into probabilistic behavior. A traditional workflow might explicitly say:
Validate
→ Charge
→ Update
→ Notify
An agent might be told: “Use the available tools to complete the subscription upgrade.” Now the sequence isn't fully encoded in the application. It's partly encoded in model behavior. That makes the system harder to reason about. If the process is business-critical, that's a trade-off worth taking seriously.

Use AI Where Uncertainty Exists

This doesn't mean agents are unnecessary. Quite the opposite. There are problems where deterministic logic becomes awkward because the input is unstructured and the path depends on interpretation.
Investigation is a good example. Research is another. Troubleshooting can be another. Planning tasks, information synthesis, multi-step analysis, and workflows that dynamically change based on discovered information can all benefit from agentic systems.
The important question isn't: “Can an agent do this?” Almost certainly, it can. The better question is: “Does this problem actually require autonomous reasoning?” If the answer is no, an agent may be unnecessary complexity.

Start With the Simplest Architecture

When designing an AI feature, I would start with the boring solution. Can an API handle it? If not, can a workflow handle it? If the workflow has explicit states, can a state machine handle it? If work needs to happen asynchronously, can a queue and workers handle it?
Only when the system genuinely needs dynamic decision-making would I introduce an agent. That progression might look like: API → Workflow → Queue → State Machine → Agent
Not every problem needs to reach the last step. In fact, most shouldn't. An agent should be an architectural choice based on the problem, not a default implementation pattern because the product happens to use AI.

The Goal Isn't to Make Everything Agentic

AI agents are powerful precisely because they can operate when the path isn't completely known. But that doesn't mean every product workflow should become agentic. If a process is predictable, encode it. If its states are known, model them. If its transitions are deterministic, make them explicit. If the system needs to reason about what to do next based on information it discovers, then consider an agent.
The simplest distinction is still the one I find most useful:
Known path → deterministic software.
Unknown path → consider an agent.
The goal isn't to maximize the amount of autonomy in a product. It's to use the right level of autonomy for the problem. Sometimes the smartest thing an AI engineer can do is not use an AI agent at all.

The Biggest Problem With AI-Generated Code Isn't Bad Code. It's Unfamiliar Code.

Hakeem Abbas — Thu, 03 Sep 2026 16:00:41 +0000

A developer opens a pull request. The feature works. The tests are green. The implementation looks reasonably clean. Then they realize the PR changes 17 files for what should have been a relatively small feature.
There is a new abstraction nobody on the team has seen before. A dependency has been added for functionality the project already had. Two similar utilities now exist in different parts of the codebase. The tests are extensive, but nobody is completely sure what assumptions they are actually testing.
And the author didn't write most of it manually. AI did. This is one of the less obvious problems created by AI coding tools. The problem isn't necessarily that AI generates bad code.
Sometimes it generates perfectly valid code. The problem is that it can generate code faster than a team can develop an understanding of it. That changes the bottleneck in software engineering.

The Cost of Writing Code Has Changed

For a long time, engineering teams were constrained by how quickly developers could implement things. A developer had to understand the requirement, design the solution, write the code, debug it, and test it. AI changes the economics of that process.
A developer can now describe a feature, generate an implementation, ask for tests, refactor it, and generate supporting code in a fraction of the time it might previously have taken. That's useful. But the amount of code entering the system can increase faster than the team's ability to review and understand it.
Consider a simple feature that requires modifying four files. An AI coding assistant might produce a solution that touches twelve. It may introduce a service layer, a helper abstraction, a configuration object, several interfaces, and a new package. None of these things are necessarily incorrect. But every additional abstraction creates another thing a future developer has to understand. This is where code familiarity becomes an engineering concern.

Working Code Isn't the Same as Understandable Code

A piece of code can compile. It can pass unit tests. It can pass integration tests. It can even work correctly in production. And still be expensive to maintain.
Imagine an engineer joining the project six months later. They encounter:
Feature
├── Controller
├── Service
├── Factory
├── Adapter
├── Strategy
├── Helper
└── Utility
The actual requirement is straightforward, but understanding the implementation requires tracing through seven different layers. The question isn't whether each layer is technically valid. The question is: Did the complexity buy us anything?
This distinction becomes especially important with generated code because AI tends to optimize for producing a complete solution, not necessarily for preserving the mental model of a specific codebase. It doesn't inherently know which abstractions your team prefers. It may not know that your repository already has a utility for the problem. It may create a new pattern because that pattern is reasonable in isolation. And reasonable in isolation can still be wrong for the system.

Large PRs Are a Warning Sign

One of the first places this becomes visible is pull requests. A developer might normally submit a 150-line change.
With AI assistance, that can become 500 lines without anyone intentionally trying to make the change larger.
The implementation may include additional error handling, abstractions, tests, types, wrappers, comments, and refactoring.
Some of those changes may be useful. Others may simply increase the review surface. This matters because code review is not about counting changed lines. It's about understanding behavior. A reviewer has to answer questions such as: Does this implementation satisfy the requirement? What assumptions does it make? What happens when the input is invalid? What state can this code modify? Does it introduce a race condition? Does it interact correctly with existing services? What happens when this dependency fails?
The larger the change, the harder those questions become. AI can make writing a 600-line PR inexpensive. It doesn't make reviewing 600 lines inexpensive.

Generated Tests Can Create False Confidence

Tests introduce another interesting problem. AI can generate tests very quickly. That's useful, but test quantity isn't the same as test quality.
Suppose the generated implementation makes an incorrect assumption about how a pricing rule works. The AI can generate ten tests based on that same assumption. All ten tests pass.
You now have a well-tested implementation of the wrong behavior. The tests aren't necessarily broken. They're testing the implementation rather than validating the actual product requirement. This is why engineers still need to define what should be tested.
The important question isn't: “Did the AI generate tests?” It's: “Do these tests protect the behavior the system is actually supposed to provide?” That requires understanding the system and its requirements.

Dependencies and Duplicate Solutions

Another common problem is unnecessary dependencies. A developer asks an AI assistant to implement a small piece of functionality. The model suggests a popular package. The package works. But somewhere else in the codebase, the team already has a dependency that provides the same capability. Now the project has two ways to solve the same problem. This creates maintenance cost.
Someone has to understand both implementations. Someone has to keep both dependencies updated. Someone has to decide which one new code should use. The same thing happens with internal utilities. Instead of finding an existing helper, generated code can create another implementation.
Over time, the codebase develops multiple answers to the same question. None of them necessarily look terrible individually. Together, they make the system harder to reason about.

Reviewability Becomes an Engineering Constraint

This is why I think reviewability deserves to be treated as an engineering constraint. When evaluating an implementation, correctness isn't the only question. We should also ask: Can another engineer understand this? Can they explain why this abstraction exists? Can they debug it without asking the original author? Can they safely modify it six months later? Can a reviewer verify the behavior without spending hours reconstructing the architecture?
If the answer is no, the implementation has a problem even if all the tests pass. This doesn't mean AI-generated code should be avoided. It means the role of the engineer changes.

AI Makes Senior Engineering More Important

AI is very good at reducing the mechanical cost of producing software. But architecture isn't just code generation. Someone still has to decide whether the abstraction should exist. Someone has to recognize that the repository already has a solution. Someone has to determine whether introducing another dependency is justified. Someone has to understand the failure modes. Someone has to decide what the tests should prove. Someone has to review the final implementation and determine whether it belongs in the system.
That makes senior engineering judgment more valuable, not less. The interesting shift is that the bottleneck moves. When writing code is expensive, engineering effort naturally focuses on implementation. When writing code becomes cheap, understanding and deciding become relatively more expensive.

The Question Isn't “Can AI Write This?”

The better question is: “Will our team understand this code later?” AI can produce an implementation in seconds. Your team may have to maintain it for years. That's the asymmetry. The cost of generating code has fallen dramatically. The cost of understanding a complicated system hasn't.
So the goal shouldn't be to maximize the amount of code AI can produce. It should be to produce the smallest, clearest implementation that solves the actual problem. Use AI to accelerate implementation. Use engineering judgment to control what enters the codebase. Because the most expensive code isn't always the code that is wrong. Sometimes it's the code that works perfectly today, but nobody understands tomorrow.

An LLM Request Shouldn't Behave Like a Normal API Request on Mobile

Hakeem Abbas — Tue, 01 Sep 2026 17:17:02 +0000

A user taps Send on a mobile AI app. The request starts normally. A few tokens appear on the screen. Then the user locks their phone. The network connection changes. The app goes into the background. A few seconds later, the user opens it again.
Now what? Did the request finish? Is the server still processing it? Should the client reconnect? Should it start the request again? What if the first request is already completed and the retry creates a duplicate operation?
This is where building an AI feature starts to look very different from building a traditional mobile API integration. For a normal API request, we often think in terms of: Tap → Request → Response
The request starts, the server processes it, and eventually the client receives a response. An LLM interaction is often closer to: Tap → Request → Processing → Tool Calls → Partial Output → Streaming → Completion
That difference isn't just an API implementation detail. It changes the architecture of the mobile application.

An LLM Request Is an Operation, Not Just a Request

Consider a traditional endpoint such as: GET /users/123
The client sends the request and waits for the response. If the connection fails, the client can usually retry according to a straightforward policy. AI interactions are less predictable.
A request might take several seconds. The model may generate output incrementally. The server might call a search service, database, or external API before producing the final response. The operation can fail after partial output has already reached the client.
That means the client isn't simply waiting for a response. It's observing the state of a long-running operation. This distinction becomes especially important on mobile because the environment itself is unreliable. Connections change. Apps are backgrounded. Processes can be suspended or terminated. Users switch networks. The operating system can reclaim resources. A mobile AI client needs to assume that these things will happen.

Streaming Changes the State Model

With a traditional request, you can often think of the response as one object. Either you received it or you didn't. Streaming changes that.
Suppose the model has generated: “The deployment failed because the service…” The connection drops. The client already has part of the answer. What should happen next? A naive implementation might simply retry the entire request.
Now the server generates the response again, potentially producing duplicated work and a different output.
The client might end up with: “The deployment failed because the service… The deployment failed because the service…”
A better design treats streamed output as an incremental state. The client should know which operation it is observing and what portion of the output it has already received.
This is where an operation identifier becomes useful. Instead of treating the interaction as: POST /generate → response
think in terms of:
operation_id = abc123
status = running
output = partial
The client can then reconnect and ask about the existing operation rather than assuming that the original request disappeared.

Connection Drops Are Normal on Mobile

A desktop application can often maintain a relatively stable connection. A mobile application can't make that assumption. The user can move from Wi-Fi to cellular. They can enter an elevator. They can switch applications. The device can temporarily lose connectivity. The application can be suspended while the server is still processing the request.
For an AI interaction, a connection drop doesn't necessarily mean the operation failed. That's an important distinction. There are at least two separate states:
Transport state: Is the client currently connected?
Operation state: Is the AI operation still running?
Those states should not be treated as the same thing. A socket can disconnect while the server continues processing.
Likewise, a connection can remain available while the underlying operation has already failed.
If the client treats every network failure as an operation failure, it can create duplicate requests and inconsistent UI state.

Reconnection Shouldn't Automatically Mean Retry

This is one of the most dangerous patterns in an AI mobile client:
Connection lost
↓
Retry request
↓
New operation
What if the original operation is still running? Now you have two operations. Both may execute tools. Both may consume model tokens. Both may modify external state. Both may eventually produce responses.
For read-only generation, this might primarily create unnecessary cost and duplicated output. For an agent that can perform actions, it can become much more serious.
Imagine the model is processing: “Send the customer an email confirming their refund.”
The connection drops immediately after the email tool executes. The client doesn't know whether the operation completed.
If it blindly retries, the email could potentially be sent twice. The solution is to make operations idempotent or explicitly trackable wherever possible. The client needs enough information to distinguish: “The request failed before execution.” From: “The request is still running.” And: “The request was completed, but I never received the final response.” Those are very different situations.

Cancellation Is More Than Closing the Connection

Now consider another common interaction. The user asks the AI a question. The model starts generating. The user taps Stop.
A naive implementation might simply close the streaming connection. But that only changes the client's transport state. It doesn't necessarily stop the server-side work.
The model may continue generating. A tool call may continue executing. A database query may still be running. The client has disappeared, but the operation hasn't necessarily been cancelled.
A proper cancellation flow should communicate cancellation to the server. Conceptually:
Client
↓
Cancel operation_id
↓
Server
↓
Stop generation / cancel tools where possible
↓
Operation = cancelled
Not every operation can be interrupted instantly. A tool that's already committed an external side effect may not be reversible. But the system should at least have an explicit concept of cancellation rather than assuming that disconnecting the client automatically stops everything.

Timeouts Need More Thought Too

Traditional API clients often have relatively straightforward timeout logic. If the server hasn't responded after a certain period, fail the request. For an LLM operation, a long delay doesn't necessarily mean failure. The server may be waiting for a tool. The model may still be generating. A response may simply be slow.
Streaming makes this even more interesting because you can have an active connection without receiving output for some period.
Instead of thinking only about a single request timeout, AI clients often need to distinguish between things such as connection timeout, inactivity timeout, and overall operation timeout.
For example, receiving tokens every few seconds might indicate that the operation is healthy even if the final answer hasn't arrived yet. A completely silent connection may indicate a problem. Again, the important part is understanding the operation rather than treating it like a single HTTP response.

Backgrounding Creates Another State Transition

Mobile applications have another problem that backend applications don't usually face in the same way: the user can simply leave. They can press the home button. They can lock the phone. They can open another application. The operating system may suspend or terminate the application.
What happens to an LLM operation during that time? The server may continue processing. The client may stop receiving streamed output. When the user returns, the application needs to recover its state. This is why AI conversations should not depend entirely on in-memory client state.
The application should be able to reconstruct something like:

Conversation
Operation ID
Operation status
Partial output
Final output
Error state after a reconnect or application restart. The user shouldn't have to resend their question simply because the operating system suspended the application.

State Recovery Is Part of the UX

Once you think about AI interactions as operations, state recovery becomes much easier to reason about.
Imagine the client reconnects after being offline. Instead of immediately sending the user's message again, it can ask the backend: “What happened to operation abc123?”
The server might respond: running. The client resumes observation. Or: completed. The client retrieves the final result. Or: failed. The client displays the error. Or: cancelled. The client restores the appropriate UI state.
This is fundamentally different from blindly retrying an HTTP request. The client is recovering the state of an operation. That small architectural distinction can prevent a lot of difficult edge cases.

AI Changes the Mobile Interaction Model

The biggest mistake is treating an LLM endpoint as just another REST endpoint. It might technically be exposed through HTTP. But the interaction behaves differently. A traditional request is often short-lived: Request → Response → Done
An AI operation can be: Start → Process → Stream → Pause → Reconnect → Resume → Complete
It can involve multiple backend services and tools. It can produce partial results. It can be cancelled. It can outlive the client connection. It can continue while the mobile application is in the background. That means the mobile architecture needs to model these states explicitly.
The client should know whether an operation is pending, running, streaming, completed, failed, cancelled, or recovering. The backend should provide enough state to make those transitions reliable. And actions with side effects should have strong guarantees around authorization and idempotency.

Design the Operation, Not Just the API Call

When building an AI feature for mobile, I wouldn't start by asking: “What's the endpoint?”
I'd start with: “What happens if the connection disappears halfway through?”
Then ask: What happens if the user taps Stop? What happens if the app goes into the background? What happens if the network changes? What happens if the server finishes while the client is disconnected? What happens if the client reconnects but doesn't know whether the operation completed? What happens if the user retries?
These aren't unusual edge cases on mobile. They're normal operating conditions.
An AI-powered mobile application therefore needs to treat an LLM interaction as a long-running asynchronous operation, not simply a request that eventually returns JSON. The API is only one part of the design.
The harder engineering problem is everything around it: streaming, cancellation, retries, timeouts, connection recovery, duplicate prevention, background execution, partial responses, and state recovery.
The model may take seconds to answer. The network may disappear in milliseconds. A good mobile AI architecture has to handle both. AI doesn't just change what your mobile app calls. It changes how the app has to think about the interaction itself.

Your React App Is Slow. React May Not Be the Reason.

Hakeem Abbas — Mon, 31 Aug 2026 16:24:24 +0000

A user clicks a button. Nothing happens for a moment. They click again. The interface finally responds. As developers, we tend to look at the most obvious suspect first: React.
Maybe the component is rendering too often. Maybe the component tree is too large. Maybe we forgot useMemo. Maybe a missing useCallback is causing a child component to render again.
So we start changing React code. Sometimes we fix the problem. Sometimes we make the code more complicated and the application feels exactly the same.
The reason is simple: “the React app is slow” doesn't tell you where the time is actually being spent. A slow interface is a symptom, not a diagnosis. Before optimizing React, you need to understand what happens between the user's interaction and the browser displaying the result.

Start With What Actually Happens

A simplified view of an interaction in a React application looks like this:
User Interaction
↓
State Update
↓
Render
↓
Commit
↓
Layout
↓
Paint

That's already more interesting than simply saying “React renders.”
A user interaction might trigger a state update. React then renders the affected component tree and determines what needs to change. Those changes are committed to the DOM. After that, the browser may need to recalculate styles, perform layout, and paint the updated pixels. And that isn't the entire story.
JavaScript may execute before, during, or after these steps. A network request may be waiting on the critical path. A large JavaScript bundle may be delaying execution. Hydration may be consuming the main thread. A third-party script may be competing for the same resources. So when a user says, “This page feels slow,” there are many possible explanations. The mistake is assuming that React must be responsible simply because the application was built with React.

The Same Symptom Can Have Completely Different Causes

Imagine a dashboard that takes 200 milliseconds to respond when a filter is changed. You inspect the code and find that the filter updates React state. It's tempting to conclude that the render is slow. But suppose profiling shows that React takes only 10 milliseconds. The remaining 190 milliseconds could be spent somewhere else.
Maybe the filter triggers an expensive data transformation. Maybe thousands of records are being sorted on the main thread. Maybe a network request is being made. Maybe the DOM update triggers expensive layout work. Maybe a large component subtree is being hydrated. The user experiences all of these as one thing: The UI is slow. The browser, however, doesn't see one operation. It sees many.
That's why performance debugging should start by breaking the symptom into its underlying work.

Profile Before You Optimize

The first tool I reach for is usually Chrome DevTools' Performance panel. Record the interaction. Click the button. Type into the input. Open the dropdown. Perform whatever action feels slow. Then inspect the resulting timeline. You're looking for where the main thread actually spent its time.
Long JavaScript tasks are an obvious signal. So are expensive rendering operations, layout calculations, painting, and other long-running tasks. You might find something like this:

Interaction
↓
120ms JavaScript
↓
8ms React
↓
15ms Layout

That's a very different problem from:

Interaction
↓
5ms JavaScript
↓
110ms React
↓
10ms Layout

Both applications feel slow. But the correct fixes are completely different.
In the first case, optimizing React won't address the 120ms JavaScript operation. In the second, investigating React's rendering behavior makes much more sense. This is why profiling is more valuable than immediately reaching for an optimization API.

Unnecessary Renders Are Real, But Don't Assume They're the Problem

React can absolutely do unnecessary work. Consider a dashboard where the parent component updates because one small piece of state changes. That update may cause children to render again even though their visible output hasn't changed.
This is where tools such as React.memo can be useful. But the important question isn't: “Can I prevent this component from rendering?”
It's: “Is this render actually expensive enough to matter?” A component rendering again isn't automatically a performance problem. If a component takes 0.2ms to render, preventing that render may accomplish almost nothing.
If a component performs expensive calculations, creates a large subtree, or triggers significant downstream work, then the render becomes much more interesting.
There is another subtle issue here: changing references. If a parent creates a new object or array on every render, a child can receive a new prop reference even when the underlying data hasn't meaningfully changed. For example:

creates a new object every time the parent renders. That can affect memoization and cause additional work. But again, the correct response isn't “add memo everywhere.” First find out whether the render is actually contributing to the user-visible delay.

Sometimes the Expensive Part Is Just JavaScript

One of the easiest things to misdiagnose is expensive JavaScript running inside a component. Consider a search interface with 100,000 records. Every time the user types a character, the component filters the entire dataset and then sorts the results. React may be doing exactly what you asked it to do.
The problem is that you've asked JavaScript to perform a lot of work every time the component renders. The performance trace might look like:

Keystroke
↓
State Update
↓
Render
↓
Large filter()
↓
Large sort()
↓
Commit

Calling this a “React performance problem” hides the actual issue. The better questions are: Can the dataset be reduced? Can the computation be avoided? Can the result be cached? Can the operation happen less frequently? Can the work move off the main thread? Can the search happen on the server instead?
The answer depends on the application, but the principle remains the same: Find the expensive operation before deciding how to optimize it.

React Can Finish Quickly and the Browser Can Still Be Slow

There is another category of performance problems that React developers often overlook: browser rendering. React commits DOM changes, but the browser still has to turn those changes into pixels. That can involve style calculation, layout, and painting.
For example, changing one element might cause the browser to recalculate the position of many other elements. Large DOM trees, complex CSS, expensive effects, animations, and forced synchronous layout can all contribute to this work.
One particularly common problem is layout thrashing. Imagine JavaScript repeatedly changing an element and then immediately asking the browser for its layout information. The browser may be forced to perform layout synchronously between those operations.
The result can be a significant amount of main-thread work. React can be perfectly optimized and the interface can still feel sluggish. A React optimization won't fix a browser layout bottleneck.

The Component Tree Isn't the Only Thing That Matters

Another common assumption is that performance problems must exist inside the component tree. They don't.
Consider a server-rendered or server-component application that still takes a noticeable amount of time before becoming interactive. Hydration can be part of the problem.
The browser may need to download JavaScript, parse it, execute it, and attach behavior to server-rendered markup.
If there is a large amount of client-side JavaScript, the user can experience delays even when the initial HTML arrives quickly. Similarly, network waterfalls can dominate the experience. One request waits for another. That request triggers another request.
A component isn't necessarily rendering slowly; the application may simply be waiting for data. The performance trace might reveal that React spent 12ms while the browser spent hundreds of milliseconds waiting on network activity.
Again, the symptoms are the same. The diagnosis isn't.

Don't Optimize the Framework. Find the Expensive Operation.

This is probably the most important habit in frontend performance work. When an application feels slow, don't start with: “Which React optimization should I use?”
Start with: “Where did the time go?” Then follow the evidence.
If the profiler shows unnecessary React renders, investigate component boundaries, state placement, props, and memoization. If JavaScript is consuming the main thread, optimize the computation. If the component tree is unnecessarily large, reduce the amount of work being rendered. If hydration is expensive, examine how much client-side JavaScript you're shipping. If the network is creating a waterfall, fix the request dependencies. If layout and paint dominate the trace, investigate browser rendering rather than React.
The optimization should follow the bottleneck. Not the other way around. A useful debugging flow is:

UI feels slow
↓
Record a performance trace
↓
Find where the time goes
↓
JavaScript? React? Network? Layout?
↓
Identify the expensive operation
↓
Optimize that operation
↓
Measure again

That last step matters. An optimization isn't successful because the code looks cleaner or because a familiar React API is now being used. It's successful when the measured bottleneck improves.

Performance Is a Diagnosis Problem First

React provides powerful tools for building complex interfaces, but it doesn't own everything that happens in the browser. Your application is a system. React rendering is one part of that system.
JavaScript execution, network requests, hydration, DOM updates, layout, painting, images, fonts, and third-party scripts can all affect what the user experiences.
That's why I don't like starting performance work with optimization. I start with measurement. Find the interaction that feels slow. Record it. Look at the timeline. Find the expensive operation. Then decide what needs to change.
Because if React takes 8ms and JavaScript takes 180ms, rewriting your React components isn't performance optimization. It's optimizing the wrong thing.
Your React app may be slow. React may not be the reason. The first step in performance debugging isn't finding a React optimization. It's finding where the time went.