DEV Community: Emma Schmidt

Your App Has AI Features. That's Not the Same Thing as Being an AI Company Anymore.

Emma Schmidt — Fri, 31 Jul 2026 10:48:07 +0000

There's a quiet but important distinction reshaping how serious software gets built in 2026, and most teams haven't caught up to it yet. Bolting a chatbot onto an existing app and designing an application where AI is the actual architecture are two completely different engineering disciplines, and the gap between them is becoming one of the clearest competitive differentiators in software today. This is exactly the shift driving demand for real custom AI development, not a plugin added after the fact, but AI woven into how a system makes decisions, processes data, and adapts from the very first architectural diagram.

Here's what actually separates AI-native software from AI-enhanced software, why the distinction matters more than it sounds, and how to think about it if you're planning a build.

AI-Enhanced vs AI-Native: The Real Difference

Most existing "AI-powered" products today are AI-enhanced. A traditional application, built the same way it always would have been, with a recommendation widget, a search-with-AI feature, or a support chatbot added on top. The underlying system, its data flow, its decision logic, its architecture, remains essentially unchanged.

An AI-native application is architected differently from day one. Intelligence isn't a feature sitting on top, it's part of how the system actually functions.

Dimension	AI-Enhanced	AI-Native
Where AI sits	Bolted on as a feature or module	Built into the core architecture
Data flow	Static, mostly predefined	Continuous, feeding real-time model decisions
Adaptability	Fixed behavior between releases	Learns and adjusts from ongoing data
Typical origin	Retrofit of an existing product	Designed around AI from the first architecture decision
Failure mode when done poorly	AI feature feels bolted on, inconsistent with the rest of the app	Poor governance and evaluation gaps compound across the whole system

Neither approach is inherently wrong. A lot of genuinely good products are AI-enhanced and don't need to be anything else. But for products where intelligence is meant to be the actual value proposition, retrofitting an existing architecture tends to be far more expensive and painful than building AI-native from the start.

Why This Shift Is Happening Now

A few forces are converging to push serious teams toward AI-native thinking rather than incremental AI feature additions.

Retrofitting is proving genuinely costly. Teams that added AI as an afterthought are running into architectural walls, data pipelines that weren't built for continuous model input, decision logic that was never designed to be adaptive, and it's turning into expensive rework rather than a quick feature addition
Evaluation and observability are now considered table stakes, not extras. Serious AI development in 2026 increasingly means having real evaluation pipelines that systematically test outputs against known-good examples, not just shipping a prompt and hoping it behaves
Security expectations have caught up with adoption. Prompt injection, data exfiltration through AI interfaces, and output manipulation are now treated as real attack vectors, which means the AI layer needs the same security discipline as any other system boundary, not a bolted-on afterthought
Multi-model orchestration is becoming a standard architectural component. Rather than routing everything through one model, AI-native systems increasingly route different tasks to different models based on cost, latency, and capability, which only works cleanly if the architecture was designed with that flexibility in mind

What AI-Native Architecture Actually Requires

Building this way touches more of the stack than a typical feature addition would.

Data pipelines designed for continuous flow, not periodic batch updates, since adaptive systems need fresh data to actually adapt
Evaluation infrastructure built in from the start, tracking output quality, catching model regressions, and validating behavior against real, known-good examples
Clear governance around AI decision-making, especially for anything touching regulated data or high-stakes outcomes, where explainability and audit trails matter
Architecture flexible enough for multi-model routing, so the system isn't locked into a single model provider or a single point of failure
Security treated as a first-class concern for the AI layer specifically, not assumed to be covered by the same protections as the rest of the application

Real-World Scenarios Where This Distinction Actually Matters

To make this concrete, here's where the AI-native versus AI-enhanced choice genuinely changes the outcome.

A support platform adding a chatbot to an existing ticketing system is AI-enhanced. A support platform where routing, prioritization, and resolution suggestions are all driven continuously by live data from the start is AI-native, and it behaves fundamentally differently as ticket volume and complexity grow
A financial analytics tool with an AI summary feature bolted onto existing dashboards is AI-enhanced. One where risk scoring and forecasting are core to how the system processes every transaction is AI-native
An internal knowledge tool with search-with-AI added to an existing wiki is AI-enhanced. One built around continuously updated embeddings and adaptive retrieval from the ground up is AI-native
A logistics platform with a recommendation widget for routing suggestions is AI-enhanced. One where routing decisions are made continuously by an adaptive model integrated into the core dispatch logic is AI-native

Where Teams Get This Wrong

Assuming AI-native means "use more AI features." It's an architectural distinction, not a feature count. A product can have fewer visible AI features and still be genuinely AI-native if intelligence is core to how it functions
Skipping the evaluation and governance layer to move faster. This is the corner that gets cut most often, and it's the one that causes the most expensive problems later, once the system is handling real production decisions
Treating security as something to add after the AI features work. By the time an AI-native system is live, retrofitting proper security around prompt handling and data access is far harder than designing it in from the start
Underestimating the data infrastructure work. Teams often plan for the model and the interface, but underestimate how much of the actual effort lives in the data pipelines that feed continuous, adaptive behavior

A Quick Framework for Deciding What You Actually Need

Before committing to a full AI-native rebuild, it's worth being honest about which category your product actually falls into.

Is intelligence the core value proposition, or a helpful addition to a product that works fine without it?
Does your system need to adapt continuously based on new data, or would periodic updates genuinely be sufficient?
Do you have, or are you willing to build, the evaluation and governance infrastructure this approach requires?
Is your team prepared for the security implications of treating the AI layer as a first-class system boundary?

If the honest answer to the first two questions is "yes, genuinely," AI-native architecture is worth the upfront investment. If AI is a nice complement to an otherwise solid, deterministic product, a well-built AI-enhanced approach is often the more practical, lower-risk choice.

Why This Is Genuinely Hard to Get Right Alone

This isn't a criticism of any team, it's just an honest reflection of how new and fast-moving this discipline still is. The specific technical profile this requires, deep experience across data pipeline design, model evaluation, multi-model orchestration, and AI-specific security, remains genuinely scarce in most hiring markets right now. That's exactly why a lot of serious AI software development work today involves custom AI development support and AI readiness assessment brought in from outside, specifically to architect the foundation correctly before a team commits to a direction that's expensive to unwind later. Getting an experienced second opinion on whether a product genuinely needs full AI-native architecture, or whether a well-built AI-enhanced approach would serve it just as well, is often the highest-leverage conversation to have before writing a single line of the core architecture.

The Future Outlook

The gap between AI-enhanced and AI-native products is likely to keep widening over the next year or two, not narrow. As evaluation tooling matures and multi-model orchestration becomes more standardized, AI-native architecture is expected to shift from a differentiator into a baseline expectation, particularly for products where intelligence genuinely is the value proposition rather than a nice-to-have layered on top.

The Takeaway

Having AI features and being architected as an AI company are not the same thing, and that distinction is only going to matter more as the products competing for the same users diverge further along these two paths. The teams thinking carefully now about whether they genuinely need AI-native architecture, versus a solid, well-built AI-enhanced product, are the ones avoiding the expensive retrofit conversation a year or two from now.

Is your product built AI-native from the ground up, or is it AI-enhanced with intelligence layered on top? Curious where most teams here actually land on that distinction once they think it through honestly.

Build Your First MCP Server: A Practical Guide to Connecting AI Agents to Real Tools

Emma Schmidt — Thu, 30 Jul 2026 06:43:25 +0000

Learn how to build a working MCP server from scratch and connect it to an AI client, step by step.

Introduction

If you've ever needed an AI agent to check a database, call an internal API, or search a knowledge base, you've probably hit the same wall: every tool integration ends up as custom, one-off code that doesn't transfer to your next project. MCP server development and MCP integration solve this by giving you one standardized way to expose any tool or data source to any compatible AI client, instead of writing bespoke glue code for every combination.

This tutorial walks through building an actual working MCP server, connecting it to a client, and testing that it works end to end. By the end, you'll have a functioning server exposing a real tool, and you'll understand exactly what's happening at each layer instead of just copying a snippet.

Prerequisites

Comfortable reading and writing basic Python
Python 3.10 or later installed
A terminal and a code editor
Roughly 30-45 minutes

What We're Building

We're building a small MCP server that exposes one practical tool: looking up the weather for a given city. It's simple enough to fully understand in one sitting, but it demonstrates every core piece of setting up MCP tool integration.

The finished server will be able to:

Advertise its available tools to any connecting MCP client
Accept a tool call with a specific input (a city name)
Return a structured response back to the calling AI agent
Run locally so you can test it directly from your terminal

Step 1: Set Up Your Environment

Create a project folder and install the MCP SDK.

mkdir mcp-weather-server
cd mcp-weather-server
python -m venv venv
source venv/bin/activate  # on Windows use venv\Scripts\activate
pip install mcp requests

This gives you the core MCP library plus requests, which we'll use to call a public weather API inside our tool.

Step 2: Define the Server and Register Your Tool

Create a file called server.py. This is illustrative, simplified code meant for learning, not a full production implementation.

# server.py
from mcp.server import Server
from mcp.types import Tool, TextContent

app = Server("weather-server")

@app.list_tools()
async def list_tools():
    return [
        Tool(
            name="get_weather",
            description="Get the current weather for a given city",
            inputSchema={
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "Name of the city"}
                },
                "required": ["city"]
            }
        )
    ]

This list_tools function is how a client discovers what your server can do. When an AI agent connects, it asks "what tools do you have," and this is the answer it gets back, in a structured format it can reason about.

Step 3: Implement What the Tool Actually Does

Now add the logic that runs when the tool is actually called.

# server.py (continued)
import requests

@app.call_tool()
async def call_tool(name: str, arguments: dict):
    if name == "get_weather":
        city = arguments["city"]
        # Simplified example call, replace with a real weather API and key
        response = requests.get(f"https://example-weather-api.test/v1/current?city={city}")
        data = response.json()
        summary = f"The weather in {city} is {data.get('condition', 'unknown')} at {data.get('temp', '?')}°"
        return [TextContent(type="text", text=summary)]

This is the execution layer. The AI model never talks to the weather API directly, it calls your tool through the protocol, and your code decides exactly what happens and what comes back.

Step 4: Run the Server Locally

Add a simple entry point so the server can actually start.

# server.py (continued)
from mcp.server.stdio import stdio_server
import asyncio

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await app.run(read_stream, write_stream, app.create_initialization_options())

if __name__ == "__main__":
    asyncio.run(main())

Run it with:

python server.py

At this point, the server is running and waiting for a client to connect over standard input and output, which is the simplest transport for local development and testing.

Step 5: Connect a Client and Test It

To actually test this, you need an MCP-compatible client configured to launch your server. Most MCP client configurations use a simple JSON entry pointing to your script.

{
  "mcpServers": {
    "weather": {
      "command": "python",
      "args": ["/full/path/to/server.py"]
    }
  }
}

Once your client is configured and connected, ask it something like "what's the weather in Lisbon." If everything is wired correctly, the client will discover your get_weather tool through the list_tools call, invoke it with the city argument, and return the result from your call_tool function back through the model's response.

Testing / Verifying It Works

A few checks to confirm your MCP server integration is actually functioning correctly:

Confirm the client's tool list shows get_weather as an available tool after connecting
Manually call the tool with a test city and check that the response text looks correct
Try an invalid or missing argument and confirm your server doesn't crash silently
Restart the client and reconnect to make sure the server initializes cleanly every time, not just on the first run

Common Pitfalls and How to Avoid Them

Forgetting to validate input. A missing or malformed city argument will crash a naive implementation. Add basic validation before making the external call.
Blocking calls inside async functions. Using a purely synchronous HTTP library inside an async tool handler can freeze your server under load. Consider an async HTTP client for anything beyond a simple learning project.
Not handling API failures gracefully. If the weather API is down, your tool should return a clear error message, not throw an unhandled exception that breaks the whole session.
Overloading one server with too many unrelated tools. Keeping a server focused on one coherent set of capabilities makes it easier for both you and the connecting AI agent to reason about.

Best Practices for Production Use

Add proper authentication and scoped permissions before exposing any server beyond local testing
Log every tool call, including inputs and outcomes, so failures are debuggable after the fact
Set reasonable timeouts on any external API calls inside your tools
Validate all inputs against your declared schema before executing any logic
Version your tool definitions so you can evolve them without silently breaking existing clients

What to Explore Next

Add a second, related tool to the same server, like a multi-day forecast, and see how tool discovery scales
Swap the stdio transport for Streamable HTTP to run your server as a proper remote service
Explore adding authentication middleware so only verified callers can invoke your tools
Look into structured output schemas so responses can be validated programmatically instead of just returned as plain text

Conclusion

Building your first MCP server strips away a lot of the mystery around MCP integration once you've actually wired up discovery, execution, and a real client connection yourself. The pattern you just built scales directly to far more complex tools, databases, internal APIs, search systems, without changing the underlying shape of what you learned here. Once this clicks, adding your next tool is a lot faster than the first one was.

There's a New Universal Plug for AI, and Most Developers Haven't Wired It In Yet

Emma Schmidt — Wed, 29 Jul 2026 12:26:39 +0000

Picture this. You've built an AI feature that needs to check a customer's order status, pull data from your CRM, and search internal documentation, three separate systems, three separate custom integrations, each with its own auth, its own data format, its own maintenance burden. Now imagine adding a fourth tool next quarter. And a fifth. This exact integration sprawl is precisely the problem the Model Context Protocol was built to solve, and it's become one of the fastest-adopted standards in recent memory across LLM & GenAI development. Understanding how to actually build with it, not just read about it, is quickly becoming a core skill for anyone shipping AI features that need to touch real data and real tools.

Here's what MCP actually is, why it's spreading this fast, and how to build your first working MCP server step by step.

Why This Protocol Is Spreading Faster Than Almost Anything in AI Tooling

The adoption numbers here are genuinely unusual, even for an industry used to fast-moving trends.

Metric	Figure
Monthly SDK downloads (March 2026)	97 million, up from roughly 100,000 at launch
Time to reach 100M+ monthly downloads	About 16 months
Comparable milestone for React's npm package	Roughly 3 years
Public MCP servers in the official registry	Around 9,600 as of a May 2026 snapshot
Fortune 500 companies with MCP deployed	Roughly 28% within 18 months of launch
Major platforms with shipped MCP support	OpenAI, Google, Microsoft, and others within about 13 months of launch

That last row is the one worth sitting with. A protocol originally released by Anthropic in November 2024 has since been adopted broadly enough across the industry that it now functions as shared infrastructure rather than one company's proprietary approach, which is exactly the kind of cross-platform momentum that turns a promising idea into a genuine standard.

What MCP Actually Solves

Before MCP, connecting an AI application to external tools meant building a custom integration for every single combination of model and tool. Ten AI applications and a hundred tools meant a potential one thousand separate integrations, each one bespoke, each one a maintenance burden of its own.

MCP replaces that with one standardized interface. Think of it the way USB-C replaced a drawer full of proprietary charging cables. Any MCP-compatible client can talk to any MCP-compatible server through the same protocol, regardless of which model or which tool is on either end.

The Three Core Pieces You Need to Understand

MCP Host. The AI application itself, the thing the end user actually interacts with
MCP Client. Lives inside the host and manages the connection to one or more MCP servers
MCP Server. Exposes specific tools, data sources, or capabilities in the standardized format the protocol expects

Most developers building on top of MCP will spend their time on the server side, exposing their own systems and tools so any compatible AI application can use them.

Building Your First MCP Server, Step by Step

Let's build a simple MCP server that exposes a single tool, checking an order status, using the official Python SDK.

Step one: install the SDK

pip install mcp

Step two: define the server and register a tool

from mcp.server import Server
from mcp.types import Tool, TextContent

server = Server("order-status-server")

@server.list_tools()
async def list_tools():
    return [
        Tool(
            name="get_order_status",
            description="Look up the current status of a customer order",
            inputSchema={
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"}
                },
                "required": ["order_id"]
            }
        )
    ]

This is the discovery step. Any MCP client connecting to this server can now ask what tools are available and get back a structured description it can reason about.

Step three: implement what the tool actually does

@server.call_tool()
async def call_tool(name: str, arguments: dict):
    if name == "get_order_status":
        order_id = arguments["order_id"]
        status = lookup_order_in_database(order_id)
        return [TextContent(type="text", text=f"Order {order_id} status: {status}")]

This is the actual execution layer. The AI model never touches your database directly, it calls the tool through the standardized protocol, and your server controls exactly what happens and what gets returned.

Step four: run the server with the appropriate transport

from mcp.server.stdio import stdio_server

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await server.run(read_stream, write_stream, server.create_initialization_options())

Local development typically uses stdio transport. Production deployments increasingly use Streamable HTTP, which lets the server run as a proper remote service rather than a local process, and it's the transport most enterprise MCP rollouts are standardizing around this year.

Step five: secure it before it goes anywhere near production

from mcp.server.auth import RequireAuth

@server.call_tool()
@RequireAuth(scopes=["orders:read"])
async def call_tool(name: str, arguments: dict, context):
    # verified caller identity is available in context
    ...

Every tool call should carry proper identity and scope verification. A recent industry analysis noted that in the MCP era, trust isn't established once at login, it's re-earned with every single tool call and data access an agent makes, which is a meaningfully different security posture than traditional API authentication.

What's Changing Right Now, as of This Week

This is genuinely current, not stale reporting. The next major MCP specification, dated 2026-07-28, has its release candidate locked and is shipping as the final spec this week. It introduces a stateless protocol core, an Extensions framework for adding capabilities without breaking existing implementations, a formal deprecation policy, and hardened authorization. If you're starting a new MCP implementation right now, building against this version rather than the November 2025 spec is the right call, since SDK maintainers are expected to support it within a defined validation window.

Common Mistakes Teams Are Making With Early MCP Adoption

Tool overexposure. Registering far more tools than a given workflow actually needs, which bloats the context an AI model has to reason over and increases the chance of it calling the wrong tool
Treating authentication as an afterthought. Standing up a functional MCP server quickly and only addressing proper scoped auth after it's already been connected to something sensitive
Ignoring observability. Deploying MCP servers into production workflows without tracking tool call success rates, latency, or error patterns, which makes debugging a failed agent workflow nearly impossible after the fact
Skipping the governance conversation. Rolling out MCP servers across a team or organization without agreeing on security, compliance, and identity controls up front, then retrofitting governance after adoption has already sprawled

Why This Matters Beyond the Hype Cycle

MCP exhibits genuine network effects. Every new MCP server makes every existing MCP client more capable, and every new client makes building an MCP server more worthwhile. That's a structurally different growth pattern than most developer tools, and it's part of why the ecosystem has grown as fast as it has.

For teams evaluating whether to invest in this now, the practical answer is usually yes, but scoped carefully. Building custom MCP connectors and servers for your own internal tools and data sources, wired into RAG development and existing LLM and GenAI workflows, is exactly the kind of foundational integration work that compounds in value as your AI features grow, rather than something that needs rebuilding from scratch every time a new tool gets added to the stack.

The Takeaway

The integration sprawl that used to make connecting AI to real business tools a slow, custom, one-off project each time is exactly what MCP was built to eliminate. With cross-vendor support, a maturing specification, and genuine network effects driving adoption, this has moved well past experimental territory into shared infrastructure that's shaping how serious AI applications get built this year.

Has your team started building or adopting MCP servers yet, or are you still wiring up custom integrations one tool at a time? Curious how far along everyone actually is with this.

Tags: #ai #llm #genai #tutorial

Your AI Feature Isn't Slow. Your Architecture Is Making It Feel Slow.

Emma Schmidt — Tue, 28 Jul 2026 10:30:54 +0000

You've seen it a hundred times. A user types a question into an AI feature, hits submit, and then just stares at a blank screen for eight seconds before a wall of text dumps onto the page all at once. That gap between "I asked something" and "something happened" is exactly where users lose patience, and it's rarely the model's fault. It's almost always a streaming problem, not a speed problem. This is one of the core patterns behind modern AI services and AI web application development, and getting it right in Next.js is simpler than most developers expect, especially now that the framework's App Router and Server Actions map almost perfectly onto how real-time AI responses actually need to flow.

Here's how to build a genuinely responsive, streaming AI feature in Next.js from the ground up, understanding what's actually happening under the hood instead of just wiring up a library and hoping.

Why Streaming Changed What "Fast" Means for AI Features

A non-streaming AI feature waits for the entire response to generate before showing anything. For short answers that's tolerable. For anything longer, a paragraph, a summary, a generated document, it means staring at a static loading spinner for several seconds with zero feedback that anything is actually happening.

Streaming flips this. Tokens appear as the model generates them, the same way ChatGPT-style interfaces work. The perceived speed difference is enormous even when the total generation time is identical, because the user sees progress immediately instead of waiting in silence.

The Real-World Difference This Makes

Picture a support tool generating a written summary of a long customer thread. The generation itself takes about six seconds either way.

Without streaming: the user submits, sees a static spinner, and waits the full six seconds staring at nothing. To someone who isn't sure the request even registered, six seconds of silence feels much longer than it actually is, and a meaningful number of users will refresh or click again out of doubt, sometimes triggering a duplicate request.

With streaming: the user sees the first words appear within a few hundred milliseconds, and text keeps flowing steadily until the summary is complete. The total time is identical, but it reads as fast because there's constant, visible progress instead of dead air.

Same backend, same model, same six seconds of actual compute. The only difference is architecture, and it's the difference between a feature that feels broken and one that feels instant.

Choosing Your Transport Mechanism

Before writing code, it's worth understanding the three real options for getting streaming data from server to client, since picking the wrong one for your use case creates real problems later.

Server-Sent Events (SSE)

One-directional, server to client only
Works over standard HTTP, no special infrastructure required
The natural fit for AI text generation, since the client only needs to receive tokens, not send data mid-stream

WebSockets

Bidirectional, full duplex communication
Better suited for genuinely interactive real-time features like collaborative editing or live multiplayer state
Overkill for most AI chat features, and adds real infrastructure complexity you don't need for one-way token streaming

Web Streams API (ReadableStream)

The lowest-level option, giving direct control over how chunks are read and processed
What the AI SDK and most Route Handler implementations are actually built on under the hood
Best when you need fine-grained control over parsing structured data mid-stream, like tool calls or JSON fragments

For the vast majority of AI chat and generation features, SSE built on top of the Web Streams API is the right choice. Reach for WebSockets only when you genuinely need bidirectional communication during generation, which is rarer than most implementations assume.

Step-by-Step: Building a Streaming AI Route Handler

Step one: set up the project

npx create-next-app@latest ai-streaming-app
cd ai-streaming-app

Select the App Router when prompted. This pattern depends on Route Handlers, which live specifically in the App Router architecture.

Step two: create the streaming Route Handler

// app/api/chat/route.ts
export async function POST(req: Request) {
  const { message } = await req.json();

  const stream = new ReadableStream({
    async start(controller) {
      const response = await callModelProvider(message);

      for await (const chunk of response) {
        controller.enqueue(new TextEncoder().encode(chunk));
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: { "Content-Type": "text/event-stream" },
  });
}

The key detail here: the API key and the call to the model provider stay entirely server-side. Nothing sensitive ever ships to the client bundle, which matters both for security and for keeping your provider costs from being exposed.

Step three: consume the stream on the client

// app/components/ChatInput.tsx
'use client';
import { useState } from 'react';

export default function ChatInput() {
  const [response, setResponse] = useState('');

  async function handleSubmit(message: string) {
    setResponse('');
    const res = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify({ message }),
    });

    const reader = res.body?.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader!.read();
      if (done) break;
      setResponse((prev) => prev + decoder.decode(value));
    }
  }

  return <div>{response}</div>;
}

This is the piece that actually creates the typing effect. Each chunk arrives and appends to state as it comes in, instead of waiting for one giant response.

Step four: add proper error and cancellation handling

const controller = new AbortController();

async function handleSubmit(message: string) {
  try {
    const res = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify({ message }),
      signal: controller.signal,
    });
    // stream reading logic here
  } catch (err) {
    if (err.name === 'AbortError') {
      console.log('Generation cancelled by user');
    } else {
      setError('Something went wrong, please try again');
    }
  }
}

function handleCancel() {
  controller.abort();
}

A stop button that actually cancels an in-flight generation is a small detail users notice immediately when it's missing, and it costs very little to implement properly.

Step five: validate structured output when you need more than plain text

import { z } from 'zod';

const responseSchema = z.object({
  summary: z.string(),
  confidence: z.number(),
});

function validateResponse(raw: string) {
  const parsed = JSON.parse(raw);
  return responseSchema.parse(parsed);
}

Instead of hoping a model returns well-formed JSON, defining a schema and validating against it catches malformed output before it ever reaches your UI, which matters a lot once an AI feature is doing more than just displaying conversational text.

Step six: add rate limiting before this ever reaches production

// app/api/chat/route.ts
const requestCounts = new Map<string, number>();

export async function POST(req: Request) {
  const userId = getUserIdFromSession(req);
  const currentCount = requestCounts.get(userId) ?? 0;

  if (currentCount >= 20) {
    return new Response("Rate limit exceeded", { status: 429 });
  }

  requestCounts.set(userId, currentCount + 1);
  // proceed with streaming logic
}

This is a simplified example, a real implementation would use a proper store like Redis rather than an in-memory map, but the principle holds regardless of scale. Without rate limiting, a single user or a scripted abuse pattern can run your model provider costs up fast, and this is one of the most commonly skipped steps in early AI feature builds.

Step seven: track cost per feature, not just in aggregate

function logUsage(feature: string, tokenCount: number, model: string) {
  const cost = tokenCount * getCostPerToken(model);
  analytics.track("ai_usage", { feature, tokenCount, cost, model });
}

Breaking cost down by feature rather than watching one aggregate monthly number lets you catch a specific feature quietly ballooning in usage long before it shows up as a surprise on the bill.

Common Mistakes Worth Avoiding

Forgetting to handle stream errors mid-response, leaving the UI stuck in a permanent loading state if the connection drops partway through
Skipping the cancellation handling entirely, which leaves users stuck watching a response they no longer want finish generating
Calling the model provider directly from client-side code, which exposes API keys and lets costs run up uncontrolled
Not validating structured output, and discovering malformed data only when it breaks something downstream in production
Shipping without rate limiting and finding out the hard way when a single user or bot script drives up provider costs overnight
Choosing WebSockets by default out of habit when SSE would be simpler, cheaper to run, and perfectly sufficient for one-way token streaming

A Quick Self-Check Before You Ship

Does your UI show visible progress within the first second, or does it sit on a static spinner while waiting for the full response?
Can a user actually cancel a generation mid-stream, or does clicking away just leave the request running in the background?
Is there a rate limit in place, or is the endpoint currently wide open to unlimited requests per user?
Are you tracking cost per feature, or would a sudden spike in usage only show up once the monthly bill arrives?

Why This Is Worth Building Properly, Not Just Quickly

Getting a demo chat working in an afternoon is genuinely easy now. Getting it production-ready, proper error handling, rate limiting, cost monitoring, structured output validation, and a UI that gracefully degrades when something goes wrong, is where most teams underestimate the actual scope of the work. Custom AI development and API integration work focused specifically on shipping these features at production scale tends to close that gap far faster than assembling it piece by piece under a deadline, especially for teams integrating multiple providers or building on top of an existing enterprise codebase.

The Takeaway

The gap between an AI feature that feels instant and one that feels sluggish usually isn't the model's raw speed, it's whether the response streams token by token or dumps all at once after a long, silent wait. Next.js's architecture, particularly Server Actions and Route Handlers, maps naturally onto this pattern, which is exactly why it's become the default framework for AI-powered web applications this year.

Has your team already shipped a streaming AI feature in production, or are you still working through the plumbing on this one? Curious what's tripped people up most.

Your ML Model Doesn't Need to Be Bigger. It Needs to Be Smaller and Smarter.

Emma Schmidt — Mon, 27 Jul 2026 06:37:54 +0000

For the last few years, the assumption in machine learning was simple: bigger models mean better results, so scale up and worry about the bill later. That assumption is quietly breaking down in 2026. The real shift happening across serious ML teams right now isn't about building bigger models, it's about deployment efficiency, running smaller, specialized models that cost a fraction as much to run while matching or beating general-purpose performance on the specific task they're actually built for. This is exactly where a lot of ML services work is heading this year, helping teams figure out which tasks genuinely need a massive general model and which ones a smaller, purpose-tuned one handles just as well for a fraction of the cost.

This shift has a name, small language models, and understanding how to actually build and deploy one is quickly becoming a core ML skill rather than a niche specialty.

Why Bigger Stopped Automatically Meaning Better

For a specific, well-defined task, a massive general-purpose model is often carrying enormous amounts of capability you never actually use. Ask a giant model to classify customer support tickets into five categories, and it's spending compute capacity it could use to write poetry or debug Python, none of which the task needs.

A few forces are pushing the industry toward smaller, specialized models instead:

Inference cost adds up fast at real scale. A model that's cheap to query once becomes expensive very quickly across millions of daily requests, and a smaller model tuned for the specific task can cut that cost dramatically
Latency matters more as AI moves into real-time features. A smaller model responds faster, which matters enormously for anything user-facing where a slow response actively hurts the experience
On-device and edge deployment is only realistic with smaller models. A model that needs to run on a phone, a browser, or an embedded device simply can't be enormous
Specialized accuracy often beats general capability. A model trained specifically on your domain's data and terminology frequently outperforms a general model on that exact task, even at a fraction of the parameter count

The Two Core Techniques Worth Actually Understanding

Two techniques make this shift practical, and both are worth understanding at a working level even if you're not building the tooling yourself.

Quantization reduces the precision of a model's internal numbers, typically from 32-bit floating point down to 8-bit or even 4-bit representations. This shrinks the model's memory footprint and speeds up inference, often with only a small, carefully managed loss in accuracy.

Distillation trains a smaller "student" model to mimic the behavior of a larger "teacher" model. Instead of learning from raw data alone, the student learns from the teacher's outputs, effectively compressing what the larger model learned into a much smaller package.

Together, these two techniques are why a well-tuned small model can now handle tasks that would have required a massive general model just a couple of years ago.

A Practical Walkthrough: Quantizing a Model for Deployment

Let's walk through a basic quantization workflow using a common open-source library, the kind of step that turns a model too large for practical deployment into one that runs efficiently in production.

Step one: load your baseline model

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "your-base-model"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

Step two: apply post-training quantization

import torch

quantized_model = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

This converts the model's linear layers to 8-bit integer precision, typically shrinking the model size significantly and speeding up inference on CPU, often with minimal accuracy loss for well-suited tasks.

Step three: benchmark before trusting the result

import time

def benchmark(model, input_ids, runs=50):
    start = time.time()
    for _ in range(runs):
        with torch.no_grad():
            model(input_ids)
    return (time.time() - start) / runs

original_latency = benchmark(model, sample_input)
quantized_latency = benchmark(quantized_model, sample_input)
print(f"Original: {original_latency:.4f}s, Quantized: {quantized_latency:.4f}s")

Never skip this step. The whole point of quantization is a measurable efficiency gain, and you need real numbers on your actual hardware and actual data, not an assumption borrowed from a blog post.

Step four: validate accuracy didn't quietly degrade

def evaluate_accuracy(model, test_dataset):
    correct = 0
    for inputs, label in test_dataset:
        with torch.no_grad():
            prediction = model(inputs).logits.argmax()
        correct += (prediction == label)
    return correct / len(test_dataset)

print(f"Original accuracy: {evaluate_accuracy(model, test_set):.3f}")
print(f"Quantized accuracy: {evaluate_accuracy(quantized_model, test_set):.3f}")

A quantized model that's faster but meaningfully less accurate isn't actually a win. This comparison step is what turns quantization from a guess into a real, measured engineering decision.

Where Small Language Models Are Already Winning

This isn't theoretical. Real deployments are already showing the pattern clearly.

Customer support ticket routing using a small, domain-tuned classifier instead of a general-purpose model call for every incoming ticket, cutting both latency and per-request cost significantly
On-device features in mobile apps, where a compact model handles tasks like text summarization or intent detection locally, without a round trip to a server at all
Manufacturing and IoT anomaly detection, where small, efficient models run directly on edge hardware near the sensors generating the data, since sending everything to the cloud for a giant model to process isn't fast enough for real-time decisions
High-volume, narrow classification tasks like spam detection or content moderation, where a small specialized model handles the bulk of routine cases and only escalates genuinely ambiguous ones to a larger model

A Quick Framework for Deciding When to Go Small

Before defaulting to the largest available model, ask these questions honestly.

Is this task narrow and well-defined, or does it genuinely require broad, general reasoning across many domains?
Do we have enough quality training or distillation data specific to this task to teach a smaller model well?
Does latency or on-device deployment actually matter for this specific feature?
What's the real cost difference at our expected production volume, not just at demo scale?

If a task is narrow, well-defined, and running at real volume, a small, specialized model is very often the better engineering choice, not a compromise.

Common Mistakes Worth Avoiding

Quantizing a model and shipping it without actually benchmarking the accuracy tradeoff on real evaluation data
Assuming distillation is a one-time step rather than an ongoing process that needs revisiting as the underlying task or data shifts over time
Defaulting to the largest general model out of convenience for tasks that a much smaller, cheaper model would handle just as well
Skipping proper monitoring after deployment, since a smaller model can drift or degrade in production just like any other model, and needs the same kind of ongoing evaluation

Why This Is Worth Doing Properly, Not Just Quickly

Getting the quantization, distillation, and evaluation pipeline right takes real ML engineering discipline, not just running a library function once and hoping for the best. For teams without a dedicated MLOps function already in place, building this properly alongside shipping product features is a genuinely heavy lift. Custom model development, deployment pipeline setup, and ongoing MLOps support built around this exact efficiency-first approach tend to get a team to a well-benchmarked, production-ready small model far faster and more reliably than assembling it piecemeal under deadline pressure.

The Takeaway

The biggest available model was never automatically the right choice, it just looked that way while nobody was measuring the actual cost and latency tradeoffs closely. As ML moves deeper into real-time, on-device, and high-volume production use cases, the teams pulling ahead are the ones treating model size as a deliberate engineering decision, matched carefully to the task, rather than a default setting left on maximum.

Is your team still defaulting to the largest available model for every task, or have you already started matching model size to the actual job? Curious how far along different teams are with this shift.

Your Chatbot Answered Perfectly Yesterday. Today It's Confidently Making Things Up. Nobody Noticed.

Emma Schmidt — Fri, 24 Jul 2026 06:54:44 +0000

Your uptime dashboard is green. Response times look normal. Error rates are near zero. And yet, somewhere in production right now, your AI feature might be giving out subtly wrong answers, hallucinated policy details, a drifted tone, an answer that's technically valid but wrong for your actual use case, and none of your existing monitoring would ever catch it. This is exactly the blind spot pushing AI services from a bolted-on feature toward something that needs the same rigor as any other production infrastructure, with dedicated observability, evaluation, and monitoring built in from the start rather than added after something goes wrong.

This gap has a name now: LLM observability, and it's becoming one of the fastest-growing corners of AI infrastructure this year for a very simple reason. Traditional monitoring was built for deterministic software. AI models aren't deterministic, and the old playbook doesn't catch the failures that actually matter.

Why Your Existing Monitoring Can't See This

Traditional application monitoring answers one question well: did the request succeed, and how fast. That's genuinely useful, and it's also completely blind to the specific way AI systems fail.

A response can be fast, formatted correctly, and still be wrong. A hallucinated fact, a policy detail that's outdated, or an answer that sounds confident but references the wrong context all return a perfectly healthy 200 status code
The same prompt can produce different quality on different days. A model that answered a question correctly last week can drift into a subtly wrong pattern this week with no code change on your end at all
Multi-step AI workflows hide their failures in the middle. When a system retrieves documents, reasons over them, and calls tools before producing a final answer, a failure in any middle step can still produce an output that looks fine on the surface
Cost and token usage can balloon silently. A workflow that used to cost a few cents per request can quietly triple in cost as prompts grow or retrieval pulls in more context than intended, with nothing in a standard dashboard flagging it

None of these show up as an outage. They show up as a slow, invisible erosion of quality that nobody notices until a customer complains, or worse, until nobody notices at all.

The Four Things Real AI Observability Actually Tracks

Getting real visibility into an AI system means watching more than the usual infrastructure signals.

System health. The traditional layer, latency, error rates, uptime, still matters and still needs watching
Model performance and quality. Accuracy against known good answers, hallucination rates, and semantic drift over time, not just whether a response was returned
Cost tracking. Token usage and per-request cost broken down by feature, so a quiet cost creep gets caught before it shows up as a surprise on the monthly bill
Behavioral tracing. Full visibility into multi-step workflows, what was retrieved, what reasoning path was taken, what tools were called, so a wrong answer can actually be debugged instead of shrugged at

A Practical Way to Start Building This

You don't need a massive platform overhaul to start closing this gap. Here's a reasonable starting sequence.

Step one: log the full trace, not just the final output

def log_llm_call(prompt, retrieved_docs, response, metadata):
    trace = {
        "timestamp": datetime.utcnow(),
        "prompt": prompt,
        "retrieved_context": retrieved_docs,
        "response": response,
        "token_count": metadata["tokens"],
        "latency_ms": metadata["latency"],
        "model_version": metadata["model"],
    }
    trace_store.save(trace)

Without the full trace, a bad output down the line is nearly impossible to debug. Was it a bad retrieval, a bad prompt, or a bad model response? You can't tell from the final answer alone.

Step two: run automated quality scoring on a sample of live traffic

def score_response(prompt, response, retrieved_docs):
    return {
        "faithfulness": check_grounded_in_context(response, retrieved_docs),
        "relevance": check_relevance(prompt, response),
        "hallucination_risk": detect_unsupported_claims(response, retrieved_docs),
    }

Scoring every single production response can be expensive, but sampling a meaningful percentage catches drift long before it becomes a visible pattern to users.

Step three: set alerts on quality signals, not just infrastructure signals

if quality_scores["faithfulness"] < 0.7:
    alert_team(f"Faithfulness score dropped: {quality_scores}")

A latency alert tells you something broke. A faithfulness or drift alert tells you something is quietly getting worse, which is the failure mode that actually damages user trust over time.

Step four: track cost per feature, not just cost in aggregate

cost_by_feature = defaultdict(float)
cost_by_feature[feature_name] += token_count * cost_per_token

A single feature quietly ballooning in token usage is easy to miss in an aggregate monthly bill and very easy to catch when broken down by feature and tracked over time.

Step five: close the loop by feeding failures back into evaluation

Every flagged failure should become part of an ongoing evaluation dataset, not a one-off fire drill. This is what actually improves the system over time instead of just reacting to the same category of failure repeatedly.

Real Numbers Worth Knowing

The scale of this gap is bigger than most teams assume. A large share of enterprises now require AI monitoring in production, yet a majority cite a lack of adequate observability tooling as a major barrier to actually having it. That gap between what teams know they need and what they've actually built is exactly where the invisible failures described above tend to live.

Evaluation overhead itself is a real cost too. Adding a full evaluation pass to a live request can meaningfully increase response latency, which is part of why the emerging best practice for 2026 is combining lightweight in-line scoring on every request with deeper evaluation sampling run asynchronously, rather than trying to fully evaluate every single response in the live path.

Common Mistakes Teams Make Getting Started

Treating LLM observability as identical to traditional APM and only tracking latency and error rates, missing quality entirely
Scoring 100 percent of production traffic with an expensive evaluation model, which quietly doubles response latency and cost
Building alerts only for hard failures and missing the slow, gradual drift that's actually the more common failure pattern
Never closing the loop between a caught failure and an updated evaluation dataset, so the same category of mistake keeps recurring

Why This Is Worth Building Properly

Getting this right touches more than one layer, evaluation design, tracing infrastructure, alerting thresholds, and cost monitoring all need to work together rather than existing as disconnected tools. For teams without dedicated MLOps or AI infrastructure specialists on staff already, building this from scratch while also shipping product features is a genuinely heavy lift. Custom AI integration and observability setup, done by people who've built this exact monitoring layer before, tends to get a team from flying blind to genuinely production-grade visibility far faster than assembling it piecemeal under deadline pressure.

The Takeaway

A green dashboard doesn't mean your AI feature is actually working well, it means the parts of it that traditional monitoring knows how to measure are working. The parts that actually matter to users, whether the answer was true, relevant, and useful, live in a layer most teams still aren't watching. That gap is exactly what LLM observability exists to close, and closing it early is a lot cheaper than discovering the failure the way most teams still do, from a customer complaint.

Is your team actually tracking AI output quality in production, or is your monitoring stack still only watching the infrastructure layer? Curious how many of us have actually closed this gap yet.

Your App Is Fast Enough. It's Also Quietly Burning More Energy Than It Needs To.

Emma Schmidt — Thu, 23 Jul 2026 12:12:35 +0000

Tags: #sustainability #cloudcomputing #webdev #discuss
Somewhere in your cloud bill this month, there's a number nobody on your team has ever really looked at: how much energy your application actually consumes to do its job. Not cost, energy. Every unnecessary database query, every bloated Docker image, every inefficient loop running at scale is drawing real power from a real data center, and most engineering teams have never once measured it. This gets even more pressing once AI services enter the picture, since AI inference is one of the most energy-hungry workloads a modern application can run, and it's exactly where a lot of this invisible waste tends to hide first.

That's starting to change fast, and not because of idealism. Green coding, writing and architecting software specifically to minimize energy consumption, has moved from a nice-to-have sustainability talking point to a genuine engineering discipline businesses are being pushed toward from multiple directions at once.

Why This Is Suddenly Everyone's Problem

A few forces are converging here, and none of them are going away.

Cloud costs and energy consumption are now directly linked in a way they weren't a few years ago, as providers increasingly price compute in ways that reflect actual power draw, not just raw usage time
Regulatory pressure is rising in multiple regions requiring companies to report on and reduce their digital carbon footprint, and software energy use is a growing line item in those reports
AI workloads specifically have made this urgent. Training and running AI models is extraordinarily energy intensive, and companies bolting AI features onto every product are watching their infrastructure energy costs climb in ways that are hard to ignore
Customers and investors are starting to ask. Sustainability reporting increasingly includes technology stack efficiency, not just the usual categories like travel and office energy use
Data center capacity itself is becoming a real constraint, with some regions seeing power grid strain directly tied to the growth of compute-heavy workloads, which puts pressure back on the software layer to use less of it

None of this requires anyone to care personally about the planet to take it seriously. It's becoming a real line item, a real compliance risk, and a real cost center.

What Inefficient Code Actually Looks Like at Scale

Here's the part that surprises people. None of this requires exotic new practices. It's mostly ordinary engineering habits that were never a priority because energy cost was invisible.

N+1 queries running millions of times a day across a large user base, each one drawing real compute and real power for work that could've been a single efficient join
Polling instead of event-driven updates, where a client checks for new data every few seconds indefinitely, burning compute around the clock for data that changes rarely
Oversized container images that take longer to build, deploy, and run than they need to, multiplied across every deployment and every autoscaling event
Unoptimized AI inference calls, sending far more tokens or running a larger model than the task actually requires, when a smaller, purpose-built model would do the job with a fraction of the energy draw
Idle infrastructure left running in staging and development environments around the clock, consuming power for workloads nobody's actually using outside business hours
Unbounded background jobs that retry endlessly or run on overly aggressive schedules, quietly consuming compute for work that could run a fraction as often

Individually, each of these looks like a minor inefficiency. At the scale most production systems run at, they add up to a genuinely significant amount of wasted energy and wasted money, quietly, every single day.

A Real-World Example of How This Adds Up

Take a mid-sized ecommerce platform serving a few million requests a day. Picture a single product page that fires an N+1 query pattern loading reviews, one that polls every three seconds for inventory updates that actually change a few times an hour, and a recommendation feature calling a large general-purpose AI model when a lightweight, purpose-built model would return the same quality of result.

None of those three issues, on their own, would ever show up as an incident. No outage, no page, no angry customer email. But multiplied across millions of page loads a day, that's a meaningful chunk of unnecessary compute running continuously, month after month, for work that could've been a fraction of the size with no visible change to the user experience at all. That's the actual shape of this problem in most real production systems: not one dramatic failure, but a thousand small, invisible ones running constantly.

A Practical Starting Point

You don't need a sustainability team to start improving this. A few concrete places to look first.

Audit your database query patterns
Look specifically for N+1 patterns and queries running far more often than the data actually changes. This is usually the single highest-impact place to start, because database load scales directly with both cost and energy draw.

Replace polling with event-driven architecture where you can
A webhook or a push notification that fires only when something actually changes uses a fraction of the compute of a client checking in every few seconds forever.

Right-size your AI model usage
Not every task needs the largest, most capable model available. Matching model size to task complexity is one of the fastest-growing cost and energy savings levers teams are finding this year, precisely because so many teams default to the biggest model out of convenience rather than necessity.

Shut down idle environments automatically
Staging and dev environments running 24/7 for workloads only used during business hours are pure waste. Automated shutdown schedules are a low-effort, high-impact fix.

Batch and schedule background work intelligently
Jobs that run every minute out of habit rather than necessity are an easy, low-risk place to cut unnecessary compute cycles without touching anything user-facing.

Measure before you optimize
Most teams have never actually measured their application's energy footprint because the tooling to do it wasn't mainstream until recently. You can't meaningfully improve what you've never measured.

A Quick Self-Check

Before deciding this doesn't apply to your team, answer these honestly.

Do you know which endpoints in your application generate the most database load relative to how often that data actually changes?
Are any of your staging or dev environments running continuously, with nobody checking whether they need to be?
Is your team defaulting to the largest available AI model for every task, regardless of how simple the task actually is?
Has anyone ever pulled a report on your cloud provider's energy or carbon metrics, or does that dashboard exist unopened?

If more than one of these gave you pause, there's likely real, low-risk savings sitting in your stack right now, unmeasured and unaddressed.

Common Questions Teams Ask When Starting This

Does this require rewriting our whole application?
Almost never. Most of the highest-impact fixes, like resolving N+1 queries or shutting down idle environments, are targeted, low-risk changes rather than architectural rewrites.

Will this slow down feature development?
An initial audit takes focused time, but the fixes that come out of it are usually incremental and can be folded into normal sprint work rather than requiring a dedicated multi-month initiative.

Is this only relevant for very large-scale applications?
The savings scale with traffic, but the underlying inefficiencies exist at any scale. Catching them early is actually easier and cheaper than untangling them after years of accumulated scale.

Why This Matters Beyond the Obvious

There's a business case here independent of any environmental motivation. Energy-efficient code tends to correlate strongly with genuinely well-architected code. The same discipline that reduces unnecessary compute cycles also reduces latency, reduces cloud spend, and reduces the blast radius of scaling problems under real traffic. Teams chasing energy efficiency usually end up with a faster, cheaper, more resilient system as a side effect, not a tradeoff.

This is also exactly the kind of work that benefits from a proper audit rather than scattered fixes. Untangling inefficient query patterns, right-sizing infrastructure, and rearchitecting around event-driven patterns across an existing production system touches database design, cloud architecture, and AI services integration all at once, which is a lot to take on internally while also shipping new features on the usual roadmap. A dedicated cloud architecture and performance audit, done by people who've mapped this exact kind of waste before, tends to surface savings that are easy to miss when you're deep inside your own codebase every day.

The Takeaway

The energy your code consumes was always real, it just wasn't visible on a dashboard anyone was watching. As regulation tightens, AI workloads grow, and cloud providers price compute closer to its real power draw, that invisibility is disappearing fast. The teams getting ahead of this now aren't doing it out of idealism, they're doing it because efficient code was always going to be cheaper, faster, and more resilient too, the environmental benefit just happens to come along for free.

Has your team ever actually measured your application's energy footprint, or is it still an invisible number buried somewhere in a cloud bill nobody's dug into? Curious how many of us are actually tracking this yet.

Prompt Engineering Is Dead. Context Engineering Is What Actually Fixes Your AI's Wrong Answers.

Emma Schmidt — Wed, 22 Jul 2026 10:54:05 +0000

You've tuned the prompt a dozen times. Added examples, adjusted the tone, tried three different phrasings of the same instruction. The AI still hallucinates a fact, misses context from earlier in the conversation, or confidently answers a question using the wrong data. If this sounds familiar, the problem was never the prompt. It's what's called context engineering, and it's quietly become one of the most important skills in AI development this year. Teams that get this right often lean on structured AI development and integration services specifically because getting the architecture right the first time avoids months of trial-and-error prompt tweaking that never actually fixes the root cause.

Here's what context engineering actually is, why it matters more than clever prompting, and how to build it into a real application step by step.

Why Prompt Engineering Alone Stops Working

Prompt engineering treats the model like a black box you talk to more cleverly. Context engineering treats the entire system around the model, what data it sees, in what order, and how it's structured, as the thing you actually design.

Here's the distinction that matters:

Prompt engineering optimizes the instruction you give the model in a single message
Context engineering optimizes everything the model has access to when it generates that response: retrieved documents, conversation history, system instructions, tool outputs, and how all of that is assembled and prioritized

A perfectly worded prompt pointed at the wrong or poorly organized context still produces a wrong answer. This is why teams that only tune prompts eventually hit a wall, no phrasing fixes a model that's missing the right information entirely.

The Core Problem You're Actually Solving

Every AI application built on top of a language model faces the same limitation: a fixed context window and no persistent memory between calls. Context engineering is the discipline of deciding what goes into that limited space and how it's organized so the model can actually use it well.

Get this wrong and you'll see familiar symptoms:

The model confidently answers using outdated or irrelevant retrieved data
Earlier parts of a long conversation get silently dropped or ignored
Retrieved documents get stuffed in with no clear structure, and the model can't tell what's actually relevant
Adding more context makes answers worse, not better, because signal gets buried in noise

Building a Real Context Pipeline, Step by Step

Let's walk through building this properly, using a support chatbot pulling from a knowledge base as the example.

Step one: separate your context into clear categories

context = {
    "system_instructions": "You are a support assistant for Acme SaaS...",
    "retrieved_docs": [],       # from your knowledge base
    "conversation_history": [], # prior turns in this session
    "user_profile": {},         # account tier, past tickets, etc
    "current_query": ""
}

Treating these as separate, labeled components instead of one giant blob of text is the single biggest shift from prompt engineering to context engineering. It lets you control, debug, and prioritize each piece independently.

Step two: retrieve only what's actually relevant

def retrieve_context(query, top_k=3):
    query_embedding = embed(query)
    results = vector_db.search(query_embedding, top_k=top_k)
    return [r for r in results if r.score > 0.75]

Notice the relevance threshold. Pulling in five loosely related documents to "give the model more to work with" usually makes answers worse, not better. Fewer, highly relevant documents consistently outperform a large pile of mediocre ones.

Step three: manage conversation history deliberately, not by just appending everything

def build_history(messages, max_turns=6):
    recent = messages[-max_turns:]
    if len(messages) > max_turns:
        summary = summarize(messages[:-max_turns])
        return [{"role": "system", "content": f"Earlier context: {summary}"}] + recent
    return recent

Long conversations without this kind of management silently push early, important context out of the window entirely. Summarizing older turns instead of dropping them keeps continuity without blowing the token budget.

Step four: assemble everything with clear structure, not a wall of text

def build_prompt(context):
    return f"""
System: {context['system_instructions']}

Relevant documentation:
{format_docs(context['retrieved_docs'])}

Conversation so far:
{format_history(context['conversation_history'])}

User profile: {context['user_profile']}

Current question: {context['current_query']}
"""

Clear section labels matter more than they seem like they should. Models handle structured, clearly delineated context noticeably better than the same information dumped in as one undifferentiated paragraph.

Step five: test with adversarial and edge-case queries, not just the happy path

test_cases = [
    "question referencing something from 10 turns ago",
    "question where retrieved docs are slightly outdated",
    "question with no relevant docs in the knowledge base at all",
]

This is the step most teams skip. A context pipeline that works great on clean, simple test queries often falls apart the moment a real user asks something messy, which is exactly when it matters most.

Common Mistakes Worth Knowing Before You Build This

Context window stuffing. Cramming in every possibly-relevant document instead of the few genuinely relevant ones, which drowns out the signal
No relevance filtering on retrieval. Returning the top results regardless of how weak the match actually is
Flat, unstructured context. Pasting everything into one block of text with no labeled sections for the model to anchor on
Ignoring token budget until it breaks. Not tracking how much of the context window is consumed by each component until responses start silently truncating
No evaluation loop. Shipping the pipeline without a way to systematically test whether context changes actually improved or hurt answer quality

The Takeaway

If your AI feature is giving inconsistent or wrong answers and you've already spent real time tuning the prompt, the prompt was probably never the actual problem. Look at what the model can see when it answers, how it's retrieved, how it's structured, and how much of it is genuinely relevant. That's where the real fix usually lives.

Have you run into this wall yet, prompt tuning stops helping and the real fix turns out to be the context pipeline underneath it? Curious what broke it open for you.

Your AI Code Review Passed. That Doesn't Mean Your Code Is Safe.

Emma Schmidt — Mon, 20 Jul 2026 05:49:39 +0000

You push a PR. The automated review runs, comes back green, no issues flagged. You merge with confidence because the tool said it was clean. Two days later a teammate finds a broken auth check sitting in that exact file, something the review tool should have caught and simply didn't.

This isn't a rare, unlucky edge case. Right now it's one of the fastest-growing complaint clusters showing up across GitHub issues, Stack Overflow, and developer forums: AI code review tools silently failing to execute properly, then reporting a clean pass anyway. Not flagging false positives. Not being too strict. Actually not running the check at all, and telling you everything's fine regardless.

That gap between "the tool said pass" and "the code is actually safe" is exactly where teams are getting burned this year.

Why This Keeps Happening

The pattern behind it is almost always the same, and it's less exciting than a model being "wrong." It's plumbing.

The review tool hits an environment misconfiguration mid-run, fails to execute the actual check, and reports success anyway instead of erroring loudly
A dependency version mismatch between the CI environment and what the review tool expects causes it to silently skip files instead of flagging the mismatch
Permissions or API scope issues cause partial runs that only cover some of the changed files, but the summary reads as if everything was reviewed
Rate limiting or timeout issues cut a review short, and the tool defaults to "no issues found" instead of "review incomplete"

None of these are the AI being fooled by clever code. The AI never actually looked. The failure happened one layer below it, in configuration, and the interface just didn't have a way to say "I couldn't finish this" instead of "this looks fine."

According to recent industry analysis tracking developer complaint patterns across GitHub, Stack Overflow, Hacker News, and Bluesky, this exact failure mode is currently one of the highest-scoring pain clusters being reported, and it's still climbing rather than leveling off. That's a strong signal this isn't one unlucky team's config mistake, it's a systemic gap showing up across a large number of independent pipelines right now.

Real Examples of How This Shows Up

To make this less abstract, here are the actual shapes this bug tends to take in production.

A review bot configured with an outdated API key silently returns an empty result set instead of an authentication error, and the pipeline reads that empty result as "nothing to flag"
A large PR gets split across multiple review batches, one batch times out, and only the completed batches get reported, with no indication that a batch was dropped
A newly added file type isn't covered by the reviewer's configuration, so it's skipped entirely, but the summary still says "all files reviewed"
A flaky network call to the review service fails on retry, the pipeline catches the exception, and defaults to a pass state instead of surfacing the failure

Every one of these looks identical from the outside. Green checkmark, no comments, merge button unlocked.

How to Know If You're Already Exposed

Ask yourself these honestly, right now, about your current setup.

Does your review tool distinguish clearly between "no issues found" and "review didn't complete," or do both look identical in your PR checks?
Have you ever actually tested what happens when the review tool hits a timeout or a permissions error mid-run?
Do you know exactly which files were covered in the last ten "passed" reviews, or are you trusting the green checkmark on faith?
Has anyone on your team ever manually re-reviewed a "passed" PR and found something the tool missed?

If you can't answer the first two confidently, you likely have a blind spot exactly like the one that let that auth bug through.

A Practical Way to Catch This

You don't need to abandon AI-assisted review, you just need to stop treating a green checkmark as proof of anything on its own.

Step one: log what was actually reviewed, not just the verdict

# .github/workflows/review-check.yml
- name: Verify review completeness
  run: |
    echo "Files changed: $(git diff --name-only origin/main | wc -l)"
    echo "Files covered by review: ${{ steps.review.outputs.files_reviewed }}"

Comparing these two numbers directly is often the single fastest way to catch a partial run before it merges.

Step two: fail loudly instead of failing silently

- name: Fail if review coverage is incomplete
  if: steps.review.outputs.files_reviewed < steps.diff.outputs.files_changed
  run: exit 1

A tool that can't complete its job should block the merge, not quietly wave it through. This one change closes most of the actual gap.

Step three: spot-check with a real second pass

Pick a random sample of "passed" PRs each week and have a human actually read the diff. Not every PR, just enough to catch drift before it becomes a pattern nobody noticed.

Step four: track the metric that actually matters

Review coverage percentage over time tells you far more than pass/fail counts ever will. A tool passing 100% of PRs while silently covering 60% of the files in them is worse than no automated review at all, because it's actively creating false confidence.

Step five: alert on silence, not just on failure

- name: Alert if reviewer produced no output at all
  if: steps.review.outcome == 'success' && steps.review.outputs.comment_count == null
  run: echo "::warning::Review completed with zero output, verify manually"

A review with literally zero comments on a large, complex PR is itself a signal worth flagging, not just quietly accepting as a clean bill of health.

Why This Matters More Than It Looks Like It Does

This isn't a minor tooling annoyance. It's currently one of the highest-volume, fastest-growing pain point clusters developers are reporting across multiple independent platforms this year, which means it's not one team's bad luck, it's a systemic gap in how a lot of review pipelines are configured right now.

Closing this gap usually touches more than one layer of the stack, and the kind of support that actually helps includes:

Auditing existing CI and code review pipelines to find where failures are getting swallowed instead of surfaced
Setting up proper CI/CD workflows with explicit failure states, so "couldn't complete" never gets reported as "passed"
QA and testing automation that adds a real second layer of verification instead of relying on a single tool's word
Ongoing DevOps support to keep the pipeline configuration from silently drifting out of sync as dependencies update

Auditing and hardening a CI and code review pipeline so failures surface loudly instead of hiding behind a green checkmark is exactly the kind of infrastructure work that's easy to deprioritize until it costs you a production incident. This kind of pipeline auditing and DevOps support, focused specifically on catching these silent failure modes, is often the fastest way to close this gap without your team losing weeks untangling it mid-sprint.

The Real Takeaway

A passing check is not the same thing as a completed review, and right now a lot of pipelines are treating those two things as identical when they're not. The fix isn't more trust in the tool, it's less blind trust and a system that fails loudly when it can't actually do its job.

Has your team ever caught a "passed" review that actually missed something? What tipped you off? Genuinely curious how widespread this is once people start comparing notes.

Your Next.js Caching Strategy Is Probably Broken and You Won't Know Until Production

Emma Schmidt — Fri, 17 Jul 2026 06:37:05 +0000

Picture this. You ship a feature on a Friday. Everything works perfectly in every test you ran. Monday morning, support tickets start piling up. Users are seeing stale prices, old inventory counts, yesterday's data on pages that are supposed to update in real time. Nobody touched the code over the weekend. The caching layer just quietly decided to serve old data, and nobody noticed until customers did.

If that story makes your stomach drop a little, you've lived some version of it. Caching in Next.js has always been powerful and also, frankly, confusing. Too many overlapping mechanisms, unclear boundaries between what's cached and what isn't, and a debugging experience that mostly involves guessing. That's exactly the problem Cache Components were built to solve, and they're one of the biggest shifts in how Next.js apps are built this year.

The Real Problem Developers Keep Running Into

Before Cache Components, caching in Next.js meant juggling several different systems at once, each with its own rules.

Here's what that actually looked like in practice.

Fetch requests had their own caching behavior, controlled through options that were easy to forget or misconfigure
Route segments had a separate caching model tied to how the route itself was structured
Revalidation logic lived in yet another place, disconnected from the component actually rendering the data
One misconfigured fetch call could silently cache data that should have been dynamic, and the bug wouldn't show up until real traffic hit it
Debugging meant tracing through multiple files and configuration layers just to answer one simple question: why is this page showing old data

The result was a system that worked fine in a demo and quietly broke in ways that only showed up under real production conditions, which is exactly the worst time to discover a caching bug.

What Cache Components Actually Change

Cache Components flip the caching model on its head. Instead of caching being an implicit side effect scattered across fetch calls and route configs, caching becomes an explicit, visible part of the component itself.

The core idea is simple. You mark exactly which parts of your UI are cacheable, right at the component level, and everything else is dynamic by default. No more guessing which layer decided to cache something. No more hunting through config files. The caching boundary lives right next to the code it affects.

This matters because it turns a debugging nightmare into something you can actually reason about by reading the component tree.

Building It Step by Step

Let's walk through what this looks like in a real project, starting with the problem from the intro: a product page that needs live inventory data but was accidentally serving stale numbers.

Step one: enable Cache Components in your project

// next.config.js
module.exports = {
  experimental: {
    cacheComponents: true,
  },
};

This flips the default behavior so components are dynamic unless you explicitly say otherwise, which is the opposite of the old implicit caching model and immediately removes an entire category of accidental staleness.

Step two: mark what should actually be cached

// components/ProductDescription.jsx
'use cache';

export default async function ProductDescription({ productId }) {
  const product = await getProductDetails(productId);
  return (
    <div>
      <h2>{product.name}</h2>
      <p>{product.description}</p>
    </div>
  );
}

The 'use cache' directive right at the top of the component is the whole point. Anyone reading this file instantly knows this data is cached, without needing to trace through fetch options or route configuration somewhere else.

Step three: keep the volatile data dynamic, explicitly

// components/InventoryStatus.jsx
export default async function InventoryStatus({ productId }) {
  const inventory = await getLiveInventory(productId);
  return <span>{inventory.count} in stock</span>;
}

No cache directive here on purpose. This component always fetches fresh, and it's obvious at a glance why, because there's no 'use cache' line sitting at the top of it.

Step four: compose them together on the same page

// app/products/[id]/page.jsx
import ProductDescription from '@/components/ProductDescription';
import InventoryStatus from '@/components/InventoryStatus';

export default function ProductPage({ params }) {
  return (
    <div>
      <ProductDescription productId={params.id} />
      <InventoryStatus productId={params.id} />
    </div>
  );
}

This is the part that actually solves the Friday-to-Monday problem from the intro. The description, which rarely changes, gets cached and served fast. The inventory count, which changes constantly, stays dynamic every single time. Both live on the same page, and the boundary between them is sitting right there in the code, not buried in a config file three folders away.

Step five: verify it under real conditions, not just locally

curl -I https://your-app.com/products/123

Check the cache headers directly instead of trusting that it's working because it looked fine in dev mode. This is the step most teams skip, and it's exactly the step that would have caught the stale pricing bug before real customers did.

Why This Is the Right Moment to Care

Cache Components landed as part of Next.js 16, and the framework team has been actively rolling out tooling specifically to help existing projects adopt the pattern gradually rather than forcing a painful rewrite. That's a strong signal this isn't an experimental side feature, it's the direction the framework is committing to going forward.

There's also a security angle worth knowing about. Next.js recently moved to a formal monthly security release schedule, with several advisories this year involving middleware and proxy bypass issues in App Router applications. Caching and routing bugs aren't just a performance annoyance anymore, they can quietly become a security gap too, which makes getting this right even more important than it used to be.

Where Teams Actually Get Stuck

Migrating an existing app to Cache Components isn't a find-and-replace job. A few things trip people up consistently.

Assuming a component is cache-safe just because it looks static, when it actually depends on request-specific data like cookies or headers
Forgetting that nested components inherit caching behavior from their parents in ways that aren't always obvious at first glance
Rolling out the change across an entire large app at once instead of migrating one route at a time and verifying behavior as you go
Skipping the production-level verification step and trusting that local dev behavior matches what actually happens under real traffic and real cache layers

Getting this migration right on a production app, especially one with a large existing codebase, real traffic patterns, and revenue riding on pages not showing stale data, is exactly the kind of work that benefits from experienced hands. Custom web application development and performance auditing support built specifically around Next.js architecture can be the difference between a smooth, gradual rollout and a repeat of that Friday-to-Monday nightmare from the top of this post.

The Takeaway

Caching bugs are uniquely painful because they're invisible until real traffic exposes them. Cache Components won't make caching disappear as a concept, but they make the boundaries visible, explicit, and sitting right next to the code that depends on them, instead of hidden three layers deep in configuration nobody remembers writing.

Have you started migrating to Cache Components yet, or are you still untangling the old fetch-level caching rules? Even if you haven't touched it yet, I'd love to hear what's held you back, drop it in the comments.

Your Deploy Process Is Held Together By One Person's Memory, and Everyone Knows It

Emma Schmidt — Thu, 16 Jul 2026 05:27:46 +0000

Ask your team what happens when someone needs to spin up a new service. If the honest answer involves a Slack message to "the person who knows infra," a half-documented runbook from eighteen months ago, and at least one manual step nobody's automated because "it's fine, it only takes five minutes," you're not alone. You're just describing most engineering teams in 2026, whether they admit it publicly or not.

The Problem That's Quietly Slowing Everyone Down

Here's the pattern I keep seeing, on my own team and everyone else's:

New service setup takes days, not because the code is hard, but because provisioning, secrets, and environment config all live in someone's head
The same Terraform module gets copy-pasted and slightly modified across a dozen repos, each one now its own snowflake
Onboarding a new engineer means shadowing someone for two weeks just to learn where things live and who to ask
Every "quick fix" in production touches infra that only one or two people actually understand end to end
Nobody remembers why a particular staging environment exists, but nobody's brave enough to delete it either

None of this shows up in a burndown chart. It shows up as a quiet tax on every single feature, paid in Slack messages, tribal knowledge, and one overworked person who can never actually take a real vacation.

Why This Got Worse, Not Better

You'd think faster tooling would fix this. Instead it made the gap more obvious. Teams are shipping code faster than ever, but the infrastructure and provisioning layer underneath it hasn't kept pace, so the bottleneck just moved. It used to be "how fast can we write the code." Now it's "how fast can we actually get this thing deployed, secured, and observable without bothering the one person who understands the pipeline."

Check how many of these sound familiar:

Your CI pipeline has grown into a script nobody fully understands anymore, edited in place for years
Getting a new environment takes a support ticket instead of a self-service click
Security and compliance checks happen at the very end, right before a deadline, instead of automatically along the way
"Works on staging" and "works in production" are two different, unrelated statements
Rolling back a bad deploy involves someone remembering the exact right sequence of commands, not clicking a button

If three or more of those are true, the actual problem isn't your code quality. It's that nobody's ever sat down and built a real, self-service platform layer between your engineers and your infrastructure.

What a Real Fix Looks Like

This is exactly what platform engineering is meant to solve, and it's picking up serious momentum this year for a simple reason: it turns infrastructure from a bottleneck into a self-service layer.

A working internal developer platform generally means:

A paved road, not a maze. New services get provisioned through a standard template, not by copying whatever repo looked closest last time.
Self-service environments. A developer can spin up a sandboxed environment without filing a ticket and waiting two days for a reply.
Security baked in, not bolted on. Compliance and security scans run automatically as part of the pipeline, not as a manual gate right before launch.
Documentation that isn't a person. If your platform's tribal knowledge lives entirely in one engineer's head, that's not documentation, that's a single point of failure with a name and a Slack handle.
Observability by default. Logs, metrics, and alerts get wired up automatically for every new service, instead of being an afterthought someone adds after the first incident.

What This Actually Looks Like Day to Day

To make this concrete, here's the difference in practice, side by side:

Spinning up a new service, the old way:

Message the one person who knows the Terraform setup
Wait for a reply, then wait again for review
Copy an existing repo and manually strip out what doesn't apply
Manually request secrets access and hope you didn't miss one
Discover a week later that monitoring was never actually wired up

Spinning up a new service, with a real platform layer:

Run a single command or click a template in an internal portal
Environment, secrets, and networking are provisioned automatically
Security and compliance scans run as part of the pipeline, not after
Logging and alerting exist from the first deploy, not after the first incident
A new hire can do this alone on day one, without pinging anyone

A Quick Gut Check

Before you convince yourself this doesn't apply to your team, answer honestly:

If your most senior infra person left tomorrow, how long before something breaks that nobody else can fix?
How many manual steps happen between "code merged" and "code running in production"?
Can a new hire spin up a working environment on day one, or day ten?
Do you know, right now, how many staging environments are running that nobody actually uses anymore?

If those answers make you a little uncomfortable, that discomfort is the actual signal, not the org chart or the sprint velocity.

Why Most Teams Never Actually Fix This

Not because it's not worth fixing. Because it's genuinely hard to prioritize. It never feels as urgent as the next feature deadline, right up until the day it becomes an outage. And building a proper platform layer, one with real self-service provisioning, security automation, and documentation that survives someone leaving, is its own significant engineering project, one that a team already underwater on feature work rarely has the bandwidth to take on internally.

That's usually the point where bringing in outside expertise makes more sense than trying to squeeze it into an already packed roadmap. The kind of help that actually moves the needle here tends to cover things like:

Cloud architecture design and infrastructure audits to find where the snowflakes and single points of failure actually live
DevOps and CI/CD pipeline setup that replaces the script nobody fully understands with something documented and repeatable
Custom internal tooling and self-service platform builds tailored to how your team actually works, not a generic template
Security and compliance automation baked into the pipeline instead of bolted on before a deadline
Ongoing support so the platform stays maintained instead of quietly rotting the way the last one did

A team that's built this exact kind of platform layer before can usually get you from "one person's memory" to "documented, self-service, secure by default" a lot faster than doing it in the margins of a sprint that was already full.

The Real Cost of Waiting

Every month this stays unfixed, the tax gets a little heavier. More snowflake environments. More tribal knowledge locked in one person's head. More engineers quietly frustrated that setting up a simple service takes three days instead of three hours.

Is your team still running on institutional memory, or have you actually built the paved road? Genuinely curious how far along everyone else is, because I suspect most of us are further behind than we'd like to admit.

I Spent Two Weeks Replacing REST With tRPC. Here's What Nobody Tells You.

Emma Schmidt — Wed, 15 Jul 2026 13:51:35 +0000

I've written more REST endpoints than I can count. Request comes in, I write a DTO, I write a validator, I write matching types on the frontend, I pray the two never drift apart. Then last month I finally gave in and rebuilt a mid-sized internal tool using tRPC instead, mostly because half of my dev.to feed wouldn't shut up about it.

Two weeks later I get it. But not for the reasons everyone's tweeting about.

The Problem That Started This

Here's the exact bug that pushed me over the edge. Someone renamed a field on the backend from userId to accountId. The API still returned 200. The frontend still rendered. It just quietly showed undefined everywhere that field was used, and nobody noticed until a customer emailed asking why their dashboard looked broken.

That's the failure mode REST allows by default: a contract that only exists in your head and in a documentation page nobody reads. Nothing crashes, nothing throws, it just silently lies to you.

The Pitch Everyone Makes

The usual sales pitch goes like this:

No more manually syncing types between frontend and backend
No more API documentation going stale the week after you write it
No more runtime surprises because your types are shared end to end
Change a field on the server, TypeScript yells at you on the client before you even save the file

All true. None of it is the actual reason I'd recommend it.

What Actually Changed My Mind

The real shift wasn't the type safety itself. It was how much dead weight quietly disappeared from the codebase once I stopped needing it.

Here's what I deleted from the project by the end of week two:

The OpenAPI spec file, which was already three fields behind reality
The separate client SDK generation step that ran in CI and occasionally just failed for no reason
The Postman collection that three people updated and forty people ignored
A folder of hand-written TypeScript interfaces that existed only to describe what the API returned

// server/router.ts
export const appRouter = router({
  getUser: publicProcedure
    .input(z.object({ id: z.string() }))
    .query(async ({ input }) => {
      return db.user.findUnique({ where: { id: input.id } });
    }),
});

export type AppRouter = typeof appRouter;

// client/api.ts
import { createTRPCClient } from '@trpc/client';
import type { AppRouter } from '../server/router';

const client = createTRPCClient<AppRouter>({ url: '/api/trpc' });

const user = await client.getUser.query({ id: '123' });
// user is fully typed, no codegen step, no .d.ts file to regenerate

That's it. That's the whole integration. No schema file to keep in sync, no build step to remember to run, no "did anyone regenerate the types after that last deploy" Slack message.

Before and After, Side by Side

This is the part that actually sold the rest of my team, so I'll lay it out plainly.

Renaming a field on the backend, REST version:

Update the field in the database model
Update the endpoint response shape
Update the OpenAPI spec, if anyone remembers
Regenerate the client SDK, if the CI step doesn't silently fail
Manually update every frontend type that referenced the old field
Find out three weeks later that one component was missed

Renaming a field on the backend, tRPC version:

Update the field in the database model
Save the file
Every frontend usage that references the old field name turns red in your editor immediately

That second list isn't shorter because I'm exaggerating for effect. It's shorter because two entire categories of human error just stopped being possible.

Where It Actually Falls Apart

Here's the part the hype posts conveniently skip.

tRPC only works when both sides speak TypeScript. The moment any of these show up, the whole pitch changes:

A mobile team building in Swift or Kotlin
A partner integration that needs a documented, language-agnostic contract
A public API with external developers who've never seen your codebase

In every one of those cases you're back to needing REST or GraphQL with real documentation, because "just import the types" doesn't work across a language boundary or a company boundary.

The migration cost is real, and it's not a weekend project. Here's what actually ate my two weeks:

Every existing fetch call needed rewriting, not just the obvious ones
Every error handling pattern had to be redone since tRPC errors don't look like REST errors
Every loading and retry state in the UI needed a second look
One legacy endpoint that three other internal tools depended on had to keep working in REST form the entire time

If you're planning this migration, budget for the boring rewrite work, not just the exciting type-safety demo.

A Quick Gut Check Before You Migrate

Ask yourself these before touching a single endpoint:

Does my frontend and backend both live in TypeScript, ideally the same repo?
Does one team own both sides, or will a separate mobile or partner team need this API too?
Am I trying to fix a real, recurring bug pattern, or just chasing what's trending this month?
Can I migrate one feature at a time instead of doing a big bang rewrite?

If you answered yes to the first two and no to the third, you're the exact use case tRPC was built for.

The Actual Verdict

For an internal dashboard, an admin panel, or any full stack app where one team owns both ends, it's genuinely worth the switch. For anything with external consumers, keep your REST or GraphQL layer and don't let a trend talk you out of a decision that was already correct.

The pattern that seems to be winning in 2026 isn't picking one paradigm forever, it's running both side by side on purpose:

tRPC for the internal application layer, where your own frontend talks to your own backend
REST for webhooks, third-party integrations, and anything public facing

Two systems, each doing the job it's actually good at, instead of forcing one tool to cover every use case.

Would I Do It Again

Yes, but only on projects where I'd draw the same boundary I just described. The type safety is nice. The disappearance of an entire category of "the docs are out of date again" bugs is the part that actually saved me time, and saved me from that one silent undefined bug that started this whole thing.

Have you made the jump yet, or are you still holding the line with REST? Curious where everyone's landing on this one, drop your setup in the comments.