DEV Community: The Unmeshed Team

Your LLM Returned a 200 OK. So Why Is the Answer Wrong?

The Unmeshed Team — Mon, 31 Aug 2026 05:18:49 +0000

The one thing that genuinely fills my brain with dopamine is watching Sherlock Holmes walk into a room and immediately know exactly what happened, who did it, and why, while everyone else is still standing around saying, "something seems off."

Watson is great. He notices when something is wrong, he raises the alarm, he keeps track of what is happening. But he cannot tell you why. Holmes can. Same room, same evidence, completely different depth of understanding.

That is exactly the difference between LLM monitoring and LLM observability. Monitoring is Watson; it tells you something broke, latency spiked, error rates climbed, costs jumped. Observability is Holmes; it shows you which step failed, what prompt it received, what the model returned, and where in the pipeline things went sideways.

This post breaks down what each one covers, what to track, and when you need both.

What Is LLM Monitoring?

LLM monitoring is the practice of continuously tracking predefined metrics to tell you whether your AI system is healthy at a given point in time. It watches from the outside and tells you when something crosses a threshold you set.

The metrics it typically covers:

Request latency at p50, p95, and p99
Error rates and API failures
Token usage per request
Cost per call
Throughput and availability

It is continuous, real-time, and threshold-based. When latency exceeds two seconds, an alert fires. When error rates spike, your pager buzzes. It is how DevOps and platform teams know the system is running.

What it cannot do is tell you why. A hallucinating model returns a 200 OK. Your dashboard stays green. Your user gets a wrong answer. LLM monitoring has no idea anything happened.

"Monitoring is great at telling you a system is down. It is terrible at telling you your AI is confidently wrong."

Chip Huyen, author of AI Engineering

What Is LLM Observability?

LLM observability is your ability to see inside a specific request and understand why it produced the output it did. Where LLM monitoring watches the system from outside, observability gets inside each request and reconstructs what actually happened.

Every request generates a trace, and LLM tracing is the mechanism that captures it: a structured record of each step from input to output. In a RAG pipeline, that might be seven steps. In an agent workflow, it could be forty. Observability captures all of it.

What LLM observability actually tracks:

The exact prompt sent to the model, fully resolved with all context injected
The model's raw response before your code touched it
Which documents retrieval returned and their relevance scores
Latency broken down by individual step
LLM evaluation scores for quality, relevance, and safety
Hallucination detection signals

This is where you find out that the retrieval step returned stale documents, or that the prompt doubled in size because a context injection bug ran twice. LLM monitoring would never catch either of those. They both return a clean 200 OK.

LLM Monitoring vs Observability: The Actual Difference

The easiest way to think about it is the question each one answers.

Dimension	LLM Monitoring	LLM Observability
Core question	Is it working?	Why did it behave that way?
Signals	Latency, error rates, token counts, cost	Traces, prompts, retrieval context, span metadata
Approach	Threshold-based alerts	Exploratory, trace-driven investigation
Who uses it	DevOps, SREs, platform teams	AI engineers, ML teams, product teams
When you need it	Continuous, real-time	Incident investigation, quality debugging
Lifecycle	Post-deployment only	Dev through production

LLM monitoring tells you the alarm went off. LLM observability tells you which wire burned and why.

The gap between these two is where most production AI problems hide. According to Gartner, only 15% of enterprises running generative AI in production have semantic quality monitoring in place alongside operational monitoring. The other 85% are flying half blind.

What to Track for LLM Monitoring?

Good monitoring covers three categories. These are the signals that tell you something changed, even if they cannot tell you why.

Performance metrics

Request latency at multiple percentiles (p50, p95, p99)
Time to first token for streaming applications
Token generation rate
Throughput

Cost and resource metrics

Input and output token counts separately (output costs four to five times more)
Cost per request
Total spend by model and by workflow
GPU and CPU utilization

Reliability metrics

Error rates by type
API failure rates per provider
Availability and uptime
Rate limit hits

These numbers give you a real-time view of system health. Set alerts on the ones that matter for your SLAs and your budget. If any of them move, monitoring is what catches it first.

What to Track for LLM Observability?

LLM observability goes into territory that operational metrics simply cannot reach. These are the signals that tell you whether your AI is actually doing what it is supposed to do.

Trace and span data

Full request traces with step-level breakdown
Inputs and outputs at each step
Prompt versions and model configurations
Tool calls and their results in agent workflows

Quality signals

Hallucination detection rates
Relevance scores for RAG outputs
Factual accuracy against ground truth
Safety and toxicity flags

Retrieval signals (for RAG)

Documents retrieved and their relevance scores
Context window utilization
Retrieval latency by query type

Drift signals

Output quality degradation over time
Semantic drift in responses
Prompt effectiveness changes

The distinction matters most for agentic systems. According to Databricks, tool-related failures are the most common category of production agent failures, and they are almost entirely invisible to LLM monitoring alone.

Which Failure Modes Does Each One Catch?

This is the table worth bookmarking. Different failures require different detection approaches.

Failure Mode	LLM Monitoring Catches It?	LLM Observability Catches It?
Latency spikes	Yes	Yes, plus tells you which step
Cost overruns	Yes	Yes, plus tells you which prompt caused it
Hallucinations	No	Yes
Silent quality degradation	No	Yes
Wrong retrieval context	No	Yes
Prompt regression after update	No	Yes
Provider outages	Yes	Partially
Agent tool-call loops	Limited	Yes

Notice how many of the hardest failures, the ones your users will actually complain about, fall entirely in the observability column.

LLM monitoring would never catch a hallucination. It would never catch retrieval returning stale documents. It would never catch a prompt regression that degraded quality across ten thousand requests while every operational metric stayed green.

This is the Watson problem. Watson keeps excellent notes. He just cannot solve the case.

Do You Need Both?

For simple, single-model applications at low volume, LLM monitoring alone might be enough. If your app sends a request to one provider and returns the result, and quality is easy to verify, start with monitoring and add observability as complexity grows.

For everything else, you need both. That includes:

RAG pipelines where retrieval quality determines output quality
Multi-step agent workflows where tool failures compound
Any application in a regulated industry where audit trails matter
High-volume production systems where production AI monitoring alone leaves quality gaps
Teams running multiple models where routing and fallback need visibility

McKinsey found that 88% of organizations use AI in at least one business function but only 1% report mature implementations. A lot of that gap lives in observability. Teams move fast and quality problems surface from user complaints rather than dashboards.

If you want to go deeper on what LLM observability specifically covers, our post on what LLM observability is and why your production AI needs it covers the full picture.

How to Start With LLM Monitoring in Production?

The practical rollout most teams follow looks something like this.

Weeks one and two

Set up the monitoring foundation. Instrument latency, error rates, token usage, and cost. Get dashboards and basic alerts in place.

This is your Watson layer. It notices when something is off.

Weeks three and four

Add request-level tracing. Capture full traces with span-level detail. Log prompt and response pairs with metadata.

Now you have the Holmes layer, the one that can actually investigate.

Weeks five and six

Layer in evaluation. Run automated quality scoring on production traffic. Set alerts when quality scores drop.

This closes the gap between "the system is running" and "the system is working."

Ongoing

Feed production traces back into your evaluation datasets, refine scoring logic as the application evolves, and improve prompts based on what the traces actually show you.

The temptation is to skip straight to observability tooling before monitoring basics are in place. Do not.

Monitoring gives you the baseline for LLM performance tracking that makes observability useful.

How Unmeshed Fits Into LLM Monitoring?

Unmeshed is a workflow orchestration platform, not a dedicated observability tool. But if you are running AI workflows in Unmeshed, you already have something most teams set up separately: step-level execution visibility baked into every workflow run.

Every step records:

What it received as input and what it returned
Whether it hit a cache or triggered an error policy
What the model was asked and what it returned for AI LLM steps specifically

This is not semantic evaluation or hallucination detection. It is the execution-level foundation that makes LLM monitoring actually useful. You know which step ran, what it received, and where things went sideways before you even open a separate observability tool.

If you want to see what that looks like in a real AI workflow, the Agentic AI page shows how AI steps sit inside production workflows. And if you are ready to build, start free, and your first workflow has step-level visibility from day one.

The Bottom Line

LLM monitoring and LLM observability are not competing approaches. They answer different questions, and they catch different problems.

Monitoring is Watson. It watches the system, notices when something changes, and raises the alarm. You absolutely need it.

Observability is Holmes. It gets inside the request, follows the evidence, and tells you exactly what happened and why. For anything beyond the simplest AI application, you need this too.

Start with monitoring. Add observability as your system gets more complex.

And never assume a green dashboard means your AI is producing good answers, because it almost certainly is not telling you either way.

Is Your Company in a Situationship with AI?

The Unmeshed Team — Fri, 28 Aug 2026 06:14:00 +0000

Over the past year, I've watched a surprising number of my friends go through something. It feels like every few weeks, someone tells me about some kind of relationship. They text every day. They spend weekends together. They know each other's coffee orders and favorite songs. Yet somehow, when someone asks, "So… are you two together?" the answer is always the same. "We're just seeing where things go."

The internet has a word for relationships that are more than casual, but never quite become serious: Situationship.

It's when two people spend time together, make plans, rely on each other, and maybe even imagine a future together, but somehow never have the conversation.

At first, I thought it was just another internet term for modern dating. But the more I listened to these stories, the more I realized situationships aren't really about romance. They're about ambiguity. About moving forward without defining expectations. About everyone assuming the relationship means the same thing, until one day it becomes painfully obvious that it doesn't.

The problem wasn't a lack of feelings. It was a lack of definition. That thought stayed with me longer than I expected.

A few days later, while reading about companies moving beyond vibe coding toward governed AI, I realized I'd seen this pattern somewhere else before. It reminded me of how many companies are approaching AI today.

The Relationship Exists. The Definition Doesn't.

Ask almost any company whether they're using AI. The answer is almost always yes.

AI is no longer a side project. It has found its way into product roadmaps, quarterly planning sessions, strategy discussions, and conversations about the future of work. In many companies, AI isn't something a single team owns. It's something almost every team wants to use.

That's because AI enters a company through different doors.

For Engineering, it's an opportunity to build something new. Product sees new ways to improve customer experiences. Leadership imagines a competitive advantage, while Security begins thinking about the risks that come with adopting another technology. Operations looks at AI and wonders which workflows could become faster or more efficient.

None of these perspectives are wrong. In fact, they're all necessary. Everyone welcomes AI, just for different reasons. And that's where the ambiguity begins.

If you've ever watched a situationship unfold, you'll notice that it rarely ends because people don't care. It ends because no one ever defines the relationship. Both people assume they're working toward the same future, until one day they realize they've been operating with completely different expectations. Companies often fall into the same pattern with AI.

Product expects Leadership to define the strategy. Leadership assumes Engineering will figure out implementation. Security waits until AI reaches production. Customer Success assumes someone else is monitoring AI-generated responses.

Everyone is involved, yet no one has clearly defined who owns which part of AI.

From the outside, it looks like a committed relationship. Inside, everyone is operating with a different understanding of what that relationship actually means.

Defining the Relationship

In dating, defining the relationship doesn't make it less exciting. It creates clarity. Everyone knows where they stand. Companies eventually need the same conversation.

Just as healthy relationships work because both people understand their roles, healthy AI adoption depends on every team understanding where their responsibilities begin and end.

A simple way to think about it looks like this:

AI Responsibility	A Question Every Company Should Answer	Typical Owner
Business Outcome	What business problem is AI solving?	Product
Workflow Execution	How does AI fit into the workflow?	Engineering
Model Performance	Is the model still accurate, reliable, and cost-effective?	AI/ML Team or Engineering
Customer Experience	How do customers experience AI, and when should a human step in?	Product & Customer Success
Risk & Compliance	Is AI being used safely and responsibly?	Security, Legal, or Governance Team

Notice that no single team owns AI. Instead, every team owns a different part of making AI successful. That's the difference between experimenting with AI and operating AI.

Ownership Is What Turns AI Into Infrastructure

The difference between experimentation and production isn't usually the model. It's accountability. The companies succeeding with AI aren't necessarily using the smartest models. They're the ones where everyone knows:

Who owns the workflow
Who measures success
Who responds when something goes wrong
And who decides when AI is no longer the right solution

Ownership transforms AI from an interesting experiment into reliable infrastructure.

From Situationship to Partnership

Eventually, every situationship reaches the same crossroads.

Not because the feelings disappear, but because uncertainty eventually catches up. The conversations that were easy to postpone become impossible to avoid. Questions about expectations, commitment, and responsibility can no longer be left unanswered.

Companies reach a similar moment as AI becomes part of everyday operations.

Experimentation gets teams started. But scaling AI across a company requires something different: clarity around ownership, visibility into how AI workflows operate, and confidence that every decision has someone accountable behind it.

After all, healthy relationships aren't built on assumptions. Neither are healthy AI systems. Both begin with the same conversation:

"What are we, and who's responsible for what comes next?"

LLM Gateways: The Missing Layer in Your Production AI Stack

The Unmeshed Team — Thu, 27 Aug 2026 05:30:39 +0000

Hey, have you seen those heaven gateway memes?

An LLM gateway is kind of the same thing, except instead of deciding who gets into heaven, it decides which LLM provider handles your request, who gets turned away when rate limits hit, and who the fallback is when your first choice is having a bad day.

Most teams building AI in production end up needing one sooner or later, usually after they have written enough provider-specific code to realize they accidentally built a worse version of it themselves.

This post breaks down how an LLM gateway actually works and when you need one.

1. What Is an LLM Gateway?

An LLM gateway is middleware that sits between your application and the LLM providers you use . Your app sends a request to the gateway, and the gateway decides where it goes, which provider handles it, what format the request should be in, and what happens if something goes wrong.

Without one, your app talks directly to each provider. OpenAI returns responses one way. Anthropic returns them another. Mistral does its own thing. You end up writing adapters for all of them, and maintaining those adapters every time a provider changes their API (which they do, often and without much warning).

The market for this infrastructure is growing fast. The LLM middleware gateway market is projected to grow at a 49.6% CAGR through 2034, and roughly 42% of enterprises already use a middleware layer to manage their AI infrastructure.

"The real cost of managing multiple LLM providers is not the API bills. It is the engineering time spent writing glue code that has nothing to do with your actual product." Andrej Karpathy, former OpenAI

An LLM gateway abstracts all of that away. One API in, one consistent response format out, every provider handled behind the scenes.

2. What Problems Do You Face Without an LLM Gateway?

The short answer is more than you expect, and teams usually find out the hard way. Running production AI without an LLM gateway tends to surface problems gradually, then all at once.

Vendor Lock-in

When you integrate directly with one provider, your entire system depends on them. If their prices spike, performance degrades, or they change their terms, migrating is a real engineering project.

You are not switching a config value. You are rewriting integrations.

Over 80% of enterprises are expected to deploy generative AI APIs by end of 2026, up from under 5% in 2023. Managing multiple providers without a unified layer becomes an operational problem at that scale.

API Fragmentation

Every provider defines request and response formats differently. Your code becomes:

A custom adapter for OpenAI
Another one for Anthropic
Another one for Mistral
And a prayer that none of them change their schema this quarter

Cost Opacity

Without a single point to track spend, costs scatter across provider dashboards. You cannot connect a cost spike to a specific feature or workflow. The invoice arrives, and nobody can explain the number.

If you are already running workflows in Unmeshed, you can see token usage and cost broken down at the step level. The bringing AI workflows into production without burning tokens post covers how that works.

Operational Fragmentation

Debugging, monitoring, and optimizing routing become genuinely painful when everything lives across separate integrations. You are jumping between dashboards, alert systems, and logs that were never designed to talk to each other.

3. How Does an LLM Gateway Work?

An LLM gateway is a middleware layer between your application and AI model providers like OpenAI, Anthropic, or self hosted models. Instead of integrating multiple SDKs, your application sends requests to a single API while the gateway handles routing, security, and cost tracking.

Here is what happens when your app sends a request through an LLM gateway:

Step	What happens	Overhead
1. Request received	Authentication validated, policies checked	1 to 2ms
2. Routing decision	Provider selected based on cost, latency, or custom rules	1ms
3. Request translation	Standardized request converted to provider format	1 to 2ms
4. API call	Request sent to LLM provider	Variable
5. Response translation	Provider response converted to standard format	1 to 2ms
6. Logging	Tokens, costs, latency recorded	1ms
Total gateway overhead	Before the model responds	5 to 10ms (before the model responds)

That 5 to 10ms is the cost of having complete visibility and control over every request. For most production systems, that is a very reasonable trade.

The unified LLM API gateway pattern means your application code never needs to know which provider it is talking to. You write to one interface, and the gateway handles the translation.

4. What Features Does Your LLM Gateway Need?

Not all gateways are built the same. Here is what actually matters when you are running AI in production.

Model Abstraction

Write once, route anywhere. Your application code stays the same regardless of which provider handles the request. Switch from GPT to Claude without touching your integration layer. This is what "LLM provider abstraction layer" actually means in practice.

Intelligent Routing and Load Balancing

LLM routing and load balancing is where most of the cost optimization lives. Send simple classification tasks to cheaper models. Route complex reasoning to the models that can actually handle it. Load balance across multiple instances when one is slow.

This is not a nice-to-have. At scale, smart routing is the difference between an AI bill you can explain and one that makes your finance team ask questions.

Cost Control

Set hard limits per user, team, or workflow. Track spend in real time rather than at month-end. According to Gartner, uncontrolled AI inference spend is now one of the top three budget surprises for enterprise engineering teams. A good gateway prevents that.

Failover and Fallback

If a provider goes down or hits rate limits, the gateway automatically retries with a different provider. Your application keeps running. Users never notice.

Observability

See every request, every provider, every cost in one place. Multi-model LLM orchestration only works if you can actually see what all the models are doing.

Security and Compliance

Centralized LLM cost tracking and policy enforcement beats managing security across five separate integrations. Enterprise teams need audit logs, role-based access, and data handling rules in one place.

5. LLM Gateway vs Direct API Integration

Here is the honest comparison:

Aspect	Direct API	LLM Gateway
Setup	Separate integration per provider	One integration point
Flexibility	Hard to switch providers	Switch in configuration
Cost tracking	Scattered across dashboards	Centralized view
Security enforcement	Per integration	One place
Failover	Custom code every time	Built in
Vendor lock-in	High	Low
Maintenance	Grows with every provider	Stays manageable

Direct integration works fine for a single provider at small scale. Once you are running multiple models in production, the maintenance cost of direct integration compounds fast. A gateway is not complexity for its own sake. It is complexity that replaces worse complexity.

6. When Should You Use an LLM Gateway?

There is no hard rule on when an LLM gateway becomes necessary, but here are the situations where teams reliably reach for one.

Cost optimization is the most common driver. Route non-critical queries to cheaper models and reserve expensive ones for high-value tasks. Teams typically see 40 to 60 percent cost reduction without touching output quality. Teams that skip the gateway layer entirely tend to see token spend climb 30 to 40 percent faster than necessary.
Multi-model applications are where gateways really earn their place. An AI assistant that uses different models for different tasks (classification on a smaller model, reasoning on a frontier one, summarization on another) needs something to coordinate all of that. That something is the gateway.
Enterprise compliance is non-negotiable in certain industries. Banks and healthcare companies need centralized control, audit logs, and data policies enforced at the request layer. Doing that across five direct integrations is an audit nightmare.
Provider flexibility matters for any team still figuring out which models are right for which tasks. A gateway lets you test OpenAI versus Anthropic versus open source without rewriting code every time.
High-volume production systems handling millions of requests need load balancing, failover, and cost attribution at scale. You cannot bolt that on as an afterthought.

Unmeshed handles exactly this kind of multi-step AI orchestration natively. The Agentic AI page shows what that looks like in practice.

7. Should You Build or Buy an LLM Gateway?

The build vs buy question comes up constantly when teams are evaluating an LLM gateway, and the math is usually pretty clear once you add it up.

Building your own means supporting every provider's API as they evolve, writing routing logic, building cost tracking, implementing failover, handling security, and maintaining all of it. That is not a weekend project. That is a team of engineers maintaining infrastructure that has nothing to do with what your product actually does.

Buying means setup in hours instead of months. You get updates when providers change their APIs (and they will). You get security patches. You get operational support.

Every hour your team spends building gateway infrastructure is an hour not spent building your actual product.

The only case for building your own is if your requirements are genuinely unusual enough that nothing on the market fits. That happens, but it is rarer than teams think.

8. How Unmeshed Handles LLM Routing?

Unmeshed is not a dedicated LLM gateway, and it is worth being clear about what that means before you read this section. It is a workflow orchestration platform that covers part of what an LLM gateway does, specifically the routing and fallback piece.

What Unmeshed does cover is the routing part of what a gateway does. Using switch steps combined with different LLM model steps, you can build routing logic directly into your workflow:

A switch step evaluates the condition (task complexity, cost threshold, whatever makes sense for your use case)
Each branch routes to a different LLM provider via a dedicated model step
Fallback logic handles what happens if the primary provider fails or rate limits

If you are already orchestrating workflows in Unmeshed, you may not need a separate gateway for the routing and fallback piece. It is already there.

What Unmeshed does not replace is the full gateway layer, things like unified API abstraction across providers, PII redaction at the request level, or a single cost dashboard across all your model spend. For those, you would still want a dedicated gateway.

The combination that works well for a lot of teams is a gateway handling the provider-level concerns (format translation, rate limits, security) and Unmeshed handling the workflow-level concerns (what runs when, which steps use AI versus deterministic code, what happens when something fails).

If you want to see how Unmeshed handles routing and fallback inside a real workflow, explore the templates or start free and build your first AI workflow today.

9. What Mistakes Do Teams Make With LLM Gateways?

Teams setting up an LLM gateway for the first time tend to run into the same patterns repeatedly.

Over-engineering the router: Simple rules outperform complex logic more often than people expect. Route based on cost or latency. Do not try to predict model performance from first principles. Let real production data drive routing decisions over time.
Not tracking costs at the task level: Knowing your total monthly spend is not enough. You need to know cost per workflow, per feature, per user. Without that, teams optimize things that do not actually move the bill.
Ignoring failover until something breaks: Test what happens when a provider goes down before it goes down. Your gateway should handle it automatically, and your users should never notice.
Assuming all providers are equal: They are not. OpenAI and Anthropic have different rate limits, different response quality on different task types, different costs per token, and different reliability characteristics. Your routing logic needs to know the difference.

The Bottom Line

An LLM gateway is what sits between your application and the chaos of managing multiple LLM providers directly, and once you have used one, you will wonder how you shipped without it. It handles routing, failover, format translation, cost tracking, and policy enforcement so your application code can stay clean and your team can focus on building things that actually matter.

Most teams start without one and add it when the pain of managing direct integrations gets expensive enough. The better move is to start with one and never accumulate that pain in the first place.

7 Insurance Workflows Every Team Should Automate in 2026 And What Each One Actually Saves

The Unmeshed Team — Mon, 24 Aug 2026 06:24:39 +0000

I am writing this at exactly 2 a.m., and I am still annoyed about a phone call from this afternoon. Forty minutes on hold with my own insurer to change one address.

One.

That is not a customer service problem. That is an insurance workflow automation problem, and it happens to be exactly what this post is about.

The person who finally picked up was not slow or bad at their job. They were buried under the same routine request a thousand other people needed that day, and not one of those needed a human to begin with.

McKinsey found that insurers leading on AI earned 6.1 times the shareholder return of the ones lagging behind — the widest gap of any industry. That gap was never about staffing. It came down to who stopped doing by hand what a rule could handle on its own.

This post walks through seven processes worth automating first, what each one saves, and how to start.

1. What Insurance Workflow Automation Actually Is (And Why It Matters Now)

Insurance workflow automation means using technology to handle the repetitive, rule-based work that eats up your team's day, without a person having to touch every step. It is not digitization. It is not scanning paper into a computer and calling it progress. It is the actual work getting done on its own.

Research suggests up to 45% of work activities across the insurance value chain can be automated with technology that already exists today.

Think about a claim coming in. The system pulls the details, checks them against your rules, sends it to the right adjuster, and confirms receipt with the customer. Nobody touches it until it hits a point that genuinely needs a human to weigh in.

Why does this matter so much right now? Because two things are pulling in opposite directions. Customer expectations keep climbing while your team's capacity stays flat.

Customers want answers in hours, not days
Regulators want more compliance proof, not less
Carriers need to grow without adding headcount at the same pace

Insurance AI deployments grew 87 percent year over year into 2025, and the carriers moving first are pulling away from the ones still deciding.

2. The 7 Insurance Workflow Automation Processes That Deliver the Biggest Returns

Your budget and your time are both limited, so start where the return is fastest. These seven represent the highest volume work in most insurance operations, and they pay back quicker than anything else you could pick.

A. FNOL Intake

First notice of loss is where speed matters most. A customer has a problem; they reach out, and the next few minutes decide whether they stay or start shopping around.

Done by hand, one call ties up a staffer for at least 15 minutes: picking up the phone and calming the customer down, asking the questions in the right order, typing notes, routing the claim. At 5,000 calls a month, that's roughly 1,250 hours of staff time.

Automated intake works differently. The system takes the call or form, asks the questions in order, and validates each answer as it lands. It routes the claim, hands the customer a claim number on the spot, and gives the adjuster a clean case to open.

Intake drops from 15 minutes to about 2. That saves more than a thousand hours a month, so one person handles what used to take a room of twenty.

If your workflows extend into lending, our loan application processing solution page covers a similar pattern.

See this exact workflow, ready to run
Get the FNOL-to-settlement template, prebuilt with AI extraction, rule-based coverage checks, and adjuster review steps.
Try the Template

B. Policy Renewals

Renewals are pure volume. Notices go out, and then the responses scatter in every direction — some call with questions, some need documents resent, some adjust coverage, and plenty just pay.

At 10,000 renewals a month, the manual version (staff working through mail, email, and phone calls while updating the system by hand) turns into a mess fast.

Automation runs the whole thing end to end: notices go out on schedule, anyone making a change moves through a guided flow instead of an email thread, payments process and confirm on their own, and quiet customers get automatic reminders by phone, text, or email.

That is more than 2,000 hours a month back on the calendar, with renewals that once took three exchanges now closing in one.

C. Quote Generation and Quote Follow-up

A producer gets a quote request and, done by hand, logs into a few systems, gathers the details, runs the numbers, formats the document, sends it, and chases the follow-up. One quote runs 20 to 30 minutes. At 2,000 requests a month, that's over a thousand hours of manual quoting.

Automation captures requests from your website, email, phone, or broker portal, gathers the information, runs it against your rules, and sends the finished quote back in minutes.

If a quote goes unanswered past a set window, the system follows up on its own. If one gets abandoned, it flags the producer to step in personally.

The producer now spends their hours closing rather than formatting. Turnaround falls from a full day to minutes, which saves more than 800 hours a month and tends to lift close rates, because speed wins deals.

"Automation does not replace your producers. It gives them time to close deals instead of pu
sh paper".

D. Underwriting Data Gathering

Underwriters run on information, and a lot of it. Medical records, financial statements, prior claims history, credit reports.

Done manually, assistants request each document, chase the applicant, receive it, organize it, and prep it. A single complex case can pull from ten sources across several rounds of back and forth.

Policy underwriting automation connects straight to your data providers and pulls what it can on its own. From there it:

Generates smart requests for whatever it cannot find
Tracks what has come back against what is still missing
Nudges applicants for the gaps

By the time the underwriter opens the file, around 80 percent of the data is already there and organized. That takes 30 to 40 percent of assistant time off the table and gets to a decision faster.

E. Policy Servicing Requests

Customers reach out constantly for routine things: a copy of the policy, proof of coverage, an address change, a payment question, a change of beneficiary. These follow a fixed pattern and need no real judgment, yet staff handle each one by hand. At 3,000 requests a month, that's over 500 hours spent on the same handful of tasks on repeat.

Automation answers these through phone, chat, or email, pulling the information from your system and sending the document back in seconds. Anything that needs approval or hits an exception gets flagged for a human, while the routine majority resolves on its own. That returns more than 400 hours a month, and customers get their answer immediately instead of waiting days for a callback.

F. Claims Documentation and Verification

Claims handlers lose real time to finding and organizing files — scanning documents, sorting them by type, matching them to the right claim, checking nothing is missing.

Claims automation ingests documents as they arrive and identifies each one using OCR and machine learning. From there it pulls the key data like dates, amounts, and parties, organizes everything by claim, and flags whatever is missing. That clears 20 to 30 percent of document handling time and speeds up every claim.

AI-assisted claims processing is approximately 75% faster than traditional methods, with some carriers cutting resolution time from 30 days down to 7.5.

G. Compliance and Audit Workflows

Regulators want documentation, internal audits want proof of process, and compliance reporting has to show what happened and when.

Done by hand, that means gathering files, building spreadsheets, and compiling reports. A single audit request can swallow 40 hours or more.

Automation keeps the audit trail on its own:

it records what decision was made, who made it, when, and why;
it generates compliance reports on demand;
it flags likely violations before a regulator ever does.

When an audit lands, the documentation is already sitting there waiting. That saves over a thousand hours a year and closes the door on findings caused by missing paperwork.

"Compliance that runs automatically is compliance that never fails."

3. How the Savings Actually Stack Up

The math is worth doing plainly. Add up the hours across all seven processes and the total is hard to ignore.

Below is a table for your reference-

Process	Monthly hours saved
FNOL intake	1,250
Renewals	2,000
Quotes	800
Underwriting	500
Servicing	400
Claims docs	300
Compliance	200
Total	5,450 hours

4. How to Start Your Insurance Workflow Automation Initiative Without Overwhelming Your Team

Do not try to automate all seven at once. Pick one process, get it running, prove the return, and use that win to fund the next.

Most teams start with FNOL or renewals because the volume is high and the rules are clear. Results tend to show up in six to eight weeks, and that early proof is what builds momentum and budget for everything after.

Before you automate anything, map how the process runs today. Look for:

Where people wait on someone else to move the file forward
Where errors slip in because data is being typed instead of pulled
Where the same question gets answered the same way a hundred times a week

Those friction points are your automation opportunities, and they are usually easy to spot once you follow one case from start to finish.

5. How Insurance Automation Fits Into the Tech Stack You Already Have

You already run a core system, probably several. None of that needs to go.

System	What it does	What automation adds
Policy administration system	Manages policies, renewals, endorsements	Auto updates from intake, renewals, and servicing workflows
Claims platform	Tracks and processes claims	Receives structured, validated data the moment a claim comes in
CRM	Manages customer relationships and communications	Gets updated automatically as customers move through workflows

Insurance automation processes sit around your existing systems and handle the connective tissue between them. The gaps your staff currently fill by hand:

capturing input from any channel
validating it against your rules
routing it to the right place
updating your core systems as it moves.

This is also where RPA for insurance has grown up. It used to mean a bot clicking through screens the way a person would. Now it means workflows that coordinate across systems, move data where it needs to go, and trigger the next action without anyone involved.

The result is that your team stops being the bridge between your tools and starts doing the work those tools were never built to handle.

6. The Real Barriers to Adoption and Why None of Them Are Dealbreakers

Cost: A good automation platform is a real expense, but the labor it removes almost always costs more than the platform itself. ROI on FNOL and renewals alone typically lands inside three to six months.
Integration complexity: Most platforms connect through APIs or direct connectors. It takes planning, not a rebuild.
Staff resistance: It tends to disappear the first week someone stops handling 200 identical inquiries a day. People genuinely like their work more when the boring part is gone.

The teams that move past these three tend to have one thing in common — they picked one process, proved it worked, and let the numbers make the case for everything after. Insurance digital transformation does not have to start big to finish that way.

"If you want to see how this works in a financial services context, our page on fraud detection workflow automation is worth a read."

7. How Unmeshed Fits Into Your Insurance Workflow

Here is where I want to be precise, because most automation pitches overpromise. Unmeshed is not a magic box that runs your entire operation for you. It is the orchestration layer that sits between AI and production and decides, step by step, what actually needs a model and what does not.

That distinction is the whole point. In a typical underwriting workflow, only a handful of steps genuinely need AI, the ones involving language or real judgment. The rest run better as plain code, rules, and API calls, which are cheaper, faster, and completely reproducible. Unmeshed runs each step on whichever of those makes sense, which is what keeps cost down and keeps every decision auditable for regulators like NAIC and NYDFS.

Here is what that looks like in practice:

AI only where it earns its place: Judgment- and language-heavy steps run on a model. Everything else runs as deterministic code.
Auditable by design: Because pricing and rules run as logged, reproducible code rather than opaque model guesses, every decision can be explained and defended when a regulator asks.
Humans stay on the calls that matter: Standard cases move through automatically while complex ones route to a senior person, who reviews the summary and makes the final decision.

The results back this up. Hiscox cut underwriting time from 72 hours to 180 seconds with no drop in decision quality.

It plugs into the systems you already run rather than replacing them, and workflows tend to go live in weeks rather than quarters.

The Bottom Line

You do not have to automate everything to see a difference. Start with these seven insurance workflow automation processes, since they carry the most volume and the fastest return.

Customers get quicker answers, your staff gets work worth doing, regulators get cleaner compliance, and you free up the budget that funds whatever comes next.

Pick one process, automate it, measure it, and prove it works. Then move to the next one. If you want help finding where the cost actually hides in your workflows, that's exactly the kind of problem Unmeshed was built to map.

LLM Cost Optimization: 9 Ways to Cut Token Usage in Production

The Unmeshed Team — Fri, 21 Aug 2026 05:16:18 +0000

There is a scene in Moneyball where Brad Pitt sits across from a room of old scouts who want to spend big on famous players. He tells them they are asking the wrong question. The point was never to buy the best players. It was to buy wins, and wins were hiding in places nobody was looking.

"That is the mindset LLM cost optimization asks of you."

The most powerful model is rarely the answer to every task, and the biggest line on your invoice is rarely the one you would guess.

The trouble is that most teams cannot see where their money actually goes. The number climbs each month, and the explanation lives somewhere nobody has looked yet.

This post gives you nine ways to bring that number down and cut your token usage in production. Every one has been tested in real systems, and not one of them asks you to change models or rebuild anything.

1. Why Does LLM Cost Optimization Matter More Than Teams Think?

The model is almost never the real problem. What teams actually lack is a clear view of what each part of their system costs to run, and that view is where LLM cost optimization begins.

Your bill really comes down to two numbers:

How many tokens move through your system
What each token costs on the model you chose

Everything in this guide moves one of those two.

Say you run a support summarizer that turns long ticket threads into short summaries for your agents. It handles 100,000 requests a day, taking in a thousand tokens and giving back five hundred.

Now, obviously, for months nobody noticed what it cost, because the number lived inside one plain line on the invoice. That is how LLM inference cost hides.

Enterprise LLM API spending hit $8.4 billion by mid-2025, more than double the $3.5 billion from late 2024.

As OpenAI CEO Sam Altman put it, "compute costs limit everything."

2. The 9 LLM Cost Optimization Techniques That Actually Work

Here are the nine LLM cost optimization techniques, starting with the ones that give you the most back for the least effort.

1. Route Simple Tasks to Cheaper Models

Your best model is overkill for most of what you send it. Model routing means matching each request to the cheapest model that can still do the job well.

Sorting tickets, pulling out fields, cleaning up formatting. None of these need a frontier model, and smaller ones handle them for a fraction of the price.

Look at the summarizer again. Condensing a ticket thread is not deep reasoning. Move it down to a solid mid-tier model and the cost of that workflow can fall by half while the summaries read the same.

2. Trim Your System Prompts

System prompts get long the same way junk drawers get full. You keep adding one more thing that seemed necessary at the time, and nobody ever goes back to clear it out.

You pay for all of it on every request, whether the model needs it or not. Prompt optimization here is the fastest structural saving most teams have within reach.

The surprising part is how little quality you give up when you cut. Trimming a prompt down to the instructions that actually shape the output usually leaves the results looking the same, with a noticeably smaller bill behind them.

3. Cache Repeated Inputs and Responses

Two kinds of caching exist, and they solve different problems.

Prompt caching cuts the cost of reprocessing a stable chunk like your system prompt. The model stops rereading it every call, so you pay a fraction for that repeated part.
Response caching skips the model completely. When a question already has a stored answer, you hand it back for nothing.

Barely anyone switches either one on, which is a shame for a workflow like the summarizer. The instruction block at the top never changes, so caching it means you stop paying to teach the model the same thing on every single run.

4. Constrain Output Length and Format

Output tokens run four to five times the price of input tokens on every major provider. That single fact should shape how you write every prompt you own.

When your code only reads one field, asking for a full explanation around it is money spent on text nothing ever uses. Tell the model to return the summary and nothing else.

It is one of the simplest fixes available and one of the most skipped, because a wordy answer never feels expensive until you multiply it out. Output length is the easiest LLM cost optimization win people walk right past.

5. Fix Retrieval Before Touching Anything Else

In a RAG setup, the prompt is usually fine. The waste comes from everything you pack into the context around it.

Dropping a whole document into context when the model only needs two paragraphs is one of the easiest ways to pour tokens down the drain. Two fixes handle most of it:

Sharper chunking so you pass smaller, more relevant pieces
Better retrieval so only the context that matters makes it into the prompt

Together they reduce token usage more than any wording change you could make. So start there. In a retrieval pipeline, this is nearly always the change that moves the needle most.

6. Batch Non-Urgent Requests

Plenty of work does not need an answer the second you ask. Batch APIs handle those requests on a delay and charge you roughly half of standard rates for the patience.

Anything on a schedule qualifies. Say the summarizer also builds a morning digest of yesterday's tickets. That digest has no reason to run at live prices, so batching it halves the cost with nothing lost.

Live chat and anything a person is waiting on stays on standard calls. The rest is worth a second look.

7. Set Hard Token Limits on Every Step

This is the plainest safeguard you can put in place, and the one teams skip most.

A ceiling on each step stops a runaway before it reaches your invoice. It catches the two failures that quietly rack up tokens:

A model that loops on itself and keeps generating
An agent that wanders off track and calls step after step

When either one hits the limit, it stops instead of billing you for a mountain of tokens nobody wanted.

It will not lower your everyday spend. What it does is save you from the bad day, and at scale that is worth a lot.

8. Replace LLM Calls With Code Where Possible

Some steps in your workflow do not need a model at all. Things like routing a request, reading a single field, or checking whether a value is formatted correctly. A few lines of normal code can do these jobs, and code does not cost you anything to run.

The problem is that reaching for a model is easy, so teams do it even when the task is simple. Take pulling a date out of a sentence. That is not a thinking task. Code can find the date every time and get it right.

Give that same job to a model, and two things happen. You pay tokens for it, and now and then it hands the date back in a format you did not ask for. More cost, less certainty, for a job code already does well.

9. Track Cost at the Workflow Level

A monthly total tells you what you spent. It says nothing about where the money went or why.

Tracking at the workflow level turns that number into something you can act on. Real AI cost management starts here, where you finally see:

Which workflow is driving the bill
Which ones pay for themselves
Which ones are just noise

This is the groundwork that makes every token optimization effort on the list measurable. You cannot fix what you cannot see.

3. Which Technique Should You Start With?

Every LLM cost optimization effort needs a starting point. Here is how the nine stack up on effort, savings, and where each one earns its place.

Technique	Effort	Typical savings	Best for
Route to cheaper models	Medium	40 to 80% on cost	Multi-step pipelines
Trim system prompts	Low	10 to 30% on input	All workflows
Caching inputs and responses	Low	Up to 90% on repeated calls	Stable prompts, repeat questions
Constrain output	Low	30 to 60% on output	Structured outputs
Fix retrieval	Medium	20 to 50% on context	RAG pipelines
Batch requests	Low	Up to 50% on batched work	Non-urgent tasks
Hard token limits	Low	Prevents runaway spend	Agentic workflows
Replace with code	Medium	100% on replaced steps	Routing, parsing, validation
Workflow level tracking	Medium	Enables everything else	All production systems

Begin with the low-effort rows near the top. Trim your prompts, tighten your output, and turn on caching.

Once those are done, move to the heavier lifts like routing and replacing steps with code. They take more work, but they keep paying you back long after.

4. Where Do Most Teams Go Wrong With LLM Cost Optimization?

The usual mistake in LLM cost optimization is cutting too deep. Teams get a taste of the savings, push harder, and quality starts slipping somewhere they do not immediately connect to the change.

Three traps catch most teams:

Cutting past the point of quality. When your token count falls, but your error rate climbs, the cleanup can cost more than you ever saved. Aim for the lowest token count before quality breaks, not the lowest number you can hit.
Fixing the wrong thing. Without workflow-level tracking, teams tend to optimize whatever feels expensive, which is often not what is actually draining the account.
Treating this as a job you finish. Prompts fill back up, new features launch, usage shifts under you. LLM cost reduction works best as a habit, not a task you close and forget.

The strange part is that prices are actually falling. Epoch AI found the cost to reach GPT-4 level performance dropped about 40x per year, yet bills keep growing because usage climbs faster than prices drop.

5. How Does Unmeshed Help You Control LLM Costs?

Most cost tools report what you spent once the invoice has already landed. Unmeshed takes a different path to LLM cost optimization and lets you draw the line before the money leaves.

Per-step token limits. Every AI step gets a budget. When it hits the cap, it stops, so no single task quietly runs up the bill.
Tool allow lists for agents. Agents only touch the tools you approved. Nothing strays past the edges you set.
Deterministic steps where AI is not needed. For routing, parsing, or validation, you swap the model call for plain code: no prompt, no tokens, no cost.

The goal is not to starve your system of tokens. Good LLM cost optimization spends them only where a model earns its keep and lets code cover the rest for free.

The Bottom Line

LLM cost optimization is not about being stingy with AI. Anyone can shrink a bill by doing less. The real work is spending only where the spend pays off and trimming everything that does not.

Brad Pitt never won by buying the priciest roster. He won by knowing what every dollar was actually doing.

Do that with your tokens. Take the easy wins first, watch what genuinely moves your bill, and build the habit of checking before the number drifts. If you want those limits enforced for you at the workflow level, that is what Unmeshed is built for.

What Is LLM Observability — and Why "No Errors" Doesn't Mean It's Working

The Unmeshed Team — Thu, 20 Aug 2026 05:55:44 +0000

I was probably not the first person to tell you that AI breaks in ways nobody warned you about. But I might be the first to admit I had no idea what was actually happening inside my own AI application until something went very wrong and I had absolutely nothing useful to look at.

There was no stack trace to follow, no error log to read, just a very unhappy user and a dashboard that had absolutely no idea anything had gone wrong.

That moment is why LLM observability has become one of the fastest-growing categories in AI infrastructure.

The market hit $2.75 billion in 2026, and Gartner expects it to cover 50% of all GenAI deployments by 2028. Teams are not spending on this because it is fun. They are spending on it because running AI blind is genuinely painful.

1. What Is LLM Observability?

By early 2026, 72% of Fortune 500 companies had at least one LLM application running in production. Most had no real visibility into what it was actually doing inside.

LLM observability is your ability to see inside your AI application while it runs: every prompt sent, every response generated, what it cost, how long each step took, and whether the output was actually correct.

The uncomfortable truth is that you need observability most in situations where nothing looks wrong. A hallucinating model does not throw errors. It just returns confident wrong answers with a 200 OK while your monitoring takes a nap.

LLM monitoring tells you something broke. LLM observability tells you why it broke and where to look first.

2. LLM Observability vs Monitoring vs Evaluation

Most people treat these as the same thing. They are not even close, and mixing them up is how you end up with blind spots that only surface when users complain.

Capability	What it actually does	What it cannot tell you
Monitoring	Tracks latency, errors, token counts. Alerts when thresholds cross.	Whether your outputs were correct
LLM observability	Captures every step inside the pipeline. Shows what happened and when.	Whether the output quality was good
Evaluation	Scores outputs for faithfulness, relevance, hallucination detection	Nothing — this is the full picture

94% of organizations with AI agents in production have observability in place. Only 44.8% run evaluations on live traffic. That gap is where most quality problems hide.

Monitoring watches from the outside. Good for infrastructure. Not good enough for AI.
LLM tracing shows every step inside your pipeline — what was retrieved, what prompt went in, what came back, and when. This is where you find out why something failed.
Evaluation scores whether the output was actually good. This is the one most teams skip entirely, and it is always the one that would have caught the problem first.

3. What LLM Observability Actually Tracks in Your Application

You do not need to observe everything. You just need visibility into the parts that tell you what went wrong and why when something breaks in production.

Inputs and Prompts
When a user reports a bad answer, the first thing you want to see is the exact prompt that went to the model, fully built out with all the context that was injected. In most cases, the problem is sitting right there the moment you can actually read it.

Outputs and Responses
You need the raw response the model gave before your code did anything with it. If the answer was already wrong at that point, the model is the problem. If it was fine and broke later, something in your code changed it. You cannot debug this without seeing both versions.

Latency by Step
How long each step actually took:

Retrieval
Model inference
Post-processing

Without step-level tracking, you are guessing where the slowness is coming from. Most teams assume it is the model. Half the time it is a retrieval step nobody thought to measure.

Token Usage and Cost
A prompt that grew by 200 tokens sounds small. At 100,000 daily requests, that is thousands of dollars a month in extra spend nobody planned for. Token-level telemetry shows you that happening in real time instead of on the invoice.

Quality Signals
This is the column most teams leave empty:

User thumbs up or down
Hallucinations detected
Validation failures
Task completion rates

Production AI monitoring without quality signals means you are collecting data you cannot act on. Expensive logging, essentially.

4. Why LLMs Fail Differently From Traditional Software

When a regular app breaks, it screams — errors, alerts, something blinking red somewhere.

LLMs do not scream. They just start giving wrong answers while looking completely healthy from the outside.

Non-determinism
Run the same prompt twice, and you get different outputs. So when a failure shows up once and disappears when you go looking for it, you have no way to reproduce it without tracing that captures exactly what went in and what came out, every time.

Context Drift
Everything looks fine — the model is running — but outputs have been getting worse for weeks because retrieval changed and nobody updated the prompt to match.

Quality Degradation With Zero Errors
The application is healthy by every measure you have. Users are just getting worse answers, and there's no signal telling you that until enough of them say something.

5. What LLM Observability Looks Like in Practice

Take a RAG-powered support bot as an example. A lot happens between a user asking a question and your application returning an answer. Without observability, you see none of it — just a question in, an answer out.

With LLM observability, you see all of this for every request:

The original user question
The documents retrieval returned and their relevance scores
The full resolved prompt with those documents injected
The model's raw response before any parsing
Latency at each step, broken down individually
Cost of the API call
Whether the user marked the answer helpful

What you actually catch by having it:

A retrieval step returning stale documents because the index hadn't updated
A prompt that doubled in size because a context injection bug ran twice
A model response that was wrong before your code even processed it
A retrieval call that took four seconds on specific query types

None of this is findable without LLM telemetry. But… all of it is obvious once you have it.

6. LLM Observability Across Different Application Types

Different applications fail in completely different ways.

RAG Applications
Most RAG failures hide in retrieval. When an answer comes back wrong, you need to know whether the model got bad information — or good information and still got it wrong. Two different problems, two different fixes.

Chatbots and Conversational AI
A single-turn log is not enough. A bad answer in turn three usually has its roots in turn one, where the model picked up an incorrect assumption and carried it through the conversation. You need the full conversation trace, not just individual snapshots.

Agents and Agentic Workflows
Agents are the hardest application type to observe, and the most expensive when something goes wrong. Agent observability means visibility into every tool call, every decision, and every loop the agent runs. According to MLflow's 2026 report, a single user request in production can trigger anywhere from 30 to 300 internal steps. Without tracing, most of that is invisible — you see the request and the response, and nothing in between.

Application type	Biggest failure risk	What debugging uncovers
RAG	Retrieval returning wrong or stale documents	Exactly what was retrieved and when
Chatbots	Context carried forward incorrectly across turns	Full conversation trace across all turns
Agents	Loops, unexpected tool calls, runaway token spend	Every step, decision, and tool invocation

7. How Unmeshed Reduces What You Need to Observe

The less your AI workflows are allowed to do without guardrails, the less you need to observe after the fact.

Unmeshed lets you set hard limits on what each AI step can do before it runs:

Per-step token limits — every AI step has a maximum token budget; when it hits that limit, it stops.
Tool allow-lists for agents — agents only call the tools you explicitly approved.
Deterministic steps where AI isn't needed — for tasks like routing or validation, you replace the LLM step with plain code. No prompt, no tokens, nothing to observe.

Observability tells you what went wrong after it happened. Unmeshed is what you put in place so fewer things go wrong in the first place.

The Bottom Line

Running AI in production without LLM observability is how you end up finding out about problems from users instead of dashboards. By then the damage is usually already done, and the fix takes longer than it should.

The teams that get this right are not necessarily the ones with the best models. They are the ones that built visibility into their workflows early enough to actually use it.

If you want to reduce what can go wrong before observability needs to catch it, that's exactly what Unmeshed is built for.

Vibe Coding Works Until Yo Put It in Production

The Unmeshed Team — Wed, 19 Aug 2026 07:22:27 +0000

Software engineering is currently being pulled between two narratives that do not fit cleanly together.

The first narrative is about acceleration. A developer can describe a feature, generate a first pass, ask for a refactor, patch the obvious errors, and get something working quickly. That is the useful part of vibe coding. It lowers the cost of starting, especially for prototypes, internal tools, scripts, and early product exploration.

The second narrative is about compression. Over the last two years, many technology companies have cut roles, reduced hiring, and pushed teams to support more software with fewer people. AI is often mentioned in that context, sometimes because it genuinely changes the work and sometimes because it is a convenient label for broader cost pressure.

Both narratives are incomplete. AI has not made software engineering irrelevant, it is relocated where the judgment calls happen. The valuable skill is moving from simply producing code to deciding what should be code, what should be a model call, what should be a deterministic rule, and how those pieces should execute safely in production.

That distinction matters for enterprises because production systems do not care how impressive the demo was. They care about latency, retries, cost, observability, approval paths, auditability, and failure behavior.

Vibe Coding Works Best Before Production Starts

Vibe coding is useful because it removes friction from the early part of building.

You can ask for a feature draft, a refactor, an API client, a migration script, a test case, or a throwaway admin page. For prototypes and low-risk internal tooling, that speed is valuable. It lets teams explore more options before committing to one.

The economics change when AI moves from assisting a developer to executing inside a customer-facing or business-critical workflow.

During prototyping, the cost of an extra prompt or retry is easy to ignore. In production, every additional context window, tool call, retry, and verbose response becomes part of the cost profile of the workflow. The demo usually exercises the happy path. The real system gets malformed inputs, larger documents, incomplete records, transient tool failures, user-specific policy checks, approval branches, and audit requirements.

A common mistake is to treat the production workflow as if it were just the demo with more traffic. It rarely is. The production version needs budgets, limits, fallback paths, observability, and a clear decision about which parts should be handled by a model at all.

AI is a useful dependency, but an expensive one when it is doing work a simple rule or database lookup could have handled instead.

The Productivity Gap Is Really a Governance Gap

One of the more interesting studies from 2025 came from METR. They ran a randomized controlled trial with experienced open-source developers working on mature repositories. Developers expected AI tools to reduce task completion time by 24%. After the study, they still believed the tools had made them about 20% faster.

The measured result went the other way. Tasks with AI access took 19% longer.

That doesn't mean AI coding tools don't work. It means generation speed and engineering throughput are two different things. On mature codebases, engineers still have to inspect the output, verify architectural fit, run tests, handle subtle regressions, and own the maintenance burden.

We have observed the same pattern in AI workflow implementations. Teams often assume the workflow is saving time because the first answer appears quickly. When they include review time, retries, prompt refinement, exception handling, and human escalation, the gain is usually smaller than the first demo suggested.

Before AI, the bottleneck was often producing the first version. With AI in the loop, it is shifted to validation: is the result correct, safe, explainable, and worth the cost of running repeatedly?

That's what makes governance an engineering problem, not a policy afterthought. The system has to make controlled decisions under real operating constraints, not just generate output.

Why Prompt Engineering Is Not Enough Anymore

The AI playbook has changed quickly.

Period	What teams focused on	What changed
2023	Prompt engineering	Teams learned that wording, examples, and formatting could change model behavior.
2024	RAG	Teams started grounding model output in internal documents and business data.
2025	AI agents	Teams gave models tools and allowed them to choose more of the execution path.
2026	Governed AI workflows	Teams now need cost control, auditability, deterministic rules, and human review.

Prompt engineering still matters. RAG still matters. Agents still matter. They are just insufficient as the primary control layer for enterprise systems.

A better prompt reduces ambiguity, it does not decide whether a workflow should exist in the first place. Grounding output in real data cuts down on hallucination, but someone still has to sign off on the action being taken. And giving an agent tools to call does not hand you budgets, ownership, audit trails, or change control for free.

Prompt optimization also has diminishing returns compared to redesigning the workflow. If the model is being called for every execution of a repeatable decision, trimming 200 tokens helps. Moving that decision into an approved rule helps more.

What is missing is not a better prompt technique. It is workflow governance.

Where Vibe Coding Breaks Down in Enterprise Workflows

Vibe coding works best when the cost of being wrong is low.

A prototype can be thrown away. A small internal script can be rewritten. A demo can tolerate rough edges if it proves the idea. Enterprise workflows have a different risk profile. Expense approvals affect reimbursement. Support escalations affect customer trust. Compliance reviews affect legal exposure. Claims workflows may determine whether money leaves the business.

In those systems, "the model decided" is not a sufficient explanation.

Production AI needs answers to questions that rarely come up in a demo:

Production question	Why it matters
Which steps actually need AI?	Prevents unnecessary token spend
What is the token budget per workflow?	Keeps cost from growing silently
What tools can the agent call?	Reduces security and operational risk
When should a human review the decision?	Protects high-risk or ambiguous outcomes
Which decisions can become rules?	Moves repeatable work out of expensive model calls
Can we explain the outcome later?	Supports auditability and enterprise governance

This is the point where AI stops being a coding shortcut and becomes part of an operational system. The engineering work shifts from generating the workflow to controlling how it executes.

What Token-Aware Engineering Looks Like

A token-aware engineer does not think only in prompts. They think in execution paths.

Some workflow steps need judgment, language understanding, or ambiguity handling. Other steps only need rules. Good AI architecture separates those two instead of sending everything through a model because the model is already available.

Use AI when the step needs judgment	Use deterministic logic when the step needs rules
Classifying ambiguous customer intent	Routing based on a known status field
Summarizing a long document	Validating an amount against a threshold
Extracting structure from messy input	Checking whether a required field is present
Drafting a response that needs tone and context	Applying an approved decision table
Helping a user translate policy into rules	Running those rules thousands of times a day

The goal is not to avoid AI. It is to spend model calls only where the model is actually earning its cost.

Take invoice exception handling.

The naive AI implementation sends the invoice, purchase order, vendor policy, approval history, and a long instruction prompt into a model and asks what to do. It may work for the initial demo. At volume, it is expensive, hard to explain, and difficult to tune because too much of the workflow is hidden inside a single model decision.

A governed implementation separates the concerns:

Invoice uploaded
  -> OCR or parser extracts invoice fields
  -> AI classifies missing or ambiguous information
  -> Decision table validates amount, vendor, PO match, and policy rules
  -> Manager approval only if the amount is above threshold or policy is unclear
  -> ERP is updated
  -> Tokens, outcome, escalation, and cost are logged against the workflow

AI still has an important job here. It handles ambiguity and unstructured input. It just doesn't own the whole business process.

The same pattern applies to customer support:

Customer email arrives
  -> LLM classifies intent and urgency
  -> Known issue categories route through deterministic rules
  -> Account-specific policy checks run without a model
  -> Unknown or high-risk cases go to an AI-assisted draft
  -> Human approves sensitive responses
  -> Case is updated and the workflow records cost and outcome

Most cost improvements come from this kind of workflow redesign, not from endlessly tuning the prompt. Repeated decisions should move toward deterministic execution. Ambiguous decisions should stay closer to AI or human review.

Businesses Already Have AI Without Enough Governance

Many companies shipped AI features before they had a complete governance model. That was a reasonable sequence. Teams were under pressure from executives, competitors, customers, and internal productivity goals. They added copilots, chatbots, summarizers, classifiers, document processors, and agents. Some of those systems are creating real value.

The problem is that many of them were built as isolated AI features rather than governed workflows.

The common gaps are predictable:

No workflow-level token budget
No per-step cost visibility
No clear owner for each AI decision
No policy for when to use a smaller model
No human review path for high-risk outcomes
No audit trail for tool calls
No process for turning repeated model decisions into deterministic rules

These gaps usually do not hurt on day one. They show up when usage grows, when the invoice arrives, when output quality drifts, or when someone asks why a workflow made a specific decision.

At that point, prompt optimization helps, but it is not the main fix. The business needs orchestration and governance around the AI.

How Unmeshed Helps Enterprises Make This Shift

Unmeshed is built for the point where an AI demo needs to become a production workflow.

Most business processes are not just model calls. They involve APIs, files, schedules, business rules, approvals, retries, notifications, and systems of record. If every piece of that process lives in a different script, prompt, dashboard, or agent framework, governance becomes difficult.

Unmeshed gives teams one place to design and operate those workflows.

Orchestrate AI and Non-AI Steps Together

In Unmeshed, an AI step can sit next to API calls, data transformations, file processing, schedules, decision tables, and human approvals. That matters because the model should not carry work that belongs to a rule, a database check, or an integration.

During implementation, this separation is usually where the architecture gets cleaner. Teams can keep AI in the steps that require judgment while keeping predictable work deterministic and observable.

Turn Repeated Decisions Into Decision Rules

If a model keeps making the same decision from the same criteria, that decision is a candidate for a rule.

Unmeshed decision tables let teams encode approved business logic and run it consistently. AI can still help generate or refine the rule set, but once the business approves it, the decision can run without paying for a model call every time.

This is a practical way to reduce token spend without reducing automation. It also makes the decision easier to review, test, and change.

Put Boundaries Around Agentic Execution

Agents are useful because they can choose the next step. That is also why they need boundaries.

With Unmeshed, teams can define which tools are available, where approval is required, how exceptions are handled, and when a workflow should move from AI to a human or deterministic path.

Autonomy is not the problem, the lack of accountability is. Unmeshed lets teams keep the autonomy while making it something the business can actually inspect.

Measure Cost Where the Business Process Runs

Provider dashboards can show aggregate token usage. That is useful for finance, but it is too coarse for engineering decisions.

Enterprises need to know which workflow consumed the tokens, which step caused the cost, and whether the outcome justified the spend. When cost is tied to workflow execution, teams can optimize the actual business process instead of guessing from a monthly bill. That is where token economics becomes operational rather than theoretical.

Governed AI Is Engineering Work

AI coding made it easier to produce software quickly. Production AI makes it necessary to govern software carefully.

Writing code still matters. So do architecture, testing, observability, security, and maintainability. AI adds another set of operational concerns: model behavior, token budgets, context management, tool access, approval paths, and auditability.

For enterprise teams, the useful question is not whether AI can generate something. It usually can. The better question is whether the generated or AI-driven workflow can be operated, explained, optimized, and trusted over time.

Unmeshed is built to support that shift: AI where it adds judgment, deterministic rules where the decision is repeatable, orchestration around the full business process, and enough governance to run the workflow in production.

AI Governance for Engineers: Knowing When Not to Ship a Feature

The Unmeshed Team — Tue, 18 Aug 2026 05:26:03 +0000

Every generation gets its own cautionary tale about shortcuts. For Gen Z, one of the latest is Obsession.

On the surface, it's a psychological horror film about Bear Bailey, a music store employee who discovers the One Wish Willow — a magical shortcut that promises to make his long-time crush, Nikki Freeman, fall in love with him. His wish comes true, but not in the way he imagined. What begins as a solution slowly turns into possession.

The Willow isn't inherently evil. It simply grants Bear exactly what he asks for, without questioning whether that's what he truly needs. AI adoption looks surprisingly similar.

As a Gen Z, I love when a piece of pop culture becomes more than entertainment. Sometimes it unexpectedly explains the way we think about technology better than another whitepaper or conference talk ever could. It gives us a language for recognizing patterns we might otherwise miss.

That's exactly what Obsession did for me. It wasn't just a psychological horror film — which is funny, because after watching it, I realized we might be doing the exact same thing with AI. Not because AI is dangerous. But because, much like Bear in the film, we sometimes become so focused on getting the outcome we want that we stop questioning the path we're taking.

Every AI Initiative Starts with Good Intentions

"Let's automate documentation."
"Let's summarize meetings."
"Let's build a support agent."

Nothing about these ideas is inherently wrong. In fact, many of them create real value. The challenge begins when AI quietly shifts from being a means to becoming the objective itself.

At some point, the conversation changes. Instead of asking,

Does this create value?

Teams begin asking,

Where else can we put AI?

That's the same version of making another wish.

AI Maturity Isn't About Building More AI

Bear doesn't lose himself because of one decision. He loses himself because every decision after the first becomes easier to justify. Teams experience the same drift.

Teams that misunderstand AI maturity measure success by adoption:

How many AI features are shipped?
How many agents are running?
How many workflows use LLMs?

Mature teams measure something entirely different:

Which workflows actually improved outcomes?
Which ones reduced operational cost?
Which ones do customers genuinely use?
Which ones should never have been built?

AI maturity is measured by how deliberately AI is introduced — and how confidently teams choose not to use it when it doesn't add value.

Governance Exists to Interrupt Momentum

One of the quietest tragedies in Obsession isn't the wish itself. It's that Bear stops questioning whether the outcome he's chasing still has meaning. He becomes focused on preserving the wish rather than understanding its consequences.

AI governance exists for exactly this reason. Its job is to protect the quality of decision-making.

Governance creates space for teams to keep asking the questions that momentum tends to erase:

Why are we using AI here?
Does this solve a customer problem?
Is AI actually the best approach?
Would a simpler workflow achieve the same result?
What happens if we remove AI entirely?

The Most Expensive Prompt Is the One That Never Needed to Exist

Engineering practices like token optimization, caching, workflow orchestration, and production-ready AI systems are incredibly valuable. But they all assume one thing: that the workflow itself deserves to exist.

The workflow deserves to exist if it's valuable. Before optimizing prompts, teams should first optimize the decision that introduced the prompt in the first place.

Using AI wisely isn't about saying yes to every opportunity. Using AI requires knowing when saying no creates more value than another workflow ever could.

The Difference Between Making Wishes and Making Decisions

The tragedy in Obsession wasn't that Bear made a wish. It was that, once the wish worked, he stopped questioning whether it was still leading him toward what he truly wanted.

AI behaves much the same way. It can generate. Summarize. Classify. Automate.

But it won't ask whether any of those things create meaningful value. That responsibility still belongs to us.

AI is incredibly good at granting wishes. Governance exists to make sure we're wishing for the right things.

How We Cut Our LLM Bill by $3,600/Month Just by Auditing System Prompts

The Unmeshed Team — Mon, 17 Aug 2026 10:58:03 +0000

The one thing about me is that I have always been more interested in what things actually cost than what they can do.

So when I started working with AI teams in production, the first thing I noticed was not the models. It was the prompts nobody had looked at in months.

According to a 2026 LLM pricing report, teams are overspending on input tokens by up to 40% simply because prompts were never revisited after launch.

Prompt optimization is not about making prompts smarter. It is about making sure you are not paying for tokens that stopped being useful the day after you shipped.

This post breaks down how to audit what you have, where the waste hides, and how to cut without breaking anything.

1. Why Does Prompt Optimization Directly Affect Your Bill?

Most teams treat prompts as a quality problem. Length, structure, and redundancy are a cost problem.

Every token in your prompt costs money on every single request. Not occasionally. Every time.

Here is what that math looks like:

A 1,200-token system prompt at 100,000 daily requests on Claude Sonnet 5 costs roughly $360 per day on input alone, before a single user message enters the picture.
Trim 400 tokens from that prompt, and you save $120 a day.
That is $3,600 a month from one edit to one prompt.

Most teams have five to ten prompts running in production. The math compounds fast.

2. The Anatomy of an Expensive Prompt

I want to tell you about a prompt I saw once that was 2,400 tokens long. The team had no idea. They thought it was maybe 600, because that is what it looked like when they wrote it. The rest had accumulated over eight months, one caveat at a time, and nobody had gone back to look.

That is how expensive prompts happen.

Take a customer support bot as an example. It launched with a clean 350-token prompt, just a role definition, response tone, and one example ticket. Within six months, that same prompt had grown to 1,400 tokens without anyone rewriting it.

A caveat got added after a tricky refund case, three examples went in when the bot started misclassifying billing tickets, and a full refund policy section got pasted in because it felt safer to have it there. The bot kept working fine. The token bill did not.

Instructions That Repeat Themselves

Teams write prompts the way they write documentation. Thorough, detailed, every edge case covered.

The model does not need all of that every time. I have seen prompts that explain the same output constraint in three different ways, just worded differently. The model gets it the first time. You are paying for the other two on every single request.

Context That Does Not Change the Output

Full company backgrounds. Product overviews. Onboarding materials. All of it is sitting in the context window when the model only ever uses one paragraph.

A simple test: remove the section and run 50 requests. If the output does not change, the model was not using it. You were just paying for it to be there.

Few Shot Examples Left Permanently

Examples were added during testing that nobody removed.

Three examples at 150 tokens each add up to 450 tokens per call, whether the model needs that context or not. I have seen teams with six examples still in their production prompt from a testing session that happened a year ago.

Output Formatting Nobody Asked for Downstream

Ask the model to return markdown tables, headers, and plain English explanations when your application reads one field.

Output tokens cost four to five times more than input tokens. This is one of the most expensive habits to leave in production and one of the easiest to fix.

Prompt Drift

This is the one nobody names. And in my experience, it causes more damage than all the others combined.

Your prompt grows quietly:

Someone adds a caveat after a bug report.
Someone else handles an edge case inline instead of in code.
A new model version needs a formatting note.
A product change makes half the instructions irrelevant, but nobody updates the prompt.

Six months later, your prompt is 2,000 tokens, and half of it describes situations that no longer exist. Every single request pays for that history.

Month	Token count	What was added	Still relevant?
Month 1	400	Core instructions	Yes
Month 3	700	Edge case handling	Partially
Month 5	1,100	New model formatting note	No
Month 7	1,800	Legacy product context	No

If your prompt has grown more than 50% since you first shipped it, audit it now.

3. How to Run a Prompt Optimization Audit?

You need to run such an optimization audit every month. Clean prompt optimization audits don't work well if you do it once and forget about it.

Measure What Your Prompt Is Actually Doing

Start by counting tokens per section. Break your prompt into named parts:

Core instructions
Context and background
Few-shot examples
Output format instructions
Edge case handling

Count what each section costs, then ask which ones fire on most requests versus rare edge cases. If 20% of your prompt only applies to 2% of requests, that 20% is a candidate for removal.

What to Cut and What to Keep

Here is the rule I follow:

Instructions that change output quality: keep.
Instructions that only affect format or style: candidates for cutting.
Instructions added for edge cases that no longer exist: cut immediately.

Remove the section, run 50 requests through both versions, and compare outputs. If quality holds, the cut is safe. If it drops, put it back and look somewhere else.

Test Cuts Without Breaking Production

Never cut in production without testing first. I learned this the hard way, watching a team shave 300 tokens off a classification prompt and quietly break edge case handling for three days before anyone noticed.

Run both versions side by side and track three metrics at the workflow level:

Tokens per successful task completion
Cost per outcome
Output quality score

Average tokens per request tells you what you spent. Tokens per successful task completion tell you whether it was worth it.

4. How to Optimize Your Prompts and Cut LLM Costs?

Five prompt optimization actions in order of impact. Pick the one that fits your situation first, not the one that sounds most impressive.

Trim your system prompt: Run the audit above. Remove anything that does not change the output. This is almost always the fastest win and the one most teams skip because the prompt feels important to touch.
Constrain your output format: Tell the model exactly what you need and nothing more. "Return only the category name" is a valid instruction. It is also cheaper than "Return the category name with a brief explanation of your reasoning and any relevant caveats."
Fix retrieval before touching the prompt: For RAG workflows, the prompt is often fine. The problem is that everything is stuffed into the context around it. Better chunking means smaller context windows, which reduce token spend without changing a single prompt instruction.
Cache stable inputs: If your system prompt is long and does not change between requests, prompt caching gives you a significant discount on that repeated input. Most major providers support it. Most teams never set it up.
Route simple tasks to cheaper models: Not every step needs your most capable model. Classification, formatting, and extraction work just as well on smaller models at a fraction of the cost.

Tactic	Effort	Typical savings	Best for
Trim system prompt	Low	10 to 30% on input	All workflows
Constrain output format	Low	30 to 60% on output	Structured outputs
Fix retrieval quality	Medium	20 to 50% on context	RAG pipelines
Prompt caching	Low	Up to 90% on repeated input	Long stable prompts
Model routing	Medium	40 to 80% on cost	Multi-step pipelines

5. What Prompt Optimization Actually Looks Like in Production?

Let’s walk through a hypothetical audit, okay?

A prompt that went from 800 tokens to 300 tokens with zero change in output quality. I want to walk through exactly what was cut and why each cut was safe, because the decisions matter more than the outcome.

That customer support bot example is a good reference here. The 1,400-token prompt we talked about earlier went through exactly this kind of audit and came out at 420 tokens with no change in ticket classification accuracy.

Here it is:

The Before

A prompt that started at around 400 tokens. Over six months, it grew:

A 200 token company background section was added during onboarding that nobody revisited
Three few-shot examples from early testing, all doing roughly the same thing
An output format instruction asking for structured JSON with an explanation for every field
Two edge case instructions for scenarios that had not come up in four months

By the time anyone counted, it was 800 tokens. At 50,000 daily requests on Claude Sonnet 5, that is roughly $150 per day on one prompt.

The Cuts and Why They Were Safe

Prompt element	Token cost	Decision	Reason
Role definition	40	Keep	Removing it changed the model's behavior
Company background	200	Cut	The output was identical without it
3 few-shot examples	300	Cut 2, keep 1	One example was enough
Output format instructions	80	Rewrote to one line	Downstream code only reads one field
Edge case instructions	150	Cut	Those scenarios no longer exist

The After

300 tokens, and the output did not change across 500 test requests. Here is what that translated to:

Cost dropped from $120 to $45 per day on one prompt
$75 saved daily, $2,250 saved every month
Zero changes to the model, the pipeline, or anything the user sees

Most production pipelines have several prompts in exactly this condition right now.

6. How Far Is Too Far With Prompt Optimization?

There is a version of this that goes wrong, and I have seen it happen.

Cutting too aggressively is its own problem. When token counts drop fast, quality usually follows. The model starts missing things it used to catch, and by the time the support queue makes it obvious, the cost of handling bad outputs has already exceeded what was saved on tokens.

The math is not complicated:

Total cost = token spend + (error rate x cost per error)

Saving 400 tokens per request saves $40 per 100,000 requests.
A 1% rise in error rate at $2 per error costs $2,000 in new handling costs.
The savings disappear and then some.

Your goal is the lowest token count before quality breaks. Not the lowest token count possible.

7. Prompt Optimization Across Different Workflow Types

The right approach depends on what your prompt does. Same problem, different starting points.

Classification and Routing Prompts

Usually, the most over-engineered prompts in any pipeline. Teams add examples, edge cases, and explanations for every possible category because classification feels fragile.

In most cases, you can cut 60 to 80% with no quality loss. This is the easiest win in prompt optimization and the best place to start if you want fast results on LLM cost optimization.

RAG Prompts

The prompt itself is usually fine. The expensive part is everything stuffed into context around it.

Fix your retrieval pipeline first:

Better chunks mean smaller context windows.
Smaller context windows mean lower token spend.
Semantic caching can cut API costs by up to 73% in high-repetition RAG use cases.

You can reduce LLM costs significantly here without touching a single line of the prompt.

Agent System Prompts

The most expensive and most neglected. Every tool description, every instruction, every example adds to every single agent call, which often runs multiple times per task.

This is where prompt drift does the most damage because agent system prompts grow the fastest and get audited the least. Every tool you add comes with a description. Every new capability gets an instruction. Nobody removes the old ones.

If you only have time to audit one prompt this month, make it your agent system prompt.

8. How Does Unmeshed Support Your Prompt Optimization Process?

Prompt optimization reduces what you pay per token. Unmeshed controls what gets to spend tokens in the first place.

Here is what that looks like in practice:

Per-step token limits. You set a ceiling on every AI step in your workflow. When a step hits that limit, it stops. A bloated prompt cannot quietly run up your bill because there is a hard limit on what each step can generate.
Deterministic steps where AI is not needed. For tasks like routing a request, parsing a field, or validating a value, you replace the LLM step with plain code. No prompt needed. No tokens spent.
You only pay for the steps where AI actually earns its cost. Everything else runs for free.

The Bottom Line

Prompt optimization is not a one time thing you do when costs spike. It is something you build into how your team works.

Everything drifts over time and by the time the bill makes it obvious, the decisions behind it are months old and spread across people who may not even remember making them.

An audit every month, clear rules on what to cut, and hard limits on what each step can spend are what keep it manageable. If you want those limits enforced at the workflow level automatically, that is exactly what Unmeshed is built for.

Incident Response Management with Google Workspace & PagerDuty Integration

The Unmeshed Team — Fri, 14 Aug 2026 06:26:46 +0000

Incident response works better when the steps are clear

When an incident starts, the team usually needs the same few things every time: a place to record what happened, a doc for notes, a page to the on-call person, and a way to keep everyone else informed. This workflow keeps those pieces in one place.

What the workflow does

The workflow is simple:

incident event
  -> get_current_oncall
  -> create_new_sheet
  -> add_incident_response
  -> create_incident_document
  -> page_oncall
  -> notify_stakeholders

Each step does one thing:

get_current_oncall looks up the current on-call owner.
create_new_sheet creates a shared incident tracker in Google Sheets.
add_incident_response writes the incident details into the sheet.
create_incident_document generates a Google Doc for notes and updates.
page_oncall triggers PagerDuty when escalation is needed.
notify_stakeholders sends a stakeholder email with links to the sheet and doc.

Workflow definition

This is the workflow shape used in the post:

orgId: 1
namespace: incident-management
name: incident_response_automation
version: 1
type: API_ORCHESTRATION

steps:
  - name: get_current_oncall
    type: HTTP
    input:
      method: GET
      url: https://devs.unmeshed.com/api/call/get_current_oncall/get_current_oncall/fixed?apiCallType=SYNC

  - name: create_new_sheet
    type: INTEGRATION
    input:
      type: google-sheets
      publishProperties:
        action: CREATE_SPREADSHEET
        title: New Incident Created

  - name: add_incident_response
    type: INTEGRATION
    input:
      type: google-sheets
      publishProperties:
        action: APPEND_ROWS
        spreadsheetId: "{{ steps.create_new_sheet.output.spreadsheetId }}"
        range: Sheet1!A1
        values:
          - [
              IncidentId,
              Severity,
              Service,
              Message,
              Status,
              AssignedTo,
              CreatedAt,
            ]
          - [
              "{{ context.incidentId }}",
              "{{ context.input.severity }}",
              "{{ context.input.service }}",
              "{{ context.input.message }}",
              OPEN,
              "{{ steps.get_current_oncall.output.response.output.oncall }}",
              "{{ context.created }}",
            ]

  - name: create_incident_document
    type: INTEGRATION
    input:
      type: google-docs
      publishProperties:
        action: CREATE_DOC
        title: New Incident Document

  - name: page_oncall
    type: INTEGRATION
    optional: true
    input:
      type: pagerduty
      messageBody:
        event_action: trigger
        dedup_key: "{{ context.input.message }}"
        summary: "{{ context.input.message }}"
        severity: "{{ context.input.severity }}"
        source: "{{ context.input.service }}"
        links:
          - href: https://docs.google.com/spreadsheets/d/{{ steps.create_new_sheet.output.spreadsheetId }}
            text: View Incident Report
          - href: https://docs.google.com/documents/d/{{ steps.create_incident_document.output.docId }}
            text: View Incident Document

  - name: notify_stakeholders
    type: INTEGRATION
    input:
      type: google-gmail-read-send
      publishProperties:
        action: sendEmail
        subject: A New Incident Reported
        to: "{{ variables.stakeholdersSharedEmail }}"
        bodyContentType: html

Workflow output

These screenshots show the result of each integration after the workflow completed.

Sheets data

This is the row that got written into Google Sheets. It keeps the incident id, severity, service, message, status, assignee.

Docs data

This is the Google Doc the workflow created for the incident. It gives the team a place for the summary, affected service, assigned owner, and follow-up notes.

Gmail message

This is the email that was sent to stakeholders. It includes the incident details plus links to the sheet and the doc so people can open the right record immediately.

Integration setup

These are the two setup screens that matter most.

Google integrations

This connects the sheet and doc steps so the incident record and notes go straight into Google Workspace.

PagerDuty integration

This is the page to the on-call person when the incident needs escalation.

How the flow works

1. Capture the incident

An alert, webhook, or manual trigger starts the workflow with a small payload such as incidentId, service, severity, and message.

2. Find the on-call owner

The workflow checks who is on call and stores it right away so the assignment is visible to everyone working the incident.

3. Create the record

Google Sheets becomes the incident tracker. The workflow creates the sheet, then writes the first row so the team has a shared record right away.

4. Generate the incident document

Google Docs gives the team a place for the timeline, notes, and follow-up. That keeps the details out of chat threads.

5. Page and notify

PagerDuty handles the escalation. Gmail sends the update with the links everyone needs.

Why this version is better

You create the sheet first, before anything else needs it. The page to on call links straight back to both the sheet and the doc, using the IDs Google returns when each one gets created.
The sheet and the doc give your team one place to check what happened. Nobody has to dig through old messages to find the incident owner.
The page to on-call and the stakeholder email pull from the same incident record. Both use the same ID, severity, service, and message. You enter these details once, and every downstream step uses that same data.
Each integration does one job. The sheet step writes rows. The doc step creates a doc. Nothing tries to do two things at once, so the workflow stays easy to follow.

Practical notes

Keep the header row fixed at Sheet1!A1. Every row you add after that lands in the wrong column if the header shifts.
Use a doc template with placeholders for the summary, owner, and timestamps. Fill them in as the incident happens.
Add the PagerDuty step only when the incident needs escalation. It's marked optional in the workflow for that reason.
Include the sheet and doc links in the stakeholder email. Your team can check the record themselves instead of asking for updates.

Closing

Incident response doesn't need to be a pile of scripts and handoffs. A small workflow handles all of it. It catches the incident and assigns the owner, then creates the record and notifies the right people. No extra coordination needed.

Stop Writing Custom Webhooks Every Time You Need a Google Doc or Sheet

The Unmeshed Team — Thu, 13 Aug 2026 05:50:45 +0000

There is a type of work that nobody puts on their resume but everyone ends up doing.

You know the kind. Someone joins the company. Someone has to make the welcome doc. Someone has to CC the manager. Someone has to update the tracker sheet. Someone has to post in the team channel. And usually, "someone" is a different person each time, which means it either gets done inconsistently or it gets forgotten.

Or a support ticket comes in. The agent reads it, mentally classifies it, copies the subject line into a spreadsheet, writes an acknowledgment email that says roughly the same thing it always says, and if the ticket looks urgent, manually pings the right channel.

None of this is hard. That's the thing. It's just repetitive, and repetitive is the exact kind of work that should not require a human.

Unmeshed connects to Gmail, Google Sheets, Google Docs, Google Drive, and Google Chat. This is what that looks like in practice.

Why Google Workspace Specifically?

Most of our customers already use it. That was the short answer.

The longer answer is that Google Workspace is a strange beast. It is both a productivity suite and an informal data layer for a huge number of teams. Sheets is where people track things. Docs is where decisions get written down. Drive is where files end up. Gmail is how things get communicated. Chat is where teams react to things in real time.

These tools are already inside most of the workflows we were being asked to automate. They just weren't connected to anything.

Before this integration, if you wanted a workflow to send an email when something happened, you had two options: maintain a separate email service integration (and its credentials, its templates, its rate limits) or use a webhook that fired into a third-party tool that then sent the email. Both worked. Neither felt right.

Now you just add a Gmail step. The connection is already there. The workflow sends the email and moves on.

What Each Integration Actually Does

Google Sheets: The Tracker That Runs Itself

Sheets is how most teams keep track of things because it requires no setup, everyone already knows how to use it, and it is readable by a human without any additional tooling.

The problem is that someone has to update it. That "someone" is usually the first person who notices something happened and remembers to do it, which is not a great system.

With Unmeshed, a Sheet can be on the receiving end of any workflow. Incident happens, a row gets appended. New hire joins, a row gets appended. Support ticket comes in, a row gets appended. Leave request submitted, the workflow reads a different sheet to check the balance before deciding what to do.

You can also read from Sheets inside a workflow, which turns a spreadsheet into a lightweight config or lookup table. We used this in our PTO approval workflow to check an employee's remaining balance before deciding whether to route the request to a manager or auto-deny it. No database needed. Just a sheet.

The value here is not just convenience. It is that the sheet becomes accurate in real time, without asking anyone to update it.

Gmail: The Email That Goes Out at the Right Moment

I want to be careful about how I describe this one because "send email from a workflow" sounds simple to the point of being boring.

What is actually interesting is the timing.

Most teams send emails when someone remembers to send them. A new hire gets a welcome email when HR finishes their onboarding checklist, which might be the morning of their start date or might be two days before. A customer gets a ticket acknowledgment when the support agent gets to it, which might be immediately or might be three hours later.

When the email comes from a workflow, it goes out at the moment the event occurs. The new hire's welcome email lands in their inbox the second the offer is accepted, not when someone gets around to it. The customer's acknowledgment is sent within seconds of the ticket being submitted, not when an agent picks it up.

That consistency is what teams actually want. And it is hard to get when humans are doing the sending.

Google Docs: Documents That Exist Before Anyone Asks for Them

Here is a thing that happens on every engineering team I have seen: an incident fires, the on-call engineer opens their laptop, and the first five minutes are spent making a postmortem doc. Picking a template. Filling in the title. Writing down the service name, the severity, the time it started. Basic information that was already in the alert.

Those five minutes are not a big deal in isolation. But they happen every single time, for every single incident, and they are happening at exactly the moment when the engineer's attention is most valuable.

We wired up the Google Docs API so that the postmortem doc can be created the moment the incident is logged. By the time the on-call engineer opens their laptop, the doc exists, the fields are filled in, and there is a structured template waiting for them. They start writing the actual timeline instead of creating the file.

The same logic applies to welcome packets for new hires, release notes from commit data, knowledge base articles for support tickets that are worth documenting, and policy rollout docs. The document exists before anyone asks for it.

Google Drive: Filed Before Anyone Gets a Chance to Lose It

Drive is one of those tools where the value is obvious and the execution is painful. Everyone knows that files should be organized. Everyone knows that a well-structured Drive folder is better than a jumbled shared drive. Nobody has time to actually do the filing.

Unmeshed can create folders and move files as part of a workflow step. This means the onboarding folder for a new hire gets created the moment the offer is accepted, not the day before they start. The postmortem doc goes into the ops folder the moment it's created, not after someone remembers to move it. The KB article lands in the support library without anyone having to drag it there.

In the onboarding workflow we built, the folder gets created, the welcome packet doc gets filed inside it, and the manager gets an email with a direct link to the folder. All before the HR team even sees the notification. That is not a meaningful optimization in terms of time. It is a meaningful optimization in terms of never having to remember to do it again.

Google Chat: Notifications That Are Actually Part of the Process

I want to draw a distinction here between a Chat message that comes from an Unmeshed workflow and a Chat message that a person sends.

When a person sends a message in a team channel, it might have incomplete information. It might come late. It might not come at all if the person is in a meeting. It is informal, and it depends on someone deciding the message is worth sending.

When an Unmeshed workflow sends a Chat message, it carries the exact data relevant to the moment: the incident ID, the severity, the affected service, the link to the postmortem doc. It goes out the second the event occurs. And it always goes out, regardless of whether anyone is watching.

For the DevOps incident workflow, this means the DevOps Chat space gets notified the moment an incident is logged, with everything the team needs to start responding. For the support ticket workflow, high-priority tickets alert the Support space before an agent manually routes them. For onboarding, HR sees a new hire notification in their space the moment the offer is accepted.

The message is part of the process, not a side channel someone manually updates.

One Thing Worth Saying Plainly

We tested a lot of approaches for the Google Chat integration specifically. The first version used a webhook URL stored as a secret. It worked, but it meant maintaining a URL, worrying about it expiring, and managing it separately from everything else.

The current version uses a proper Google Chat integration step with its own space ID and configured connection. It is cleaner, it is observable the same way every other step is, and it does not require anything to be managed outside the workflow.

The same is true for Gmail. It is not an SMTP connection or a third-party email service. It is a step in the workflow, set up as its own integration, just like every other Google Workspace step.

The goal was always to make adding these steps feel as natural as any other step in a workflow. I think we got there.

How to Use It

Each Google Workspace app connects as its own integration in Unmeshed. Sheets, Gmail, Docs, Drive, and Chat are set up individually, so you only connect the ones you actually need.

From there, you add a step to whatever workflow it belongs in, and point it at the integration. Each integration supports multiple operations, so the same step can be configured differently depending on what the workflow needs it to do.

Once an integration is set up, it's not tied to a single workflow. The same Google Sheets integration used to log incidents in one workflow can be used by a Sheets step in a completely different workflow to check PTO balances. The same Gmail integration used for onboarding emails can be used by a Gmail step in the support ticket workflow. Set up the integration once, use it across as many workflows as you need.

Every step is logged. Every run is visible. You can see exactly what each step did, what data it used, and whether it succeeded. No separate logging setup needed.

The Bottom Line

Most teams are not looking for new tools. They are looking for the tools they already have to work together without someone manually bridging the gaps.

Google Workspace is already where your team works. Unmeshed connects it into your workflows so that the things that are supposed to happen automatically, actually do.

If you are building workflows that involve documents, emails, spreadsheets, file storage, or team messaging, and your team runs on Google Workspace, this is worth trying.

How to Calculate Your LLM Pipeline's Token Efficiency Score

The Unmeshed Team — Wed, 12 Aug 2026 07:38:34 +0000

Running AI in production has a funny way of humbling you.

Most teams I have worked with had no idea what their AI features actually cost to run until the bill showed up and made it very clear.

According to Deloitte's 2026 enterprise AI report, AI is now the fastest-growing expense in corporate technology budgets, with some firms reporting it consumes up to half of their IT spend. And most of that spending is invisible until it is not.

Token efficiency is what separates teams that scale AI sustainably from ones that end up in a budget conversation they did not plan for. This post breaks down what it means, where teams lose the most money, and how to get ahead of it.

1. What Tokens Are and Why They Cost Money

Every time you call an LLM, you are not paying per request. You are paying per token, and those add up faster than most people expect.

A token is roughly four characters of text. The model reads your input in tokens and generates output in the same way. What most teams miss early on is that input and output tokens are priced very differently, and the gap is significant.

Here is what that looks like across the major providers right now (as of June 2026):

Model	Input (per 1M tokens)	Output (per 1M tokens)	Output multiplier
GPT-5.5	$5.00	$30.00	6x
Claude Sonnet 4.6	$3.00	$15.00	5x
Gemini 3.5 Flash	$1.50	$9.00	6x
DeepSeek V4 Flash	$0.14	$0.28	2x

Output tokens cost four to five times more than input tokens across every major provider. That one fact changes how you think about what is actually driving your bill.

A 500 word prompt with a 300 word response is around 1,000 tokens, roughly $0.018 per call on Claude Sonnet 4.6. At 100,000 requests a day, that is $1,800 daily on a single workflow, and output tokens are doing most of the damage.

2. What Token Efficiency Actually Means

Token efficiency is the ratio of useful output to total tokens consumed.

A workflow that returns a correct answer in 200 tokens costs a quarter of what a workflow costs when it returns the same answer padded with 800 tokens of preamble and repetition. The output quality is identical. The cost is not.

There are two dimensions to this:

Input efficiency is about how lean your prompts are.

Every unnecessary sentence in your system prompt costs money.
Every redundant example costs money.
Every piece of context the model does not need costs money.

And it costs you on every single request.

Output efficiency is about how much of what the model returns you actually use.

If your downstream code reads one JSON field and you ask the model to return a formatted table with explanations, you are paying for tokens your product never uses.
If the model adds a preamble and you strip it in post-processing, you already paid for it.

Both matter, and most teams I have worked with are only fixing one of them.

How to Measure Token Efficiency

Most teams track total token usage. That tells you what you spent but not whether it was worth it.

A more useful way to think about it:

Efficiency Score = Useful output tokens ÷ Total tokens consumed

The higher the score, the better. A high score means more of what the model returned was actually useful to your product.

Score	What it means	What to do
Above 0.8	Most of what the model returns is useful	You are in good shape, keep monitoring
0.5 to 0.8	Some padding, but mostly usable	Review output constraints and prompt length
Below 0.5	More than half your tokens are noise your product never uses	Audit prompts, retrieval, and output format now

Track this at the workflow level over time. If the score keeps dropping, something in your pipeline has changed, and it is worth looking at.

Token efficiency also affects speed, and this one caught me off guard.

Output tokens are generated sequentially, not in parallel, so a verbose 800 token response is not just more expensive; it is slower.

I spent an embarrassingly long time staring at latency dashboards before someone pointed out the responses were just too long.

Reducing AI inference cost and reducing response time are the same fix.

3. Where AI Teams Lose Tokens Without Knowing It

The most common sources of token waste are overloaded context windows, bloated system prompts, and redundant retrieval. Each request looks fine individually. The cost only becomes visible when you look at the workflow as a whole.

Bloated System Prompts

This is probably the one I see most often. System prompts grow quietly.

Someone adds a caveat, someone else adds an edge case instruction, and a new model version needs a formatting note. Six months later, your system prompt is 2,000 tokens, and half of it has nothing to do with what most requests actually need.

The problem is that every single request pays for the full prompt every time.

A 1,000 token system prompt across 100,000 daily requests is 100 million input tokens before a single user message even comes in. Trimming it is one of the fastest wins you can get on AI inference cost.

Sending Full Documents When Summaries Work

Passing an entire PDF or knowledge base into the context window when the model only needs two relevant paragraphs is one of the most common and expensive habits teams develop.

I think most companies underestimate how much of their token spend sits here. Poor retrieval is a big part of the problem:

Bad retrieval means large context windows.
Large context windows mean wasted tokens.
Better chunking and better retrieval pipelines reduce costs quickly.

Semantic caching can cut API costs by up to 73% in high-repetition use cases. Fixing retrieval is usually the highest-impact change a team can make before touching anything else.

Verbose Output Formatting

If you ask the model to return a markdown table with column headers, descriptions, and a plain English summary, but your application only reads one field, you are generating output tokens you never use.

Output tokens cost four to five times more than input tokens. Telling the model exactly what you need, and nothing more, is one of the fastest ways to reduce AI costs without changing your model or your pipeline.

Redundant Chaining in Multi-Step Pipelines

In agent workflows with multiple steps, each step often receives the full conversation history.

By step four, the model is re-processing the context it already handled in steps one, two, and three. You are paying to re-read the same tokens repeatedly. The fix is to summarize history between steps rather than passing the full thread forward.

The cost grows with each step:

A five-step pipeline where each step carries the full history does not cost five times the first step.
It costs significantly more because the context window gets larger with each hop.
Most teams do not notice until the bill arrives.

No Caching on Repeated Inputs

There are two types of caching worth understanding, and they work very differently.

Prompt caching saves you money on inputs. If your system prompt is long and stable, the model does not need to reprocess it on every call. You pay a fraction of the normal input cost for that repeated portion. A 3,000 token system prompt across 10,000 daily requests adds up to 30 million tokens a day. With caching, most of that is free.
Response caching goes further. If a user asks the exact same question and you have a stored answer, you skip the model call entirely. Zero tokens, zero cost. This works well for high-volume use cases where questions repeat frequently.

Most teams never set up either. Once you run the numbers on how often your inputs repeat, it becomes an obvious fix.

4. Practical Ways to Improve Token Efficiency

These are not theoretical. Each one directly reduces what you spend.

Tactic	Effort	Typical savings	Best for
Audit system prompts	Low	10–30% on input	All workflows
Fix retrieval quality	Medium	20–50% on context	RAG pipelines
Set output constraints	Low	30–60% on output	Structured outputs
Model routing	Medium	40–80% on cost	Multi-step pipelines
Prompt caching	Low	Up to 90% on repeated input	Long stable prompts
Response caching	Medium	100% on repeated calls	High-repetition use cases
Max token limits	Low	Prevents runaway spend	Agentic workflows

A few of these deserve more detail:

Audit your system prompts every month: Remove instructions that no longer reflect how the product works. If a caveat was added for an edge case that no longer exists, it is costing you money on every request.
Route by model, not by habit: Not every step needs your most capable and most expensive model. Classification, formatting, and extraction tasks often work just as well on smaller models at a fraction of the cost. Running a simple task through a frontier reasoning model when a smaller model would do it just as well is one of the most expensive habits I see in production pipelines.
Set a max token limit on every AI step: A hard ceiling on output length prevents verbose responses from quietly inflating your bill. It is the simplest guardrail you can add, and most teams skip it entirely.
Track at the workflow level: Total monthly token usage tells you what you spent. Workflow-level tracking tells you where and why.

5. How Unmeshed Helps You Control Token Spend

Provider dashboards tell you what you spent after the invoice arrives. Unmeshed lets you set hard limits before the spend happens.

Here is what that looks like in practice:

Per-step token limits. Every AI step in your workflow has a maximum token budget. Once a step hits that limit, it stops. This means no single step can quietly generate thousands of tokens and inflate your bill without you knowing.
Tool allow-lists for agents. You decide upfront which tools each AI agent is allowed to use. If a tool is not on the list, the agent cannot call it. No unintended actions, no surprise costs from tools running in the background.
Deterministic steps where AI is not needed. Not every step in a workflow needs a language model. Tasks like routing a request, parsing a field, or validating a value can run as plain code at zero token cost. You only pay for the steps where AI is actually doing something a function could not.

The goal is not to use fewer tokens across the board. It is to use AI where it earns its cost and replace everything else with code that runs for free.

The Bottom Line

Token efficiency is not about being cheap with AI.

Anyone can reduce a bill by doing less. What actually matters is understanding which workflows are worth the spend and which ones are just generating noise.

Tracking token usage early, at the workflow level, is how you stay ahead of that.

If you want hard limits on what every AI step can spend, Unmeshed is a good place to start.