I cut my agent bill by shrinking the prompt, not the model

I kept trying to lower my agent costs by switching models.

GPT-5 for one step. Claude Opus for planning. Gemini Flash for classification. Maybe Qwen for background work. Maybe Llama for cheap branches.

That was not the main problem.

The main problem was that my workflows were hauling too much junk into every request.

An n8n agent. A Make scenario. A LangChain worker. A custom retry loop. None of them looked expensive alone. But together they kept rebilling the same bloated context on every step, retry, and branch.

That was the leak.

If you run LLM-powered automations, prompt compaction usually saves more money than another round of model shopping.

The expensive part usually isn't the model

Model pricing is visible, so everyone fixates on it.

Prompt bloat is worse because it hides in places that feel harmless:

• full chat history
• giant tool schemas
• repeated document payloads
• stale memory from previous steps
• retries that resend almost the same prefix

In an agent workflow, long context behaves like compound interest.

One oversized prompt is annoying.
A prompt that gets replayed across 20 steps, 3 retries, and 2 branches is a billing pattern.

LangChain's short-term memory docs are pretty honest about this: long conversations make models slower, more expensive, and often worse because stale context distracts the model.

That matches what I see in real workflows. Long context is not automatically intelligence. A lot of the time it's just clutter with a premium price tag.

What providers actually bill as context

A lot of people think they're paying for "the user message."

They're not.

Providers count the full rendered request context. That usually includes:

• system or developer messages
• tool definitions
• JSON schemas
• memory/history
• retrieved documents
• images or other multimodal payloads
• the actual user input

So if your agent step only needs search_tickets and update_crm_record, but you're sending 12 tool schemas and a full transcript anyway, you're paying for dead weight.

The 3 biggest prompt bloat problems I keep seeing

Across n8n, Make, OpenClaw, LangChain, and custom workers, the same problems show up over and over.

1. Full chat transcripts on every step

Most steps do not need the entire conversation.

Usually they need:

• a compact state summary
• the latest user instruction
• maybe the last 1-3 turns

Passing 40 turns of history into a tool-selection step is how you quietly light money on fire.

2. Huge tool schemas everywhere

This one is common in agent frameworks.

You register a big toolset once, then every request drags all of it along.

If a step only needs two tools, send two tools.

Bad:

{
  "tools": [
    "search_tickets",
    "update_crm_record",
    "send_email",
    "create_invoice",
    "sync_calendar",
    "fetch_slack_thread",
    "query_warehouse",
    "generate_contract",
    "log_incident",
    "create_jira_issue",
    "archive_conversation",
    "notify_webhook"
  ]
}

Better:

{
  "tools": [
    "search_tickets",
    "update_crm_record"
  ]
}

3. Re-sending big document payloads

A lot of automations still paste giant docs into prompts instead of retrieving only the relevant chunks.

If the model needs one policy paragraph, don't send the whole handbook.

If it needs one support ticket, don't send the whole account history.

Prompt caching helps, but not as much as people hope

Prompt caching is real.

But caching rewards stability, not chaos.

OpenAI

OpenAI says cached-input discounts can reach up to 90%.

That's great.

The catch is that cache reuse depends on the full rendered prefix matching exactly. If you change something early in the prompt, mutate tool definitions, reorder context, or inject slightly different memory, you can miss the cache.

Anthropic

Anthropic prompt caching is useful too, but the default cache lifetime is 5 minutes.

That works well for tight loops.

It helps less when your automation:

• pauses between steps
• retries later
• branches asynchronously
• wakes up on delayed triggers

Gemini

Gemini implicit caching is enabled by default on newer models, but requests still need to cross token thresholds before it matters.

So yes, caching is useful.

But "maybe the cache saves me" is not a cost strategy.

Compacting the prompt is.

Provider caching reality, side by side

Provider	What actually matters
OpenAI Prompt Caching	Exact rendered prefix match is required; tools, schemas, developer messages, and history all affect reuse; cached-input discount can reach up to 90%
Anthropic Prompt Caching	Helpful for repeated prefixes, but default cache lifetime is 5 minutes; better for tight loops than delayed automations
Google Gemini Implicit Caching	Enabled by default on newer Gemini models, but only helps after minimum token thresholds are crossed

The pattern is simple:

Caching is great when your workflow is repetitive and tightly controlled.

Most real-world automations are not.

n8n already points you toward trimming memory

This part matters for anyone building agents in n8n.

n8n documents memory patterns pretty clearly, including memory backends and the Chat Memory Manager node. The docs explicitly point you toward inspecting and reducing memory before handing it to an Agent node.

That's not an advanced trick.

That's routine maintenance.

The n8n pattern I like

If I'm building an n8n agent now, I try to do this:

1. Keep short-term memory separate from durable app data
2. Summarize or trim history before the Agent node
3. Pass only the tools needed for that step
4. Retrieve only the document chunks relevant to the current action
5. Save a compact state summary instead of the full transcript

That usually improves both cost and output quality.

Because agents are terrible roommates. Give them too much stuff and they stop finding what matters.

How memory bloat spreads in LangChain-style agents

The architecture makes the problem obvious.

Memory gets read before the next action, then updated after the action. If you don't manage it, every bad step becomes future baggage.

Example:

from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="google_genai:gemini-3.6-flash",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

That setup is convenient.

It's also how one noisy tool call turns into repeated prompt cost for the rest of the workflow.

A failed attempt gets written.
Then a retry.
Then another tool result.
Then a side branch.
Then a transcript chunk nobody needs anymore.

Soon your "memory" is just a landfill the model has to search before answering a simple question.

A practical compaction pass

Here's the kind of cleanup I do before touching model selection.

Before: bloated request assembly

def build_prompt(state, tools, docs):
    return {
        "system": SYSTEM_PROMPT,
        "history": state.full_chat_history,
        "tools": tools.all_tools,
        "documents": docs.full_results,
        "input": state.current_task,
    }

After: compact request assembly

def build_prompt(state, tools, docs):
    return {
        "system": SYSTEM_PROMPT,
        "history": state.summary + state.last_turns[-3:],
        "tools": tools.required_for_step,
        "documents": docs.top_k_chunks,
        "input": state.current_task,
    }

That change is boring.

It also tends to save real money.

Quick ways to inspect prompt size

If you're not measuring, you'll miss the problem.

A simple first step is logging approximate payload size before each model call.

Python example

import json

def approx_chars(payload):
    return len(json.dumps(payload))

payload = build_prompt(state, tools, docs)
print(f"prompt_payload_chars={approx_chars(payload)}")

Node.js example

function approxChars(payload) {
  return JSON.stringify(payload).length;
}

const payload = buildPrompt(state, tools, docs);
console.log(`prompt_payload_chars=${approxChars(payload)}`);

This isn't token-accurate, but it's enough to catch obvious growth.

If one workflow step is shipping 10x more payload than the others, that's where I'd start.

A useful workflow audit checklist

Before I debate GPT-5 vs Claude Opus vs Gemini Flash, I ask:

• Does this step really need the full conversation history?
• Can I replace transcript history with a rolling summary plus the last few turns?
• Are unused tool schemas being sent anyway?
• Are retrieved documents narrowed to only the chunks needed right now?
• Are retries resending giant prefixes that changed just enough to miss cache reuse?
• Is this automation delayed enough that Anthropic's 5-minute cache window probably won't help?
• Is this Gemini request even large enough to qualify for implicit caching?

That checklist has saved me more money than most model-routing debates.

When model switching actually does matter

Model choice still matters.

If you've already cleaned up context and a cheaper model can do the same job, switch models.

That can be a clean win:

• classification on Gemini Flash
• extraction on a smaller open model
• simple branch logic on Qwen or Llama
• premium reasoning only where it actually matters

But I would do that after prompt cleanup.

If the prompt is bloated, you're just moving the same garbage to a cheaper truck.

One shell-level habit that helps

If you run your agents as services, log request size and retry count together.

Something as simple as this can surface the real cost pattern fast:

grep "prompt_payload_chars\|retry_count\|workflow_step" app.log | tail -n 100

Or aggregate it properly in your normal observability stack.

The point is the same: don't just track model name. Track how much context each step is dragging around.

Where Standard Compute fits

This is also why flat-rate inference is appealing for agent builders.

If you're running lots of automations across n8n, Make, Zapier, OpenClaw, or custom workers, per-token pricing punishes every messy workflow habit.

Standard Compute takes a different approach: one predictable monthly price, OpenAI-compatible API, and routing across models like GPT-5.4, Claude Opus 4.6, and Grok 4.20.

That doesn't mean prompt compaction stops mattering. It still improves latency, quality, and throughput.

It just means you're not babysitting token spend every time an agent retries or a workflow fans out.

For teams running agents all day, that tradeoff is pretty compelling.

The takeaway

If your agent bill feels weirdly high, don't start with model shopping.

Start by auditing context.

The usual waste is not mysterious:

• full chat history
• oversized tool schemas
• repeated document dumps
• memory that keeps growing between steps

Do prompt compaction first.
Then reduce context-window waste.
Then optimize model selection.

The fastest way to cut agent spend is often not finding a cheaper model.

It's teaching your workflow to shut up.