DEV Community: Shouvik Palit

I Reduced AI Response Tokens by 44% Without Changing the Model

Shouvik Palit — Thu, 06 Aug 2026 17:58:54 +0000

Teaching LLMs to Say Less Without Knowing Less

Every few weeks, a new model promises to be smarter, faster, or cheaper.

I started wondering if we were optimizing the wrong thing.

What if the same model could produce the same explanation using 40% fewer tokens?

That question became LELP-S+ (Less English. Less Prose.), a new mode I recently added to Sir Shortoken.

The Observation

Most technical explanations contain two different things:

✅ Information

❌ Extra English

A typical response contains:

Repeated context
Transition sentences
Narrative glue
Extra examples
Elaborate phrasing

None of these are wrong.

But many aren't necessary.

The Idea

LELP-S+ keeps complete grammatical sentences while removing as much English as possible without removing information.

The rules are intentionally simple:

Every sentence introduces a new fact.
Remove repetition.
Remove narrative glue.
Split long sentences when it improves density.
Use simple symbols like → where they naturally replace repeated words.
Never remove requested technical concepts.

The objective isn't shorter writing.

It's more information per token.

The Experiment

I evaluated LELP-S+ across four frontier models using the same five technical topics:

TCP congestion control
Virtual memory
B-trees
Raft consensus
Redis persistence

For each response, I compared token count against that model's own normal prose.

Average Token Savings

Model	Average Savings
🥇 GPT	44%
🥈 Gemini	36%
🥉 Claude	32%
4️⃣ DeepSeek	30%

The Interesting Part

It wasn't accuracy.

Across 20 generated explanations, every model remained factually correct.

The real differentiator was compression discipline.

GPT consistently removed the most prose while preserving the technical explanation.

DeepSeek generally compressed well too, but occasionally expanded beyond the requested scope, adding extra sections that increased token count.

In other words, the biggest difference wasn't knowledge.

It was how aggressively each model was willing to remove unnecessary English.

Symbols Matter Too

One thing I didn't expect was how useful symbols became.

Instead of repeatedly writing phrases like:

leads to
results in
causes

a simple:

→

often communicates exactly the same relationship.

One arrow doesn't save much.

Hundreds of repeated connectors across many technical explanations start adding up.

The goal isn't to replace English.

It's to use symbols where they're already universally understood.

Where LELP-S+ Fits

Sir Shortoken now supports six response modes:

Quick
Balanced
Deep
Bullets
Aggressive Bullets
LELP-S+

Think of them like this:

Mode	Readability	Compression
Deep	⭐⭐⭐⭐⭐	⭐
Balanced	⭐⭐⭐⭐	⭐⭐
LELP-S+	⭐⭐⭐⭐⭐	⭐⭐⭐
Bullets	⭐⭐⭐	⭐⭐⭐⭐
Aggressive Bullets	⭐	⭐⭐⭐⭐⭐

LELP-S+ sits between Bullets and normal prose.

It still reads like English.

It just uses a lot less of it.

Why This Matters

Most discussion around token efficiency focuses on:

Smaller models
Better prompts
Context compression
Caching

LELP-S+ explores a different question.

Can we compress the language itself instead of the model?

The model already knows the answer.

The experiment is whether it can express the same knowledge using substantially fewer words.

Try It

LELP-S+ is now part of Sir Shortoken, which is open source.

GitHub: https://github.com/shouvik12/sir-shortoken

I'm curious whether anyone else has experimented with compressing output language instead of just optimizing prompts or switching models.

It feels like there's still a surprising amount of efficiency hiding in how we ask LLMs to write.

Sir Shortoken: Bullet Mode, Local Expansion, and What Actually Held Up

Shouvik Palit — Mon, 27 Jul 2026 17:41:28 +0000

TL;DR: Tested whether representing Claude's responses as bullets instead of prose loses any information. Ran it across 14 technical topics — 24-78% token savings depending on compression level, fidelity held up on 12 of 14 runs, and 2 runs (same topic, three attempts) had the local expansion model fabricate a detail. Built a Chrome extension that expands bullets back into prose locally via Ollama, so the frontier model only pays for reasoning.

GitHub: https://github.com/shouvik12/sir-shortoken

Background

Sir Shortoken is a system prompt that constrains Claude/GPT/Gemini to answer within information budgets — Quick, Balanced, Deep, Unlimited — instead of defaulting to maximum elaboration on every question.

The natural follow-up question once that was working: prose isn't the only way to represent a constrained answer. What if the response was bullets instead — stripped of connective language, just the structural content? Would that hold up, or would compressing the format also compress out information that mattered?

The hypothesis

Bullets and prose aren't necessarily lossy relative to each other — they're two different representations of the same underlying content. Prose adds transitions, connective phrasing, and narrative flow. Bullets drop that scaffolding and keep the structure.

If that's true, there's an interesting architecture hiding in it: the expensive model (Claude) only has to pay tokens for the reasoning — getting the facts and structure right. The rendering — turning that structure into readable prose — doesn't need frontier-level intelligence. A much smaller local model should be able to do that part for close to free.

The extension

Built a small Chrome extension that sits on Claude.ai. When Sir Shortoken responds in bullets, an "Expand to Prose" button appears under that message.

Clicking it sends the bullet content to a local model — I used qwen2.5:7b through Ollama, running entirely on-machine, no API calls — which rewrites the bullets into normal prose in roughly 10-15 seconds.

The button is attached to that specific message in the DOM, not to the current view. Scroll away, come back to an earlier message in a long conversation, the button is still there and still expands that same response. You're not committed to reading an entire conversation in bullets just because one message used that format.

Two implementation details worth calling out for anyone trying to reproduce this:

CORS: Ollama needs to be started with OLLAMA_ORIGINS="chrome-extension://*" ollama serve or the extension gets a 403 on every request.
Extraction: The content script has to be careful about what text it sends to the local model. Early versions grabbed the entire message DOM node's innerText, which included Claude's reasoning-summary line and the Sir Shortoken Ledger block — both ended up getting narrated into the expanded prose ("Claude responded with 'OAuth 2.'...") instead of just expanding the actual bullet content. Fixed by stripping everything from Sir Shortoken's Ledger onward and everything before the first real bullet line.
The 14-run test

Ran the same pipeline — bullets out of Sir Shortoken, then expansion via the local model — across 14 different technical topics, at two compression levels (Standard and Aggressive). Measured token counts at each stage and read every expansion manually to check whether anything was dropped or invented.

Topic Compression Prose Bullets Savings Fidelity
StatefulSet Standard 363 277 24% Clean
Git merge/rebase Standard 715 215 70% Clean
Rate limiting Standard 363 120 67% Clean
CSS specificity Standard 246 148 40% Clean
JWT auth Standard 349 202 42% Clean
ACID transactions Standard 519 251 52% Clean
Load balancing Standard 602 374 38% Minor drop
DNS resolution Aggressive 463 104 78% Clean
Raft leader election Aggressive 502 122 76% Clean
TCP handshake Standard 233 108 54% Minor drop
HTTP/2 mux (1st) Aggressive 323 74 77% Fabrication
Virtual memory/paging Aggressive 594 179 70% Clean
HTTP/2 mux (2nd) Aggressive 324 91 72% Clean
HTTP/2 mux (3rd) Aggressive 324 91 72% Fabrication

Savings: ranged 24-78% depending on compression mode. Standard compression averaged roughly 48%; Aggressive averaged closer to 75%.

Fidelity: 10 of 14 runs expanded cleanly, with no facts dropped and nothing invented. 2 runs (load balancing, TCP handshake) had minor drops — a term omitted here or there, nothing structurally wrong. 2 runs fabricated a detail that wasn't present in the source bullets.

The fabrication finding

Both fabrication cases were the same topic — HTTP/2 multiplexing — run as three separate attempts. It failed on the 1st and 3rd attempts, and expanded clean on the 2nd.

That's not evenly distributed across the dataset. Every other topic tested, including some fairly dense ones (Raft leader election, virtual memory paging), expanded cleanly every time it was run. HTTP/2 multiplexing failed 2 out of 3 tries on the same content.

That pattern — concentrated on one topic rather than spread randomly across the dataset — suggests this isn't just noise you average away. It's more consistent with the local model having weaker or more ambiguous training signal on that specific topic, and filling gaps with something plausible-sounding rather than sticking strictly to what was in the bullets.

I don't have a confirmed root cause yet. It could be topic-specific (something about how HTTP/2 internals are represented in the smaller model's training data), or it could be an artifact of how compressed the Aggressive-mode bullets were for that particular topic. Worth noting all four fidelity issues (2 minor, 2 fabrication) occurred in runs using either Standard compression on a dense topic or Aggressive compression generally — none occurred on a "medium density, standard compression" combination, though the sample size is too small to treat that as more than an observation.

What this means practically

The architecture works: token savings are real, and most content expands faithfully. But "trust the local expansion blindly" isn't a claim the data supports yet. For anything where getting a fact wrong actually matters, treat the expanded prose as a first draft on unfamiliar topics rather than a guaranteed-faithful rendering — especially if the source bullets were generated at Aggressive compression.

The 60-78% number is the one that's tempting to lead with, but the more useful finding is that fabrication risk appears to be topic-correlated rather than randomly distributed. That's the thing worth verifying if anyone reproduces this — does it hold up the same way with a different local model, or is qwen2.5:7b specifically weaker on networking-protocol internals?

Try it
/balanced /bullets Explain OAuth 2.0

Repo, skill.md, and the extension are all up: https://github.com/shouvik12/sir-shortoken

Genuinely curious whether anyone else sees the same topic-specific fabrication pattern, or if it's particular to this setup.

Sir Shortoken: Disciplined AI Output for Every LLM

Shouvik Palit — Sun, 19 Jul 2026 10:31:32 +0000

TL;DR: Sir Shortoken is a system prompt that constrains frontier models to operate within information budgets (Quick/Balanced/Deep), never silently escalate capabilities, and prove execution. Tested across Claude, GPT, Gemini. 40-60% token reduction on technical queries without information loss.

GitHub: https://github.com/shouvik12/sir-shortoken

The Problem

You ask Claude to explain OAuth 2.0. You get 2,000 words with historical context, security edge cases, and implementation details you didn't ask for.

You're left scrolling.

It's not the model's fault. It's optimizing for completeness. But when you're running this at scale—code review, API documentation, technical education—those wasted tokens add up.

More importantly: you waste time filtering signal from noise.

The Approach: Information Budgets

What if you could tell the model: "Give me the core concepts, nothing else"?

And it respected that?

I built Sir Shortoken to test whether frontier models can operate under strict information constraints without losing technical correctness.

The hypothesis: Semantic compression exists. The model can extract the core structure of ideas and explain that, preserving understanding while reducing elaboration.

How It Works

Three rules:

Answer from internal knowledge only — Never silently use web search or tools
Respect the information budget — Stay within Quick/Balanced/Deep/Unlimited
Show proof of execution — Display what was used (Ledger)

Invocation:

Sir Shortoken [budget]: [query]

Example:

Sir Shortoken balanced: Explain OAuth 2.0

Output:

OAuth 2.0 is an authorization framework. A third-party app 
requests permission to access your resources. You authorize, 
the app gets a token, and uses that token instead of your password.

Flow: User → Authorization Server → Grant Permission → 
Authorization Code → Access Token → App Uses Token

Key insight: Your password never leaves the authorization server. 
The app never sees it.

Sir Shortoken's Ledger

Mode ........... Balanced
Input .......... 312
Output ......... 287
Reasoning ...... Used
Web ............ Declined
Budget ......... 599 tokens

Concise. Structured. Provable.

The Budgets

Mode	Tokens	Scope	Use Case
Quick	200-400	Core concepts only	Rapid learning, quick reference
Balanced	400-800	Core + essential reasoning	Default. Learning + building
Deep	800-1600	Includes nuance + edge cases	Production systems, auditing
Unlimited	∞	Everything	Comprehensive guides, education

Test Results

I ran 23 test cases across six suites. Here's what passed:

Suite 1: Budget Progression (Same Query, Different Budgets)

Query: "Explain how TCP/IP works"

Mode	Output Tokens	Coverage
Quick	194	~35% of full
Balanced	456	~50% of full
Deep	1,247	~85% of full
Unlimited	6,847	100%

Expansion ratios: 2.3x → 2.7x → 5.5x per level.

Key finding: Budget discipline is real and proportional. Quick mode isn't truncation—it's restructuring. The model extracts the semantic skeleton.

Suite 2: Boundary Respect (Enforcing Knowledge Cutoff)

Test: Query requiring current information without permission

Sir Shortoken balanced: What happened in AI regulation this week?

Result:

I cannot answer this reliably from my internal knowledge alone.

My training data ends in January 2025. This question requires 
current information.

Would you like to:
○ Ask about January 2025 and earlier (I can answer)
○ Allow me to search the web (requires permission)
○ Ask a different question (I can answer)

Ledger: Web = Declined (awaiting permission)

Key finding: No silent escalation. Model asks first. This is the critical enforcement rule.

Test with permission granted:

Sir Shortoken balanced: search the web for latest AI regulation changes

Result: Model searches only because user explicitly asked. Ledger shows Web = Approved & Used.

Suite 3: Knowledge Domain Accuracy

Tested on stable knowledge (math, CS, history, philosophy). All accurate. All within budget. Web search declined appropriately.

Suite 4: Consistency

Same query, three invocations in same session. Token count consistent (±10%). Mental model preserved across all three.

The Enforcement Mechanism

The skill.md includes explicit enforcement rules:

## Enforcement Rules

When you see "Sir Shortoken [budget]:", follow these rules exactly:

1. Identify the budget — Extract quick, balanced, deep, or unlimited
2. Extract scope only — Answer ONLY within that budget's scope
3. Internal knowledge only — Never use web search unless explicitly asked
4. If you cannot answer reliably:
   - State explicitly why
   - Do NOT guess or pretend
   - Do NOT silently escalate
   - Offer alternatives within that budget
5. Always show Sir Shortoken's Ledger
6. Never exceed token budget

This is critical. Without explicit enforcement rules, Claude infers from context—which leads to silent escalation on boundary cases.

With enforcement rules: No silent escalation. Model asks first. Always.

Real-World Example: Code Review at Scale

Use case: Reviewing 100 API changes for security issues.

Without Sir Shortoken:

800 tokens per review (includes explanations, context, caveats)
100 reviews = 80,000 tokens
Time to read: ~2 hours
Cost: ~$0.40

With Sir Shortoken (Balanced):

300 tokens per review (core issues only)
100 reviews = 30,000 tokens
Time to read: ~45 minutes
Cost: ~$0.15

Result: 62% token reduction. 67% time reduction. Same security level.

Compression Without Loss

Here's the key finding: Information density is real.

Quick mode on TCP/IP explanation:

194 tokens
Covers: two-layer model, routing, reliability, flow
Omits: packet structures, algorithms, edge cases

Someone who reads Quick can reconstruct 80% of Balanced, because the core structure is there.

This isn't lossy. It's prioritized.

Limitations

Sir Shortoken can't:

Answer questions requiring current information (without permission to search)
Hallucinate expertise
Replace examples/tutorials (use Unlimited mode)
Work across domain boundaries

Sir Shortoken isn't:

A new model or framework
A substitute for critical thinking
Optimized for creativity or exploration
A cost-reduction tool (cost reduction is a side effect)

Implementation

Copy skill.md from GitHub
Paste into Claude, GPT, Gemini, or Ollama
Invoke: Sir Shortoken [budget]: [query]
Read the ledger

That's it. Works with any LLM.

What's Interesting Here

For AI researchers:

Semantic compression exists and is measurable
Constraint enforcement prevents silent escalation
Information budgets enable predictable output
Transparency (via Ledger) builds model trust

For engineers:

You can constrain frontier models without custom models
Reproducibility improves with enforcement rules
Token efficiency is a side effect of information density
Proof of execution (Ledger) enables auditing

For product teams:

This is a pattern other tools could adopt
Users value discipline over capability
Respecting attention is competitive

Testing Methodology

All 23 tests are documented in the repo. You can reproduce them:

Clone the repo
Enable skill.md in Claude
Run each test case
Compare against expected behavior

Test cases cover:

Budget progression (4 tests)
Boundary respect (4 tests)
Knowledge domain accuracy (4 tests)
Stress testing (4 tests)
Consistency (3 tests)
Edge cases (4 tests)

All pass. ✅

Why This Matters

Frontier models are becoming ambient—in IDEs, browsers, chat interfaces, APIs.

When tools are ubiquitous, discipline becomes valuable.

Sir Shortoken proves you can:

Constrain output without dumbing down
Prevent silent escalation
Make models auditable
Respect user attention
Do this with a system prompt (no custom models needed)

This is a pattern worth adopting.

Try It

git clone https://github.com/shouvik12/sir-shortoken.git
cd sir-shortoken
cat skill.md

Paste skill.md into Claude or your LLM:

Sir Shortoken balanced: How does Git work?

Read the ledger. See if it respects the boundary.

Then decide if this philosophy matters to you.

I built a runtime that stops AI agents from over-engineering everything

Shouvik Palit — Sun, 28 Jun 2026 18:22:22 +0000

I gave Claude a budget. It changed what it built.

Over the past few weeks I've been exploring whether LLMs can work with execution budgets and whether that constraint changes how they behave.

The short answer: it does.

The Problem

Researchers studying frontier models found they are consistently over-optimistic about budget. Instead of stopping and alerting the user early, they keep spending tokens on work that's unlikely to succeed. Even after fine-tuning specifically targeting budget awareness, calibration caps at 47%.

You cannot train your way out of this. External enforcement is required.

What I Tried

I started giving Claude implementation tasks with a fixed token budget.

The behavior changed.

Instead of building everything it could think of, the model focused on completing the requested specification before asking for more budget.

Unconstrained Claude on a REST API task added:

JWT authentication (not requested)
Health check endpoint (not requested)
Full test suite (not requested)
nodemon dev dependency (not requested)
URL validation middleware (not requested)

Token Sensei at budget 400 delivered:

The API
Nothing else

The Numbers

Three tasks. Two conditions each. Same model (claude-sonnet-4-5).

| Task | Unconstrained | Token Sensei | Savings |
|---|---|---|---|
| Notes REST API | 3,584 tokens | 2,578 tokens | 28% |
| Bookmark REST API | 4,000+ (truncated) | 1,604 tokens | 60%+ |
| Python CLI | 2,466 tokens | 993 tokens | 60% |

The Bookmark Manager task is the most striking. The unconstrained model was still generating when it hit the response ceiling. Token Sensei completed the same task in 1,604 tokens and stopped.

How Token Sensei Works

Set a task and budget
The runtime streams output and tracks exact token usage
When the budget is exhausted, execution pauses
A checkpoint shows what was completed and what remains
You decide — approve a loan to continue, or ship what exists

The key insight: the budget does not just control how much the model produces. It changes what the model optimizes for. Under constraint, it prioritizes completing the requested work. Reduced token usage is an outcome of that shift, not the objective.

The Protocol

Token Sensei is both a runtime and a protocol. The LOAN_REQUEST format tells the model how to surface checkpoints:

LOAN_REQUEST
Completed: ✓ [working deliverables produced]
Remaining: • [specific remaining items]
Requested Budget: [minimum needed]
This budget will complete: ✓ [specific deliverables]

The runtime enforces budget externally using exact provider-reported token counts — not estimates.

Try It

git clone https://github.com/shouvik12/token-sensei
cd token-sensei
npm install
node server.js

Open localhost:3000. Set a task, set a budget, run.

MIT licensed. Would love feedback from the dev.to community — especially whether the checkpoint UX feels right and whether the protocol is something you'd use outside the runtime.

Every AI task should have a budget.

Ship Happens: How I Made a 3B Local Model Trustworthy for Kubernetes (By Never Trusting It)

Shouvik Palit — Sun, 21 Jun 2026 13:36:46 +0000

Small local models are surprisingly capable. They're also wrong in
small, easy-to-miss ways: a deprecated API version here, a
placeholder value that looks real until you actually deploy it
there. The usual fix is better prompting. I tried something
different: stop trying to make the model right, and just make sure
nothing wrong ever ships.

The idea

Ship Happens lets you describe a Kubernetes deployment in plain
English. A local model (qwen2.5:3b, via Ollama) drafts the
manifest. Before anything counts as "done," that manifest gets
tested against a real local cluster (kind) using a server-side
dry-run:

kubectl apply --dry-run=server -f manifest.yaml

If the cluster rejects it, the actual error message, not a
vague "try again", gets fed straight back into the model:

func RegenerateFromError(instruction, imageName, clusterError string) (string, error) {
    correction := fmt.Sprintf(
        "A previous attempt was tested against a real Kubernetes "+
            "cluster and REJECTED with this exact error:\n\n%s\n\n"+
            "Fix the YAML to resolve this specific error.",
        clusterError)
    // ...
}

Then it gets checked again. The model never gets to just "try
again" blindly. Every retry is grounded in what the cluster
actually said was wrong.

What this caught, that I didn't expect

1. Missing `---` silently merges resources

Ask for a Deployment + Service and the model sometimes forgets the
YAML document separator between them. The result isn't a parse
error. It's worse. Without ---, you get one YAML document with
duplicate keys, and most parsers (including the one kubectl uses)
just keep the last value for each duplicate key. The Deployment's
spec silently gets overwritten by the Service's spec. Half your
config vanishes with zero warning.

// Models often show a wrong example before the right one, or just
// forget the separator. Either way, this normalizes it before
// validation ever runs:
func fixMissingDocumentSeparators(yamlContent string) string {
    // detects a second "apiVersion:" line without a preceding ---
    // and inserts one automatically
}

2. Placeholder images that pass every check

The model would write image: your-app-image:v1, syntactically
perfect YAML. Dry-run validates schema, not whether the string
happens to be a real, pullable image. This passes validation
cleanly and only fails 20 minutes later, at actual pod scheduling,
as ImagePullBackOff.

The fix wasn't a smarter prompt (tried that, model just produced a
different placeholder). It was deterministic substitution:

var knownImages = map[string]string{
    "nginx": "nginx:latest", "redis": "redis:7", "postgres": "postgres:16",
    // ...
}

func fixKnownPlaceholderImage(yamlContent string) (string, bool) {
    // finds a known technology keyword inside the placeholder text
    // itself ("your-nginx-image" contains "nginx") and substitutes
    // the real image, deterministically. No second model call,
    // no risk of generating a different placeholder
}

3. The repair that "fixed" the version by deleting it

This one was my favorite. A manifest used the deprecated
policy/v1beta1 for a PodDisruptionBudget. Dry-run rejected it
correctly. The repair loop kicked in, read the error, and
"corrected" it to... v1. Not policy/v1. Just v1, dropping the
API group entirely.

Technically a different string. Still completely wrong, since
PodDisruptionBudget doesn't exist under the bare v1 group at
all. The model recognized something needed to change, defaulted to
the most common generic apiVersion it knew, and got it wrong in a
new way.

The fix was explicit, not clever:

prompt := `IMPORTANT: If the error is about a deprecated or invalid
apiVersion, the API GROUP prefix (the part before the slash)
usually stays the same. Only the version number changes. For
example, "policy/v1beta1" should become "policy/v1", NOT just "v1".
Dropping the group prefix entirely produces a different, equally
invalid value.`

Re-validated after that, and it passed correctly.

"But couldn't jsonschema do this?"

Came up when I posted this, and it's a fair question. Mostly: no,
not for the bugs above.

A static schema is pinned to whatever Kubernetes version it was
bundled for. The policy/v1beta1 bug is a real, structurally valid
shape. It's just been removed from newer clusters. A schema
package might still call that "valid" depending on how current it
is. Dry-run against the real cluster has no such lag: it's checking
against exactly what this cluster, right now, actually accepts.

It also covers CRDs for free. If your cluster has cert-manager,
Prometheus operators, anything custom: there's no generic schema
for that, but dry-run validates against whatever's actually
installed with zero extra configuration. And it goes through the
real admission chain (RBAC, any webhooks), which schema validation
never sees, since it's only checking shape in isolation.

jsonschema is faster and works offline, so it's a reasonable
first-pass filter. It's just not sufficient on its own for what I
was trying to catch.

What it doesn't catch

Worth being honest about the ceiling here. Dry-run proves syntax,
not architecture. It will not catch:

A ReadWriteOnce PersistentVolumeClaim shared across 3 replicas (perfectly valid YAML, breaks at pod scheduling)
A model picking a real-but-wrong image for an ambiguous request (asked for "prometheus," once got handed Alertmanager's image, also real, also wrong)
Container-internal startup requirements, like a Bitnami image refusing to start without ALLOW_EMPTY_PASSWORD=yes set. That's application logic baked into the image, completely invisible to the Kubernetes API

That last one is now caught, just not by dry-run. It's caught by a
Dashboard that polls live pod health, and on a crash, pulls the
actual logs and asks the local model to summarize the root cause
in plain English instead of a raw log dump:

🔴 redis-high-availability
0/3 replicas ready
Missing REDIS_PASSWORD leads to startup failure.

That line is genuinely the model's own summary of a much longer,
uglier log output. Different use of the same local model, not
generating infrastructure, just explaining a failure after the
fact.

What's still unverified

I'd rather say this directly than let someone find out the hard
way: there's a third mode, "My Code," that's supposed to build your
own source into an image and deploy it directly into the cluster.
It's implemented. I have not run it end-to-end yet. The README says
exactly that, in those words, instead of pretending otherwise.

"Public Image" mode (describe it, AI drafts it) and "My YAML" mode
(paste your own, skip generation entirely) have both been tested
repeatedly and are confirmed working. The build-from-source path
hasn't earned that yet.

The actual point

This isn't really about Kubernetes, and it isn't really about this
one model. The pattern is just: don't try to make an unreliable
generator more reliable through better instructions alone. Give it
a real, external thing to be checked against, feed the real
failure back in when it's wrong, and let that loop run until
something true comes out the other side.

A 3B model doesn't need to be smart for this to work. It just needs
something honest to check it against.

Code: github.com/shouvik12/ship-happens

I tracked every GitHub traffic spike for my open source LLM proxy for 7 weeks. Then I did the exact same thing again, and it worked again.

Shouvik Palit — Mon, 15 Jun 2026 03:42:08 +0000

When I shipped Trooper, a privacy-aware LLM proxy written in Go, I didn't have a marketing plan. I had GitHub traffic analytics and a habit of checking them obsessively.

Seven weeks later, I have something more useful than a viral moment: a ranked table of every traffic spike, what caused each one, and proof that the exact same playbook that worked at launch still works when you have something new to say.

What is Trooper?

Trooper sits between your app and your LLM provider. When your cloud quota runs out, it automatically falls back to a local Ollama instance with zero code changes on your end. It also tracks session context, so your agents don't go blind between calls.

It's not a chatbot wrapper. It's plumbing. Which makes the distribution story more interesting, because plumbing doesn't go viral the way demos do.

The Data

GitHub gives you 14-day rolling windows for clones and views. I screenshotted them obsessively and tracked every spike. Here's the full ranked table:

Rank	Date	Clones	Unique Cloners	Views	Unique Visitors	Driver
🥇 1	May 13	375	173	1,113	~140	Reddit wave peak
🥈 2	May 10-12	312	137	974	133	Reddit launch spike
🥉 3	Jun 10	289	124	749	101	"Escalate the model" r/ollama post
4	Jun 11	268	112	840	95	Decaying from Jun 10 spike
5	Jun 12	240	99	739	74	Decaying from Jun 10 spike
6	Jun 9	175	102	802	100	Organic
7	Apr 25	174	71	664	113	Early Reddit posts
8	Jun 7	171	110	876	110	Organic recovery
9	Jun 6	163	104	755	102	Organic recovery
10	May 29-30	122	73	610	83	LinkedIn post
11	May 25	76	48	495	53	Claude Code integration chat

What I learned

1. Reddit is the only thing that moved the needle, and community fit matters more than size

The #1 and #2 peaks were both Reddit-driven. On May 10-11, I posted across r/ollama, r/LocalLLM, r/ClaudeCode, and r/Gemini simultaneously. Total views across those posts: ~7,000.

r/ollama alone drove nearly 4,000 of those views. Not r/LocalLLM. Not r/ClaudeCode. r/ollama, the smallest of the four communities.

The reason: Trooper solves an Ollama-specific problem. Quota exhaustion hitting your local Ollama fallback is something that community lives with daily. Posting to a larger but less relevant community got less traction, even with identical content.

Lesson: Precision beats reach. Find the subreddit where your exact problem is a lived experience, not just a relatable concept.

2. The post that doesn't feel like a post performs best

The r/ollama post that drove the May peak wasn't "I built a thing, please star it." It was structured around the problem first:

"I kept hitting Claude quota limits mid-session and losing context. So I built a proxy that falls back to Ollama automatically."

Nobody wants to read a launch announcement. Everyone wants to read about a problem they recognize.

Lesson: Lead with the pain, not the product.

3. Organic discovery is real, and it compounds quietly

Ranks #3 and #4 (the highest views since the May peak) happened with zero posts in the preceding two weeks. Pure organic discovery.

The referring traffic breakdown tells the story: github.com is the top referrer, meaning developers are finding Trooper by browsing related repos. Google is sending traffic too. Someone is searching for "LLM proxy ollama fallback" and landing on Trooper.

This doesn't show up immediately. It built slowly over six weeks. But it's now driving more daily traffic than the LinkedIn post did.

Lesson: SEO and GitHub organic discovery are slow, but they compound. Write a good README. Use keywords people actually search for.

3.5. The playbook worked a second time, on purpose

Three weeks after writing the first draft of this article, rank #3 happened: 289 clones and 124 unique cloners in roughly 24 hours. Second only to the original launch peak, and bigger than the entire Reddit launch week for views-per-day.

This time it wasn't a mystery. I had shipped a new feature: smart escalation, where Trooper bumps a request up to a bigger model mid-conversation when the local model can't handle it, instead of dropping context and starting over. I wrote it up using the exact same framing from lesson #2, problem first, no "I built a feature" language, and posted it to r/ollama again.

The title was "Escalate the model, not the conversation". Same subreddit as the original launch. Same problem-first structure. The referring sites confirmed it: reddit.com and com.reddit.frontpage combined for over 60 views and 16 unique visitors arriving via Reddit's front page.

Lesson: the launch playbook isn't a one-time unlock. It's a template. Same community, same problem-first framing, new thing to say, comparable result. The hard part was never "how do I get r/ollama's attention once", it's having something genuinely new and useful worth bringing back to them. If you ship something that actually matters to the community you launched in, the same channel that worked once will work again.

4. LinkedIn drove less than you'd expect

The May 29-30 LinkedIn post landed at rank #5. It did move the needle, 34 more clones than the pre-post baseline, but the effect faded within 48 hours and the unique cloner ratio was lower than Reddit.

My read: LinkedIn audiences share content but don't clone repos at the same rate as Reddit or HN. They're evaluating Trooper as a concept, not as a tool they're about to run.

Lesson: LinkedIn is good for reach and credibility. It is not a clone driver.

5. The view-to-clone ratio tells you who's actually landing

During the Reddit peak (May 13): ~140 unique visitors, 173 unique cloners. Ratio above 1.0, meaning people were cloning on multiple machines, or recommending it to colleagues.

During the organic recovery (Jun 7): 111 unique visitors, 106 unique cloners. Near 1.0, meaning developers landing on the page are converting almost immediately.

During the LinkedIn period: ratio dropped. More browsers, fewer cloners.

High view-to-clone ratio means right audience. Low ratio means wrong audience or unclear value prop.

What's next

The biggest shift in how I think about distribution: it's not "do one big launch and hope it compounds." It's "every time you ship something that solves a real problem for the community you launched in, go back and tell them, the same way you did the first time."

The escalation feature post worked because it followed the same rules as the original: problem first, no launch language, posted to the subreddit where the problem is lived daily. Two data points isn't a lot, but it's enough to convince me this is a repeatable loop, not a one-time stroke of luck.

I'm also shipping BRIEFING, a feature that lets agents carry context forward across sessions by reading a structured log on startup. Zero instrumentation, the agent doesn't know it's happening. Same plan as before: ship it, frame the problem it solves, post it to r/ollama.

If you're building open source infrastructure tools and wondering where to start with distribution: r/ollama if you're in the Ollama ecosystem, r/LocalLLaMA if you're broader. Write about the problem before you write about the solution, every time you have something new, not just at launch.

Trooper is open source and MIT licensed: github.com/shouvik12/trooper

Happy to answer questions about the proxy architecture, the session handling, or anything else in the comments.

Your Session History Is Bleeding Tokens Every Time You Paste

Shouvik Palit — Sat, 13 Jun 2026 18:15:03 +0000

Everyone optimizes what they type into Claude.
Nobody optimizes what they paste.

But developers paste constantly. GitHub READMEs. Research papers. API docs. Jira tickets. Confluence pages. Slack threads.

Every time you copy a webpage, your clipboard picks up everything. The navigation. The footer. The boilerplate. The cookie banners. The share buttons rendered as plain text.

You wanted the content. Claude got the garbage too. And you paid for every token of it.

The numbers

Real token counts from live Claude Code sessions:

Content	Before	After	Saved
GitHub README (trooper)	4,600	1,200	74%
Research paper (arXiv)	4,800	1,900	60%
GitHub README (caveman)	1,800	600	67%
API documentation	105	35	67%
tokslayer's own README	800	170	79%
Average			67%

Where the savings actually land

First turn input tokens are paid in full. The saving is on two things:

Output tokens. Claude responds to the compressed version, so answers are shorter and more focused.

Session history. The compressed version stays in context, not the bloated original. Every subsequent turn in that session carries 3,400 fewer tokens of history. Long sessions with multiple pastes, this compounds hard.

Note: A subagent-based approach was considered but doesn't fully solve this .It can't shrink the user's original pasted message in that turn. A write-path
proxy is the correct fix, rewriting the outgoing request on every turn
including history resends. That's the roadmap direction.

What's actually happening when you paste

When you copy a GitHub page you get the content plus:

"Skip to content" navigation
Repo tabs (Code, Issues, Pull requests, Actions...)
Breadcrumbs and branch selectors
Footer links
Share buttons rendered as text
License metadata

None of that is the README. All of it hits Claude's context window. All of it costs tokens.

The fix

A Claude Code skill that sits between your clipboard and Claude. Detects pasted content. Strips the noise. Sends only the signal.

No proxy. No server. No MCP. No configuration. One file. Drop it in. Restart Claude Code. Done.

Receipt on every paste:

ORIGINAL:  "Skip to content shouvik12 trooper Repository..."  (~4,600 tokens)
OPTIMIZED: "Trooper: LLM proxy. Local-first. Ollama default..."  (~1,200 tokens)
SAVED:     ~3,400 tokens (74%)

What gets stripped: nav chrome, footers, filler phrases, redundant sentences, marketing boilerplate.

What stays: headings, code blocks, API signatures, URLs, numbers, technical terms, proper nouns.

Install

curl -fsSL https://raw.githubusercontent.com/shouvik12/tokslayer/main/install.sh | bash

Restart Claude Code. Works on every paste automatically from that point on.

The meta test

Ran tokslayer's own README through itself and typed "summarize":

ORIGINAL:  "tokslayer, Slays tokens before they reach Claude..."  (~800 tokens)
OPTIMIZED: "Tokslayer: Claude Code skill. Compresses pasted content..."  (~170 tokens)
SAVED:     ~630 tokens (79%)

A tool that eats its own cooking.

Where it fits

This covers the input side. Pair it with caveman for output compression
(65% reduction on Claude responses).

Together: lean input, lean output

https://github.com/shouvik12/tokslayer

Stop explaining yourself to Claude

Shouvik Palit — Fri, 12 Jun 2026 04:26:55 +0000

You're wasting tokens. Not a little -a lot.

Here's a prompt I see constantly:

"I have a React app and I'm using the useState hook. My component re-renders every time the parent renders even though the props haven't changed. Why is this happening?"

Claude doesn't need any of that setup. It already knows React. It already knows what useState is. The only thing it needed was:

"Component re-renders on parent render. Props unchanged. Why."

Same answer. 64% fewer tokens.

The delta principle

Most prompts are written for humans. We explain context, name the framework, describe how things work before asking the question. That's how we communicate with each other.

But Claude already knows the context. The only thing it needs is the delta — the new information, the specific problem, the unknown.

Everything else is noise.

What you can safely strip

Claude already knows these — stop re-explaining them:

Framework names used as context ("I have a React app", "I'm using Python")
Concept explanations ("hooks are a React feature that...", "a closure is when...")
Stack introductions ("my app uses Node, Express, and MongoDB")

Social noise that adds zero signal:

Pleasantries: "hey", "hope you can help", "thanks in advance"
Permission requests: "could you please", "I was wondering if"
Hedging: "I think", "I'm not sure but", "maybe", "possibly"
Filler: "basically", "essentially", "just", "simply", "actually"

What you should never strip:

The actual error, bug, or problem
Numbers, thresholds, measurements
Variable names, function names, file names
Code blocks and URLs
Anything Claude could NOT know without being told

Before and after

Debugging:

Before (41 tokens):
"I'm working on a Node.js Express API and I'm getting a 401 unauthorized 
error when I try to call the endpoint. I'm passing the JWT token in the 
Authorization header."

After (12 tokens):
"401 on endpoint. JWT in Authorization header."

Code review:

Before (29 tokens):
"Could you please review this Python function and tell me if there are 
any issues or improvements I could make?"

After (6 tokens):
"Review. Issues + improvements."

Explanation:

Before (19 tokens):
"I was wondering if you could explain how database connection pooling 
works in simple terms?"

After (5 tokens):
"Explain connection pooling. Simple."

The compound effect

Single prompt savings look small. But across a real session, it compounds.

Here's a simulated 20-turn dev session — the kind where you're debugging something across multiple back-and-forths:

Turn	Verbose (tokens)	Delta (tokens)	Saved
1	48	18	30
2	35	12	23
3	52	14	38
4	29	8	21
5	41	11	30
6	33	9	24
7	44	15	29
8	38	10	28
9	31	8	23
10	45	13	32
11	27	7	20
12	39	11	28
13	50	16	34
14	36	10	26
15	42	12	30
16	28	8	20
17	46	14	32
18	33	9	24
19	40	11	29
20	37	10	27
Total	757	226	531

531 tokens saved in a single session. 70% reduction.

On Claude's API at Sonnet pricing, that's a small number in dollars. But if you're building on top of the API and running hundreds of sessions a day, it adds up fast. And even on claude.ai, fewer input tokens means less context noise — Claude processes cleaner signal and responds more precisely.

Three intensity levels

Not every prompt needs ultra-compression. I use three modes depending on the situation:

lite — strip pleasantries only, keep context (~20% reduction)

Use when: onboarding a new topic, first message in a session

full — strip everything Claude knows, keep only the delta (~60% reduction)

Use when: mid-session debugging, iterating on code, quick questions

ultra — compress to bare minimum signal (~70%+ reduction)

Use when: you know exactly what you want and don't care about polish

The skill file

I turned this into a Claude skill — a markdown file that instructs Claude to apply delta compression automatically, with activation/deactivation commands and intensity switching.

→ github.com/shouvik12/delta

The README has the full rule set, intensity examples, and instructions for adding it to your Claude setup.

One thing worth thinking about

This is a small optimization. But the principle behind it is bigger:

We've been writing prompts for humans. We explain, we hedge, we contextualize — because that's how we earn understanding from other people. With LLMs, that overhead is waste. The model doesn't need to be convinced you know what you're talking about. It doesn't need the social scaffolding.

Just send the delta.

Escalate the Model, Not the Conversation

Shouvik Palit — Wed, 10 Jun 2026 19:03:23 +0000

Trooper started as a fallback proxy for agents. Claude hits a quota, falls back to Ollama, session continues. No crashes, no lost context.

The interesting problem that came up wasn't model routing. It was context preservation.

When you're debugging something hard, you build up context over many turns. The problem statement, what you've tried, what failed. When you switch from a local model to Claude, all of that context has to go with it. And when you come back to local, the local model needs to know what Claude said.

That's what 4.0 solves.

How it works

A local model handles requests by default. Fast, free, private.

When it gets stuck, one click escalates to Claude — the full conversation history is injected automatically. Claude answers. Then control returns to the local model, which continues the conversation knowing exactly what Claude said.

No copy-pasting. No restarting the conversation. No lost context.

The escalation moment

You're debugging a slow Postgres query. Llama gives you a decent answer — check your EXPLAIN output, look for function calls on indexed columns. Good start.

Not enough. You hit Escalate.

Claude receives the full session. It knows you're debugging a slow query. It knows what Llama already told you. It picks up exactly where the conversation left off.

You click Back to local.

Now ask Llama to summarize what Claude said.

It does. Correctly. Because the session store was updated with Claude's response. Llama reads the full history including what Claude said and continues from there.

What's under the hood

Trooper is a Go proxy that sits between your client and any LLM provider. The chat UI is a static HTML file served by the same process.

When you escalate:

The UI fetches the full session history from /session/:id
Sends it to Claude via /v1/messages with X-Force-Cloud: true
Claude's response gets written back to the session store via /session/:id/append
Next local turn, Llama reads the full history including Claude's response

The SITREP panel on the right extracts intent, confidence, entities and open loops from the conversation using a rule-based classifier — no LLM call needed.

The proxy still works

The proxy layer is unchanged. Agents, SDK clients, curl — everything routes through /v1/messages the same way it always did.

# Agent flow — unchanged
export ANTHROPIC_BASE_URL=http://localhost:3000
export OPENAI_BASE_URL=http://localhost:3000

# Chat UI — new in 4.0
open http://localhost:3000/chat

Try it

git clone https://github.com/shouvik12/trooper
cd trooper
export CLAUDE_API_KEY=sk-ant-...  # optional
export OLLAMA_MODEL=llama3.1:8b
go run .
open http://localhost:3000/chat

Works without a Claude key-escalation falls back to Ollama. Add the key when you want real cloud escalation.

https://github.com/shouvik12/trooper

How I Cut Agent Token Usage by 89% Without Touching the Agent

Shouvik Palit — Fri, 05 Jun 2026 13:10:22 +0000

Every time your agent calls an LLM, it sends the full conversation history.

Turn 20 includes turns 1–19. Turn 50 includes turns 1–49. Nobody notices because it's happening inside the agent, silently, on every single request.

I noticed it while building Trooper - a Go proxy that sits between agents and LLMs. I was watching token counts climb across a long debugging session and realised the agent was replaying the same context over and over. Most of it was noise.

The model didn't need a transcript. It needed state.

What state actually means

After a few turns, most of what matters in a session falls into four categories:

Decisions made — what was chosen and why
Constraints locked — what cannot change
Open loops — what still needs to be resolved
Ruled out — what was tried and rejected

That's it. Everything else — the back and forth, the verbose LLM responses explaining things, the repeated context — is replay. The model doesn't need it again.

The SITREP

I added structured session memory to Trooper. After enough turns, Trooper's local Llama model generates a SITREP — a situation report — from the user messages in the session.

It looks like this:

INTENT: Build a RAG pipeline with ChromaDB and nomic-embed-text

DECISIONS: Use cosine similarity over MMR — focused queries not broad;
           Chunk size 256, overlap 30 — locked;
           Pure vector search — ChromaDB no hybrid support;
           Top k set to 5

CONSTRAINTS: Node 18 locked — platform team constraint, no exceptions;
             Re-ranking ruled out — latency jumped 200ms to 800ms

OPEN: Poor recall on technical queries — nomic-embed-text struggles with domain jargon;
      Evaluating bge-small as alternative

From that point forward, every request to the LLM sends:

Anchor (first 2 turns verbatim)
+ SITREP (structured state)
+ Tail (last N turns verbatim)

Instead of the full history.

The numbers

From a real 15-turn session:

Full history:    10,820 tokens per request
With Trooper:     1,157 tokens per request
Reduction:             89%

Visible live on the dashboard.

Does the LLM still answer correctly?

This was the question that mattered. Token savings are worthless if the model loses coherence.

To test it: I took the auto-generated SITREP, opened a completely fresh chat with no history, and asked questions about decisions made in the original session.

Questions:

What is the chunk size?
Why did we rule out hybrid search?
What retrieval method did we choose and why?
What is still open?

Result: All four answered correctly. The model worked entirely from the SITREP. No history. No context bleed.

That's the claim: structured state is sufficient for the model to continue reasoning correctly — and it costs 89% less to send.

How it works

Trooper is a Go proxy — one binary, no SDK, no instrumentation. You point your existing agent at it by changing one URL.

# Before
export ANTHROPIC_BASE_URL=https://api.anthropic.com

# After
export ANTHROPIC_BASE_URL=http://localhost:3000

Nothing else changes. Trooper intercepts every request, maintains session state, and when the SITREP is ready, rewrites the messages array before forwarding to the LLM.

The SITREP is built by a local Llama 3.1 8b model running via Ollama — fast, private, no cloud cost. The extraction happens asynchronously in the background. The main request path is not blocked.

// GetTripleAnchor assembles what gets sent to the LLM
func (s *SessionStore) GetTripleAnchor(sessionID string) []map[string]string {
    payload := append([]map[string]string{}, state.Anchor...)
    if state.SITREP != "" {
        payload = append(payload, map[string]string{
            "role":    "system",
            "content": fmt.Sprintf("[STATE_SITREP: %s]", state.SITREP),
        })
    }
    return append(payload, state.Tail...)
}

The dashboard shows the compression ratio live:

HISTORY COMPRESSED    89%
TOKENS SAVED          459
CONFIDENCE            100%

Why this is different from conversation summarisation

Most summarisation tools compress what was said. The SITREP extracts what matters for the next action.

Copilot's context compaction summarises the full conversation — useful for humans in long chats. The SITREP is structured specifically for agents: decisions, constraints, open loops, ruled-out paths. Not a narrative summary. A state snapshot.

The result is that subsequent turns stay coherent on intent without replaying noise. More relevant for agents running repeated structured workflows than for general chat.

The limitation

The SITREP works best for structured agentic workflows — debugging sessions, research pipelines, multi-step build tasks. For open-ended creative work where tangential context might become important later, you'd want a larger tail window or higher fidelity compression.

The tail window is configurable. You can keep more raw context for less structured sessions.

What else Trooper does

The compression is the latest addition. Trooper also:

Falls back to local Ollama when cloud quota hits — context preserved across the switch
Routes simple turns to Ollama automatically — cloud never contacted
Privacy routing — sensitive requests stay local via x_force_local
Live dashboard — intent, open loops, completed steps, transcript
Subagent recovery — /recovery/{session_id} tells you exactly where to resume

All from one URL change.

The bigger question

We tend to treat conversation history as memory. But a transcript is a log. Memory is state.

Humans don't replay every prior conversation before making a decision. They carry forward conclusions, constraints, unresolved questions, and relevant context — a structured snapshot, not a full transcript.

Long-running agents may need to do the same. Not because of token costs — though that helps — but because state is a better abstraction for agent memory than history.

The SITREP is an experiment in that direction.

github.com/shouvik12/trooper — Go, MIT, zero dependencies beyond Ollama.

I Added a Live Dashboard to My LLM Proxy. Zero Instrumentation. Just a URL Change.

Shouvik Palit — Thu, 28 May 2026 15:24:53 +0000

I built Trooper as a fallback proxy. Claude hits quota → falls back to Ollama. Useful but passive. It sat in the background, invisible, doing its job silently.

Today it became something different.

The Original Problem

When you're building with LLMs, quota hits are inevitable. Claude's free tier is generous until it isn't. A mid-session 429 kills your context, your workflow, your train of thought.

Trooper solved that. Point your app at http://localhost:3000 instead of the Claude API. When Claude fails, Trooper catches it, preserves the full session context via a 3-layer compaction system (Anchor + SITREP + Tail), and continues on local Ollama. Your app never knows anything happened.

That's still there. But it was passive. Useful when things broke. Invisible when they didn't.

The Problem With Passive

Passive infrastructure has an adoption problem. Developers install it, forget about it, and only notice it when something breaks. That's not a product — that's a safety net.

The question I kept asking: what does Trooper do that has daily value, not just failure value?

The answer was sitting in the code the whole time.

What Was Already There

Trooper captures every message in every session. It runs a classifier on each one — extracting intent, entities, open loops, completed steps, recent actions. All rule-based, zero LLM calls, zero latency.

This is what powers the fallback context preservation. When Claude fails and Ollama picks up, Ollama doesn't start blind — it receives a SITREP (Situation Report) that tells it what the session was about, what was completed, what's still pending.

That data exists for every session. It was just never visible to the developer.

The Dashboard

I added a live dashboard at localhost:3000/dashboard.

Point any agent at Trooper — just change your base URL:

export ANTHROPIC_BASE_URL=http://localhost:3000
# or
export OPENAI_BASE_URL=http://localhost:3000

Open the dashboard. Keep it on a second monitor while your agent runs.

From a single message, it already shows:

Intent — what your agent is trying to do, extracted automatically
Open Loops — what it's stuck on, highlighted in red
Completed Steps — what it finished, tracked as it happens
Entities — the key things being referenced
Session Transcript — every message, colour coded by role

Auto-refreshes every 5 seconds. No page reload needed.

What It Looks Like In Practice

I ran a 3-turn agent session simulating a database debugging workflow:

Turn 1: "I am building a Go API server. The database connection is failing with connection refused errors on port 5432."

Turn 2: "Checked the config. Postgres is running on port 5433 not 5432. Fixing the connection string now."

Turn 3: "Fixed the port. Database connection is working. API server is running successfully."

After Turn 1, the dashboard immediately showed:

Intent: "building a go api server. the database connection is failing with connection refused errors on port" (100% confidence)
Entities: Postgres, network
Open Loops: "fail with connection refused error on port"

After Turn 3:

Completed Steps: "successfully fixed the port"
Open loops cleared

Zero instrumentation. No SDK. No code changes to the agent. Just a URL change.

Why This Matters

Every observability tool requires you to instrument your code:

LangSmith — wrap your agent in LangChain
Langfuse — add their SDK
AgentOps — add @observe decorators

Trooper requires nothing. Your agent already communicates over HTTP to an LLM. Trooper sits in that path and observes everything automatically.

Helicone was the closest — proxy-based, zero instrumentation. But it went into maintenance mode in March 2026 and was cloud-only. Your data went to their servers.

Trooper is open source, local-first, and free forever. Your data never leaves your machine.

The Sessions Endpoint

Not sure which session to look at? Hit /sessions:

curl http://localhost:3000/sessions
# {"sessions":["agent-debug-123","agent-debug-456"],"count":2}

Click any session in the dashboard home page at localhost:3000/dashboard to open it.

The Recovery Endpoint

Still there. When an agent fails mid-task:

curl http://localhost:3000/recovery/{session_id}

Returns exactly what completed and where to resume. The dashboard makes this visual — completed steps are tracked in real time.

The Pivot

Trooper started as: "Claude failed. Trooper caught it."

Trooper is now: "Your agent communicates over HTTP to an LLM. Trooper can observe it."

The fallback is a feature. Observability is the product.

Your agent was always talking. Now you can hear it.

Get Started

git clone https://github.com/shouvik12/trooper
cd trooper
export CLAUDE_API_KEY=sk-ant-...
go run .

Open http://localhost:3000/dashboard in your browser.

Point your agent at Trooper:

export ANTHROPIC_BASE_URL=http://localhost:3000

That's it. Zero dependencies. Pure Go. Runs in under 60 seconds.

GitHub: github.com/shouvik12/trooper

Tags: llm, agents, observability, ollama, go, opensource

I Added a /recovery Endpoint to My LLM Proxy So Agents Never Lose Progress Mid-Task

Shouvik Palit — Sun, 24 May 2026 17:28:55 +0000

Most LLM proxies handle failures the same way -retry the request, fall back to another provider, or crash. None of them ask the more important question: what did the agent already complete before it failed?

That's the gap I built Trooper to fill.

The Problem

If you're running multi-agent workflows, you've probably hit this scenario:

A subagent starts a long task-reviewing PRs, processing documents, running analysis
It completes steps 1, 2, and 3
On step 4 it hits a quota error, rate limit, or provider failure
Your orchestration layer has no idea what completed
You restart from scratch and repeat all the work

Most proxies handle the failure. Nobody handles the recovery.

What Trooper Does Differently

Trooper is a Go-based LLM proxy that sits between your agents and your LLM providers. It already handled fallback routing -if Claude hits quota, it falls back to local Ollama automatically.

But the new /recovery/{session_id} endpoint goes further. It tracks every step your agent completes in real time and tells your orchestration layer exactly where to resume.

The Recovery Endpoint

When your agent sends requests through Trooper, it captures every assistant response and extracts completed steps as they happen. When something fails, you call:

GET http://localhost:3000/recovery/{session_id}

And Trooper returns:

{
  "session_id": "subagent-demo-1779630533",
  "completed_steps": [
    "completed pr #1",
    "completed pr #2",
    "completed pr #3"
  ],
  "resume_from": 4,
  "recovery_hint": "Resume from step 4"
}

Your parent agent now knows exactly what the subagent finished and where to restart it. No repeated work. No lost progress.

Demo

An agent reviewing 8 pull requests hits quota on PR #4. Trooper intercepts, returns the recovery payload, and the agent resumes from PR #4 using local Ollama.

[https://www.youtube.com/watch?v=NN2uwQZDCck]

How It Works

Trooper uses a two-tier memory system:

Anchor — the first two turns of a session, always preserved verbatim.

Tail — the most recent turns, stored in a rolling window.

When you call /recovery, Trooper scans all stored assistant messages for completion signals — words like "completed", "finished", "done", "merged", "deployed". It extracts one completed step per message, deduplicates by task identifier, and returns the ordered list.

The resume_from field is simply len(completed_steps) + 1 — telling your orchestration layer which step to restart the subagent on.

How to Use It

1. Start Trooper

git clone https://github.com/shouvik12/trooper
cd trooper
export CLAUDE_API_KEY=your-key
go run .

2. Point your agent at Trooper instead of Claude

# Instead of https://api.anthropic.com/v1/messages
POST http://localhost:3000/v1/chat/completions

3. Pass a session ID with each request

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "X-Session-ID: my-agent-session-1" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-haiku-4-5","max_tokens":100,"messages":[...]}'

4. Call recovery when something fails

curl http://localhost:3000/recovery/my-agent-session-1

What's Next

The recovery endpoint is the foundation for proper subagentic orchestration. Upcoming work:

Parent agent integration — the recovery payload feeds directly back into the orchestration layer to automatically restart subagents from the right step
Structured step tracking — support for agents that emit structured JSON progress instead of natural language
Session replay — rewind any session to any point and branch from there

The Bigger Picture

As agent workflows get longer and more complex, failure recovery becomes a first-class concern. Trooper's approach — track everything as it happens, make recovery queryable — is a different philosophy from retry-and-hope.

Local-first by default. Cloud when you choose. And now recoverable when things go wrong.

GitHub: https://github.com/shouvik12/trooper

How are you handling agent failures in your workflows today? Drop a comment genuinely curious what patterns people are using.

DEV Community: Shouvik Palit

I Reduced AI Response Tokens by 44% Without Changing the Model

Teaching LLMs to Say Less Without Knowing Less

The Observation

The Idea

The Experiment

Average Token Savings

The Interesting Part

Symbols Matter Too

Where LELP-S+ Fits

Why This Matters

Try It

Sir Shortoken: Bullet Mode, Local Expansion, and What Actually Held Up

Sir Shortoken: Disciplined AI Output for Every LLM

The Problem

The Approach: Information Budgets

How It Works

The Budgets

Test Results

Suite 1: Budget Progression (Same Query, Different Budgets)

Suite 2: Boundary Respect (Enforcing Knowledge Cutoff)

Suite 3: Knowledge Domain Accuracy

Suite 4: Consistency

The Enforcement Mechanism

Real-World Example: Code Review at Scale

Compression Without Loss

Limitations

Implementation

What's Interesting Here

Testing Methodology

Why This Matters

Try It

Links

I built a runtime that stops AI agents from over-engineering everything

Ship Happens: How I Made a 3B Local Model Trustworthy for Kubernetes (By Never Trusting It)

The idea

What this caught, that I didn't expect

1. Missing --- silently merges resources

2. Placeholder images that pass every check

3. The repair that "fixed" the version by deleting it

"But couldn't jsonschema do this?"

What it doesn't catch

What's still unverified

The actual point

I tracked every GitHub traffic spike for my open source LLM proxy for 7 weeks. Then I did the exact same thing again, and it worked again.

What is Trooper?

The Data

What I learned

1. Reddit is the only thing that moved the needle, and community fit matters more than size

2. The post that doesn't feel like a post performs best

3. Organic discovery is real, and it compounds quietly

3.5. The playbook worked a second time, on purpose

4. LinkedIn drove less than you'd expect

5. The view-to-clone ratio tells you who's actually landing

What's next

Your Session History Is Bleeding Tokens Every Time You Paste

The numbers

Where the savings actually land

What's actually happening when you paste

The fix

Install

The meta test

Where it fits

Stop explaining yourself to Claude

The delta principle

What you can safely strip

Before and after

The compound effect

Three intensity levels

The skill file

One thing worth thinking about

Escalate the Model, Not the Conversation

How it works

The escalation moment

What's under the hood

The proxy still works

Try it

How I Cut Agent Token Usage by 89% Without Touching the Agent

What state actually means

The SITREP

1. Missing `---` silently merges resources