Shouvik Palit

Posted on Jul 19

Sir Shortoken: Disciplined AI Output for Every LLM

#ai #llm #promptengineering

TL;DR: Sir Shortoken is a system prompt that constrains frontier models to operate within information budgets (Quick/Balanced/Deep), never silently escalate capabilities, and prove execution. Tested across Claude, GPT, Gemini. 40-60% token reduction on technical queries without information loss.

GitHub: https://github.com/shouvik12/sir-shortoken

The Problem

You ask Claude to explain OAuth 2.0. You get 2,000 words with historical context, security edge cases, and implementation details you didn't ask for.

You're left scrolling.

It's not the model's fault. It's optimizing for completeness. But when you're running this at scale—code review, API documentation, technical education—those wasted tokens add up.

More importantly: you waste time filtering signal from noise.

The Approach: Information Budgets

What if you could tell the model: "Give me the core concepts, nothing else"?

And it respected that?

I built Sir Shortoken to test whether frontier models can operate under strict information constraints without losing technical correctness.

The hypothesis: Semantic compression exists. The model can extract the core structure of ideas and explain that, preserving understanding while reducing elaboration.

How It Works

Three rules:

Answer from internal knowledge only — Never silently use web search or tools
Respect the information budget — Stay within Quick/Balanced/Deep/Unlimited
Show proof of execution — Display what was used (Ledger)

Invocation:

Sir Shortoken [budget]: [query]

Example:

Sir Shortoken balanced: Explain OAuth 2.0

Output:

OAuth 2.0 is an authorization framework. A third-party app 
requests permission to access your resources. You authorize, 
the app gets a token, and uses that token instead of your password.

Flow: User → Authorization Server → Grant Permission → 
Authorization Code → Access Token → App Uses Token

Key insight: Your password never leaves the authorization server. 
The app never sees it.

Sir Shortoken's Ledger

Mode ........... Balanced
Input .......... 312
Output ......... 287
Reasoning ...... Used
Web ............ Declined
Budget ......... 599 tokens

Concise. Structured. Provable.

The Budgets

Mode	Tokens	Scope	Use Case
Quick	200-400	Core concepts only	Rapid learning, quick reference
Balanced	400-800	Core + essential reasoning	Default. Learning + building
Deep	800-1600	Includes nuance + edge cases	Production systems, auditing
Unlimited	∞	Everything	Comprehensive guides, education

Test Results

I ran 23 test cases across six suites. Here's what passed:

Suite 1: Budget Progression (Same Query, Different Budgets)

Query: "Explain how TCP/IP works"

Mode	Output Tokens	Coverage
Quick	194	~35% of full
Balanced	456	~50% of full
Deep	1,247	~85% of full
Unlimited	6,847	100%

Expansion ratios: 2.3x → 2.7x → 5.5x per level.

Key finding: Budget discipline is real and proportional. Quick mode isn't truncation—it's restructuring. The model extracts the semantic skeleton.

Suite 2: Boundary Respect (Enforcing Knowledge Cutoff)

Test: Query requiring current information without permission

Sir Shortoken balanced: What happened in AI regulation this week?

Result:

I cannot answer this reliably from my internal knowledge alone.

My training data ends in January 2025. This question requires 
current information.

Would you like to:
○ Ask about January 2025 and earlier (I can answer)
○ Allow me to search the web (requires permission)
○ Ask a different question (I can answer)

Ledger: Web = Declined (awaiting permission)

Key finding: No silent escalation. Model asks first. This is the critical enforcement rule.

Test with permission granted:

Sir Shortoken balanced: search the web for latest AI regulation changes

Result: Model searches only because user explicitly asked. Ledger shows Web = Approved & Used.

Suite 3: Knowledge Domain Accuracy

Tested on stable knowledge (math, CS, history, philosophy). All accurate. All within budget. Web search declined appropriately.

Suite 4: Consistency

Same query, three invocations in same session. Token count consistent (±10%). Mental model preserved across all three.

The Enforcement Mechanism

The skill.md includes explicit enforcement rules:

## Enforcement Rules

When you see "Sir Shortoken [budget]:", follow these rules exactly:

1. Identify the budget — Extract quick, balanced, deep, or unlimited
2. Extract scope only — Answer ONLY within that budget's scope
3. Internal knowledge only — Never use web search unless explicitly asked
4. If you cannot answer reliably:
   - State explicitly why
   - Do NOT guess or pretend
   - Do NOT silently escalate
   - Offer alternatives within that budget
5. Always show Sir Shortoken's Ledger
6. Never exceed token budget

This is critical. Without explicit enforcement rules, Claude infers from context—which leads to silent escalation on boundary cases.

With enforcement rules: No silent escalation. Model asks first. Always.

Real-World Example: Code Review at Scale

Use case: Reviewing 100 API changes for security issues.

Without Sir Shortoken:

800 tokens per review (includes explanations, context, caveats)
100 reviews = 80,000 tokens
Time to read: ~2 hours
Cost: ~$0.40

With Sir Shortoken (Balanced):

300 tokens per review (core issues only)
100 reviews = 30,000 tokens
Time to read: ~45 minutes
Cost: ~$0.15

Result: 62% token reduction. 67% time reduction. Same security level.

Compression Without Loss

Here's the key finding: Information density is real.

Quick mode on TCP/IP explanation:

194 tokens
Covers: two-layer model, routing, reliability, flow
Omits: packet structures, algorithms, edge cases

Someone who reads Quick can reconstruct 80% of Balanced, because the core structure is there.

This isn't lossy. It's prioritized.

Limitations

Sir Shortoken can't:

Answer questions requiring current information (without permission to search)
Hallucinate expertise
Replace examples/tutorials (use Unlimited mode)
Work across domain boundaries

Sir Shortoken isn't:

A new model or framework
A substitute for critical thinking
Optimized for creativity or exploration
A cost-reduction tool (cost reduction is a side effect)

Implementation

Copy skill.md from GitHub
Paste into Claude, GPT, Gemini, or Ollama
Invoke: Sir Shortoken [budget]: [query]
Read the ledger

That's it. Works with any LLM.

What's Interesting Here

For AI researchers:

Semantic compression exists and is measurable
Constraint enforcement prevents silent escalation
Information budgets enable predictable output
Transparency (via Ledger) builds model trust

For engineers:

You can constrain frontier models without custom models
Reproducibility improves with enforcement rules
Token efficiency is a side effect of information density
Proof of execution (Ledger) enables auditing

For product teams:

This is a pattern other tools could adopt
Users value discipline over capability
Respecting attention is competitive

Testing Methodology

All 23 tests are documented in the repo. You can reproduce them:

Clone the repo
Enable skill.md in Claude
Run each test case
Compare against expected behavior

Test cases cover:

Budget progression (4 tests)
Boundary respect (4 tests)
Knowledge domain accuracy (4 tests)
Stress testing (4 tests)
Consistency (3 tests)
Edge cases (4 tests)

All pass. ✅

Why This Matters

Frontier models are becoming ambient—in IDEs, browsers, chat interfaces, APIs.

When tools are ubiquitous, discipline becomes valuable.

Sir Shortoken proves you can:

Constrain output without dumbing down
Prevent silent escalation
Make models auditable
Respect user attention
Do this with a system prompt (no custom models needed)

This is a pattern worth adopting.

Try It

git clone https://github.com/shouvik12/sir-shortoken.git
cd sir-shortoken
cat skill.md

Paste skill.md into Claude or your LLM:

Sir Shortoken balanced: How does Git work?

Read the ledger. See if it respects the boundary.

Then decide if this philosophy matters to you.

DEV Community