A linter that finds wasted tokens in your prompts (fork of ritenv/tokensift)

joyofish — Sat, 31 Oct 2026 10:35:00 +0000

Fork of ritenv/tokensift — full credit to Riten Vagadiya for the original library. I packaged it for npm; I did not build it from scratch.
I kept noticing the same thing in my own prompts: repeated instructions, filler words, bloated JSON, IDs and tables that cost tokens without adding meaning. Editing by feel never showed how much it actually cost — so I packaged this linter for npm to make it easy to install and use.
Token Sift is a local command-line tool that finds avoidable token use in prompts, message arrays, and tool schemas. It ships with 20 built-in rules for common inefficiencies — repeated text, oversized identifiers, verbose JSON, duplicated message content, unhelpful filler, dynamic data that should not be baked into a prompt, inefficient tables — plus 2 CI gate rules for budgets and baselines.
Install it with:

npm install --global @yuhond/token-sift

Then point it at a prompt file:

token-sift prompts/support-ticket.md --model gpt-4o

Every finding shows the affected text, severity, token counts, a suggested change, and a confidence label. If a finding supports a safe automatic fix, you can apply it with --write — but review the result first on important files. --format json exports structured results for scripts and CI.
The part I use most in real projects is the CI gate. You can set a token baseline and a budget, then fail the build when a prompt regresses:

token-sift prompts/*.md --model gpt-4o --update-baseline

token-sift budget init prompts/*.md --model gpt-4o

token-sift check prompts/*.md --model gpt-4o

Two things I was strict about. First, privacy: analysis runs fully locally. The only commands that touch the network are the explicit pricing update and calibrate commands — no code or prompts are ever uploaded. Second, honesty about Claude: Claude has no public tokenizer in this tool, so every Claude count is labeled estimate, never presented as exact. OpenAI-model counts use their corresponding tokenizer.
Fork details: MIT, attribution kept. I renamed the package to @yuhond/token-sift and the command to token-sift, and isolated its config as token-sift.config.json and .token-sift/ so it does not collide with the original.
Try it on your longest prompt and see what it finds: https://github.com/yhdhappy/TokenSift

DEV Community: joyofish

A linter that finds wasted tokens in your prompts (fork of ritenv/tokensift)