Every time I wanted to paste a whole project into Claude or ChatGPT, I ended up guessing whether it would even fit — and often found out the hard way, mid-conversation, that it didn't.
So I built Tokenazire, a small CLI tool that solves exactly that.
What it does
Scans a local folder or a GitHub repo (just pass the URL, it clones it for you)
Counts tokens per file using tiktoken (the same tokenizer OpenAI models use, a solid approximation across most LLMs)
Shows a color-coded breakdown (green → yellow → orange → red) so you instantly see which files are "heavy"
Calculates what percentage of a model's context window (default 200k, configurable) your whole project takes up
Ignores .git, venv, node_modules, and other noise automatically
Has an --export flag that bundles the entire project — folder structure plus every file's content — into a single text file, ready to paste straight into an LLM chat
I kept hitting the same annoying loop: copy a project into a chat, get cut off or told the input's too long, then manually trim files and try again. This automates the "will it fit, and if not, what's taking up the most space" question up front.
The --export step came later — once I knew what would fit, I still had to manually copy-paste files one by one into the chat. Now it just spits out one clean file with a project tree on top and clearly separated file contents, ready to paste.
Tech stack
Plain Python, tiktoken for tokenization, rich for the terminal output (tables, colors, progress bar). No config files, no external services beyond git for cloning.
Try it
Repo: https://github.com/DeKlain4ik/token-counter (MIT licensed)
Still early — feedback, issues, and PRs are welcome.
Top comments (0)