If you only want the number: 1,000 words of ordinary English is roughly 1,300 tokens. Going the other way, 1,000 tokens is roughly 750 words, or about 4,000 characters.
That is the whole answer for estimating. The rest of this post is the table, the cases where the ratio breaks, and how to get an exact count when an estimate is not good enough.
The cheat sheet
For plain English prose, using the common rule of thumb that 1 token is about 4 characters and about 0.75 words:
| You have | Roughly this many tokens |
|---|---|
| 1 word | 1.3 |
| 100 words | 130 |
| 500 words (about 1 page) | 650 |
| 1,000 words | 1,300 |
| 10 pages | 6,500 |
| 100 characters | 25 |
| 1,000 characters | 250 |
| 1 paragraph (about 100 words) | 130 |
And in reverse, which is the direction you usually need when you are staring at a model's context limit:
| Token budget | Roughly this much English |
|---|---|
| 1,000 tokens | 750 words |
| 4,000 tokens | 3,000 words |
| 8,000 tokens | 6,000 words |
| 128,000 tokens | 96,000 words (a short novel) |
Where the ratio stops working
Those numbers are for prose. Tokenizers split on statistical frequency, not on words, so anything unusual costs more:
- Code, JSON and long URLs break into many small pieces. Punctuation, braces, camelCase and random ID strings all tokenize badly. Budget noticeably more than the prose ratio.
- Non-Latin scripts are the big one. Arabic, Chinese, Japanese, Hindi and others can run several tokens per character in some tokenizers, so the same meaning costs multiples of the English price.
- Numbers and tables split in ways that look arbitrary. A long column of figures is not cheap.
- Different models, different tokenizers. A prompt is not a fixed number of tokens; it is a number per tokenizer.
So: rules of thumb are fine for "will this roughly fit". They are not fine for a hard limit or a cost estimate you are going to rely on.
Getting an exact count in code
For OpenAI models, tiktoken is the direct route:
import tiktoken
enc = tiktoken.get_encoding("o200k_base")
print(len(enc.encode("your prompt here")))
Anthropic and Google both expose token-counting endpoints in their APIs, which is the better option when you want the count for the exact model you are about to call rather than an approximation of it.
Getting an exact count without writing anything
Most of the time you are not in a script. You have a block of text in front of you and you want to know whether it fits before you paste it. That is the case I built iLostCount for: paste the text, read the token, word and character counts as you type. No signup, and the text is not uploaded anywhere, since the counting runs in the page.
Why the number is worth knowing
- The context window is shared. It has to hold your system prompt, the conversation so far, anything you retrieved, and the answer. Fill the input and there is no room left for the output.
- You pay per token, both directions. Re-sending a large document on every turn is the quiet way a bill grows.
- Truncation is silent. Plenty of tools drop the oldest turns when you run over the limit. That does not look like an error. It looks like the model forgetting.
A cheap habit that saves all three: count a long document before it goes into a prompt. If a 40-page PDF turns into 30,000 tokens, you know to chunk it or summarise it first, instead of finding out from a 400 error or an invoice.
Disclosure: this post is from the iLostCount project. The tool is free and the source is public at github.com/ahmad-almazeedi/token-counter.
Top comments (1)
Korean tokenization has genuinely surprised us more than anything else for budget work. The ratio changes depending on which characters are being combined in the same tokenizer, and it's variable enough that running estimates from character count is just not reliable. We had a FAQ ingestion job where we ended up at roughly double the expected cost, and the culprit was specific character sequences in certain sections. We've since just hardcoded "run tiktoken on the actual content" into our pipeline setup and stopped trying to estimate.