"1M context" is now a normal line in model specifications. But it is a terrible human-scale unit. Most of us cannot look at:
1,000,000 tokens
and immediately understand how much information that represents.
One Million Tokens is an interactive site that fixes that problem.
🧠It visualizes one million tokens as a huge stack of text and then walks through the history of context-window growth from GPT-3 onward.
The basic scale
The site uses the following approximate conversions:
1M tokens
~ 750K words
~ 3,000 printed pages
~ 83 hours of conversation
~ 75,000 lines of code
Those numbers are meant to be intuitive rather than universal. Tokenization varies significantly by content. But they give us the right order of magnitude.
A million-token window is not merely "a very long prompt."
It is a small library.
The timeline starts at 2,048 tokens
The site's story begins in June 2020.
GPT-3
2,048 tokens
That is visualized as roughly six pages. Then:
ChatGPT
Nov 2022
4,096 tokens
Then:
GPT-4 32K
Mar 2023
32,768 tokens
Then:
Claude
May 2023
100,000 tokens
The graph starts getting steep.
Gemini crosses the million-token line
The site marks February 2024 as the point where Gemini 1.5 Pro reached 1M tokens.
Google's own launch material from 2024 also described Gemini 1.5 Pro as supporting up to a 1 million-token context in preview.
That was the moment "million-token context" stopped being a research curiosity and became a product capability.
The site then continues into models with multi-million-token windows.
From 2K to 10M
The visualization's complete timeline reaches a cited 10,000,000-token window.
It calculates the increase from GPT-3's 2,048 tokens to 10M as roughly:
4,883x
That is a six-year change in the amount of information a model can accept in its working context.
The visual curve is more revealing than the number. For several years, context capacity barely moves. Then it starts climbing by orders of magnitude.
Why context matters for software development
For developers, context length changes the amount of a system that an AI can inspect at once.
A rough progression:
2K:
small snippet / short conversation
32K:
large file or several files
100K+:
substantial subsystem
1M:
large code collections / potentially repo-scale input
The site's rough 1M-token code equivalent is 75,000 lines at ~13 tokens per line.
That number should not be treated as a hard engineering rule. Code token density varies dramatically.
Python, minified JavaScript, JSON, comments, generated source, and strongly typed languages will all behave differently.
But the order of magnitude is still useful.
Does long context replace RAG?
Not necessarily.
A larger window reduces one reason we use retrieval: the inability to fit all relevant source material.
But retrieval also solves other problems. It helps:
remove irrelevant material
lower input cost
reduce latency
focus the model
update source data independently
So long context and retrieval are complementary.
A 1M-token window lets you choose to include far more source material.
It does not mean including everything is always optimal.
Context window != perfect memory
This is the biggest conceptual mistake I see in discussions of long context.
Maximum context length is a capacity limit. It is not a guarantee of perfect use. A model with a 1M-token window may still:
- miss a fact that appears in the prompt
- perform differently depending on where information appears
- degrade on complicated cross-document reasoning
- become slower or more expensive with huge inputs
- retrieve some details better than others
This is why long-context benchmarks test retrieval and reasoning quality, not merely whether the tokenizer accepts the input.
I would phrase it like this:
context size = size of the desk
reasoning/retrieval quality = how well the model uses what is on the desk
A larger desk is useful. It does not automatically make the worker better.
The page comparison is approximate
The site uses:
1 page ~ 333 tokens
and therefore:
3,000 pages ~ 1M tokens
This is a visualization convention. Real documents vary. Tokenizers vary. Languages vary.
The site's 83-hour conversation estimate similarly assumes around 150 spoken words per minute.
These are reference points, not exact conversions.
The open-vs-closed view is interesting too
The timeline can be filtered between:
ALL
OPEN
CLOSED
That makes it easy to see how long-context capabilities moved from closed frontier APIs into open-weight models.
The page credits Hassan at Together AI as the creator.
Why I like the project
Model cards are full of abstract specifications.
128K context
1M context
2M context
This site converts those numbers into something human.
The next time someone asks what a million-token context window actually means, I would rather send them this visualization than another table.
Because after scrolling through 3,000 pages, the number finally feels real.
How much of a million-token window have you actually been able to use?
Top comments (1)
The desk metaphor is the part I'd keep, and I'd push one step further: the desk is not flat. In the runs I have done, the failure was never that the tokenizer refused the input, it was that a fact sitting in the middle of a large prompt stopped being reachable while the same fact at the head or the tail still worked. So the number I actually track is not "how big is the window" but "how far in can I still place something and have it come back out" — that measurement moves between models and between content types, and it moves well before the advertised limit.
On the RAG paragraph: I think retrieval survives long context for a less glamorous reason than cost. It gives you a record of which fragments you chose and why. When an answer is wrong, "the model had 900 pages and picked badly" is much harder to debug than "it was handed these six passages". Did you find a threshold where stuffing the whole corpus beat retrieving a subset in your own tests, or was it always a wash?