Part 1 of Knitli's 101 introductions to AI and the economics of AI
Tokens are Parts of Words
Most people think AI, like ChatGPT, reads words. It doesn't.
It reads tokens invisible chunks of text that power every interaction.
When you type something like:
Hello, world!
The model doesn't see two words. It sees four tokens:
Hello 1 token
, 1 token
world (note the space) 1 token
! 1 token
That simple greeting is 4 tokens , not 2 words. Code fragments break into even more tokens because punctuation, brackets, and symbols all get split up. (What is and isn't a token and what becomes one actually depends on the model, so our example isn't exact.)
Tokens Aren't Expensive. Processing them is.
When you send your tokens to get processed, each one must be run through billions of math operations on very expensive GPUs every single time. Thats where the cost comes from:
Power-hungry hardware
Data center space
Cooling
Staff to maintain and secure it
More tokens more GPU time higher costs.
Fewer tokens less GPU time lower costs.
Right now, you probably don't see the meter running. You pay a flat subscription; someone else covers the token bill.
Under the hood, tokens are the biggest driver of compute costs at every AI company.
Tokens are the Foundation for Everything Else
Context windows , or how much a model can see at one time, are measured in tokens.
API pricing is per million tokens (API access is when companies or developers access an AI model to provide their own service, like a chatbot on a website, or just for internal use).
Memory , efficiency, and much of prompt engineering are all about how tokens are used.
If you want to understand AI, and how it really works, or why it sometimes costs so much. You have to start with tokens.
Learn more about how Knitli is tackling the hidden economics of AI at the source, visit us at knitli.com and subscribe to our waitlist for updates!
Top comments (0)