Tokens are an important foundational concept that underlie how we interact with LLMs. @greggyb and myself recorded a quick video to demystify what a token actually is and unpack several other related concepts.
What's in the video
What is a token? Tokens are the basic unit of input and output that LLMs operate on.
What is retrieval-augmented generation (AKA RAG)? Providing a model with extra data that isn't already in the model's training dataset, and is relevant to a specific query.
What are context windows? Context windows are how many tokens a model can use for input and output. Context window size varies from model to model, but it has increased a lot in recent years.
What is tokenmaxxing? Does using more tokens really mean you're being more productive? (Spoiler alert: no.)
To paint a bigger picture: more context you give a large language model, the better your responses are likely to be. The design patterns for providing that context have evolved at a rapid pace, and are likely to continue doing so.
What AI terms should we demystify next? Let us know!
Top comments (0)
Subscribe
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Top comments (0)