Over the past week, I’ve been building the Agent Glossary, collecting fundamental terms that explain how LLMs work and how their capabilities relate to the context items they consume and produce.
Let’s walk through these terms one by one.
Model
A model is a trained AI system that generates content from the context it receives. In agent systems, it consumes context items, such as instructions, user messages, and tool outputs, and produces new items, such as model answers and tool calls.
Most modern language models use the Transformer architecture. Their capabilities determine which context items they can consume and produce, and how those items are delivered:
Tool calling enables the model to produce tool call items containing a function name and arguments.
Reasoning enables the model to work through a problem before producing an answer or tool call. Providers may expose reasoning items or summaries. Some models offer reasoning effort settings to control how much computation they spend reasoning.
Streaming delivers generated context items incrementally, before the entire response is complete.
Structured output constrains the content of a model answer to a defined schema, such as a JSON structure.
Model usage distinguishes input tokens, representing the context processed, from output tokens, representing the content generated, including reasoning tokens where applicable. Provider API pricing is mainly based on the number of input and output tokens used, usually with separate rates for each.
Context
Context is the information available to a model during a generation, including instructions, messages, tool interactions, and reasoning state.
Context is composed of context items: system prompt, user prompt, tool call, tool call output, reasoning, and assistant message.
Reasoning models spend additional computation generating intermediate reasoning before answering or calling tools, making them useful for complex planning, coding, mathematics, and decisions involving multiple constraints; this can increase latency and token usage. Reasoning-capable APIs can return Reasoning items, while non-reasoning models do not produce these dedicated items; their exact representation depends on the provider.
Context represents working memory, while agent memory provides long-term storage, including episodic memory of past experiences.
Context Items
Context items are the individual pieces of information that make up a model's context.
Each provider has its own representation of these context items.
Some model providers support streaming, delivering context items incrementally as they are generated instead of waiting for the complete response.
This allows applications to display text as it arrives or react to events before the full response is available.
Context Window
The context window is the maximum number of tokens a model can process during generation, including input and output.
A token is a unit of text or other data processed by the model—not necessarily a whole word.
Input tokens — Information provided to the model, including instructions, messages, and tool interactions.
Output tokens — Tokens generated by the model, including reasoning tokens where applicable.
Input and output must fit within the context window; models may also have a separate output limit.
When the limit is exceeded, the system may truncate context, removing information to make room, or reject the request. Losing earlier instructions or details can cause the model to repeat actions, miss requirements, or produce inconsistent answers.
Tool Call
A tool call or function call is a directive issued by a large language model to execute a function with specific arguments, to obtain information or perform an action.
The developer tells the model which functions it can request by providing tool definitions. Each definition describes the function's name, purpose, and parameters, including what each parameter means.
The model returns the function name, concrete arguments, and a call ID identifying the call.
The client executes the tool locally or remotely. In the next inference request, the developer provides the tool call output item alongside the matching call ID, so the model can connect the result to the original call and continue reasoning, request more tools, or provide a final answer.
The model may request multiple function calls in one response or across successive requests to complete a task.
Tool calls are a key step toward agent autonomy, allowing agents to request actions such as using a computer, browsing the internet, or executing other available functions as needed to complete a task.
User Prompt
A user prompt or user message is input provided to a model by a user or an application. It can contain instructions, questions, text, images, or other supported content.
For example: “Summarize this document in three sentences.”
The word prompt often refers to the user prompt, but it can also mean the broader input sent to the model, including system instructions and conversation history.
System Prompt
A system prompt is additional context that describes how the agent should behave.
Some providers split this into a developer message and a system message. Others include only a system message.
Reasoning
Reasoning is a capability of reasoning models to work through a problem before producing an answer or requesting a tool call. It helps with tasks that require multiple steps, such as planning, coding, and solving complex problems.
During generation, these models may produce reasoning items alongside other context items. They contain reasoning content related to how the model approaches a problem and arrives at an answer or tool call. They are not necessarily a complete or faithful explanation of the model's internal process.
Depending on the model and provider, returned reasoning content may be exposed directly, summarized, or hidden entirely.
Some models support reasoning effort, a setting that controls how much effort the model spends reasoning. Higher effort may improve results on difficult tasks, but can increase response time and token usage.
Assistant Message
An assistant message or model answer is a context item containing the model's final response to the user. It may provide a result, explain something, or ask for clarification.
The model can produce an answer directly or after requesting tool calls and processing their results. Depending on the model, the answer may contain text, images, audio, or other supported content.
The answer becomes part of the conversation context and can be included in future requests so the model knows what it previously communicated.
That’s all for now. Next, I’ll expand the collection with agent-related terms, including loops, harnesses, memory, and sub-agents, before exploring more advanced patterns for agent collaboration.
The Agent Glossary is available here: https://www.miodragvilotijevic.com/agent-glossary
Top comments (0)