In my last speech, I wanted the audience to leave with something they could use beyond a particular AI tool.
So I focused on how to think with AI.
While building ReThynk AI, one lesson has become central to my thinking: start with the problem. Understand what needs to happen, who needs it, and how you will know whether your solution works.
Technical knowledge becomes valuable when it helps us answer those questions.
That is why I want developers to understand these ten terms. Each gives you a way to examine your application, identify a weakness, and make a better decision.
Let us connect them through one example: building an AI customer-support assistant.
1. Context window
The context window is the amount of information a model can work with during a response, measured in tokens. Its capacity must accommodate the relevant input and generated output, subject to the model’s limits. It is different from permanent memory.
Imagine your support assistant needs the customer’s question, conversation history, and refund policy. Those compete for space alongside instructions and tool results.
A larger window gives you more room. You still need to decide what deserves that room.
2. Context engineering
Context engineering means selecting, organizing, and maintaining the information available to the model as it works. This includes instructions, documents, conversation history, and tool results.
For our assistant, loading every policy document could introduce outdated or irrelevant information. Selecting the current policy for the customer’s region gives it a more useful starting point.
My question here is simple: Have I given the system what it needs to make this particular decision?
3. Embeddings
Embeddings represent information as numerical vectors. Suitable embedding models allow us to compare text by semantic similarity, which is useful for search, clustering, and retrieval.
A customer writes, “My parcel never arrived.” Your documentation calls this a “missing delivery.” Embedding-based search can help connect the two despite their different wording.
However, a similar document can still be the wrong document. You must also consider details such as product, region, and policy date.
4. Retrieval-augmented generation, or RAG
RAG retrieves relevant information and supplies it to a model to help generate an answer. It connects generation with external sources such as documents or databases.
Our assistant could retrieve a delivery policy before explaining the available options.
That sounds straightforward, but there are two questions to investigate: did retrieval find the right information, and did the model use it correctly?
Connecting a document store is the beginning of that work.
5. Tool calling
Tool calling allows a model to request an external function using structured arguments. For custom functions, your application executes the function and returns its result to the model.
For example, the assistant might request:
get_order_status(order_id="12345")
Your application must check whether the customer can access that order before executing the request.
The model can propose an action. Your software remains responsible for enforcing permissions and validating inputs.
6. Workflows and agents
In Anthropic’s architectural distinction, workflows follow predefined code paths, while agents dynamically direct their processes and tool use. Different products may use these labels differently, so the underlying behavior matters.
A support workflow might always check the order, retrieve the policy, and draft a response.
An agent might decide which investigation to perform next based on what it discovers.
When building, I would ask how much flexibility the problem requires. A predictable process can be easier to inspect, test, and maintain.
7. Model Context Protocol, or MCP
MCP standardizes how AI applications connect with capabilities exposed by servers, including tools, resources, and prompts. Its architecture separates the host application, clients, and servers.
For our support assistant, an MCP server could expose order-search tools or policy resources.
This helps define the connection. You still need to design access controls, tool behavior, and error handling.
Understanding the protocol should help you build clearer interfaces between your AI application and the systems it uses.
8. Evaluations, or evals
Evaluations are structured checks of how a system performs on representative tasks. For a RAG application, useful dimensions include correctness, relevance, completeness, and whether the response is supported by its supplied context.
Our assistant needs test cases for missing deliveries, conflicting policies, unavailable order data, and questions it cannot answer.
My Six Sigma background makes this especially meaningful to me. Before improving a process, we need a clear definition of acceptable performance.
A convincing demonstration gives us something to investigate. Evaluation gives us evidence across cases.
9. Observability
Observability helps us understand a system’s behavior through telemetry such as traces, metrics, and logs. A trace can follow a request through multiple operations.
Suppose our assistant gives a poor answer after taking fifteen seconds. We need to discover where the time went, which documents were retrieved, and whether a tool failed.
For AI applications, I would track the relevant retrieval, model, and tool steps while protecting sensitive customer information.
We can improve more effectively when we can locate the failure.
10. Idempotency
An idempotent operation can be repeated without producing additional effects beyond the intended first operation. This matters when requests are retried after timeouts or interrupted connections.
Imagine a refund succeeds, but the response never reaches the assistant. It retries.
The system should recognize the same logical refund request and avoid issuing another payment. Implementing that requires coordination with the service performing the action.
A retry is a small technical detail until it changes someone’s account balance.
These concepts belong together.
Context engineering determines what the model sees. Retrieval supplies information. Tools connect it to actions. Workflows or agents coordinate the work. Evaluations check performance, observability helps explain behavior, and idempotency helps make retries safe.
When I speak about thinking with AI, this is part of what I mean: understanding enough of the whole system to ask better questions.
Choose one AI project you are building. Trace a single user request through it.
Where could it retrieve the wrong information, take an unauthorized action, or repeat an operation?
Share the term you want me to explore next, and the problem you are trying to solve with it.
Top comments (2)
While building ReThynk AI, one lesson has become central to my thinking: start with the problem.
a great cheat sheet and a great read, thanks!