This article expands on the “Clarify the need and build a shared mental model” section of Before the First Line of Code: How AI Helps Turn Requirements into a Shared Design.
Have you ever wondered why an LLM does not return the result you expected?
Our first reaction is often to blame the model:
- It did not understand the requirement.
- It used the wrong design pattern.
- It missed an authorization check.
- It returned the wrong data.
- It made unnecessary assumptions.
Sometimes the model is the problem. But often, the real issue is that important context was never provided.
When information is missing, the LLM must either stop and ask questions or make assumptions. Because LLMs are designed to provide useful answers, they often choose the second option.
The result may look technically convincing while being inconsistent with the organization’s architecture, security policies, API standards, or business rules.
The same thing would happen with a developer.
Imagine asking a developer who is new to the team:
Build an API that allows a store manager to retrieve customer orders.
The developer would need more information before writing code:
- What are the organization’s API standards?
- Is there a similar API to follow?
- How is the caller identified?
- How do we determine which stores the manager can access?
- Which customer information may be returned?
- How should large results be handled?
- What performance and availability expectations apply?
We should not expect an LLM to know information that would also need to be explained to a developer.
The goal is therefore to give the LLM the same domain, engineering, security, and operational context that a developer would need.
Build a small context pack
A context pack is a focused collection of sources needed to understand a change. It should link to authoritative information rather than copy large amounts of documentation into the prompt.
What the context pack might tell us
The context pack may establish that:
- The API must follow the organization’s existing standards.
- The caller’s identity and organization come from a trusted source.
- The caller must be authorized for the requested store.
- Only permitted order and customer information may be returned.
- Large results must be broken into manageable groups.
- Customer and payment information must follow the organization’s privacy rules.
- The service must follow existing monitoring and error-handling practices.
The pack may also reveal missing information:
- What response-time target applies?
- What is the expected volume for a high-traffic store?
- What should happen when a required supporting service is unavailable?
- Is audit recording required?
Missing information should remain visible. The LLM should not silently invent an answer.
Ask the LLM to explain its understanding
Before asking for code, ask the LLM to interpret the requirement using the context pack.
For example:
Using the supplied context pack, explain your understanding of the store-order retrieval requirement. Reference the source behind each important conclusion. Separate evidence, inferences, assumptions, and unknowns. Identify missing information that could change the scope, security model, API design, or implementation. Do not generate code until this interpretation has been reviewed.
This gives the team an opportunity to correct misunderstandings before they become implementation decisions.
What a good response should look like
A useful response might say:
Understanding
An authenticated store manager should be able to retrieve permitted customer orders for a store they are authorized to manage.
The API should follow the organization’s existing standards for structure, pagination, errors, security, and monitoring.
Only the information required for store operations should be returned.
Evidence
- The API must follow the organization’s governance standards.
- The caller’s identity must come from the approved identity system.
- Store access must be verified using the established authorization process.
- Similar store-facing APIs provide an approved implementation pattern.
Each conclusion should reference the source that supports it.
Inferences
- The new API should probably reuse the design and authorization approach used by similar store-facing APIs.
- The order search should be limited by both the caller’s organization and authorized store access.
- Large order volumes will probably require pagination.
- Existing logging, monitoring, and testing patterns should be reused.
These conclusions appear reasonable but still require review.
The LLM should recommend resolving the important questions before implementation.
Why this improves the result
This approach helps the LLM:
- Follow organizational standards
- Reuse approved design patterns
- Respect security and privacy boundaries
- Avoid making hidden business decisions
- Identify missing requirements
- Explain how it reached its conclusions
- Produce an implementation that can be reviewed against evidence
It also makes review easier. The team can correct a short interpretation before reviewing hundreds of lines of generated code.
The principle is simple:
Before concluding that the LLM failed, check whether it had the context required to succeed.
An LLM should receive the same domain, engineering, security, and operational context that we would provide to any developer asked to implement the change.
Top comments (0)