How to Build Efficient Context for Your AI Client to handle incidents

XT — Thu, 23 Jul 2026 17:32:48 +0000

Most teams that adopt an AI coding or operations assistant eventually reach the same conclusion:

The model is rarely the bottleneck. The bottleneck is context.

A capable model with the wrong context produces a confident, generic answer. The same model with the right context can produce something an engineer can actually use.

So the practical question is not only:

Which model should we use?

It is also:

What should the context contain, and how should we build it?

A useful context needs more than a large prompt or a folder full of documents. It needs three distinct components:

Rules
Knowledge
Retrieval

Each component solves a different problem.

1. Rules: What the AI is expected to do

Rules contain your team’s specific expectations about how work must be performed.

Examples include:

Follow these coding and review standards.
Never fail over a saturated but healthy primary.
Escalate this class of error to the infrastructure team.
Never include customer data in generated output.
Prefer this internal library over the public alternative.
Require human approval before performing a production change.

Rules operate differently from general reference material.

Knowledge tells the AI what is true. Rules tell it what it is allowed and expected to do.

A general-purpose model cannot automatically know your team’s boundaries, escalation paths, safety requirements, or architectural conventions. Those constraints have to be provided explicitly.

Because rules change the model’s behavior—not merely the information available to it—a few well-written rules can improve an answer more than several pages of background material.

This is often the highest-leverage part of a context, and it is also the part most likely to be missing.

2. Knowledge: How your systems actually work

Knowledge is the standing material that explains your systems and operational practices.

It commonly includes the following categories.

Troubleshooting guides

Troubleshooting guides capture service-specific diagnostic knowledge that experienced engineers often carry in their heads:

Which signals matter together
Which alerts are usually noisy
How a failure mode presents itself in your environment
Which apparent symptoms are unrelated
What should be checked before escalating

Runbooks and procedures

Runbooks describe the actions needed to operate a system:

How to roll back a service
How to drain a queue
How to rotate credentials
What the deployment pipeline expects
Which steps must happen in a specific order

Architecture and domain notes

These documents explain information that may be obvious to your team but invisible to a general model:

Service boundaries
Ownership
Data flows
Dependencies
Naming conventions
Business constraints
Important historical decisions

This knowledge is specific to your organization, and it changes over time.

A runbook may be rewritten after a postmortem. A troubleshooting guide may become incorrect after a migration. An architecture document may stop reflecting the current service topology.

Good context therefore treats knowledge as a living, versioned source, not as something permanently embedded into a model and forgotten.

3. Retrieval: What is true right now

Rules and standing knowledge are not sufficient for many real engineering tasks.

During an investigation, the AI may also need to answer questions such as:

What changed in the most recent deployment?
What is the current error rate?
When did the incident begin?
What does the work item actually request?
Which services are currently unhealthy?
What appears in the relevant log file?
Was a similar incident resolved before?

This information already lives in the tools your team uses:

Git repositories
Logging systems
Monitoring dashboards
Incident-management platforms
Work-item systems
Internal documentation
Custom APIs

A good context should not attempt to copy all of this information into one giant document.

Instead, it should provide the AI with a scoped, preferably read-only way to retrieve the specific information required for the current task.

The three parts of context therefore answer three different questions:

Context component	Question it answers
Rules	What am I allowed and expected to do?
Knowledge	How does this system work?
Retrieval	What is true right now?

An assistant without retrieval reasons from stale assumptions.

An assistant without rules may perform the wrong action very efficiently.

An assistant without domain knowledge may produce advice that sounds reasonable but does not apply to your systems.

For dependable results, you need all three—and they should be scoped to the task at hand.

You can build this manually—at first

A single engineer can assemble a useful context without adopting a new platform.

For example, you can:

Write a Markdown file containing the team’s rules and conventions.
Attach the relevant runbook and troubleshooting guide.
Give the assistant access to the directories related to the task.
Paste in the relevant log excerpt.
Include the incident or work-item description.
Explain which systems and actions are out of scope.

This approach works well.

It is often the correct first step because it forces you to decide what the AI actually needs. That decision is where much of the value comes from.

Anyone who has used an AI coding assistant seriously has probably already done some version of this.

The problem is not that manually assembled context is wrong.

The problem is that it does not scale beyond one person and one moment.

Why manually assembled context breaks down

Several failure modes appear as soon as context becomes a team practice.

It becomes stale

The runbook copied into a prompt last month may have since been updated.

Nobody remembers to replace the old copy, so the AI continues reasoning from instructions that no longer reflect reality.

This creates the same danger as outdated documentation, except the outdated material is now being used to generate confident recommendations.

It does not transfer between engineers

A carefully assembled context may exist only in one engineer’s:

Local files
Chat history
Prompt templates
Personal notes
Memory

When the next engineer goes on call, they have to rebuild it.

It drifts between people

Two engineers may assemble slightly different versions of “the payments context.”

As a result, they can ask the same question and receive different answers—not because the model changed, but because the inputs did.

Retrieval becomes manual and lossy

Pasting a log excerpt is easy once.

Repeating that process for every question creates ongoing overhead. Engineers have to remember:

Which system contains the relevant information
Which time range to select
Which credentials to use
How much data to include
Which sensitive fields must be removed

It is also easy to paste either too little information or far more than the AI should receive.

There is no reliable record

When an answer turns out to be wrong, it may be difficult to determine what the model was actually given.

Was it using the current runbook?

Did it see the deployment history?

Which directories were available?

Which rules were active?

Without a record of the context, debugging an AI answer becomes guesswork.

These are not primarily model problems. They are organizational problems.

Ad hoc context works for one engineer working on one task. It does not hold up as a durable team practice.

Over time, teams need a way to define, version, scope, and share context without migrating all of their knowledge into another heavyweight platform.

Turning good context habits into a repeatable system

This is the problem we are working on with NeatContext.

NeatContext provides a structured way to assemble the three parts of context—rules, knowledge, and retrieval—while keeping the source material close to the teams that own it.

Rules live in domain profiles

A domain profile is a Markdown-based definition of a team’s:

Guardrails
Conventions
Escalation logic
Operational expectations
Important constraints

The relevant profile is connected for a specific task so that the AI works within the boundary defined by that team.

Knowledge remains versioned source material

Runbooks, troubleshooting guides, architecture notes, and other documents stay as local or version-controlled source files.

They are not permanently baked into a model.

When a runbook changes, the source changes. There is no retraining step and no hidden copy inside model weights that can silently become stale.

Current information comes from read-only extensions

Instead of copying every external system into a context document, teams can connect scoped, read-only extensions for tools such as:

Git repositories
Internal documentation
Incident systems
Monitoring platforms
Custom APIs

The context defines which extensions are available for a particular task.

Credentials remain in the operating system’s credential storage rather than being placed directly into prompts.

Context is scoped per task

A production incident does not need access to every document, repository, and tool in the company.

For each task, you can choose the relevant:

Domain profiles
Knowledge directories
External tools
Data sources

This reduces background noise, unrelated information, and signal dilution.

The same context can be reused by a team

A team can define its context once and reuse it across investigations.

The next engineer on call receives the same rules, knowledge, and retrieval boundaries instead of rebuilding them from memory.

NeatContext’s team library also allows domain profiles, knowledge, and extensions to be shared read-only across team members.

There is a record of what was provided

NeatContext keeps a local activity log showing what context was handed to the AI.

When an answer needs to be reviewed, engineers can trace it back to its inputs instead of trying to reconstruct the context afterward.

NeatContext is not another AI model

One design choice is important to clarify:

NeatContext does not contain an AI model.

It does not read your documents, rank them, or summarize them before your assistant receives them.

Instead, it defines the context boundary and connects the selected profiles, folders, and read-only tools to the AI client you already use, such as:

Claude Code
Codex CLI
Claude Desktop
ChatGPT Desktop

The approved AI client performs the actual reading and reasoning.

The context is explicitly defined by a person rather than inferred by a second model.

In other words, NeatContext turns the useful habits behind manually assembled prompts into something durable:

Defined
Versioned
Scoped
Shareable
Connected to current systems

See it in action

The following demonstration shows how the same incident can require different actions from different teams when each team provides its own domain context.

The takeaway

Efficient context is not simply a bigger prompt.

It is a deliberate combination of:

Rules that define what the AI is expected and allowed to do
Knowledge that explains how your systems work
Retrieval that provides current information from the systems your team depends on

All three should be scoped tightly to the task.

You can assemble this context manually today, and doing so is worthwhile. It helps identify which information actually changes the quality of an AI-generated answer.

But if AI-assisted engineering is going to become a team practice rather than a personal trick, context needs to become:

Versioned
Scoped
Shareable
Repeatable
Connected to real systems

That is the problem we are working on with NeatContext.

You can also try the hands-on NeatContext demo on GitHub.

How are you currently providing domain-specific context to your AI tools? I would be interested to hear what works—and where the process starts to break down.

DEV Community: XT