From Prompt Engineering to Context Engineering: The Skill AI Developers Actually Need

The most dangerous failure in an LLM application is not a bad prompt.

It is a perfect prompt wrapped around the wrong context.

You can spend hours tuning the wording: “Be precise,” “Use only the provided sources,” “If you are unsure, say so.” Then the system still answers confidently because the retrieved policy document is stale, the tool schema is vague, the user profile is outdated, or the model was given three conflicting examples and no clear priority order.

That is the shift from prompt engineering to context engineering.

Prompt engineering asks:

How should I phrase this instruction?

Context engineering asks:

What information should the model see, in what order, with what constraints, from which sources, with what permissions, and at what cost?

For AI developers building real products in 2026, that second question is usually the harder one.

TL;DR

• Prompt engineering is still important, but it is only one part of the system.
• Context engineering is the design of everything the model sees: instructions, retrieved data, tools, memory, user facts, output constraints, and omissions.
• The hard problems are selection, priority, freshness, authority, scope, and validation.
• Production systems need context budgets, retrieval filters, tool schemas, memory expiration, redaction, evals, and traces.
• If your LLM app behaves differently every week, the issue is probably not the prompt alone. It is the context pipeline.

📋 Table of Contents

• Prompt Engineering Is the Visible Part
• 1. The Context Window Is a Budget, Not a Storage Unit
• 2. Put the Contract Before the Knowledge
• 3. Retrieve for Decision Quality, Not Just Similarity
• 4. Tool Schemas Are Prompts, Too
• 5. Memory Needs Expiration, Confidence, and Provenance
• 6. Replace One Giant Prompt With Structured Intermediate Artifacts
• 7. Scope the Context With Permissions, Redaction, and Denylists
• 8. Evaluate Context Failures, Not Just Prompt Wording
• 9. Trace Context Assembly Like a Build Pipeline
• Where Context Engineering Does Not Help
• A Practical Context-Engineering Checklist

Prompt Engineering Is the Visible Part

Prompt engineering is not dead. It is just no longer sufficient.

A good prompt can make a model more consistent, more constrained, and easier to integrate. But in a production system, the prompt is one artifact among many. The model also sees:

• System instructions.
• Retrieved documents.
• User profile data.
• Conversation history.
• Tool definitions.
• Previous tool outputs.
• Memory records.
• Output schemas.
• Safety constraints.
• Redaction results.
• Token-budget truncation decisions.

Those pieces are often assembled from multiple services: a database, a vector store, a CRM, an internal policy wiki, an API gateway, a user-settings service, and maybe an agent loop. The model does not know which source is trustworthy unless your context pipeline makes that explicit.

A useful way to frame the difference:

Dimension	Prompt Engineering	Context Engineering
Primary focus	Wording, examples, instructions	Entire information environment
Typical artifact	Prompt template	Context assembly pipeline
Failure mode	Vague instruction	Wrong, stale, or excessive context
Testing method	Manual prompt iteration	Eval suites, traces, regression tests
Ownership	Individual developer	Cross-functional system design
Production concern	Output quality	Correctness, safety, latency, cost, auditability

The prompt tells the model what to do. The context determines what it has to do it with.

The following patterns are the practical skills that separate a toy integration from a production-grade AI system.

1. The Context Window Is a Budget, Not a Storage Unit

Scenario:

Your support assistant is answering customer questions about refunds. To make it “more accurate,” you retrieve twenty policy documents, paste in the customer’s full account history, add three examples, and include the entire tool documentation. The answer gets worse, not better.

This happens because context is not neutral. Every additional chunk competes for attention, increases cost, increases latency, and may introduce contradictions.

Large context windows are useful, but they do not remove the need for selection. If anything, they make the problem more subtle because teams assume that “more” is automatically safer.

Why it matters:

A model can only reason over what you give it. But giving it everything is rarely the same as giving it the right thing. Irrelevant context can dilute constraints, surface outdated policies, or cause the model to reconcile conflicting information in unpredictable ways.

Solution:

Treat the context window like a budget. Every source needs a priority, a size estimate, and a reason to be included.

from dataclasses import dataclass
from typing import Iterable


@dataclass(frozen=True)
class ContextItem:
    label: str
    priority: int
    content: str
    required: bool = False


def estimate_tokens(text: str) -> int:
    # Rough estimate. Use your provider's tokenizer for precise budgeting.
    return max(1, len(text) // 4)


def assemble_context(
    items: Iterable[ContextItem],
    token_budget: int,
    reserve_tokens: int,
) -> tuple[list[ContextItem], int]:
    selected: list[ContextItem] = []
    used_tokens = 0
    available = max(0, token_budget - reserve_tokens)

    ordered = sorted(items, key=lambda item: (not item.required, item.priority))

    for item in ordered:
        tokens = estimate_tokens(item.content)

        if item.required or used_tokens + tokens <= available:
            selected.append(item)
            used_tokens += tokens

    return selected, used_tokens

The exact token estimate is not the important part. The important part is that the system makes inclusion decisions explicitly.

A context item might be:

• priority=0: task contract and output schema.
• priority=1: active policy document directly relevant to the question.
• priority=2: user account summary.
• priority=3: similar but older support cases.
• priority=4: general product documentation.

If the budget runs out, lower-priority context is dropped first.

Why this works:

It forces the team to define what matters. A support answer backed by one active policy is usually better than an answer that tries to reconcile five outdated policies.

⚠️ Gotcha: If a “required” item exceeds the budget by itself, do not silently include it and hope. Compress it, summarize it, split it, or fail the request. Silent overflow is how production prompts become unmaintainable.

2. Put the Contract Before the Knowledge

Scenario:

You give the model a pile of retrieved documents and then ask it to answer the user’s question. The answer sounds plausible, but it does not cite sources, it invents a policy exception, and it ignores your internal rule that ambiguous cases should go to a human.

The problem is not that the model failed to read the documents. The problem is that the task contract was not established strongly enough before the evidence arrived.

Why it matters:

Models are sensitive to ordering and framing. If the first thing they see is raw evidence, they may infer that the task is “summarize this material.” If the first thing they see is a contract, they are more likely to treat the material as evidence under constraints.

Solution:

Separate the contract from the context. The contract defines the job, the constraints, the output format, and the failure behavior.

TASK_CONTRACT = """
You are a support assistant for a billing product.

Objective:
Answer the customer's question using only the provided policy excerpts.

Constraints:
- Do not invent prices, deadlines, or exceptions.
- If the correct policy is missing or conflicting, set needs_human_review to true.
- Cite policy_id for every factual claim.
- Do not use deprecated policies unless explicitly marked as active.

Output format:
Return JSON with keys: answer, citations, needs_human_review.
""".strip()

Then assemble the prompt with the contract first:

def build_prompt(contract: str, context_items: list[ContextItem], question: str) -> str:
    parts = [contract]

    for item in context_items:
        parts.append(f"[{item.label}]\n{item.content}")

    parts.append(f"Customer question:\n{question}")

    return "\n\n".join(parts)

A good contract usually includes:

• The role.
• The objective.
• The allowed sources.
• The forbidden behaviors.
• The output format.
• What to do when information is missing.
• When to escalate.

Why this works:

The contract becomes the rule set for the rest of the context. Retrieved documents are no longer free-floating truth; they are inputs to be used under specific constraints.

This is especially important in regulated or high-risk domains. “Do not invent” is not enough. You need a behavior for uncertainty: return a flag, ask a clarifying question, or request human review.

3. Retrieve for Decision Quality, Not Just Similarity

Scenario:

Your assistant answers a pricing question using a document that is semantically very similar to the user’s query. Unfortunately, that document describes a pricing plan that was retired two years ago.

This is one of the most common failures in RAG systems.

Embedding similarity is useful, but it is not the same as decision relevance. A document can be topically related and still be the wrong source for the answer.

Why it matters:

In production, retrieval needs to consider more than semantic distance. It needs to consider:

• Product version.
• Customer segment.
• Locale.
• Document status.
• Effective date.
• Authority.
• Permissions.
• Freshness.
• Conflict with newer sources.

A vector search that only optimizes for similarity will happily surface deprecated content if that content is textually close to the question.

Solution:

Use retrieval as a filtered candidate-generation step, then apply business rules before the context reaches the model.

from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class PolicyDoc:
    policy_id: str
    body: str
    product: str
    status: str
    locale: str
    authority: int
    updated_at: datetime


def select_policies(
    candidates: list[PolicyDoc],
    user_product: str,
    user_locale: str,
    limit: int = 4,
) -> list[PolicyDoc]:
    eligible = [
        doc
        for doc in candidates
        if doc.product == user_product
        and doc.locale == user_locale
        and doc.status == "active"
    ]

    eligible.sort(key=lambda doc: (doc.authority, doc.updated_at), reverse=True)

    return eligible[:limit]

In a real system, candidates might come from a vector store, keyword search, hybrid search, or a metadata-first lookup. The key point is that the final context is not just “the top four chunks.” It is the top four chunks after filtering for operational relevance.

Why this works:

The model is less likely to use stale or irrelevant information if that information never enters the context. Filtering before generation is usually safer than asking the model to ignore bad material after it has already been included.

💡 Practical note: Retrieval quality is only as good as your metadata. If your documents do not have reliable status, product, locale, and effective-date fields, no amount of prompt wording can fully fix the problem.

4. Tool Schemas Are Prompts, Too

Scenario:

Your agent is supposed to create a refund. Instead, it calls the refund tool with amount: "45.50", omits the currency, or invents a reason that your backend does not understand.

The team tweaks the natural-language prompt: “Please call the refund tool correctly.” The problem improves slightly, then returns.

The real issue is the tool schema.

Why it matters:

Tool definitions are context. Their names, descriptions, parameter types, enums, and constraints shape what the model believes is possible. A vague tool description can produce invalid calls even when the surrounding prompt is clear.

Solution:

Design tool schemas with the same care as a public API.

CREATE_REFUND_TOOL = {
    "name": "create_refund",
    "description": (
        "Create a refund for a paid order. Call this only after confirming "
        "the order is refundable and the amount is verified."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "order_id": {
                "type": "string",
                "description": "The order identifier, for example ord_8f3k2.",
            },
            "amount_cents": {
                "type": "integer",
                "minimum": 1,
                "description": "Refund amount in cents. Use integers only.",
            },
            "currency": {
                "type": "string",
                "enum": ["USD", "EUR", "GBP"],
            },
            "reason": {
                "type": "string",
                "enum": [
                    "duplicate",
                    "fraud",
                    "customer_request",
                    "service_issue",
                ],
            },
        },
        "required": ["order_id", "amount_cents", "currency", "reason"],
        "additionalProperties": False,
    },
}

This schema communicates several important things:

• The amount is in cents.
• The amount must be an integer.
• The currency is restricted.
• The reason must be one of a known set.
• Extra fields are not allowed.
• The tool should only be called after verification.

Why this works:

The model is less likely to hallucinate parameter names or values when the valid space is tightly defined. Enums are especially useful because they prevent free-form values that your backend cannot process.

This also improves safety. If a tool can perform a destructive action, the description should state when it may be used. If a tool requires prior confirmation, say so explicitly.

Where teams get this wrong:

• They expose too many tools at once.
• They use vague names like process_action.
• They describe parameters only with types, not semantics.
• They allow string fields where enums should exist.
• They fail to validate tool calls before execution.

A good rule: if a junior engineer would need clarification before calling the tool safely, the model probably does too.

5. Memory Needs Expiration, Confidence, and Provenance

Scenario:

Your assistant remembers that a user prefers sandbox environments. That was true three months ago during onboarding. Now the user is working in production, but the assistant keeps suggesting sandbox-only workflows.

Memory can make an application feel personalized. It can also make the application confidently wrong.

Why it matters:

Long-lived memory is not just a storage problem. It is a truth-management problem. A memory record should answer:

• Where did this come from?
• How confident are we?
• When does it expire?
• What scope does it apply to?
• Can the user inspect or delete it?
• Should it override newer evidence?

If memory is treated as an eternal fact store, it becomes a source of subtle bugs.

Solution:

Model memory as structured records with metadata.

from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class Memory:
    memory_id: str
    scope: str
    key: str
    value: str
    confidence: float
    source: str
    created_at: datetime
    expires_at: datetime | None


def active_memories(
    memories: list[Memory],
    scope: str,
    now: datetime,
    limit: int = 5,
) -> list[Memory]:
    eligible = [
        memory
        for memory in memories
        if memory.scope == scope
        and (memory.expires_at is None or memory.expires_at > now)
    ]

    eligible.sort(key=lambda memory: (memory.confidence, memory.created_at), reverse=True)

    return eligible[:limit]

The scope field matters. A memory that is valid for one project, workspace, environment, or conversation type should not leak into another.

For example:

• scope="workspace:analytics"
• scope="user:global_preferences"
• scope="project:billing-migration"
• scope="conversation:support-ticket-4310"

Why this works:

Memory becomes evidence, not absolute truth. The context assembler can choose the most relevant, recent, and confident memories instead of dumping everything into the prompt.

Practical memory rules:

• Prefer explicit user-provided facts over inferred ones.
• Store the source of each memory.
• Give inferred memories lower confidence.
• Expire operational facts faster than stable preferences.
• Let users view and delete memories.
• Avoid storing secrets, tokens, or sensitive PII unless absolutely necessary.

🚨 Production warning: If your system cannot explain why a memory was included in a response, you will have a hard time debugging unexpected behavior. Memory observability is not optional once personalization becomes persistent.

6. Replace One Giant Prompt With Structured Intermediate Artifacts

Scenario:

You ask the model to refactor a module, write tests, preserve backward compatibility, and explain the migration in a single response. It produces something that looks impressive, but the changes are inconsistent: the tests refer to functions that were renamed incorrectly, and the explanation describes a different approach from the code.

One-shot generation is tempting because it feels simple. But complex tasks usually benefit from decomposition.

Why it matters:

A single prompt that asks for everything at once forces the model to solve planning, reasoning, implementation, and communication simultaneously. Structured intermediate artifacts let you separate those steps.

Solution:

Generate explicit artifacts between steps.

For example, before writing code, generate a plan:

PLAN_SCHEMA = {
    "type": "object",
    "properties": {
        "goal": {"type": "string"},
        "assumptions": {"type": "array", "items": {"type": "string"}},
        "files_to_change": {"type": "array", "items": {"type": "string"}},
        "risks": {"type": "array", "items": {"type": "string"}},
        "steps": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["goal", "assumptions", "files_to_change", "risks", "steps"],
    "additionalProperties": False,
}

Then use that plan as context for implementation:

def planning_prompt(contract: str, repo_context: str) -> str:
    return "\n\n".join(
        [
            contract,
            repo_context,
            "Produce a plan only. Do not write implementation code.",
            "Return JSON matching PLAN_SCHEMA.",
        ]
    )


def implementation_prompt(contract: str, approved_plan: dict, relevant_files: str) -> str:
    return "\n\n".join(
        [
            contract,
            "Approved plan:",
            str(approved_plan),
            "Relevant files:",
            relevant_files,
            "Implement the plan. Do not change files outside the plan.",
        ]
    )

Intermediate artifacts can include:

• Plans.
• Assumption lists.
• File-change proposals.
• API contracts.
• Test matrices.
• Migration checklists.
• Review notes.
• Risk assessments.

Why this works:

Each artifact narrows the problem. The planning step can be reviewed before code is generated. The implementation step can be constrained by the approved plan. The review step can compare the diff against the original plan.

This also improves debugging. If the final code is wrong, you can inspect whether the plan was wrong, whether the plan was ignored, or whether the context supplied to the implementation step was incomplete.

When not to use this pattern:

Do not add intermediate artifacts for trivial tasks. If the user asks for a one-line utility function, a five-stage agent pipeline is probably overhead. Use decomposition where the task has meaningful risk, ambiguity, or multi-step dependencies.

7. Scope the Context With Permissions, Redaction, and Denylists

Scenario:

A support assistant retrieves an internal account note that says, “Customer suspected of fraud; do not offer self-service refund.” The model then mentions that note directly to the customer.

The model did not “leak” because it was malicious. It used the context you gave it.

Why it matters:

Context engineering is also security engineering. If sensitive information enters the model context, it can influence the response even if the model does not quote it verbatim. The safest approach is often to keep restricted data out of the context entirely.

Solution:

Apply permission checks and redaction during context assembly.

import re

EMAIL_RE = re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b")


def redact_emails(text: str) -> str:
    return EMAIL_RE.sub("[email]", text)


def allowed_for_support(docs: list) -> list:
    return [
        doc
        for doc in docs
        if doc.visibility in {"public", "support"}
        and not doc.contains_restricted_pii
    ]

This is a simplified example, but it shows the pattern: context sources should carry visibility metadata, and the assembler should filter based on the caller’s role.

A more complete context-scoping layer might include:

• Role-based access checks.
• Tenant isolation.
• Document classification labels.
• PII redaction.
• Secret scanning.
• Denylists for certain phrases or topics.
• Human-review flags for sensitive categories.
• Audit logs showing which sources were included.

Why this works:

It moves security decisions out of the model and into the application boundary. The model should not be the only thing deciding whether a user is allowed to see a document. That decision belongs to your authorization layer.

Common mistakes:

• Assuming the model will “know” not to repeat sensitive information.
• Redacting only after the response is generated.
• Including internal notes because they might be “useful.”
• Letting retrieval ignore document permissions.
• Logging full raw context in plaintext without masking.

Regex-based redaction can help, but it is not sufficient by itself. Names, addresses, account numbers, and domain-specific identifiers often require more robust tooling.

8. Evaluate Context Failures, Not Just Prompt Wording

Scenario:

You change the prompt from “Answer using the sources” to “Answer strictly using the sources.” One test case improves. Another test case regresses. A third case starts refusing valid requests. Now you are tuning words without knowing which failure you are actually fixing.

This is where prompt engineering often becomes fragile. Without evals, every prompt change is a guess.

Why it matters:

Production LLM systems fail in categories, not one-off sentences. You need tests for the failure modes that matter:

• Missing source.
• Stale source.
• Conflicting sources.
• Unauthorized source.
• Invalid tool call.
• Missing user permission.
• Ambiguous request.
• Policy exception not allowed.
• PII present in retrieved context.
• Output schema violation.

Solution:

Build eval cases around context conditions.

from dataclasses import dataclass


@dataclass(frozen=True)
class ContextEval:
    name: str
    question: str
    docs: list[str]
    user_role: str
    expect_needs_human_review: bool
    forbidden_substrings: tuple[str, ...]


def run_eval(case: ContextEval, build_context, run_model) -> None:
    context = build_context(case.question, case.docs, case.user_role)
    result = run_model(context)

    assert result.needs_human_review == case.expect_needs_human_review

    text = result.answer.lower()

    for banned in case.forbidden_substrings:
        assert banned not in text

The build_context function should be the same one used in production. That is important. If your evals use a simplified context builder, you are testing a different system.

Examples of useful eval cases:

EVALS = [
    ContextEval(
        name="missing_policy",
        question="Can I get a refund after 90 days?",
        docs=[],
        user_role="customer",
        expect_needs_human_review=True,
        forbidden_substrings=("yes, you can", "no, you cannot"),
    ),
    ContextEval(
        name="deprecated_policy",
        question="Can I get a refund after 90 days?",
        docs=["policy_v1_deprecated"],
        user_role="customer",
        expect_needs_human_review=True,
        forbidden_substrings=("policy_v1",),
    ),
]

Why this works:

You stop asking, “Does the prompt sound better?” and start asking, “Does the system behave correctly under known context conditions?”

This is also how you make context engineering a team discipline. Once evals exist, retrieval changes, prompt changes, tool schema changes, and memory changes can all be reviewed against the same regression suite.

9. Trace Context Assembly Like a Build Pipeline

Scenario:

A user reports that the assistant gave a wrong answer yesterday. You try to reproduce it today, but the answer is now correct. The retrieved documents changed, the memory state changed, or the prompt template was updated overnight.

Without traces, debugging LLM applications becomes guesswork.

Why it matters:

A response is the result of a pipeline. If you cannot reconstruct the pipeline inputs, you cannot understand the output.

Solution:

Log a context manifest for each request.

from dataclasses import dataclass, field


@dataclass(frozen=True)
class ContextTrace:
    request_id: str
    model: str
    prompt_template_hash: str
    retrieved_doc_ids: list[str]
    memory_ids: list[str]
    tool_names: list[str]
    token_estimates: dict[str, int]
    redactions_applied: list[str] = field(default_factory=list)

A useful trace usually includes:

• Model identifier.
• Prompt template version or hash.
• Retrieved document IDs.
• Retrieval query.
• Retrieval filters.
• Memory record IDs.
• Tool definitions used.
• Token estimates by section.
• Redaction rules applied.
• User role or permission scope.
• Feature flags.
• Timestamp.

You do not always need to store the full raw context. In many systems, storing document IDs, hashes, and redacted snippets is safer and more practical. The goal is reproducibility without creating another privacy risk.

Why this works:

When a bad answer occurs, you can ask precise questions:

• Which documents were included?
• Were the correct filters applied?
• Was the active policy missing?
• Did memory override newer evidence?
• Did the tool schema allow an invalid call?
• Did the prompt template change?
• Did token truncation remove a required instruction?

This turns context engineering from a theoretical concept into an operational capability.

🔍 Why this matters: If your team cannot answer “What context produced this response?” then the system is not ready for serious production use.

Where Context Engineering Does Not Help

Context engineering is powerful, but it is not a cure-all.

It does not fix:

• Bad underlying data.
• Missing product decisions.
• Unclear business rules.
• Broken authorization systems.
• Unreliable source-of-truth documents.
• A product requirement that needs deterministic guarantees.
• A workflow where any hallucination is unacceptable.

If your internal knowledge base contains five conflicting refund policies, context engineering can help prioritize them, but it cannot magically decide which policy the business actually follows. If your tool API accepts invalid input without validation, a better tool schema reduces failures but does not replace server-side checks.

There are also cases where an LLM is the wrong tool entirely. If you need exact arithmetic, deterministic routing, strict compliance logic, or hard financial calculations, put that logic in ordinary code. Use the model for interpretation, extraction, planning, or communication, not as the final authority for critical business rules.

A good context-engineering mindset is not “give the model more.” It is “give the model the right information, under the right constraints, with the right audit trail.”

A Practical Context-Engineering Checklist

If you are moving from prompt engineering to context engineering, start with the system you already have. Do not redesign everything at once.

Use this checklist in order:

1. Inventory the context sources.
   
   List everything that can enter the model context: prompts, retrieved documents, user data, memory, tool schemas, conversation history, and prior tool outputs.

2. Assign priority to each source.
   
   Decide what should survive when the budget is tight. The task contract and active source-of-truth data usually outrank generic background material.

3. Add a token budget.
   
   Reserve space for the model’s response. Do not let retrieval or memory consume the entire window.

4. Make the contract explicit.
   
   Define the objective, constraints, output format, and uncertainty behavior before including raw evidence.

5. Filter retrieval by metadata.
   
   Use product, locale, status, effective date, authority, and permission scope. Do not rely on semantic similarity alone.

6. Tighten tool schemas.
   
   Use enums, required fields, integer units, clear descriptions, and validation before execution.

7. Add memory governance.
   
   Store scope, source, confidence, creation time, and expiration. Prefer recent, explicit, high-confidence memories.

8. Redact and scope sensitive data.
   
   Keep restricted information out of the context when possible. Apply permission checks before retrieval results reach the model.

9. Create evals for context failures.
   
   Test missing sources, stale sources, conflicting sources, unauthorized sources, invalid tool calls, and schema violations.

10. Trace every production request.
    
    Log the context manifest so bad outputs can be reproduced and diagnosed.

The skill AI developers actually need is not memorizing clever prompt phrases. It is learning to design the information environment around the model.

A good prompt can improve an answer. Good context engineering makes the system easier to reason about, safer to operate, and less expensive to debug when the answer is wrong.