The most common failure mode in agent workflows is not a timeout or a bad API call. It is the agent confidently doing the wrong thing because it filled in missing information with a plausible guess.
A practitioner building their first AWS Bedrock agent for customer support hit this exact problem. The agent was supposed to collect three pieces of information before creating a bug ticket: problem description, reproduction steps, and environment details. In two test cases, the agent understood the intent and created the ticket anyway, even though one required field was missing. The evaluation correctness score was 0.83, which sounds decent until you realize the 17% failure rate came from the agent assuming it had enough context when it did not.
This is not a prompt engineering problem. It is an architecture problem. The agent needs validation gates that block execution when constraints are not met, not prompts that politely ask the model to be careful.
The Core Problem: Understanding vs. Sufficient Information
LLMs are trained to be helpful. When you ask a question, they generate an answer. When you describe a workflow, they try to complete it. This behavior breaks down in agentic systems where partial information should trigger a refusal, not a best guess.
In the bug ticket scenario, the agent had three options:
- Create the ticket with incomplete data (what happened)
- Ask the user for the missing field (correct behavior)
- Refuse to proceed and escalate to a human (also correct, depending on policy)
The agent chose option 1 because the model understood the user's intent and the prompt did not enforce a hard boundary. The fix is not better prompting. The fix is moving validation out of the model's decision space entirely.
Validation Gates in AWS Bedrock AgentCore
Bedrock AgentCore orchestrates multi-step workflows using action groups, which are Lambda functions the agent can invoke. The naive approach is to let the agent decide when it has enough information to call the "create ticket" action. The safer approach is to split the workflow into two stages with a validation gate in between.
Stage 1: Information Gathering
The agent collects user input and stores it in session state. No ticket creation happens here. The action group for this stage only writes to DynamoDB or session attributes.
Validation Gate
A separate Lambda function checks whether all required fields are present. This function does not use the LLM. It is a schema validator:
def validate_bug_report(event):
required_fields = ['description', 'reproduction_steps', 'environment']
collected = event.get('sessionAttributes', {})
missing = [f for f in required_fields if not collected.get(f)]
if missing:
return {
'status': 'incomplete',
'missing_fields': missing,
'next_action': 'prompt_user'
}
return {
'status': 'complete',
'next_action': 'create_ticket'
}
Stage 2: Ticket Creation
The agent can only invoke the ticket creation action if the validation gate returns status: complete. If the gate returns incomplete, the orchestration flow routes back to the information gathering stage with a specific prompt about the missing fields.
This architecture removes the decision from the model. The agent cannot guess its way past the gate.
Refusal Patterns and Escalation Triggers
Validation gates handle missing information. Refusal patterns handle requests the agent should not attempt at all.
In the customer support agent, three request types were defined:
- Bug reports (agent handles with validation)
- FAQ questions (agent answers from knowledge base)
- Feature requests or account changes (agent escalates to human)
The refusal logic sits in the action group router. Before the agent invokes any action, a classifier function (which can be a lightweight model or rule-based logic) determines the request type. If the type is "escalate," the agent does not try to answer. It returns a structured response that triggers a handoff.
def route_request(user_input, session_state):
intent = classify_intent(user_input) # Lightweight classifier
if intent == 'escalate':
return {
'action': 'create_handoff_ticket',
'reason': 'requires_human_judgment',
'context': session_state
}
if intent == 'bug_report':
return {'action': 'start_bug_collection'}
if intent == 'faq':
return {'action': 'query_knowledge_base'}
The key is that the refusal decision happens before the agent enters a reasoning loop. The model does not get a chance to be helpful in a way that breaks policy.
Observability for Silent Assumptions
The hardest failures to catch are the ones where the agent completes the workflow but with incorrect assumptions baked in. The ticket gets created, the user gets a confirmation, and nobody notices until a human reviews the ticket and realizes the reproduction steps are missing.
You need three observability hooks:
1. Session State Snapshots
Log the full session state before and after every action group invocation. This lets you replay the workflow and see exactly what information the agent had at each decision point.
2. Validation Gate Metrics
Track how often the validation gate returns incomplete and which fields are most commonly missing. If 40% of bug reports are missing environment details, your information gathering prompt needs work.
3. Assumption Flags
Instrument your action groups to log when they receive partial data. Even if the validation gate passes, the action group itself should check its inputs and flag any fields that are present but suspiciously generic (like "unknown" or "not specified"). These flags do not block execution, but they surface in your monitoring dashboard.
Architecture Comparison: Prompt-Based vs. Gate-Based Validation
| Approach | Failure Mode | Observability | Testability |
|---|---|---|---|
| Prompt-based ("Please ensure you have X, Y, Z before proceeding") | Model ignores instruction under ambiguous input | Requires LLM trace analysis to see why it proceeded | Hard to write deterministic tests |
| Gate-based (schema validation in Lambda before action) | Gate logic bug (rare, deterministic) | Clear pass/fail in CloudWatch logs | Standard unit tests on validation function |
| Hybrid (prompt + gate) | Redundant, but catches model drift | Both LLM trace and gate logs | Best coverage, higher complexity |
The hybrid approach is overkill for most workflows. Start with gate-based validation. Add prompt-level guidance only if you see the model repeatedly trying to invoke actions it should know are blocked.
Testing Refusal Behavior
You cannot test agent reliability by only checking happy paths. You need a test suite that explicitly tries to trick the agent into acting on incomplete information.
Test Case 1: Missing Required Field
Input: "The app crashes when I open it."
Expected: Agent asks for reproduction steps and environment.
Failure: Agent creates ticket with only description.
Test Case 2: Ambiguous Intent
Input: "Can you help me with my account?"
Expected: Agent escalates to human (account changes require verification).
Failure: Agent tries to answer from FAQ or asks clarifying questions it cannot act on.
Test Case 3: Partial Information with Confidence
Input: "I am on iOS and the app crashes. I think it is a memory issue."
Expected: Agent asks for reproduction steps (the user's theory is not a substitute for steps).
Failure: Agent creates ticket with user's theory in the reproduction steps field.
Run these tests after every prompt change and after every action group update. If your correctness score drops, check whether the failures are in the "agent refused correctly" category or the "agent assumed incorrectly" category. The first is usually acceptable. The second is not.
Deployment Shape and State Management
The customer support agent used:
- Amazon Bedrock AgentCore for orchestration
- AgentCore Gateway for API access
- AWS Lambda for action groups and validation gates
- DynamoDB for session state persistence
- S3 + Bedrock Knowledge Base for FAQ retrieval
Session state is the critical piece. If you lose session state between turns, the agent cannot track what information it has already collected. DynamoDB is the standard choice here because it integrates natively with Bedrock session attributes and supports TTL for automatic cleanup.
The validation gate Lambda should be stateless. It reads from session attributes, runs schema validation, and returns a routing decision. No side effects. This makes it easy to test in isolation and easy to replace if you need to change validation logic.
Likely Failure Modes
1. Session State Drift
If the agent updates session state in the prompt but the Lambda function reads from a stale snapshot, the validation gate might pass when it should fail. Always write session state updates synchronously and confirm the write before proceeding to the next turn.
2. Overly Strict Validation
If your gate requires exact field names or specific formats, minor variations in how the agent structures the data will cause false negatives. Use flexible schema validation (check for presence and type, not exact keys).
3. Escalation Loops
If the refusal pattern triggers too easily, users get stuck in a loop where the agent keeps saying "I cannot help with that" without explaining what it can help with. Always pair a refusal with a concrete next step (either a clarifying question or a handoff with context).
Technical Verdict
Use gate-based validation when:
- Your agent performs actions with side effects (creating tickets, updating records, triggering workflows)
- You have a clear schema for required information
- You need deterministic refusal behavior that does not depend on model reasoning
Avoid gate-based validation when:
- Your agent only answers questions (no side effects, so assumptions are low-risk)
- The required information is fuzzy or context-dependent (gates work best with hard boundaries)
- You are prototyping and need to iterate quickly on what "complete" means
The lesson from this first agent build is simple: if you want an agent to admit ignorance, do not ask it politely. Build a gate that blocks execution when constraints are not met. The model will thank you by not having to guess.
Top comments (0)