Prompt-Driven Routing & Managed Agent Architecture with Amazon Bedrock AgentCore

Part 3 of the series: “Engineering Next-Gen AI Agents: Reflections on the Future AWS Agent Engineer Nanodegree”

In Part 2 of this series, we covered the foundational phase—the AWS AI Practitioner Challenge and rapid prototyping using AWS PartyRock. While low-code sandboxes are great for early proof-of-concepts, enterprise production systems require strict control, robust state management, and rigorous evaluation frameworks.

Crossing into Project 1 of the Future AWS Agent Engineer Nanodegree , the engineering stakes were raised significantly. The objective was to build an enterprise-grade customer support chatbot for an online shop using the Amazon Bedrock AgentCore managed harness.

Unlike traditional rule-based chatbots that rely on explicit condition nodes, if-else logic, or hardcoded classification pipelines, this application routes incoming customer intents entirely through system prompt engineering.

Here is a deep-dive breakdown of how the architecture was built, tested, and systematically evaluated using automated LLM-as-a-judge metrics.

1. Architectural Overview & System Flow

The pre-provisioned infrastructure leverages a cloud-native serverless architecture consisting of the AgentCore Gateway , AWS Lambda , Amazon DynamoDB , and Amazon S3.

Core Architectural Components

• Amazon Bedrock AgentCore Gateway: Acts as the managed entry point and orchestration harness for the LLM runtime. It manages session context, wraps tool definitions, and enforces system prompt boundaries.
• AWS Lambda & Simple RAG: The backend logic executes inside AWS Lambda. The function injects a localized FAQ.json document into the prompt payload to serve as a lightweight Retrieval-Augmented Generation (RAG) knowledge base.
• Amazon DynamoDB: Serves as the high-throughput, low-latency operational data store for filed bug reports.
  • Why DynamoDB over MySQL? For semi-structured operational logs like bug reports—where individual records may vary in metadata fields (e.g., varying browser flags or client environments)—DynamoDB’s schemaless document model offers instant, key-value writes without the schema migration overhead or connection pool management required by traditional relational databases like MySQL.

High-Level End-to-End Workflow

Before diving into execution details, the project lifecycle follows a strict three-stage engineering loop:

1. Build Phase: Configuring the core system prompt, integrating the Lambda action groups, and setting up the DynamoDB persistence layer.
2. Test Phase: Deploying test harness suites (harness-tests.json) to validate intent classification and tool execution against edge cases.
3. Evaluation Phase: Running automated LLM-as-a-judge evaluations using Amazon Nova Pro to score generation correctness.

To understand how user inputs move through the runtime system, the architectural flow is mapped below:

2. Infrastructure Sanity Check: Testing Lambda & DynamoDB

Before crafting the core prompt, we performed an isolated infrastructure verification test using the AWS Lambda console to ensure the backend toolchain and database persistence were functioning.

Lambda Test Event Payload

We triggered the underlying bug-reporting Lambda function using a structured JSON test event containing the three required bug metadata fields:

{
  "description": "The checkout page crashes when I click the Pay button",
  "stepsToReproduce": "1. Add an item to the cart. 2. Go to checkout. 3. Click Pay.",
  "environment": "Chrome 120 on macOS Sonoma"
}

Verification via DynamoDB Console

A successful execution returns a JSON payload containing a newly generated ticketId and a "status": "OPEN".

To verify data persistence:

1. Navigate to the Amazon DynamoDB Console -> Explore items.
2. Select the bug-report-tool-stack-bug-reports table.
3. Click Run to execute a table scan.
4. The newly created item appears with its unique ticketId, timestamp, description, environment details, and initial OPEN state.

3. Core Mission: Prompt-Driven System Routing (system_prompt.txt)

The core challenge of this project was enforcing strict routing behavior without hardcoded conditional logic. The system prompt acts as both a state machine and a guardrail controller.

Below is the full production system prompt (system_prompt.txt) engineered for the application:

You are the customer support assistant for an online shop.

For every customer message, you must first classify it into exactly ONE of the following three categories.  
Make the classification BEFORE doing anything else.

------------------------------------------------------------
1. BUG REPORT
------------------------------------------------------------
The customer says something on the website or app is broken, malfunctioning, or not working as expected.

You MUST collect all three required fields:

1. description — Description of the bug, in the customer’s own words  
2. stepsToReproduce — Steps someone should follow to reproduce the issue  
3. environment — Customer’s environment (browser, OS, device)

Rules:
- Customers rarely provide all three up front.
- If any item is missing, ask for EXACTLY ONE missing item at a time.
- Do NOT call the create_bug_report tool until ALL THREE fields are collected.
- Once all three are gathered, call the tool with:
    - description
    - stepsToReproduce
    - environment
- After the tool succeeds, it returns a ticketId.
  You MUST tell the customer their ticket ID and inform them the engineering team will follow up.

------------------------------------------------------------
2. PLATFORM QUESTION
------------------------------------------------------------
Questions about orders, shipping, returns, refunds, payments, products, accounts, or privacy.

Rules:
- Answer ONLY using the FAQ document provided.
- NEVER invent policies.
- If the FAQ does NOT contain the answer, reclassify the message as Category 3.

------------------------------------------------------------
3. ANYTHING ELSE
------------------------------------------------------------
Any message that is NOT a bug report and NOT answerable from the FAQ.

Rules:
- Politely explain you cannot help with that here.
- Refer the customer to the human support line at 1‑800‑555‑0199 (Mon–Fri).

------------------------------------------------------------
GENERAL RULES
------------------------------------------------------------
- Always be polite and concise.
- Never fabricate information.
- Routing is a STRICT classification problem: pick ONE category only.
- Category definitions must be applied crisply.
- Follow the bug‑report checklist exactly and ask for missing items ONE AT A TIME.

--- FAQ document ---
{{FAQ}}

Key Design Highlights of the System Prompt:

• Explicit Intent Ordering: Mandates that the LLM performs internal classification before choosing an output strategy.
• Single-Question Collection Strategy: Prevents the LLM from overwhelming users by enforcing a strict rule: “ask for EXACTLY ONE missing item at a time”.
• Strict Fallback Mechanisms: Eliminates hallucinated store policies by forcing any unanswerable platform question to fall back to Category 3 (Human Support Routing).

4. Constructing the Test Harness (harness-tests.json)

To evaluate the agent’s behavior across edge cases, we constructed a test suite in harness-tests.json. Each test item defines an id, a customer prompt, and an expected intent description that serves as the ground-truth reference for evaluation:

{
  "tests": [
    {
      "id": "t1_bug_report_full",
      "prompt": "The checkout page crashes when I click the Pay button. Steps: 1. Add an item to the cart. 2. Go to checkout. 3. Click Pay. Environment: Chrome 120 on macOS Sonoma.",
      "expected": "create a bug report"
    },
    {
      "id": "t2_bug_report_missing_environment",
      "prompt": "I can't upload my profile picture; it shows an error. Steps: Go to account settings and try uploading any image. Environment: not provided.",
      "expected": "ask for missing environment"
    },
    {
      "id": "t3_platform_question_shipping",
      "prompt": "How long does shipping take to New York?",
      "expected": "answer from FAQ"
    },
    {
      "id": "t4_platform_question_refund",
      "prompt": "Can I return an item if I lost the receipt?",
      "expected": "answer from FAQ"
    },
    {
      "id": "t5_anything_else_general_chat",
      "prompt": "Can you recommend a good laptop for gaming?",
      "expected": "refer to human support line"
    },
    {
      "id": "t6_anything_else_not_supported",
      "prompt": "Can you delete my account permanently?",
      "expected": "refer to human support line"
    }
  ]
}

Executing the test suite pairs the model’s actual response against the reference response. For instance, running a test on an incomplete bug report yields:

{
  "prompt": "the app crashes every time I try to upload a file...",
  "referenceResponse": "Acknowledges the issue and asks for steps to reproduce...",
  "modelResponses": [
    {
      "response": "I'm sorry to hear about the crash. Could you tell me...",
      "modelIdentifier": "my-support-chatbot"
    }
  ]
}

5. Automated Evaluation via Amazon Bedrock Evaluations

Relying on manual spot-checks is insufficient for enterprise deployment. We automated quality assessment using Amazon Bedrock Model Evaluation Jobs , using Amazon Nova Pro (amazon.nova-pro-v1:0) as an LLM-as-a-Judge evaluator model.

AWS CLI Evaluation Job Command Breakdown

The evaluation command utilizes a set of robust configuration blocks:

• --evaluation-config: Defines the automated evaluation parameters. It points to a dataset configuration located on S3 (output_eval_dataset.jsonl) under a general task type, specifies the evaluation metric (Builtin.Correctness), and designates Amazon Nova Pro (amazon.nova-pro-v1:0) as the independent judge model via evaluatorModelConfig.
• --inference-config: Points to the precomputed inference output (my-support-chatbot) generated by our test harness runs.
• --output-data-config: Specifies the destination S3 bucket path where evaluation reports and logs will be written.

aws bedrock create-evaluation-job \
  --job-name support-chatbot-eval-run-1 \
  --role-arn <BedrockEvalRoleArn> \
  --evaluation-config '{
    "automated": {
      "datasetMetricConfigs": [{
        "taskType": "General",
        "dataset": {
          "name": "support-chatbot-eval-dataset",
          "datasetLocation": {
            "s3Uri": "s3://<EvalDatasetBucketName>/output_eval_dataset.jsonl"
          }
        },
        "metricNames": ["Builtin.Correctness"]
      }],
      "evaluatorModelConfig": {
        "bedrockEvaluatorModels": [{
          "modelIdentifier": "amazon.nova-pro-v1:0"
        }]
      }
    }
  }' \
  --inference-config '{
    "models": [{
      "precomputedInferenceSource": {
        "inferenceSourceIdentifier": "my-support-chatbot"
      }
    }]
  }' \
  --output-data-config '{"s3Uri": "s3://<EvalDatasetBucketName>/results/"}' \
  --region us-east-1

Analyzing Results in the Console

Once the job completes:

1. Navigate to Amazon Bedrock Console -> Assess -> Evaluations.
2. Open support-chatbot-eval-run-1.

As shown in the evaluation report dashboard:

• Metrics Summary: The system achieved a normalized Correctness score of 1.00 across all test prompts.
• Generation Metrics Breakdown: The distribution histogram shows that 100% of the prompts (6 out of 6) hit the maximum score of 1.0, confirming that prompt-driven routing, FAQ retrieval, missing item collection, and fallback handoffs operated with 100% precision.

Key Takeaways for Tech Leaders

1. System Prompts ARE Business Logic: Deterministic routing doesn’t always require rigid state machines. Well-structured prompt guardrails can handle multi-category intent routing gracefully.
2. Evaluation-Driven Development: Treating LLM testing with the same discipline as traditional CI/CD unit testing—using precomputed test suites and automated LLM-as-a-judge scoring—is essential before deploying AI agents to production.

In Part 4 , we will step beyond single-agent routing into AI Support Agent using Amazon Bedrock AgentCore and the Strands SDK.

The post Prompt-Driven Routing & Managed Agent Architecture with Amazon Bedrock AgentCore appeared first on Behind the Build.