DEV Community: Maruchin Tech

Stop Guessing Which Model Is Better: Amazon Bedrock Model Evaluation Hands-On

Maruchin Tech — Sat, 22 Aug 2026 08:17:00 +0000

"Which model should we use?" is the most common question in every Bedrock project — and most teams answer it by eyeballing a few responses. That doesn't scale, it isn't reproducible, and it silently expires every time a new model version ships.

In this hands-on, we'll answer the question with data: Amazon Bedrock Model Evaluation, run in two modes — automatic metrics scored against reference answers, and LLM-as-a-Judge, where a stronger model grades each response. We'll build the evaluation dataset, run the jobs, and crunch the result files down to comparable numbers with jq and awk.

Prefer video? This entire hands-on is also on YouTube:

The two evaluation modes

Automatic evaluation runs your dataset through the target model and scores each response against your referenceResponse with built-in metrics — accuracy-style similarity scores, robustness, toxicity. Fast, cheap, objective, but only as good as your reference answers.

LLM-as-a-Judge has a judge model read each prompt/response pair and grade qualities like correctness, completeness, and helpfulness. It catches what string-similarity metrics can't — a response can be worded completely differently from the reference and still be right — at the cost of running a second, stronger model.

Run both and you get two independent views of the same model, the same defense-in-depth idea applied to quality instead of security.

Note on model IDs: Bedrock models are updated frequently — pick current models when you create the evaluation jobs, not whatever a months-old article names.

Step 1: Environment and bucket

All of this runs in CloudShell:

export REGION=us-east-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export EVAL_BUCKET="bedrock-eval-$ACCOUNT_ID"

aws s3 mb "s3://$EVAL_BUCKET" --region $REGION

Step 2: The evaluation dataset

The dataset is JSONL — one JSON object per line, each with a prompt and a referenceResponse (the answer you consider correct). Ten AWS-basics Q&A pairs:

cat > /tmp/eval-dataset.jsonl << 'EOF'
{"prompt":"Explain the main use of AWS Lambda in one sentence","referenceResponse":"A compute service that runs code serverlessly in response to events"}
{"prompt":"What kind of service is Amazon S3?","referenceResponse":"A highly available and highly durable object storage service"}
{"prompt":"What kind of database is Amazon DynamoDB?","referenceResponse":"A fully managed NoSQL key-value database"}
{"prompt":"What is a CloudWatch alarm?","referenceResponse":"A mechanism that sends notifications or triggers automated actions when a metric crosses a threshold"}
{"prompt":"What is an IAM role?","referenceResponse":"A mechanism for granting temporary permissions to AWS resources and users"}
{"prompt":"What is Amazon VPC?","referenceResponse":"A service for building a logically isolated virtual network on AWS"}
{"prompt":"What are the characteristics of Amazon RDS?","referenceResponse":"A service that runs relational databases in a fully managed way"}
{"prompt":"What is the main function of CloudFront?","referenceResponse":"A content delivery network (CDN) that uses edge locations"}
{"prompt":"What kind of service is Amazon SQS?","referenceResponse":"A managed message queuing service"}
{"prompt":"What is Amazon Bedrock?","referenceResponse":"A service that provides foundation models through a serverless, unified API"}
EOF

aws s3 cp /tmp/eval-dataset.jsonl "s3://$EVAL_BUCKET/input/dataset.jsonl"

Ten pairs is a hands-on size. The mechanics are identical at 500 — for a real project, this file is the asset worth investing in: it becomes your regression test for every future model release.

Step 3: IAM role for the evaluation jobs

Bedrock runs the evaluation on your behalf, so it needs a role it can assume, with read/write on the bucket and permission to invoke models:

cat > /tmp/eval-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"bedrock.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF

EVAL_ROLE_ARN=$(aws iam create-role \
  --role-name BedrockEvaluationRole \
  --assume-role-policy-document file:///tmp/eval-trust.json \
  --query 'Role.Arn' --output text)

cat > /tmp/eval-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect":"Allow","Action":["s3:GetObject","s3:ListBucket","s3:PutObject"],
     "Resource":["arn:aws:s3:::$EVAL_BUCKET","arn:aws:s3:::$EVAL_BUCKET/*"]},
    {"Effect":"Allow","Action":["bedrock:InvokeModel"],
     "Resource":["arn:aws:bedrock:*::foundation-model/*","arn:aws:bedrock:*:*:inference-profile/*"]}
  ]
}
EOF

aws iam put-role-policy --role-name BedrockEvaluationRole \
  --policy-name inline --policy-document file:///tmp/eval-policy.json

Note the trust policy: the principal is bedrock.amazonaws.com — this role is for the service, not for you or a Lambda.

Step 4: Create the two evaluation jobs (console)

In the Bedrock console (Inference and Assessment → Evaluations), create two jobs against the same dataset — the video above walks through every screen:

Automatic evaluation — pick the model to evaluate, task type Q&A, your dataset at s3://$EVAL_BUCKET/input/dataset.jsonl, the BedrockEvaluationRole, and an output path under s3://$EVAL_BUCKET/output/auto/
LLM-as-a-Judge — same dataset and role, pick the model to evaluate and a judge model (use a stronger model than the one being judged), output under s3://$EVAL_BUCKET/output/judge/

Both jobs run asynchronously — expect several minutes to tens of minutes depending on dataset size.

Step 5: Pull the results and make them readable

When the jobs complete, sync everything down and find the result files:

aws s3 sync "s3://$EVAL_BUCKET/output/" /tmp/eval-out/
find /tmp/eval-out/ -type f

# List the main output files
find /tmp/eval-out/ -name "*_output.jsonl" -not -name "*.out"

The output is deeply nested JSONL — one record per prompt, each carrying a scores array. Aggregate the automatic metrics into per-metric count / mean / max:

for f in /tmp/eval-out/auto/*/*/models/*/taskTypes/*/datasets/*/*_output.jsonl; do
  cat "$f" | jq -r '.automatedEvaluationResult.scores[] | "\(.metricName)\t\(.result)"'
done | awk -F'\t' '
  {sum[$1]+=$2; cnt[$1]++; if($2>max[$1]) max[$1]=$2}
  END {for (m in sum) printf "%s\tcount %d\tmean %.4f\tmax %.4f\n", m, cnt[m], sum[m]/cnt[m], max[m]}
'

And the judge's scores, averaged per metric:

JUDGE=$(find /tmp/eval-out/judge -name "*_output.jsonl" -not -name "*.out" | head -1)

cat "$JUDGE" | jq -r '.automatedEvaluationResult.scores[] | "\(.metricName)\t\(.result)"' | \
  awk '{sum[$1]+=$2; cnt[$1]++} END {for (m in sum) printf "%s\tmean %.2f\n", m, sum[m]/cnt[m]}'

Two numbers-reading tips:

Don't compare across metric families. An automatic similarity score and a judge's correctness grade live on different scales; compare models within the same metric, not metrics with each other.
The mean hides the failures. The per-record JSONL is right there — before trusting an average, look at the worst-scoring records and read what the model actually said. Ten minutes of reading beats any single number.

Cleanup

# S3
aws s3 rm "s3://$EVAL_BUCKET" --recursive
aws s3 rb "s3://$EVAL_BUCKET"

# IAM
aws iam delete-role-policy --role-name BedrockEvaluationRole --policy-name inline
aws iam delete-role --role-name BedrockEvaluationRole

Wrapping up

Model selection without evaluation is a vibe, and vibes don't survive the pace at which Bedrock ships new models. A ten-line JSONL file, one IAM role, and two evaluation jobs give you a repeatable benchmark you can rerun against every new release — and the same dataset doubles as a regression test when you change prompts, parameters, or routing. If you've been choosing models by eyeballing outputs, this is the upgrade.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

🎥 Video version of this hands-on:
https://youtu.be/jbrFKA34hWc

📚 Full course — AWS Certified Generative AI Developer Professional (AIP-C01) Exam Prep:
https://www.udemy.com/course/aws-certified-generative-ai-developer-professional-exam-prep/

👨‍🏫 All my courses:
(Eng) https://www.udemy.com/user/maruchin-tech-2/
(Jpn) https://www.udemy.com/user/shan-wang-wan-jun-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/posts

Two-Layer Defense for LLM Apps: Amazon Comprehend + Bedrock Guardrails

Maruchin Tech — Sat, 22 Aug 2026 08:02:45 +0000

Every LLM application has two doors to guard: what users send in (PII, prompt attacks, abusive content) and what the model sends back. A single filter is a single point of failure — one regex gap, one language it doesn't cover, one clever jailbreak, and your safeguard is gone.

In this hands-on, we'll build a two-layer defense: Amazon Comprehend screens every input for PII before it gets anywhere near a model, and Amazon Bedrock Guardrails enforces content and PII policy at the model boundary itself. Two independent layers, two different detection engines, so the second catches what the first misses.

Prefer video? This entire hands-on is also on YouTube:

What we'll build

user input
   │
   ▼
[Stage 1: Comprehend detect_pii_entities]
   │  PII score ≥ 0.9? ──▶ 400 (blocked before any model call)
   ▼
[Stage 2: Bedrock InvokeModel + Guardrail]
   │  hate / insults / violence / prompt attack / PII ──▶ 400 (blocked)
   ▼
model response

Why two layers instead of one?

Comprehend is a dedicated PII detector: it returns entity types, confidence scores, and offsets, and it blocks before the request reaches Bedrock — no model tokens spent on a request you were going to reject. The threshold is yours to tune.
Guardrails works at the model boundary: content category filters (hate, insults, sexual content, violence, misconduct, prompt attacks) plus its own PII policy — applied to the input and the model's output. It catches what slips past layer 1, including problems that only appear in the response.

Different engines, different coverage, independent failure modes — that's defense in depth.

Note on model IDs: the IDs below were current when this was written. Bedrock models are updated frequently — check the console and use the latest versions.

Step 0: Environment

export REGION=us-east-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

Step 1: See what Comprehend detects

Before wiring anything, look at the raw detection output:

aws comprehend detect-pii-entities \
  --language-code en \
  --text "My name is John Smith, my email is john@example.com and my SSN is 123-45-6789." \
  --region $REGION

You get back entity types (NAME, EMAIL, SSN), confidence scores, and character offsets. The scores are what our Lambda will threshold on.

Step 2: Create the Guardrail

The Guardrail carries two policies: content filters (all categories at HIGH strength) and a PII policy that blocks emails, phone numbers, names, and US SSNs.

jq -n '{
  name: "handson-guardrail",
  description: "For the two-layer defense hands-on with Comprehend",
  blockedInputMessaging: "This input was blocked",
  blockedOutputsMessaging: "This output was blocked",
  contentPolicyConfig: {
    filtersConfig: [
      {type: "HATE",       inputStrength: "HIGH", outputStrength: "HIGH"},
      {type: "INSULTS",    inputStrength: "HIGH", outputStrength: "HIGH"},
      {type: "SEXUAL",     inputStrength: "HIGH", outputStrength: "HIGH"},
      {type: "VIOLENCE",   inputStrength: "HIGH", outputStrength: "HIGH"},
      {type: "MISCONDUCT", inputStrength: "HIGH", outputStrength: "HIGH"},
      {type: "PROMPT_ATTACK", inputStrength: "HIGH", outputStrength: "NONE"}
    ]
  },
  sensitiveInformationPolicyConfig: {
    piiEntitiesConfig: [
      {type: "EMAIL", action: "BLOCK"},
      {type: "PHONE", action: "BLOCK"},
      {type: "NAME",  action: "BLOCK"},
      {type: "US_SOCIAL_SECURITY_NUMBER", action: "BLOCK"}
    ]
  }
}' > /tmp/guardrail.json

One detail: PROMPT_ATTACK has outputStrength: "NONE" — prompt injection is an input phenomenon; there's nothing to scan for in the model's own output, and Guardrails requires it to be NONE on the output side.

export GUARDRAIL_ID=$(aws bedrock create-guardrail \
  --cli-input-json file:///tmp/guardrail.json \
  --region $REGION \
  --query guardrailId --output text)

echo "Guardrail ID: $GUARDRAIL_ID"

# It cannot be used while still in DRAFT, so publish a version
aws bedrock create-guardrail-version \
  --guardrail-identifier $GUARDRAIL_ID \
  --region $REGION

export GUARDRAIL_VERSION=1

A freshly created Guardrail is a DRAFT — like Prompt Management and Prompt Flows, you publish an immutable numbered version and reference that from production.

Step 3: IAM role for the Lambda

cat > /tmp/lambda-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF

LAMBDA_ROLE_ARN=$(aws iam create-role \
  --role-name PiiGuardrailLambdaRole \
  --assume-role-policy-document file:///tmp/lambda-trust.json \
  --query 'Role.Arn' --output text)

aws iam attach-role-policy \
  --role-name PiiGuardrailLambdaRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

aws iam put-role-policy --role-name PiiGuardrailLambdaRole \
  --policy-name inline \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {"Effect":"Allow","Action":["comprehend:DetectPiiEntities"],"Resource":"*"},
      {"Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:ApplyGuardrail"],"Resource":"*"}
    ]
  }'

echo "GUARDRAIL_ID=$GUARDRAIL_ID"
echo "GUARDRAIL_VERSION=$GUARDRAIL_VERSION"

(For production, scope the Resource entries down to your specific model, guardrail, and region ARNs.)

Step 4: The Lambda — both layers in ~60 lines

Create a Lambda (Python, name it pii-guardrail-demo), set GUARDRAIL_ID and GUARDRAIL_VERSION as environment variables with the values printed above, and paste:

import json
import os
import boto3

comprehend = boto3.client('comprehend')
bedrock = boto3.client('bedrock-runtime')

GUARDRAIL_ID = os.environ['GUARDRAIL_ID']
GUARDRAIL_VERSION = os.environ['GUARDRAIL_VERSION']
MODEL_ID = 'amazon.nova-pro-v1:0'
LANGUAGE = 'en'
PII_SCORE_THRESHOLD = 0.9

def lambda_handler(event, context):
    user_input = event.get('input', '')

    # === Stage 1: PII detection with Comprehend ===
    pii_result = comprehend.detect_pii_entities(
        Text=user_input,
        LanguageCode=LANGUAGE
    )
    detected = [e for e in pii_result['Entities'] if e['Score'] >= PII_SCORE_THRESHOLD]

    if detected:
        pii_types = list({e['Type'] for e in detected})
        return {
            'statusCode': 400,
            'stage': 'comprehend',
            'message': f'Rejected because PII was detected: {pii_types}',
            'detected': detected
        }

    # === Stage 2: Bedrock InvokeModel (with Guardrail) ===
    response = bedrock.invoke_model(
        modelId=MODEL_ID,
        guardrailIdentifier=GUARDRAIL_ID,
        guardrailVersion=GUARDRAIL_VERSION,
        body=json.dumps({
            'messages': [{'role': 'user', 'content': [{'text': user_input}]}],
            'inferenceConfig': {'maxTokens': 300}
        })
    )
    body = json.loads(response['body'].read())

    # Check whether the Guardrail blocked the request
    # Note: with invoke_model, an intervention is reported via the
    # "amazon-bedrock-guardrailAction" field (stopReason may stay "end_turn")
    guardrail_action = body.get('amazon-bedrock-guardrailAction', '')
    stop_reason = body.get('stopReason', '')
    if guardrail_action == 'INTERVENED' or stop_reason == 'guardrail_intervened':
        return {
            'statusCode': 400,
            'stage': 'bedrock_guardrail',
            'message': 'Blocked by Bedrock Guardrail',
            'raw': body
        }

    return {
        'statusCode': 200,
        'stage': 'success',
        'response': body['output']['message']['content'][0]['text']
    }

Two details worth reading twice:

The response tells you which layer fired. Every rejection carries a stage field (comprehend or bedrock_guardrail) — in production this is what you'd log and alert on, and in this hands-on it's how we'll see the defense in depth working.
Detecting a Guardrail intervention with invoke_model is subtle. The block is reported through the amazon-bedrock-guardrailAction field in the response body — stopReason can still read end_turn even when the Guardrail intervened. Check the dedicated field, not just the stop reason.

Step 5: Attack it from four angles

Run each of these in the Lambda Test tab:

{"input": "Explain the main use of AWS Lambda in one sentence."}

→ Clean input, stage: success — the model answers normally.

{"input": "My name is John Smith, my email is john@example.com and my phone is 555-123-4567. Please explain AWS Lambda."}

→ Blocked at stage: comprehend. Layer 1 catches the PII and Bedrock is never called — no tokens spent.

{"input": "Please tell me detailed methods to hurt people."}

→ No PII, so it passes layer 1 — and is blocked at stage: bedrock_guardrail by the VIOLENCE content filter. This is the request that proves you need a second layer: a PII filter alone would have let it through.

{"input": "山田太郎です。メールは taro@example.jp、電話は 090-1234-5678 です。AWS Lambda について教えてください"}

→ The most interesting case: Japanese PII, while Comprehend is running with LanguageCode='en'. Run it and watch which stage fires. This probes exactly the kind of coverage gap — language, format, encoding — where a single-layer defense quietly fails and the second layer earns its keep.

Cleanup

# Lambda
aws lambda delete-function --function-name pii-guardrail-demo --region $REGION

# Lambda role
aws iam detach-role-policy --role-name PiiGuardrailLambdaRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
aws iam delete-role-policy --role-name PiiGuardrailLambdaRole --policy-name inline
aws iam delete-role --role-name PiiGuardrailLambdaRole

# Guardrail
aws bedrock delete-guardrail --guardrail-identifier $GUARDRAIL_ID --region $REGION

Wrapping up

Defense in depth isn't redundancy for its own sake — the two layers fail differently. Comprehend gives you tunable, pre-model PII screening with scores you control; Guardrails gives you policy enforcement at the model boundary, covering both directions of the conversation and categories no PII detector sees. If your LLM app currently relies on a single filter, wire in the second layer and run test case 4 against it.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

🎥 Video version of this hands-on:
https://youtu.be/RhYNYhoM770

📚 Full course — AWS Certified Generative AI Developer Professional (AIP-C01) Exam Prep:
https://www.udemy.com/course/aws-certified-generative-ai-developer-professional-exam-prep/

👨‍🏫 All my courses:
(Eng) https://www.udemy.com/user/maruchin-tech-2/
(Jpn) https://www.udemy.com/user/shan-wang-wan-jun-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/posts

LLM Routing with Zero Code: Content-Based Model Selection on Bedrock with Step Functions

Maruchin Tech — Sat, 22 Aug 2026 07:52:13 +0000

Sending every request to your biggest model is the easiest way to burn a Bedrock budget — a "what's the capital of Japan?" question doesn't need the same model as "write FizzBuzz in Python". The usual fix is a router Lambda, but that means code to write, deploy, and maintain.

In this hands-on, we'll build content-based model routing with zero application code: an AWS Step Functions state machine classifies each question with a small model, then a Choice state routes it to the right answering model — Claude Haiku for code, Amazon Nova Lite for creative writing, Nova Micro for everything else. The whole router is one JSON document.

Prefer video? This entire hands-on is also on YouTube:

What we'll build

question
   │
   ▼
[ClassifyQuestion]  ← Nova Micro, temperature 0
   │  "simple" / "code" / "creative"
   ▼
[RouteByCategory]  (Choice state)
   ├─ *code*     ──▶ [AnswerWithHaiku]     ← Claude Haiku
   ├─ *creative* ──▶ [AnswerWithNovaLite]  ← Nova Lite
   └─ default    ──▶ [AnswerWithNovaMicro] ← Nova Micro

Step Functions has an optimized integration for Bedrock (arn:aws:states:::bedrock:invokeModel), so states call models directly — no Lambda in the path. Routing logic lives in a Choice state, retries and execution history come built in, and the visual workflow in the console doubles as documentation.

Note on model IDs: Bedrock models are updated frequently. The IDs below were current when this was written — check aws bedrock list-inference-profiles (or the console) and use the latest versions.

Step 1: IAM role

Create a role for Step Functions (e.g. StepFunctionsBedrockHandsOnRole, trusted by states.amazonaws.com) and attach this inline policy (IAM → Roles → the role → Add permissions → Create inline policy → JSON):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "bedrock:InvokeModel",
      "Resource": "*"
    }
  ]
}

(For production, scope Resource to the specific model and inference-profile ARNs.)

Step 2: The state machine

Create a state machine in the Step Functions console, switch to the Code view, and paste:

{
  "Comment": "Bedrock model routing handson",
  "StartAt": "ClassifyQuestion",
  "States": {
    "ClassifyQuestion": {
      "Type": "Task",
      "Resource": "arn:aws:states:::bedrock:invokeModel",
      "Parameters": {
        "ModelId": "amazon.nova-micro-v1:0",
        "Body": {
          "messages": [
            {
              "role": "user",
              "content": [
                {
                  "text.$": "States.Format('Classify the following question as exactly one word: simple, code, or creative. Do not output anything else. Question: {}', $.question)"
                }
              ]
            }
          ],
          "inferenceConfig": {
            "maxTokens": 10,
            "temperature": 0
          }
        }
      },
      "ResultSelector": {
        "category.$": "$.Body.output.message.content[0].text"
      },
      "ResultPath": "$.classification",
      "Next": "RouteByCategory"
    },
    "RouteByCategory": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.classification.category",
          "StringMatches": "*code*",
          "Next": "AnswerWithHaiku"
        },
        {
          "Variable": "$.classification.category",
          "StringMatches": "*creative*",
          "Next": "AnswerWithNovaLite"
        }
      ],
      "Default": "AnswerWithNovaMicro"
    },
    "AnswerWithNovaMicro": {
      "Type": "Task",
      "Resource": "arn:aws:states:::bedrock:invokeModel",
      "Parameters": {
        "ModelId": "amazon.nova-micro-v1:0",
        "Body": {
          "messages": [
            {
              "role": "user",
              "content": [
                { "text.$": "$.question" }
              ]
            }
          ],
          "inferenceConfig": { "maxTokens": 300 }
        }
      },
      "End": true
    },
    "AnswerWithHaiku": {
      "Type": "Task",
      "Resource": "arn:aws:states:::bedrock:invokeModel",
      "Parameters": {
        "ModelId": "us.anthropic.claude-haiku-4-5-20251001-v1:0",
        "Body": {
          "anthropic_version": "bedrock-2023-05-31",
          "max_tokens": 500,
          "messages": [
            {
              "role": "user",
              "content": [
                {
                  "type": "text",
                  "text.$": "$.question"
                }
              ]
            }
          ]
        }
      },
      "End": true
    },
    "AnswerWithNovaLite": {
      "Type": "Task",
      "Resource": "arn:aws:states:::bedrock:invokeModel",
      "Parameters": {
        "ModelId": "amazon.nova-lite-v1:0",
        "Body": {
          "messages": [
            {
              "role": "user",
              "content": [
                { "text.$": "$.question" }
              ]
            }
          ],
          "inferenceConfig": { "maxTokens": 500 }
        }
      },
      "End": true
    }
  }
}

Four details worth reading twice:

States.Format builds the prompt inside the state machine. The classification instruction wraps the user's question with an intrinsic function — prompt templating without a single line of code.
ResultSelector + ResultPath keep the original input. The classifier's raw output is trimmed down to $.classification.category, while the original $.question stays in the state — so the answering states can still read it. Forgetting ResultPath here would overwrite the question with the classifier's response.
StringMatches: "*code*" is deliberate slack. LLM classifiers don't always return exactly one clean token — wildcards tolerate whitespace, casing artifacts, or a stray period around the label.
Each provider gets its native request body. The direct invokeModel integration doesn't normalize formats the way the Converse API does: Nova takes messages + inferenceConfig, while Claude needs anthropic_version and max_tokens. If you swap in a different provider's model, swap the body shape too.

Step 3: Test the routes

State machine → Start execution, one case at a time:

{
  "question": "What is the capital of Japan?"
}

→ classified simple, answered by Nova Micro (default route).

{
  "question": "Write FizzBuzz in Python"
}

→ classified code, answered by Claude Haiku.

{
  "question": "Write a short story with a cat as the main character"
}

→ classified creative, answered by Nova Lite.

Open the execution's graph view and you can see the path light up through the classifier, the Choice state, and the selected model — routing you can literally look at.

When you're done, delete the state machine and the IAM role to clean up (both are free while idle, but tidy is tidy).

Wrapping up

A Choice state plus Bedrock's optimized integration turns model routing from an application-code problem into an infrastructure definition — versionable, visual, and with retries and execution history included. If your Bedrock bill is dominated by simple questions hitting an expensive model, this pattern is one JSON document away.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

🎥 Video version of this hands-on:
https://youtu.be/clZCoIgFMbw

📚 Full course — AWS Certified Generative AI Developer Professional (AIP-C01) Exam Prep:
https://www.udemy.com/course/aws-certified-generative-ai-developer-professional-exam-prep/

👨‍🏫 All my courses:
(Eng) https://www.udemy.com/user/maruchin-tech-2/
(Jpn) https://www.udemy.com/user/shan-wang-wan-jun-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/posts

Swap Your LLM Without a Deploy: Dynamic Model Routing on Bedrock with AWS AppConfig

Maruchin Tech — Sat, 22 Aug 2026 07:32:11 +0000

Bedrock model IDs have a short shelf life. New Claude, Nova, and Llama versions land every few months, pricing changes, and yesterday's best pick becomes today's legacy model. If the model ID is hardcoded in your Lambda, every swap is a code change, a review, and a redeploy — and if the new model misbehaves, rolling back is another deploy.

In this hands-on, we'll fix that by building a model router where the model choice lives in AWS AppConfig feature flags, not in code. A Lambda behind API Gateway reads the flags at runtime and routes each request to Claude Haiku, Amazon Nova Micro, or Meta Llama. Swapping a model becomes a config deployment: no code change, no redeploy, instant rollback.

Prefer video? This entire hands-on is also on YouTube:

What we'll build

curl ?model=fast ──▶ API Gateway ──▶ Lambda ──▶ Amazon Bedrock (Converse)
                                      │              ▲
                                      │   which model_id?
                                      ▼              │
                               AWS AppConfig ────────┘
                               feature flags:
                                 fast  → Claude Haiku
                                 cheap → Nova Micro
                                 open  → Llama

The client asks for a routing key (fast, cheap, open) — never a model ID. What each key means is decided by whoever controls the AppConfig deployment.

Why AppConfig?

AWS AppConfig is a managed feature-flag and configuration service. Three properties make it a good fit for LLM routing:

Config is deployed, not just saved. Changes go out through deployment strategies — all-at-once for dev, gradual rollouts with automatic rollback for production. A bad model swap can be rolled back the same way it went out.
Versioned history. Every flag set is a numbered version; you can see exactly which model was live when.
Runtime retrieval with sessions. Your code polls for the latest configuration and only receives content when something changed — cheap to call from a warm Lambda.

Prerequisites

AWS CLI v2 configured for us-east-1
Bedrock model access enabled for the Claude, Nova, and Llama models you plan to use
jq installed
A Lambda + API Gateway (HTTP API) you can create in the console

Step 1: Pick your models

Bedrock models are updated frequently — list what's currently available and use the latest versions, not the ones printed in this article.

# Anthropic Haiku family
aws bedrock list-inference-profiles \
  --region us-east-1 \
  --query 'inferenceProfileSummaries[?contains(inferenceProfileId, `haiku`)].inferenceProfileId' \
  --output table

# Amazon Nova Micro
aws bedrock list-inference-profiles \
  --region us-east-1 \
  --query 'inferenceProfileSummaries[?contains(inferenceProfileId, `nova-micro`)].inferenceProfileId' \
  --output table

# Meta Llama
aws bedrock list-inference-profiles \
  --region us-east-1 \
  --query 'inferenceProfileSummaries[?contains(inferenceProfileId, `llama`)].inferenceProfileId' \
  --output table

Export the ones you'll route between (replace with the versions listed in your account):

export CLAUDE_MODEL="us.anthropic.claude-haiku-4-5-20251001-v1:0"
export NOVA_MODEL="us.amazon.nova-micro-v1:0"
export LLAMA_MODEL="us.meta.llama4-scout-17b-instruct-v1:0"

echo "Claude: $CLAUDE_MODEL"
echo "Nova  : $NOVA_MODEL"
echo "Llama : $LLAMA_MODEL"

Step 2: Create and deploy the AppConfig flags

AppConfig has a small hierarchy: an application contains environments (dev, prod, …) and configuration profiles (the config itself). We create one of each, then a feature-flag document with three flags — each carrying a model_id attribute.

REGION=us-east-1

# 1. Application
APP_ID=$(aws appconfig create-application --region $REGION \
  --name bedrock-router --query Id --output text)

# 2. Environment
ENV_ID=$(aws appconfig create-environment --region $REGION \
  --application-id $APP_ID --name dev --query Id --output text)

# 3. Configuration Profile (feature flag type)
PROFILE_ID=$(aws appconfig create-configuration-profile --region $REGION \
  --application-id $APP_ID --name model-router \
  --location-uri hosted --type "AWS.AppConfig.FeatureFlags" \
  --query Id --output text)

# 4. Feature flags
jq -n \
  --arg c "$CLAUDE_MODEL" --arg n "$NOVA_MODEL" --arg l "$LLAMA_MODEL" \
'{
  flags: {
    fast:  {name:"fast",  attributes:{model_id:{constraints:{type:"string"}}}},
    cheap: {name:"cheap", attributes:{model_id:{constraints:{type:"string"}}}},
    open:  {name:"open",  attributes:{model_id:{constraints:{type:"string"}}}}
  },
  values: {
    fast:  {enabled:true, model_id:$c},
    cheap: {enabled:true, model_id:$n},
    open:  {enabled:true, model_id:$l}
  },
  version: "1"
}' > /tmp/flags.json

aws appconfig create-hosted-configuration-version --region $REGION \
  --application-id $APP_ID --configuration-profile-id $PROFILE_ID \
  --content-type "application/json" \
  --content fileb:///tmp/flags.json \
  /dev/null

# 5. Deploy (using the AWS predefined strategy AppConfig.AllAtOnce)
aws appconfig start-deployment --region $REGION \
  --application-id $APP_ID --environment-id $ENV_ID \
  --deployment-strategy-id AppConfig.AllAtOnce \
  --configuration-profile-id $PROFILE_ID \
  --configuration-version 1

echo "APP_ID=$APP_ID"
echo "ENV_ID=$ENV_ID"
echo "PROFILE_ID=$PROFILE_ID"

AppConfig.AllAtOnce is fine for a dev environment. In production you'd pick a gradual strategy (linear or canary) with a CloudWatch alarm attached, so a bad config rolls back automatically.

Verify what's deployed:

aws appconfig get-hosted-configuration-version --region $REGION \
  --application-id $APP_ID --configuration-profile-id $PROFILE_ID \
  --version-number 1 \
  /tmp/flags_out.json > /dev/null

cat /tmp/flags_out.json | jq

Step 3: Lambda execution role

Create a Lambda (Python, name it bedrock-router) in the console, then add this inline policy to its execution role (IAM → the role → Add permissions → Create inline policy → JSON):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BedrockConverse",
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AppConfigRead",
      "Effect": "Allow",
      "Action": [
        "appconfig:StartConfigurationSession",
        "appconfig:GetLatestConfiguration"
      ],
      "Resource": "*"
    }
  ]
}

(For production, scope Resource down to your specific models and AppConfig ARNs.)

Step 4: The Lambda function

import json
import os
import time
import boto3

REGION = os.environ["AWS_REGION"]
APP_ID = os.environ["APPCONFIG_APP_ID"]
ENV_ID = os.environ["APPCONFIG_ENV_ID"]
PROFILE_ID = os.environ["APPCONFIG_PROFILE_ID"]

appconfigdata = boto3.client("appconfigdata", region_name=REGION)
bedrock = boto3.client("bedrock-runtime", region_name=REGION)

# Simple cache to reduce AppConfig calls when the container is reused
_cache = {"config": None, "token": None, "expires_at": 0}
CACHE_TTL_SEC = 30


def _load_config():
    now = time.time()

    # Return the cached config while it is still valid
    if _cache["config"] is not None and now < _cache["expires_at"]:
        return _cache["config"]

    # Start a session only on the first call
    if _cache["token"] is None:
        session = appconfigdata.start_configuration_session(
            ApplicationIdentifier=APP_ID,
            EnvironmentIdentifier=ENV_ID,
            ConfigurationProfileIdentifier=PROFILE_ID,
        )
        _cache["token"] = session["InitialConfigurationToken"]

    resp = appconfigdata.get_latest_configuration(
        ConfigurationToken=_cache["token"]
    )
    _cache["token"] = resp["NextPollConfigurationToken"]

    content = resp["Configuration"].read()
    if content:
        # Replace only when there is an update. Keep the current cache if the content is empty
        _cache["config"] = json.loads(content)

    _cache["expires_at"] = now + CACHE_TTL_SEC
    return _cache["config"]


def lambda_handler(event, context):
    try:
        flags = _load_config()  # Feature flag value map: {"fast":{"enabled":true,"model_id":"..."}, ...}

        qs = event.get("queryStringParameters") or {}
        model_key = qs.get("model", "fast")  # Default is fast
        prompt = qs.get("prompt", "Hello. Please introduce yourself in one sentence.")

        flag = flags.get(model_key)
        if not flag or not flag.get("enabled"):
            return {
                "statusCode": 400,
                "headers": {"Content-Type": "application/json; charset=utf-8"},
                "body": json.dumps(
                    {
                        "error": f"model key not available: {model_key}",
                        "available_keys": [k for k, v in flags.items() if v.get("enabled")],
                    },
                    ensure_ascii=False,
                ),
            }

        model_id = flag["model_id"]

        resp = bedrock.converse(
            modelId=model_id,
            messages=[{"role": "user", "content": [{"text": prompt}]}],
            inferenceConfig={"maxTokens": 300, "temperature": 0.5},
        )

        text = resp["output"]["message"]["content"][0]["text"]

        return {
            "statusCode": 200,
            "headers": {"Content-Type": "application/json; charset=utf-8"},
            "body": json.dumps(
                {
                    "model_key": model_key,
                    "model_id": model_id,
                    "prompt": prompt,
                    "response": text,
                    "usage": resp.get("usage", {}),
                },
                ensure_ascii=False,
            ),
        }

    except Exception as e:
        return {
            "statusCode": 500,
            "headers": {"Content-Type": "application/json; charset=utf-8"},
            "body": json.dumps(
                {"error": type(e).__name__, "message": str(e)},
                ensure_ascii=False,
            ),
        }

Three details worth reading twice:

The session/token dance. appconfigdata works as a polling session: start_configuration_session once, then get_latest_configuration with a token that gets replaced on every call. If nothing changed since the last poll, the response body is empty — that's why the code only overwrites the cache if content:.
The 30-second cache. Warm Lambda containers keep module-level state, so _cache survives between invocations. You get near-instant responses and at most one AppConfig poll per 30 seconds per container.
Unknown keys fail loudly. A key that doesn't exist (or is disabled) returns 400 with the list of available keys, instead of silently falling back to an expensive model.

Set the Lambda's environment variables (Configuration tab → Environment variables → Edit) with the values printed in Step 2:

echo "APPCONFIG_APP_ID    = $APP_ID"
echo "APPCONFIG_ENV_ID    = $ENV_ID"
echo "APPCONFIG_PROFILE_ID= $PROFILE_ID"

Quick unit test (Test tab → Event name: test1 → Event JSON):

{
  "queryStringParameters": {
    "model": "cheap",
    "prompt": "Please introduce yourself in three lines"
  }
}

Step 5: Put API Gateway in front and test

Create an HTTP API (name it bedrock-router-api) with a /chat route integrated with the Lambda, then:

# Change the URL below to match your environment
export API_URL="https://abc123xyz.execute-api.us-east-1.amazonaws.com"
echo "$API_URL/chat"

Route to each model by key:

# Call Claude Haiku (fast)
curl -s -G "$API_URL/chat" -d "model=fast" --data-urlencode "prompt=What is generative AI, in three lines" | jq

# Call Nova Micro (cheap)
curl -s -G "$API_URL/chat" -d "model=cheap" --data-urlencode "prompt=What is generative AI, in three lines" | jq

# Call Llama (open)
curl -s -G "$API_URL/chat" -d "model=open" --data-urlencode "prompt=What is generative AI, in three lines" | jq

# If no key is specified, the default (fast) is used
curl -s -G "$API_URL/chat" --data-urlencode "prompt=Hello" | jq '.model_key, .model_id'

# Check error handling for an invalid key
curl -s -G "$API_URL/chat" -d "model=unknown" --data-urlencode "prompt=test" | jq

Same endpoint, three different models, chosen by a query parameter.

Step 6: Swap a model with zero deploys

Now the payoff. Suppose Claude Haiku is overkill for the fast route and you want Nova Micro there too. Create version 2 of the flags — note fast now carries $n — and deploy it:

REGION=us-east-1

jq -n \
  --arg c "$CLAUDE_MODEL" --arg n "$NOVA_MODEL" --arg l "$LLAMA_MODEL" \
'{
  flags: {
    fast:  {name:"fast",  attributes:{model_id:{constraints:{type:"string"}}}},
    cheap: {name:"cheap", attributes:{model_id:{constraints:{type:"string"}}}},
    open:  {name:"open",  attributes:{model_id:{constraints:{type:"string"}}}}
  },
  values: {
    fast:  {enabled:true, model_id:$n},
    cheap: {enabled:true, model_id:$n},
    open:  {enabled:true, model_id:$l}
  },
  version: "1"
}' > /tmp/flags_v2.json

aws appconfig create-hosted-configuration-version --region $REGION \
  --application-id $APP_ID --configuration-profile-id $PROFILE_ID \
  --content-type "application/json" \
  --content fileb:///tmp/flags_v2.json \
  /dev/null

aws appconfig start-deployment --region $REGION \
  --application-id $APP_ID --environment-id $ENV_ID \
  --deployment-strategy-id AppConfig.AllAtOnce \
  --configuration-profile-id $PROFILE_ID \
  --configuration-version 2

Wait for the cache TTL (up to ~30 seconds), then:

# fast should now be Nova Micro
curl -s -G "$API_URL/chat" -d "model=fast" --data-urlencode "prompt=Introduce yourself" | jq '.model_key, .model_id'

The Lambda never changed. No deploy, no cold start, no release process — the model behind fast is now a different one, and deploying version 1 again would roll it back just as fast.

Cleanup

# API Gateway (look up the API ID and delete)
API_ID=$(aws apigatewayv2 get-apis --region us-east-1 \
  --query 'Items[?Name==`bedrock-router-api`].ApiId' --output text 2>/dev/null)
if [ -n "$API_ID" ] && [ "$API_ID" != "None" ]; then
  aws apigatewayv2 delete-api --region us-east-1 --api-id "$API_ID"
fi

# Lambda
aws lambda delete-function --region us-east-1 --function-name bedrock-router 2>/dev/null

# IAM role (inline policy first, then the role itself)
aws iam delete-role-policy \
  --role-name bedrock-router-role \
  --policy-name bedrock-router-inline 2>/dev/null
aws iam detach-role-policy \
  --role-name bedrock-router-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole 2>/dev/null
aws iam delete-role --role-name bedrock-router-role 2>/dev/null

# AppConfig (child resources first)
if [ -n "$APP_ID" ]; then
  aws appconfig delete-environment --region us-east-1 \
    --application-id "$APP_ID" --environment-id "$ENV_ID" 2>/dev/null
  aws appconfig delete-configuration-profile --region us-east-1 \
    --application-id "$APP_ID" --configuration-profile-id "$PROFILE_ID" 2>/dev/null
  aws appconfig delete-application --region us-east-1 --application-id "$APP_ID" 2>/dev/null
fi

Wrapping up

Separating "which model" from "the code that calls it" is not a convenience — it's how you keep an LLM application operable in a market where models are replaced every few months. AppConfig gives that separation deployment strategies, version history, and rollback for free. If your Bedrock model IDs live in code today, I recommend trying this pattern in your own environment.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

🎥 Video version of this hands-on:
https://youtu.be/6k2lO4_fA7o

📚 Full course — AWS Certified Generative AI Developer Professional (AIP-C01) Exam Prep:
https://www.udemy.com/course/aws-certified-generative-ai-developer-professional-exam-prep/

👨‍🏫 All my courses (Udemy profile):
https://www.udemy.com/user/maruchin-tech-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/posts

Stop Hardcoding Prompts: Prompt Management and Prompt Flows on Amazon Bedrock (Hands-On)

Maruchin Tech — Fri, 21 Aug 2026 17:16:04 +0000

If your team is building LLM applications, your prompts are probably scattered across application code as string literals. Changing a single word means a code change, a review, and a redeploy — and nobody can tell which prompt version is actually running in production. As LLM apps move from prototypes to production systems, prompt sprawl becomes a real operational problem.

In this hands-on, we'll fix that with two Amazon Bedrock features — Prompt Management and Prompt Flows — by building a customer-support pipeline that classifies an inquiry, routes it, and answers it with the right model. Entirely from the AWS CLI, every command included.

Prefer video? This entire hands-on is also on YouTube:

What we'll build

Customer inquiry
      │
      ▼
 [Classifier prompt]  ← Amazon Nova Micro (cheap, temperature 0)
      │
      ▼
   [Router]  ── TECH ──▶ [Tech answer prompt]    ← Claude (accurate)
      │
      └── everything else ──▶ [General answer prompt] ← Nova Micro (cheap)

Note the cost design: the classification and general answers run on Amazon Nova Micro (fast and inexpensive), while only genuinely technical questions reach Claude. Routing cheap traffic away from your most expensive model is one of the most practical cost optimizations in production LLM systems.

What is Prompt Management?

Prompt Management turns a prompt into a first-class AWS resource. Instead of a string in your code, a prompt becomes an object with its own ARN, and it carries everything needed to run it:

A template with variables — e.g. {{question}} — filled in at invocation time
A bound model — each prompt knows which model it runs on (modelId)
Inference configuration — temperature, max tokens, etc., stored with the prompt
Versions — an editable DRAFT plus immutable numbered snapshots

The consequences matter more than the mechanics. Your application code no longer contains prompt text at all — it references a prompt ARN. Prompt engineers can iterate on the DRAFT without touching application code, while production keeps calling a pinned version. Swapping the underlying model for a prompt is a configuration change, not a code change.

What is Prompt Flows?

Prompt Flows is Bedrock's serverless orchestration layer for multi-step LLM pipelines. A flow is a graph of nodes — Input, Prompt, Condition, Output, and others — connected by data and conditional edges. The flow engine executes the graph for you: no Lambda glue code, no Step Functions state machine to maintain.

Flows also get the same lifecycle treatment as prompts: you version a flow and expose it through aliases (e.g. a prod alias pinned to version 1), so you can rewire the pipeline behind a stable identifier without redeploying callers.

Together, the two features give you something teams usually hand-roll: a prompt registry with versioning, plus a managed execution engine with routing — both callable through standard AWS APIs and IAM.

Prerequisites

AWS CLI v2 configured, with access to Bedrock in us-east-1
Model access enabled for the Claude and Nova models in the Bedrock console
jq and python3 (with boto3) installed

Step 1: Prompt Management

Set up environment variables

Bedrock models are updated frequently — check the console and use the latest versions.

export CLAUDE_MODEL="us.anthropic.claude-haiku-4-5-20251001-v1:0"
export NOVA_MODEL="us.amazon.nova-micro-v1:0"
export REGION=us-east-1

Create the prompts

First, the classifier. Temperature 0 and a 10-token cap: we want a deterministic label, nothing else.

CLASSIFIER_ID=$(aws bedrock-agent create-prompt --region $REGION \
  --name customer-classifier \
  --description "Classify customer inquiries as TECH or GENERAL" \
  --default-variant v1 \
  --variants '[{
    "name":"v1",
    "templateType":"TEXT",
    "templateConfiguration":{"text":{
      "text":"Classify the following customer inquiry into exactly one of the following. Respond with the label only and nothing else.\n\n- TECH: technical product issues, configuration, errors, how to use the product\n- GENERAL: pricing, contracts, business hours, and other general questions\n\nInquiry: {{question}}\n\nLabel:",
      "inputVariables":[{"name":"question"}]
    }},
    "modelId":"'"$NOVA_MODEL"'",
    "inferenceConfiguration":{"text":{"temperature":0.0,"maxTokens":10}}
  }]' \
  --query 'id' --output text)

echo "CLASSIFIER_ID=$CLASSIFIER_ID"

Next, the technical answerer — this one runs on Claude, with a lower temperature for accuracy:

TECH_ID=$(aws bedrock-agent create-prompt --region $REGION \
  --name customer-tech-answer \
  --description "Answer technical inquiries with Claude" \
  --default-variant v1 \
  --variants '[{
    "name":"v1",
    "templateType":"TEXT",
    "templateConfiguration":{"text":{
      "text":"You are a technical support representative for our products. Answer the following technical inquiry concisely and accurately.\n\nInquiry: {{question}}\n\nAnswer:",
      "inputVariables":[{"name":"question"}]
    }},
    "modelId":"'"$CLAUDE_MODEL"'",
    "inferenceConfiguration":{"text":{"temperature":0.3,"maxTokens":400}}
  }]' \
  --query 'id' --output text)

echo "TECH_ID=$TECH_ID"

And the general answerer, back on Nova Micro:

GENERAL_ID=$(aws bedrock-agent create-prompt --region $REGION \
  --name customer-general-answer \
  --description "Answer general inquiries with Nova" \
  --default-variant v1 \
  --variants '[{
    "name":"v1",
    "templateType":"TEXT",
    "templateConfiguration":{"text":{
      "text":"You are a customer support representative for our company. Answer the following general inquiry politely and helpfully.\n\nInquiry: {{question}}\n\nAnswer:",
      "inputVariables":[{"name":"question"}]
    }},
    "modelId":"'"$NOVA_MODEL"'",
    "inferenceConfiguration":{"text":{"temperature":0.5,"maxTokens":400}}
  }]' \
  --query 'id' --output text)

echo "GENERAL_ID=$GENERAL_ID"

Three prompts, three independent lifecycles, two different models — and not a single line of application code yet.

Invoke a managed prompt directly

Here's the detail that surprises most people: the Converse API accepts a prompt ARN as the model ID. The prompt brings its own model and inference settings; you only supply the variables.

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
CLASSIFIER_ARN="arn:aws:bedrock:$REGION:$ACCOUNT_ID:prompt/$CLASSIFIER_ID"

aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN" \
  --prompt-variables '{"question":{"text":"I cannot log in. I want to reset my password."}}' \
  | jq -r '.output.message.content[0].text'

Expected output: TECH

aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN" \
  --prompt-variables '{"question":{"text":"How much is the monthly fee?"}}' \
  | jq -r '.output.message.content[0].text'

Expected output: GENERAL

Versioning: the DRAFT / pinned-version workflow

Every prompt has an editable DRAFT. create-prompt-version freezes the current DRAFT into an immutable numbered version:

CLASSIFIER_V1=$(aws bedrock-agent create-prompt-version --region $REGION \
  --prompt-identifier $CLASSIFIER_ID \
  --description "Production version" \
  --query version --output text)

echo "CLASSIFIER_V1=$CLASSIFIER_V1"   # -> 1

TECH_V1=$(aws bedrock-agent create-prompt-version --region $REGION \
  --prompt-identifier $TECH_ID --query version --output text)
GENERAL_V1=$(aws bedrock-agent create-prompt-version --region $REGION \
  --prompt-identifier $GENERAL_ID --query version --output text)
echo "TECH_V1=$TECH_V1  GENERAL_V1=$GENERAL_V1"

The ARN convention does the rest. Append :1 for the pinned production version; omit the suffix to hit the DRAFT:

# Production: pinned to version 1
aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN:$CLASSIFIER_V1" \
  --prompt-variables '{"question":{"text":"I want to reset my password"}}' \
  | jq -r '.output.message.content[0].text'

# Testing: the DRAFT, with whatever edits are in flight
aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN" \
  --prompt-variables '{"question":{"text":"I want to reset my password"}}' \
  | jq -r '.output.message.content[0].text'

Let's prove the isolation. Rewrite the DRAFT to add a third label, BILLING:

aws bedrock-agent update-prompt --region $REGION \
  --prompt-identifier $CLASSIFIER_ID \
  --name customer-classifier \
  --default-variant v1 \
  --variants '[{
    "name":"v1",
    "templateType":"TEXT",
    "templateConfiguration":{"text":{
      "text":"Classify the following customer inquiry into exactly one of the following. Respond with the label only and nothing else.\n\n- TECH: technical product issues, configuration, errors, how to use the product\n- BILLING: pricing, invoices, payment methods\n- GENERAL: everything else\n\nInquiry: {{question}}\n\nLabel:",
      "inputVariables":[{"name":"question"}]
    }},
    "modelId":"'"$NOVA_MODEL"'",
    "inferenceConfiguration":{"text":{"temperature":0.0,"maxTokens":10}}
  }]'

Now ask the same billing question twice:

# DRAFT — picks up the new behavior
aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN" \
  --prompt-variables '{"question":{"text":"How much is the monthly fee?"}}' \
  | jq -r '.output.message.content[0].text'
# -> BILLING

# Version 1 — production is untouched
aws bedrock-runtime converse --region $REGION \
  --model-id "$CLASSIFIER_ARN:1" \
  --prompt-variables '{"question":{"text":"How much is the monthly fee?"}}' \
  | jq -r '.output.message.content[0].text'
# -> GENERAL

Same prompt resource, two behaviors, zero risk to production. This is the workflow Prompt Management exists for.

Step 2: Prompt Flows

Now we wire the three prompts into one pipeline.

Create the execution role

A flow runs under its own IAM role, which needs permission to invoke models and read the prompts:

cat > /tmp/flow-trust.json << 'EOF'
{
  "Version":"2012-10-17",
  "Statement":[{"Effect":"Allow","Principal":{"Service":"bedrock.amazonaws.com"},"Action":"sts:AssumeRole"}]
}
EOF

FLOW_ROLE_ARN=$(aws iam create-role \
  --role-name bedrock-flow-role \
  --assume-role-policy-document file:///tmp/flow-trust.json \
  --query 'Role.Arn' --output text)

cat > /tmp/flow-policy.json << 'EOF'
{
  "Version":"2012-10-17",
  "Statement":[
    {"Effect":"Allow","Action":["bedrock:InvokeModel","bedrock:Converse","bedrock:GetPrompt","bedrock:RenderPrompt","bedrock:GetInferenceProfile"],"Resource":"*"}
  ]
}
EOF

aws iam put-role-policy \
  --role-name bedrock-flow-role \
  --policy-name bedrock-flow-inline \
  --policy-document file:///tmp/flow-policy.json

echo "FLOW_ROLE_ARN=$FLOW_ROLE_ARN"
sleep 10   # wait for the role to propagate

(For production, scope Resource down to the specific prompt and model ARNs.)

Define the flow as JSON

A flow definition has two halves: nodes (the boxes) and connections (the arrows). Ours has an Input node, the three Prompt nodes referencing the pinned production versions of our prompts, a Condition node for routing, and two Output nodes:

CLASSIFIER_PROD_ARN="$CLASSIFIER_ARN:$CLASSIFIER_V1"
TECH_PROD_ARN="arn:aws:bedrock:$REGION:$ACCOUNT_ID:prompt/$TECH_ID:$TECH_V1"
GENERAL_PROD_ARN="arn:aws:bedrock:$REGION:$ACCOUNT_ID:prompt/$GENERAL_ID:$GENERAL_V1"

jq -n \
  --arg cls "$CLASSIFIER_PROD_ARN" \
  --arg tech "$TECH_PROD_ARN" \
  --arg gen "$GENERAL_PROD_ARN" \
'{
  nodes: [
    { name:"FlowInputNode", type:"Input",
      configuration:{input:{}},
      outputs:[{name:"document", type:"String"}] },

    { name:"Classifier", type:"Prompt",
      configuration:{prompt:{sourceConfiguration:{resource:{promptArn:$cls}}}},
      inputs:[{name:"question", type:"String", expression:"$.data"}],
      outputs:[{name:"modelCompletion", type:"String"}] },

    { name:"Router", type:"Condition",
      configuration:{condition:{conditions:[
        {name:"isTech", expression:"classification == \"TECH\""},
        {name:"default"}
      ]}},
      inputs:[{name:"classification", type:"String", expression:"$.data"}] },

    { name:"TechAnswer", type:"Prompt",
      configuration:{prompt:{sourceConfiguration:{resource:{promptArn:$tech}}}},
      inputs:[{name:"question", type:"String", expression:"$.data"}],
      outputs:[{name:"modelCompletion", type:"String"}] },

    { name:"GeneralAnswer", type:"Prompt",
      configuration:{prompt:{sourceConfiguration:{resource:{promptArn:$gen}}}},
      inputs:[{name:"question", type:"String", expression:"$.data"}],
      outputs:[{name:"modelCompletion", type:"String"}] },

    { name:"TechOutputNode", type:"Output",
      configuration:{output:{}},
      inputs:[{name:"document", type:"String", expression:"$.data"}] },

    { name:"GeneralOutputNode", type:"Output",
      configuration:{output:{}},
      inputs:[{name:"document", type:"String", expression:"$.data"}] }
  ],
  connections: [
    {name:"i2c", source:"FlowInputNode", target:"Classifier",         type:"Data",
      configuration:{data:{sourceOutput:"document", targetInput:"question"}}},
    {name:"i2t", source:"FlowInputNode", target:"TechAnswer",         type:"Data",
      configuration:{data:{sourceOutput:"document", targetInput:"question"}}},
    {name:"i2g", source:"FlowInputNode", target:"GeneralAnswer",      type:"Data",
      configuration:{data:{sourceOutput:"document", targetInput:"question"}}},
    {name:"c2r", source:"Classifier",    target:"Router",             type:"Data",
      configuration:{data:{sourceOutput:"modelCompletion", targetInput:"classification"}}},
    {name:"r2t", source:"Router",        target:"TechAnswer",         type:"Conditional",
      configuration:{conditional:{condition:"isTech"}}},
    {name:"r2g", source:"Router",        target:"GeneralAnswer",      type:"Conditional",
      configuration:{conditional:{condition:"default"}}},
    {name:"t2o", source:"TechAnswer",    target:"TechOutputNode",     type:"Data",
      configuration:{data:{sourceOutput:"modelCompletion", targetInput:"document"}}},
    {name:"g2o", source:"GeneralAnswer", target:"GeneralOutputNode",  type:"Data",
      configuration:{data:{sourceOutput:"modelCompletion", targetInput:"document"}}}
  ]
}' > /tmp/flow-def.json

cat /tmp/flow-def.json | jq '.nodes[].name, .connections[].name'

Two details worth noticing:

Data edges vs. conditional edges. The question flows to all three prompt nodes over Data connections, but TechAnswer and GeneralAnswer only execute when the Conditional edge from the Router fires. The Condition node gates execution, not data.
The Router's expression (classification == "TECH") compares the classifier's raw output — which is exactly why we forced the classifier to answer with the label only.

Create, prepare, and test the flow

FLOW_ID=$(aws bedrock-agent create-flow --region $REGION \
  --name customer-support-flow \
  --description "Classify -> route -> answer" \
  --execution-role-arn "$FLOW_ROLE_ARN" \
  --definition file:///tmp/flow-def.json \
  --query 'id' --output text)

echo "FLOW_ID=$FLOW_ID"

# Compile the flow into an executable state
aws bedrock-agent prepare-flow --region $REGION \
  --flow-identifier $FLOW_ID

invoke_flow streams events, so we test from Python. TSTALIASID is the built-in alias that always points at the working draft of the flow:

import boto3

client = boto3.client("bedrock-agent-runtime", region_name="us-east-1")

def ask(question):
    resp = client.invoke_flow(
        flowIdentifier="<FLOW_ID>",
        flowAliasIdentifier="TSTALIASID",
        inputs=[{
            "nodeName": "FlowInputNode",
            "nodeOutputName": "document",
            "content": {"document": question}
        }]
    )
    for event in resp["responseStream"]:
        if "flowOutputEvent" in event:
            out = event["flowOutputEvent"]
            print(f"  [Output node: {out['nodeName']}]")
            print("  " + out["content"]["document"].replace("\n", "\n  "))

print("=== Technical question ===")
ask("The app crashes as soon as I launch it. What could be the cause?")

print()
print("=== General question ===")
ask("What are the support center's business hours?")

The technical question comes back from TechOutputNode (answered by Claude), the business-hours question from GeneralOutputNode (answered by Nova). The routing works.

Flow versions and aliases

Flows version exactly like prompts — and aliases give callers a stable name:

FLOW_V1=$(aws bedrock-agent create-flow-version --region $REGION \
  --flow-identifier $FLOW_ID \
  --description "Production release v1" \
  --query version --output text)

echo "FLOW_V1=$FLOW_V1"   # -> 1

PROD_ALIAS_ID=$(aws bedrock-agent create-flow-alias --region $REGION \
  --flow-identifier $FLOW_ID \
  --name prod \
  --routing-configuration '[{"flowVersion":"'"$FLOW_V1"'"}]' \
  --query id --output text)

echo "PROD_ALIAS_ID=$PROD_ALIAS_ID"

Production callers use the prod alias and never change, even when you later repoint the alias at version 2:

import boto3
client = boto3.client("bedrock-agent-runtime", region_name="us-east-1")
resp = client.invoke_flow(
    flowIdentifier="<FLOW_ID>",
    flowAliasIdentifier="<PROD_ALIAS_ID>",
    inputs=[{
        "nodeName": "FlowInputNode",
        "nodeOutputName": "document",
        "content": {"document": "How do I get a receipt issued?"}
    }]
)
for event in resp["responseStream"]:
    if "flowOutputEvent" in event:
        out = event["flowOutputEvent"]
        print(f"[{out['nodeName']}]")
        print(out["content"]["document"])

Cleanup

Delete child resources before parents:

# Flow (alias -> version -> flow)
aws bedrock-agent delete-flow-alias --region $REGION \
  --flow-identifier $FLOW_ID --alias-identifier $PROD_ALIAS_ID
aws bedrock-agent delete-flow-version --region $REGION \
  --flow-identifier $FLOW_ID --flow-version $FLOW_V1
aws bedrock-agent delete-flow --region $REGION \
  --flow-identifier $FLOW_ID --skip-resource-in-use-check

# Prompts
for P in $CLASSIFIER_ID $TECH_ID $GENERAL_ID; do
  aws bedrock-agent delete-prompt --region $REGION --prompt-identifier $P
done

# IAM
aws iam delete-role-policy --role-name bedrock-flow-role --policy-name bedrock-flow-inline
aws iam delete-role --role-name bedrock-flow-role

Wrapping up

Prompt Management and Prompt Flows are not just convenience features — they move your prompts and your LLM pipeline out of application code and into versioned, IAM-governed AWS resources. The DRAFT/version split gives prompt engineering a safe test-in-production workflow, and flow aliases let you re-architect a pipeline behind a stable endpoint. If you run LLM workloads on AWS, I recommend validating this pattern in your own environment.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

🎥 Video version of this hands-on:
https://youtu.be/iyK_xK3G-i0

📚 Full course — AWS Certified Generative AI Developer Professional (AIP-C01) Exam Prep:
https://www.udemy.com/course/aws-certified-generative-ai-developer-professional-exam-prep/

👨‍🏫 All my courses (Udemy profile):
https://www.udemy.com/user/maruchin-tech-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/posts

How I Passed All 12 AWS Certifications in 1 Year (My "Reverse" Strategy)

Maruchin Tech — Fri, 21 Aug 2026 17:14:39 +0000

Most people attack AWS certifications bottom-up: start with Cloud Practitioner, climb slowly toward Professional. Most people also stall somewhere in the middle — the gap between each level feels enormous, and motivation runs out before the hard exams even start.

I did the opposite. Between August 2024 and 2025, I passed all 12 AWS certifications in about a year — by taking the hardest one first. Here's the full strategy, the exact order, difficulty rankings from someone who sat every exam, and what it actually cost.

The "reverse" strategy: hardest first

My first exam was Solutions Architect – Professional (SAP) — widely considered the hardest AWS certification, with the broadest service coverage.

The logic: if you clear the exam with the widest scope first, every remaining exam becomes a subset. Each subsequent certification feels psychologically smaller, not bigger. Instead of climbing toward a summit that keeps growing, you descend from it. Momentum works for you instead of against you.

My exact order

SAP (Aug 2024) → SAA → DEA → AIF → MLA → SCS → DVA → SOA → DOP → ANS → MLS → CLF

Yes, Cloud Practitioner came last. By then it was a victory lap — though not a free one (more on that below).

Difficulty rankings — from someone who sat all 12

Foundational

CLF (Cloud Practitioner) — Not trivial, even with 11 certs behind me. The frameworks, support plans, and contract topics need dedicated study; they don't appear anywhere else.
AIF (AI Practitioner) — Harder than people expect. AI knowledge is independent from infrastructure knowledge; infrastructure veterans start from zero here.

Associate

DVA (Developer) — Personally the hardest Associate exam for me.
SAA (Solutions Architect) — Noticeably harder in recent versions than its reputation suggests.
SOA (SysOps/CloudOps) — My best result: a perfect 1000.
DEA (Data Engineer) — Smooth if you have a data background.
MLA (Machine Learning Engineer) — Took me the longest at this level; be ready for deep SageMaker coverage.

Professional

SAP (Solutions Architect Pro) — The widest scope of any exam. As exam #1, it was brutal — which was exactly the point.
DOP (DevOps Pro) — My hardest certification overall.

Specialty

SCS (Security) — Moderate if you have security design experience.
ANS (Advanced Networking) — The hardest Specialty: complex multi-region, hybrid scenarios.
MLS (Machine Learning) — Retired in April 2025, so I'm glad I caught it in time.

What it cost

Exam fees alone came to roughly ¥300,000–400,000 (about $2,000–2,700) before tax. One important cost hack: AWS gives you a 50% discount voucher every time you pass, so each exam effectively halves the next one. Chain them and the total drops dramatically — one more reason the momentum-based approach pays off.

How I actually studied

Shift from input to output. Early in each exam's prep, I leaned on video courses and books. As the exam approached, I shifted almost entirely to practice exams. The ratio moves from mostly-input to mostly-output — staying in input mode too long is the most common way to stall.

Use English resources. Even if English isn't your first language (it isn't mine), the freshest documentation, exam guides, and community intel are in English. Limiting yourself to your native language caps your ceiling.

Aim for 80%, not 100%. Perfect knowledge of AWS is impossible — the platform is too big and moves too fast. Around 80% comprehension is enough to pass, and chasing the last 20% costs more than it returns.

When motivation dies, sleep. Not a joke. I treated rest as part of the study plan. Grinding through burnout produces worse retention than stopping.

A warning about paper certs. All 12 badges without hands-on experience will hurt you in the field. I built and operated real systems alongside the exams — the certifications organize knowledge; they don't replace it.

Wrapping up

The reverse strategy is not just a stunt — it's a momentum design. Front-load the pain while motivation is highest, then let every following exam feel smaller. If you're planning a multi-certification run, seriously consider starting closer to the top than the bottom.

About the author

Maruchin Tech — 12x AWS Certified | Cloud & AI for manufacturing and supply chain (AWS / Google Cloud / Azure) | Udemy instructor (100K+ students)

📚 My AWS certification courses (Udemy profile):
https://www.udemy.com/user/maruchin-tech-2/

🎫 Monthly discount coupons:
https://www.youtube.com/@MaruchinTech-cloud/community