Amazon Bedrock Service Tiers: Priority vs Flex

The Problem: Every Request Gets the Same Tier

When I first saw Bedrock's new service tiers, I thought the decision was going to be
pretty straightforward: use Priority when latency matters, Default for normal traffic,
and Flex for anything that can wait.

But before putting that logic into a router, I wanted to answer a simpler question:
does Priority actually make my requests faster?

The pricing table makes the case seem obvious — Priority costs roughly 1.75× Default,
Flex costs roughly 0.5× Default. The implication baked into that pricing is that you
get something proportional in return. I didn't want to build a router on top of that
assumption without testing it first. So I tested it.

The results changed how I thought about the whole problem.

---

Why This Experiment, Why Now

AWS documentation tells you what each tier is intended for. Pricing tells you the
obvious economic incentive. But neither tells you what happens with your specific
workload, region, and traffic pattern.

I didn't want to build another "Flex is for batch, Priority is for real-time" demo.
I wanted to see what actually happened when I ran real requests through all three
tiers, modeled the economics across different traffic distributions, and built a
routing layer I could actually read, reason about, and change without guessing at
the consequences.

That is what led to this project: a measurable, reproducible experiment rather than
a confident assumption presented as a tutorial.

---

Architecture: Four Small Modules, One Clean Pipeline

I intentionally kept the router boring.

No LangChain. No LLM classifier. No abstraction layers. Just four small Python
modules doing one thing each: validate the workload label, choose a tier, check
whether that tier is available, and return a decision. I wanted to look at any
routing outcome and immediately understand why it happened.

classify → policy → capability check → routing decision

Component responsibilities:

Module	Role
router/classifier.py	Validates and normalizes workload category strings
router/policy.py	Maps workload categories to preferred service tiers
router/capability.py	Checks tier availability; selects deterministic fallbacks
router/router.py	Orchestrates classify → policy → capability → decision

Why four modules instead of one function? I split the concerns because the
policy is the part I expect to change most often. By isolating it in a frozen
dataclass, I can swap in a different policy — database-backed, A/B split,
per-tenant — without touching the capability checker or the orchestrator.

AWS services used:

• Amazon Bedrock Runtime (bedrock-runtime) — the converse API with the serviceTier parameter. This is how you specify which tier to use; omitting it defaults to Default.
• boto3 — the AWS SDK for Python.

Why not a framework? The routing logic is a pure decision function. Adding
LangChain or a similar framework would add indirection without adding value. The
router is a component that plugs into your existing Bedrock call, not a
replacement for it.

---

Implementation Walkthrough

Step 1: Validate the workload category

I wanted invalid workload labels to fail early rather than silently becoming a
potentially expensive routing decision.

# router/classifier.py

class WorkloadCategory(str, Enum):
    CRITICAL    = "critical"
    INTERACTIVE = "interactive"
    BACKGROUND  = "background"


def classify_workload(workload_category: str | WorkloadCategory) -> WorkloadCategory:
    if workload_category is None:
        raise InvalidWorkloadCategoryError("Workload category is required.")

    if isinstance(workload_category, WorkloadCategory):
        return workload_category

    normalized = workload_category.strip().lower()

    try:
        return WorkloadCategory(normalized)
    except ValueError as exc:
        valid = ", ".join(c.value for c in WorkloadCategory)
        raise InvalidWorkloadCategoryError(
            f"Unknown workload category: {workload_category!r}. "
            f"Expected one of: {valid}."
        ) from exc

The str enum base class means a WorkloadCategory behaves like a string in
comparisons and CSV output — no .value everywhere.

Step 2: Apply the routing policy

Keeping the policy in its own module was deliberate. I expect this to be the part
I change most often as I learn more about the actual workload.

# router/policy.py

@dataclass(frozen=True)
class RoutingPolicy:
    mapping: dict[WorkloadCategory, ServiceTier]

    @classmethod
    def default(cls) -> "RoutingPolicy":
        return cls(
            mapping={
                WorkloadCategory.CRITICAL:    ServiceTier.PRIORITY,
                WorkloadCategory.INTERACTIVE: ServiceTier.DEFAULT,
                WorkloadCategory.BACKGROUND:  ServiceTier.FLEX,
            }
        )

    def preferred_tier(self, workload: WorkloadCategory) -> ServiceTier:
        try:
            return self.mapping[workload]
        except KeyError as exc:
            raise UnsupportedPolicyError(
                f"No routing policy exists for workload: {workload!r}."
            ) from exc

Step 3: Check capability and resolve fallbacks

This ended up being more important than I initially expected. Choosing a tier is
one thing; actually being able to use it is another. The capability checker
resolves to an available fallback using a configured chain:

# router/capability.py (fallback defaults)
#   priority → default
#   flex     → default
#   default  → no fallback (RuntimeError if unavailable)

def resolve(self, requested_tier: str | ServiceTier) -> tuple[ServiceTier, bool]:
    requested = self.normalize_tier(requested_tier)

    if self.is_available(requested):
        return requested, False       # (resolved_tier, used_fallback)

    fallback = self._fallback_order.get(requested)

    if fallback is not None and self.is_available(fallback):
        return fallback, True

    raise RuntimeError(
        f"Requested tier '{requested.value}' is unavailable and no "
        "available fallback is configured."
    )

Step 4: Compose in the Router

At this point the router is intentionally simple: it turns a workload label into
a concrete tier decision.

# router/router.py

class Router:
    def __init__(
        self,
        policy: RoutingPolicy | None = None,
        capability_checker: TierCapabilityChecker | None = None,
    ) -> None:
        self.policy = policy or RoutingPolicy.default()
        self.capability_checker = capability_checker or TierCapabilityChecker()

    def route(self, workload_category: str | WorkloadCategory) -> RoutingDecision:
        workload       = classify_workload(workload_category)
        requested_tier = self.policy.preferred_tier(workload)
        resolved_tier, used_fallback = self.capability_checker.resolve(requested_tier)

        return RoutingDecision(
            workload=workload,
            requested_tier=requested_tier,
            resolved_tier=resolved_tier,
            used_fallback=used_fallback,
        )

Step 5: Pass the resolved tier to Bedrock

This is the part that makes the project more than a routing abstraction — the
decision ultimately becomes a real serviceTier parameter in a live Bedrock call.

decision = router.route("background")   # → RoutingDecision(resolved_tier=FLEX, ...)

response = bedrock.converse(
    modelId="apac.amazon.nova-pro-v1:0",
    messages=[{"role": "user", "content": [{"text": prompt}]}],
    inferenceConfig={"maxTokens": 128, "temperature": 0},
    serviceTier={"type": decision.resolved_tier.value},   # ← this is the key param
)

The serviceTier parameter was added to the converse API alongside the tier
launch. If you are on an older boto3 version (pre-1.34), it will not exist —
upgrade to at least boto3==1.43.

IAM permissions required — no new permissions beyond standard model invocation:

{
  "Effect": "Allow",
  "Action": ["bedrock:InvokeModel"],
  "Resource": "arn:aws:bedrock:ap-southeast-1::foundation-model/amazon.nova-pro-v1:0"
}

Tier selection is a parameter in the existing converse call, not a separate API.

---

The Gotcha: Two Things I Expected That Turned Out to Be Wrong

Gotcha 1: Priority did not demonstrate lower latency in the experiment

This was the assumption I was most confident about going in: Priority should be
faster. The documentation describes it as the fastest tier and says it prioritizes
requests over Default and Flex. I fully expected to see that in the data.

The controlled tier experiment ran 300 requests across all three tiers in
randomized order. Here is what came back:

Tier	Mean (ms)	P95 (ms)
Priority	~803	~1,080
Default	~733	~1,045
Flex	~711	~903

Honestly, I was surprised. The documentation says Priority is the fastest tier,
but the experiment didn't show a statistically significant latency advantage for it.

I don't think those two things are actually contradictory — I think the experiment
just didn't create conditions where the difference becomes visible. The requests
were intentionally tiny: "Explain Amazon SQS in exactly three concise sentences."
That is roughly 10 input tokens and ~58 output tokens per request, running at 20 RPM
under light load. There is no real demand contention for Priority to resolve in your
favor under those conditions.

My hypothesis is that the difference would become more observable with larger prompts
and at higher request rates, where tiers would actually compete for inference capacity.
I haven't tested that yet, so I won't claim it — but it is the next experiment I would
run.

For now, if I had to explain Priority tier to a stakeholder, I would frame it this
way: you are paying for prioritized access, not raw speed. That distinction matters
more when there is actual demand competing for capacity. For short, low-frequency
requests, you are unlikely to feel it.

Gotcha 2: Workload-aware routing can cost more, not less

My second assumption was even simpler: if Flex is cheaper, sending batch workloads
there should obviously save money.

It does — but only if the rest of your workload does not contain too much Priority
traffic.

Here is the break-even rule I derived from the observed token profile:

~1.48 Flex-routed background requests are required to offset the cost of
one Priority-routed critical request.

The policy sensitivity model confirmed the consequence. Of 231 simulated workload
mixes, only 96 (42%) produced a cheaper result than All-Default. 134 (58%)
were more expensive. The model makes the extremes concrete:

Workload mix	Savings vs. All-Default
0% critical / 100% background	+50.00%
100% critical / 0% background	−75.00%

If your batch workload is genuinely deferrable, Flex is a compelling default. But
I wouldn't route something to Flex just because it's called "batch." The real
question is how much latency and availability variance you are willing to trade for
the lower cost. "Batch" tells you how the job runs; it doesn't tell you how tolerant
that job is to degraded service. Those are different questions, and the routing
decision should answer the second one.

---

Results: What the 100-Request Benchmark Actually Showed

The benchmark ran 100 live Bedrock requests in ap-southeast-1 against
apac.amazon.nova-pro-v1:0:

• 20 critical → Priority
• 50 interactive → Default
• 30 background → Flex

Routing

This was the part I most wanted to verify: did the router actually do what I told
it to do?

Metric	Result
Policy mapping accuracy	100.00%
Tier resolution accuracy	100.00%
Request success rate	100.00%
Fallback rate	0.00%

In this run, yes. Every workload resolved to the intended tier, every request
succeeded, and the fallback path was never needed.

Cost

Metric	Value
Estimated router cost	$0.019490
All-Default baseline	$0.019523
Savings	$0.000034 (0.17%)

This was smaller than I expected. The Priority premium on 20 critical requests
almost exactly cancelled the Flex savings on 30 background requests. The takeaway
is not that the router fails — it is that workload composition matters more than
simply "using tiers." The router gives you the lever; your traffic distribution
determines whether that lever saves money or costs you more.

Token usage

The benchmark deliberately used a small, standardized prompt so the tier comparison
stayed controlled: 1,000 input tokens and 5,851 output tokens across 100 requests,
averaging about 10 input and 58 output tokens per call. That also means the result
should not be generalized to large-context workloads.

---

Lessons Learned: What I Would Do Differently

1. Randomize request order from the start

Running critical → interactive → background in contiguous blocks means workload
category is correlated with request time. Any time-varying service behavior
contaminates the comparison. In the next experiment, I randomize.

2. Don't leave classification until the end

I built the router around an explicit label first because it made the experiment
cleaner. But that exposed a gap: in a real application, something still has to
decide whether a request is critical, interactive, or background. That problem is
non-trivial, and I would solve it before calling this production-ready. A
lightweight heuristic, or a Haiku-class model classifying on request metadata, is
probably the next piece.

3. Instrument the workload mix before you ship

This became obvious only after I saw the 0.17% result. The economic value of this
router is entirely determined by your critical-to-background ratio. Without a
dashboard showing that ratio in real time, you have no idea whether the router is
saving or costing money on any given day. Add the metric before the feature,
not after.

4. Stress-test Flex before relying on it

Flex is the only tier with an explicit throttling caveat in the AWS documentation.
I tested it at 20 RPM under light load and saw zero fallbacks. That is not a stress
test. Before routing anything important to Flex, verify how it behaves at or above
expected peak load.

5. Re-verify pricing per region before any billing claim

The pricing table I used reflects ap-southeast-1 at project time. Cross-region
inference profiles, other regions, and tier pricing can all differ. Any cost claim
in production needs current figures from the AWS pricing page for the exact model
and configuration.

6. Test different prompt sizes

This is probably the biggest open question I have after finishing the experiment.
Tiny prompts are good for controlled comparison but limit how much the latency
result generalizes. A follow-up should stratify by token count to test whether tier
differences become more observable as input and output sizes grow — which is also
where the cost differences become more meaningful.

---

What This Is, and What It Isn't

After running the experiments, this is the claim I am comfortable making:

A deterministic workload-aware router can successfully map critical,
interactive, and background workloads to Priority, Default, and Flex service
tiers respectively — with 100% routing accuracy in a 100-request benchmark,
an explicit fallback mechanism, and a measurable economic model whose value
is determined by workload distribution.

It is not a claim that the router universally saves money, that Priority is
reliably faster, or that this policy is production-optimal. Those claims would
require more evidence than one benchmark run against one model in one region
at light load.

I started this project thinking the tier decision would be straightforward.
The experiments made that mental model considerably less simple.

Priority didn't produce the latency advantage I expected under small-prompt
conditions. Flex can reduce cost, but only when enough of the workload can safely
tolerate it. And the economics can flip quickly once Priority traffic becomes a
large share of the mix.

That is ultimately why I think the router is useful — not because it magically
makes Bedrock cheaper, but because it forces the tier decision to become explicit,
measurable, and testable. Before building this, I was making implicit assumptions.
The router turned those assumptions into a policy I can read, change, and validate
against data.

---

Get the Code

The full project — router, benchmark scripts, analysis, and results — is on
GitHub: bedrock-tier-router

To run the unit tests locally (no AWS credentials needed):

git clone https://github.com/<your-org>/bedrock-tier-router.git
cd bedrock-tier-router
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pytest -v

Questions, corrections, and production war stories about Bedrock tier behavior
are welcome in the repo issues or in the AWS Community Builders Slack.

---

Kayne Rodrigo — AWS Community Builder, AI/ML Engineering Track

---

Appendix: Pricing Assumptions

These are the analytical rates used in all cost calculations in this post.
Re-verify against current AWS pricing before using for any billing estimate.

Tier	Input / 1M tokens	Output / 1M tokens
Default	$0.80	$3.20
Priority	$1.40	$5.60
Flex	$0.40	$1.60

Model: apac.amazon.nova-pro-v1:0, Region: ap-southeast-1,
cross-region inference profile.