DEV Community: Fernando Azevedo

SnapStart on containers: the bake-off against provisioned concurrency

Fernando Azevedo — Wed, 02 Sep 2026 16:53:03 +0000

On September 2, 2026 AWS extended Lambda SnapStart to functions packaged as container images — the format many organizations adopted because of a corporate deployment standard, or simply because they needed more than the 250 MB a .zip allows. The promise is honest: instead of pulling image layers and initializing the runtime for every new execution environment, Lambda snapshots the already-initialized environment when you publish the version and resumes from there. But trading initialization for a snapshot is not a neutral performance flag. It changes when you pay, what breaks, and where the deployment fails. This is the bake-off I would run before turning it on in production.

What actually got unlocked

The blocker was never the snapshot mechanism — Firecracker had been freezing memory and disk since the original Java SnapStart. The blocker was the lifecycle contract: a container image can carry any runtime, and Lambda needed a standard way to know when initialization was done and when the environment came back from the freeze.

That contract is now public, and it lives in the Runtime API. At the end of your initialization, if AWS_LAMBDA_INITIALIZATION_TYPE equals snap-start, you run your before-snapshot hooks and call GET /runtime/restore/next. The call blocks — the process literally parks there — until Lambda restores that environment from the snapshot and returns HTTP 200. Then you run the after-restore hooks and enter the normal invoke loop. Errors go to /runtime/init/error (which fails PublishVersion) or /runtime/restore/error (which fails the in-flight invocation and tears the environment down).

If you use the AWS base images for Java 11+, Python 3.12+ or .NET 8+, Lambda coordinates all of it and the experience is identical to .zip. If you bring your own base image, your own Runtime Interface Client, or the base images for provided.al2023, Node.js and Ruby, there are two paths: implement /restore/next, or declare LABEL com.amazonaws.lambda.feature.snapstart="Allow" in the Dockerfile. Without one of the two, publishing the version simply fails.

The SnapStart lifecycle for a container image

Initialization moves off the request path and onto the deployment path. The cost meters follow the snapshot, not the invocation.

🏗️ Build e publicação

Dockerfile LABEL ...snapstart=Allow (ci)
Amazon ECR imagem ≤ 10 GB descomprimida (storage)
PublishVersion ApplyOn=PublishedVersions (ci)

❄️ Init único (tempo de deploy)

Init runtime + pesos do modelo (compute)
beforeCheckpoint GET /runtime/restore/next (compute)
Snapshot Firecracker cifrado, cacheado, replicado (storage)

⚡ Invocação (tempo de request)

Restore retoma do snapshot (compute)
afterRestore re-semeia CSPRNG, reconecta (security)
Handler resposta ao cliente (compute)

📊 Sinais de observabilidade

INIT_REPORT Init Duration (data)
REPORT Restore + Billed Restore (data)
X-Ray subsegmento Restore (data)

💸 Medidores de cobrança

Cache do snapshot US$0,0000015046/GB-s (mín. 3h) (external)
Restauração US$0,0001398 por GB (external)

Flows

dockerfile -> ecr: docker push (same Region)
ecr -> publish: image optimization → Active
publish -> init: Lambda initializes exactly once
init -> hook: max(timeout, 130 s) ceiling
hook -> snap: blocks until frozen
snap -> restore: resume per new environment
restore -> after: HTTP 200 on /restore/next
after -> handler: enters the invoke loop
init -> initreport: init cost, at deploy time
restore -> report: cold start = Restore + Duration
restore -> xray: replaces Initialization
snap -> cache: per active published version
restore -> restorecost: per restored environment

The physics: what the snapshot removes and what it does not

SnapStart removes three expensive things from the request path: downloading and optimizing image layers, loading the runtime, and your own initialization — imports, framework wiring, schema compilation, loading model weights into memory. All of it now happens once per published version, at deployment time.

What it does not remove is whatever is genuinely per-environment. Network connections opened during init have no guaranteed state after the resume; the AWS SDK usually reconnects on its own, the rest is your job in the after-restore hook. Entropy is the dangerous case: if you generated a UUID, a secret or a seed during init, that value goes into the snapshot and comes out identical in every restored environment. Lambda already helps on one sensitive point — with SnapStart active the runtime switches to container credentials (AWS_CONTAINER_CREDENTIALS_FULL_URI) instead of the access-key variables, precisely so credentials do not expire frozen inside the snapshot.

And there is a design inversion almost nobody catches on first reading: with SnapStart, /tmp is capped at 512 MB, against the 10,240 MB a normal function can configure. The classic inference pattern — pulling weights from S3 into /tmp during init — no longer fits. What is left is baking the weights into the image itself (up to 10 GB uncompressed) and loading them into memory, where the snapshot captures them already materialized. That is exactly the scenario AWS cites, but it requires rewriting how you load, not just flipping the flag.

Four ways to kill cold start on a large artifact

Criterion	Container + SnapStart	Container + Provisioned concurrency	.zip + SnapStart	Always-on ECS Fargate
Cold start	Sub-second (Restore + Duration)	Double-digit ms, no cold start	Sub-second, no image pull	Zero at steady state; minutes to scale
Fixed monthly cost (2 GB, us-east-1)	~US$7.80 per published version	~US$21.60 per provisioned unit	~US$7.80 per published version	vCPU + GB per second, 24×7
Cost per startup	US$0.00028 per restore (2 GB)	None; you already paid all along	US$0.00028 per restore (2 GB)	None, but idle time is billed
Artifact ceiling	10 GB uncompressed, via ECR	10 GB uncompressed, via ECR	250 MB unzipped	No practical ceiling
Ephemeral storage (/tmp)	512 MB maximum	Up to 10,240 MB	512 MB maximum	Task volume, configurable
Runtimes covered	Java 11+, Python 3.12+, .NET 8+ and custom bases with hooks	Any runtime	Java 11+, Python 3.12+, .NET 8+ only	Any runtime
Invocation target	Published version or alias only	Provisioned version or alias	Published version or alias only	Load balancer endpoint
Typical failure mode	PublishVersion fails; non-unique state leaks across environments	Concurrency spillover falls back to cold start	Dependencies do not fit in 250 MB	Idle capacity becomes the biggest line item

The math, using the published prices

The us-east-1, x86 numbers: snapshot cache at US$0.0000015046 per GB-second, restore at US$0.0001397998 per GB, provisioned concurrency at US$0.0000041667 per GB-s and on-demand duration at US$0.0000166667 per GB-s. For Java managed runtimes the documentation waives the SnapStart price; the math below applies to Python and .NET.

A 2 GB function with one published version kept active for a full month costs roughly US$7.80 in cache alone. A single provisioned concurrency unit at the same 2 GB costs roughly US$21.60 a month. Each 2 GB restore costs US$0.00028 — meaning you would need approximately 77,000 restores per month for SnapStart's variable cost to match one provisioned unit. For the overwhelming majority of APIs, SnapStart wins by a wide margin.

The problem is not the unit price; it is the multiplication. The cache charge is per published version and continues for as long as the version exists, with a 3-hour billing minimum. A pipeline that publishes a version on every merge and never prunes leaves dozens of paid snapshots running behind it. Twenty forgotten 2 GB versions cost more than US$150 a month to serve exactly zero traffic. Add that Lambda periodically regenerates snapshots to apply runtime patches, and each re-run of your init is billed. A version retention policy stops being hygiene and becomes financial control.

How I would decide, by workload profile

Container + SnapStart

Pros

Best latency-per-dollar for bursty traffic with a large artifact
Keeps the corporate container deployment standard and the ECR pipeline intact
Model weights loaded into memory enter the snapshot already materialized

Cons

The 512 MB /tmp cap kills the download-from-S3-during-init pattern
Requires a uniqueness audit: entropy, IDs and connections opened during init
Incompatible with provisioned concurrency — no hedging possible

Verdict: The default for interactive APIs and inference with weights baked into the image.

Container + provisioned concurrency

Pros

Double-digit millisecond startup, with no resume step at all
Works with any runtime and with /tmp up to 10 GB
No uniqueness requirements on the initialization code

Cons

Roughly 2.8× the snapshot cache cost per equivalent GB-hour
Sizing it wrong is expensive; spillover falls back to a full cold start

Verdict: Only when the SLO is strict enough that the resume step does not fit inside it.

.zip + SnapStart

Pros

No image optimization step and no container Pending/Inactive state
Lifecycle coordinated by the managed runtime, zero hook code

Cons

250 MB unzipped is not much for ML dependencies
Moving from container to .zip requires a brand-new function — package type is immutable

Verdict: Still the simplest path when dependencies fit. Not somewhere to migrate to.

Always-on ECS Fargate

Pros

No runtime, /tmp or state-uniqueness restrictions
Persistent connections, warm local cache and long-running processes

Cons

You pay for idle 24×7 and inherit scaling, patching and a load balancer
Scaling out takes minutes, not milliseconds

Verdict: Justified by high sustained utilization, not by fear of cold starts.

What now breaks in the pipeline, not in the request

This is the part that interests me most as an architect: SnapStart moves initialization to deployment time, and it moves the failure modes along with it.

Init and the before-snapshot hooks share a combined timeout of max(function_timeout, 130 seconds). Blow past it and PublishVersion fails. An inference function that loads 4 GB of weights, and previously was merely slow on the first call, now breaks the release pipeline — which, honestly, is the right place to break, provided the pipeline treats it as a deployment error and not as a flake.

Second: SnapStart only exists on a published version or an alias pointing at one. $LATEST is never SnapStart. That means the development loop invoking $LATEST exercises a different code path than production; contract tests need to hit the alias. Third: package type is immutable. You do not convert a container function to .zip — you create another function, with another ARN, another alias, other event source mappings. Plan it as a migration, not as a toggle.

And there are the states that only show up under rare traffic. A container function left uninvoked for weeks has its optimized image reclaimed, returns to Pending and rejects the first invocation. For Java runtimes, the snapshot is deleted after 14 days without invocation and you get SnapStartNotReadyException. Both are caller-retryable errors — as long as the caller has retry with backoff and the operation is idempotent. Without that, your low-traffic function has an error built into the calendar.

How to measure this without fooling yourself

The first side effect of SnapStart is that your cold-start dashboards silently stop working. The Init Duration field disappears from REPORT, because initialization no longer happens at invocation; it moves to a separate record, INIT_REPORT, along with the duration of the before-snapshot hooks. Two new fields appear in REPORT: Restore Duration and Billed Restore Duration. They are not the same thing — the first includes work done outside the microVM, which the user waits for but you are not charged for; the second covers only runtime load and the after-restore hooks.

The formula that matters is simple and needs to land in your SLO: cold start = Restore Duration + Duration. If your alarm only looks at @duration, it will show an improvement that does not exist. In X-Ray the change is analogous: there is no more Initialization subsegment, there is Restore. Through the Telemetry API you receive platform.restoreStart, platform.restoreRuntimeDone (with status success, failure or timeout) and platform.restoreReport.

Two practical details. The AWS_LAMBDA_LOG_GROUP_NAME and AWS_LAMBDA_LOG_STREAM_NAME variables do not exist in a SnapStart function — logging libraries that depend on them break quietly. And the honest measure of impact is not function duration: it is API Gateway's IntegrationLatency or the function URL's UrlRequestLatency, at p99 and p99.9. That is the only place you see what the client actually felt.

The five mistakes I would expect in the first three months

Turning on ApplyOn=PublishedVersions without auditing init: seeds, UUIDs and secrets generated before the snapshot become identical across every restored environment.
Publishing a version per merge and never pruning — snapshot cache is billed per active version, with a 3-hour minimum, forever.
Keeping the pattern of pulling model weights from S3 into /tmp during init, ignoring the 512 MB cap SnapStart imposes.
Testing against $LATEST and concluding that 'nothing changed' — $LATEST never uses SnapStart.
Declaring victory based on @duration, without adding Restore Duration or looking at integration latency at the edge.

The real trade-off is not latency versus cost: It is determinism versus uniqueness. Provisioned concurrency keeps N independent environments, each with its own entropy, its own connections and its own lifecycle. SnapStart keeps one canonical initial state and clones it. Everything that was accidentally unique per environment becomes deliberately shared. In financial-grade systems that stops being a performance detail and becomes a correctness question: a cloned pseudorandom generator, a pre-computed idempotency token or a frozen connection pool produce bugs that never show up in load testing — they show up in reconciliation.

Reading it through the pillars

security: The snapshot is encrypted, but any secret materialized during init gets cloned; fetch credentials in the handler and re-seed CSPRNGs on after-restore.
reliability: SnapStartNotReadyException and the container Pending state require caller-side retry with backoff and idempotent operations.
performance: Measure Restore Duration + Duration and validate against IntegrationLatency at the edge; the improvement is only real at p99.9.

Curator's note: I would turn this on, but in week three, not week one. The order I use is always the same: first instrument Restore Duration in a staging environment against the real alias, then audit init hunting for entropy, credentials and connections — it is always that audit that surfaces the uncomfortable finding. Only then do I touch production. I once saw a payments system whose init pre-computed an idempotency-key suffix 'to save time'; under provisioned concurrency it never collided, because each environment had its own. With a cloned snapshot it would have collided, and the incident would not have shown up in latency — it would have shown up as a duplicated transaction, two days later, at close of books. Performance you cannot reconcile is not performance, it is debt.

Recommendation

For interactive APIs and inference packaged as container images, on Java 11+, Python 3.12+ or .NET 8+ over the AWS base images, SnapStart is now the default choice: it delivers sub-second startup at roughly a third of the monthly cost of an equivalent provisioned concurrency unit, and the restore-cost breakeven sits so far out (~77,000 restores a month at 2 GB) that it rarely matters. Reserve provisioned concurrency for endpoints whose SLO cannot absorb the resume step, or that need /tmp above 512 MB. Do not migrate to .zip just to get SnapStart — package type is immutable and the benefit does not pay for the ARN swap. And before anything else, do two thoroughly unglamorous chores: a uniqueness audit of your init code, and an automated pruning policy for published versions. SnapStart is won or lost in those two.

Rating: 8.5/10

References

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Apps in Amazon Quick vs. App Studio vs. a custom build

Fernando Azevedo — Wed, 02 Sep 2026 12:21:48 +0000

Every company I have worked with carries the same invisible debt: somewhere between thirty and three hundred spreadsheets nobody versions, fed by manual exports from four different systems, and yet feeding credit decisions, capacity plans and the monthly close. The September 1, 2026 announcement — building apps in Amazon Quick by describing them in natural language — aims squarely at that debt. The risk is trading an ungoverned spreadsheet for an ungoverned app, now with a write connector and an AWS invoice attached. So I did not write a feature review: I wrote the bake-off I would run before approving the first app, across the four routes that genuinely compete for this work.

What the September 1 GA actually ships

The Apps in Quick documentation says more than the announcement does. Seven capabilities matter architecturally: conversational authoring (the agent writes the code while you watch), live Quick Sight visual embeds inside the app, action connectors that call external APIs, built-in foundation-model inference for summarisation and classification, reading documents from spaces, persistent key-value storage across sessions, and one-click publishing under Quick's SSO.

Read that combination again: action connectors plus persistent key-value storage. This is not a prettier dashboard. It is an application with its own state and a write path into third-party systems, authored by conversation and published with one click. In a financial-grade environment, that is precisely the class of artefact a change board expects to see with an owner, a version and a rollback.

Positioning matters too. Quick is the agentic-workspace evolution of QuickSight, and the docs already offer BYOI — Bring Your Own Amazon Q Business Index plus attaching managed Bedrock knowledge bases. AWS is stitching the Q Business index and Bedrock knowledge into Quick. If your company has invested in either, the lowest-friction route now runs through Quick — and that weighs more in the decision than any "minutes to first app" benchmark.

The four routes that actually compete

Comparing Apps in Quick against "low-code" in the abstract helps nobody. In practice, when someone asks me for an internal tracker, there are four possible destinations.

Apps in Amazon Quick. Authored by the business user, inside the workspace where they already chat, research and consume BI. Seat-based pricing. Available to Plus, Professional and Enterprise since September 1, 2026.

AWS App Studio. AWS's other natural-language builder, GA since November 2024, aimed at the "technical professional" — IT project manager, data engineer, architect. Connects to 200+ AWS services and to third parties through API/OpenAPI connectors. Charges $0.25 per user-hour of the published app; the builder environment is free.

A custom build. Amplify/AppSync + Lambda + DynamoDB + Cognito, code in a repo, PRs, tests, pipeline. Expensive in engineering, cheap in audit.

No app at all. A Quick Sight dashboard plus a Quick Flow. Half the "internal app" requests I receive are really one number on a screen and an email fired when it crosses a threshold. That route has to be on the table, or the comparison degenerates into a contest between builders when the right answer was to not build.

Head-to-head: the dimensions that decide

Criterion	Apps in Amazon Quick	AWS App Studio	Custom build	No app (Quick Sight + Flows)
Who builds it	The business user, conversationally	Technical professional (IT, data, architecture)	An engineering team with a backlog and priorities	A BI analyst, in hours rather than weeks
Billing unit	Seat/month ($20 Plus and Pro; $40 Enterprise)	$0.25 per user-hour of the published app	Service consumption plus engineering cost	Already covered by the existing Quick seat
Fixed floor	$250/account/month infrastructure fee on Pro and Enterprise	None; the builder environment is free	Low on services, high on people	No additional floor
Where the code lives	Generated and hosted inside Quick	Generated and operated by App Studio	In your Git, with diff and rollback	There is no application code
Identity and authorization	Inherited from Quick; RBAC/SSO listed as Enterprise-tier	App Studio's own model layered over connectors	Cognito/IAM with row-level entitlement if you write it	Quick Sight dataset rules, already audited
Writes into a system of record	Yes, via conversationally authored action connectors	Yes, via API/OpenAPI connectors	Yes, with idempotency and retry you control	No; read and notify only
Audit evidence	To be confirmed: diff between generated app versions	Builder versioning, no repository of your own	Commit, PR, pipeline — evidence by construction	Dataset lineage; minimal surface
Regional footprint	Follows the Quick subscription Region	US West (Oregon) and Europe (Ireland) only	Any Region, including sa-east-1	Follows the Quick subscription Region
Time to v1	Minutes, by the process owner	Hours to days, with a technical person driving	Weeks, competing with the roadmap	Hours
Exit cost	High: logic and state locked to the workspace	High: no portable artefact of the app	Low: it is your code	Low: SQL and datasets are portable

The arithmetic almost nobody runs: seat versus user-hour

The two AWS routes bill in incomparable units, and that is where the decision becomes FinOps. A Quick Enterprise seat costs $40/user/month; App Studio charges $0.25 per user-hour. The break-even is plain arithmetic: $40 ÷ $0.25 = 160 user-hours per month. A full working month is roughly 168 hours. So if the app is the only reason to buy the seat, App Studio only loses when the person lives inside the app full time.

The realistic scenario is the opposite. Forty occasional users at 3 hours a month each: 120 user-hours — $30/month on App Studio. The same forty on Enterprise cost 40 × 40 plus the $250 infrastructure fee = $1,850/month, or $22.2k a year. Sixty times more for the same app.

Now invert it: if 250 users already hold Quick seats because they use chat, research and BI, the marginal cost of the app trends to zero and App Studio becomes a new line on the invoice. The right question is not which service is cheaper — it is whether the seat is already bought.

Two hidden details. First, the $250/account/month fee on Professional and Enterprise is a floor: at 10 users the effective rate jumps to $65/user. Second, the pricing page meters agent hours — 4 h/month on Professional, 8 h on Enterprise, $3 per overage agent hour. Iterating an app by conversation consumes exactly that resource. Confirm with your account team whether those hours are per user or per account before turning a hundred people loose to "describe whatever they want".

Triage: three questions before you pick a builder

The common mistake is starting from the tool. I start from what the app actually does — writes, evidence and usage profile — and let the tool fall out of the three answers.

🧭 Triagem / Triage

Escreve no SoR? / Writes to SoR? Salesforce, ServiceNow, ERP (security)
Vira evidência? / Regulated evidence? fechamento, risco, auditoria (security)
Usuários × horas / Users × hours break-even: 160 h/user/month (compute)

🟧 Apps in Amazon Quick — seat-based

Apps in Quick conversational authoring, 1-click publish (ai)
Action connectors + knowledge bases MCP, S3, Drive, OneDrive (external)
Key-value storage state across sessions (storage)

🟦 AWS App Studio — user-hour

App Studio $0.25 / user-hour (compute)
Oregon + Ireland only no sa-east-1 (network)

🟩 Custom build — engineering owns it

Amplify / AppSync + Lambda code in Git, PR, tests (frontend)
DynamoDB + Cognito row-level entitlement (data)
CI/CD pipeline diff, rollback, evidence (ci)

📊 No app — the cheapest route

Quick Sight + Quick Flows dashboard + automation (data)

Flows

req -> q1: 1. nature of the action
q1 -> q2: read-only
q1 -> amplify: writes to SoR → engineering
q2 -> q3: not regulated evidence
q2 -> cicd: is evidence → needs diff and rollback
q3 -> quick: users already hold Quick seats
q3 -> appstudio: many occasional users
q3 -> qsight: it is just a number on a screen
quick -> qconn: identity propagated per connector
quick -> qkv: app state
appstudio -> asreg: data sovereignty blocker
amplify -> ddb: data + authorization
amplify -> cicd: controlled promotion

Decision matrix

Apps in Amazon Quick

Pros

Near-zero marginal cost when the seats already exist
Native Quick Sight embeds plus spaces and knowledge-base integration
The process owner builds and iterates with no IT queue
It is the strategic direction: BYOI pulls the Q Business index inside

Cons

Seats are expensive for occasional users; $250/account floor on Pro and Enterprise
RBAC/SSO and data sovereignty only appear on Enterprise
Conversational iteration burns agent hours ($3/h overage)
High exit cost: logic and state stay in the workspace

Verdict: Best choice when the population already holds Quick seats and the app is read-mostly.

AWS App Studio

Pros

$0.25/user-hour is unbeatable for sporadic, long-tail usage
Free builder environment and 250 free user-hours to trial
Reach into 200+ AWS services and third parties via API/OpenAPI connectors

Cons

Oregon and Ireland only — a non-starter for data that must stay in Brazil
No new Regions since the November 2024 GA
Needs a technical person driving — not business self-service

Verdict: Wins on cost per occasional user, loses on regional footprint and product trajectory.

Custom build (Amplify/AppSync)

Pros

Diff, PR, test and rollback: audit evidence by construction
Row-level entitlement and idempotency under your control
Any Region, including sa-east-1; low exit cost

Cons

Weeks of engineering competing with the product roadmap
You inherit operations, patching and on-call for one more internal app

Verdict: Reserve it for anything that writes to a system of record or becomes regulated evidence.

Build no app (Quick Sight + Flows)

Pros

Zero incremental cost and minimal risk surface
Reuses dataset lineage and permissions that are already audited
Ships in hours and dies painlessly when it stops being useful

Cons

Covers neither data entry nor approval workflow
Frustrates whoever asked for an "app" and got a chart

Verdict: The default option. Only leave it when there is real data entry or a real workflow.

"Respects your access policies" — what that means and what it does not

That line from the announcement is true and insufficient. What the documentation describes is identity propagation at the connector boundary: the Salesforce or ServiceNow integration carries the user's identity, and what they cannot see there, they do not see here. That is good, and it is different from row-level entitlement inside the app. If the app joins Salesforce pipeline with warehouse cost and returns margin per customer, the authorization of that combined result is a new property no single connector guarantees.

The test I require before publishing is not functional, it is negative: take a user with no access to account X in the CRM, open the published app and prove they cannot see account X — not in the table, not in the total, not in the output of the inference block. Aggregates leak. Foundation-model summarisation leaks elegantly.

Then come the operational questions, worth more than any demo: is there a diff between today's version of the generated app and March's? Who approves promotion? What signal reaches CloudTrail and your SIEM when an action connector writes into a system of record? What is the failure behaviour of that connector — retry, idempotency, partial write?

And there is the licensing layer turning into an architectural constraint: the pricing page places RBAC/SSO and data sovereignty on Enterprise. In a regulated institution, identity federation is not optional — so the real price is not $20, it is $40 plus the $250 infrastructure fee. Decide the tier during architecture, not at renewal.

The real break-even is not technical: A $40 Enterprise seat equals 160 user-hours on App Studio — nearly a full working month inside the app. So, on a per-app basis, App Studio almost always wins on cost. Quick only wins when the seat was already bought for another reason (chat, research, BI). The platform decision therefore precedes the app decision: if Quick is already the company workspace, building outside it means paying twice. If it is not, one app does not justify adopting it.

How this goes wrong in production

One-click publishing "to the entire organization" under a seat model — distribution becomes a cost event, not an adoption event.
Treating the conversational app as a disposable spreadsheet, then discovering at close that a regulated number comes from an artefact with no version and no owner.
Confusing connector-level identity propagation with authorization of the aggregated result — and never running the negative test.
Enabling write-capable action connectors before defining retry, idempotency and the signal reaching the SIEM when the app mutates a system of record.
Choosing App Studio for data that must stay in Brazil, ignoring that it exists only in Oregon and Ireland.
Letting a hundred people iterate apps conversationally without knowing whether the included agent hours are per user or per account.

A Well-Architected reading

security: Inherited identity is a starting point, not a conclusion. Require a negative test per persona, review write-capable action connectors as changes to a system of record, and treat the licence tier (RBAC/SSO on Enterprise) as a security requirement, not a procurement one.
reliability: The app depends on third-party systems through connectors. Define the expected behaviour when Salesforce or ServiceNow degrades: visibly stale data, explicit failure, or queued writes with idempotency.

Three questions that always come up

Does this replace AWS App Studio?

Formally no: App Studio does not appear in the March 2026 service availability update and remains GA. In practice it has not gained a Region since November 2024 and still runs only in Oregon and Ireland, while Quick is receiving BYOI, Bedrock knowledge bases and now apps. I read that as investment direction and plan accordingly.

Can I build apps on the Free tier?

The announcement says app building is available to Plus, Professional and Enterprise starting September 1, 2026, while the pricing page lists "no-code apps" among Free features. The two pages are not aligned — confirm with your account team before planning a rollout on top of Free.

What is the first app I would let someone build?

One that only reads, has fewer than twenty named users, replaces an existing spreadsheet, and produces no number that enters an external report. If it survives a quarter of real usage, then I will discuss a write connector.

What I would do on Monday: I would not open app building to the whole company; I would open it to a two-process pilot with a named owner and a 90-day review date. The lesson that cost me came from elsewhere: at a bank we killed a critical spreadsheet by replacing it with a better internal tool — and six months later found the risk team back on the spreadsheet, because the tool could not show how March's number had been calculated. The ability to reproduce the past is worth more than the speed of building the present. So my bar for any conversational app is a single question: if the auditor asks for the difference between today's version and the one from three months ago, can I show it? If the answer is no, the app may exist — but it cannot be the source of an official number.

Recommendation

If your company already pays for Amazon Quick seats, Apps in Quick is the right choice for read-mostly apps — near-zero marginal cost, Quick Sight embeds, identity already handled at the connector boundary, and alignment with where the product is going (BYOI, Bedrock knowledge bases). If you are not a subscriber, one app does not justify $40/user plus $250/account: App Studio delivers the same outcome at $0.25 per user-hour — provided the data can live in Oregon or Ireland, which rules out a good share of regulated Brazilian cases. For anything that writes to a system of record or backs an externally reported number, stay with the custom build: you are buying diff, rollback and evidence, not technology. And before all of that, test the cheap hypothesis — half these requests die with a Quick Sight dashboard and a Quick Flow, and that is the most elegant architecture of the four.

Rating: Adote com piloto controlado / Adopt with

References

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Amazon Connect compact mode: operational density with caution

Fernando Azevedo — Tue, 01 Sep 2026 20:57:43 +0000

AWS announced on September 1, 2026 that Amazon Connect Customer analytics dashboards now support compact mode. On the surface, the change reduces spacing, font size, widget height, and visible filter footprint so more information fits on screen. In practice, for teams operating financial contact centers, insurers, fintechs, or regulated service desks, this is an operational design decision: the supervisor cockpit must show more context without turning the screen into noise.

What the official sources confirm

15s — Real-time dashboard refresh. The documentation states a 15-second refresh for real-time dashboards, except time-series widgets, which refresh every 15 minutes.
10 — Widgets per dashboard. The documented customization model allows up to 10 widgets on each dashboard.
$0.038 — Published voice price. The Connect Customer pricing page lists voice at $0.038 per voice minute, with standard telephony charges applying.

The feature is simple; the operational implication is not

I read this launch as an operational ergonomics improvement, not as a new analytics capability. Compact mode does not change the Amazon Connect Customer data model, does not create a new metric, does not solve queue cardinality, and does not replace alarms. Its value is in reducing cognitive friction when a supervisor needs to see more agents, queues, adherence, wait times, and exceptions without navigating across multiple parts of a page.

In financial contact centers, the supervisor screen is often a human control point between automated signals and contingency decisions. A spike in abandonment, a fraud queue aging abnormally, agents spending too long in after-call work, or an adherence drop during a critical window can require action within minutes. If the accountable person only notices the issue after scrolling, changing filters, or opening another report, the system has already consumed part of its operational error budget.

The change also follows a clear AWS direction: console analytics closer to operations, as seen in the MediaTailor analytics dashboard and CloudWatch alarm warm-up periods. The pattern is to reduce noise, shorten the path between telemetry and decision, and place actionable context where operations already work. I like that direction, but it requires discipline: density without semantics is just a tighter spreadsheet.

How I would position compact mode in a critical service operation

The diagram shows the right role for compact mode: a human perception layer, fed by governed metrics and complemented by incident automation.

👥 Operação / Operations

Supervisor 13-inch laptop / wallboard (user)
Agents queues and adherence (user)

🟧 Amazon Connect Customer

Connect instance voice, chat, email, messaging (compute)
Analytics dashboards compact mode enabled (frontend)
Widget filters queue, agent, proficiency (data)
Custom metrics service-level definitions (data)

📈 Observabilidade / Observability

CloudWatch alarms and dashboards (compute)
EventBridge incident workflow trigger (messaging)
Runbook Step Functions or SSM (ci)

🔐 Governança / Governance

Security profiles least privilege (security)
Export evidence CSV/PDF for review (storage)

Flows

agents -> connect: interactions and operational states
connect -> dashboards: real-time and historical metrics
filters -> dashboards: narrows visible scope
custommetrics -> dashboards: defines what matters
dashboards -> supervisor: higher density, less scrolling
connect -> cloudwatch: signals for alarms
cloudwatch -> events: actionable anomaly
events -> runbook: orchestrates response
iam -> dashboards: profile-based permission
dashboards -> audit: CSV/PDF as evidence

Where compact mode shines

The best use case is tactical supervision: people monitoring an operation in near real time who need to compare many similar rows. The documentation confirms that real-time dashboards refresh every 15 seconds, while time-series widgets refresh every 15 minutes. That is enough for queue management, adherence, abandonment trends, and capacity tracking; it is not enough for subsecond automated incident control, and it should not be treated that way.

In an operation with 80 to 150 agents spread across squads, the gain is not seeing everything. It is seeing the right set: non-adherent agents, queues with threatened SLA, active contacts, contacts in queue, oldest contact, agents in error, available agents, and after-call work. If the compact screen can show a full team without scrolling, the supervisor saves a chain of visual microdecisions. That reduces human latency, which in real operations is often larger than service latency.

I also see value in peak-period war rooms: Black Friday, payroll closing, market events, digital banking incidents, collection campaigns, or regulatory windows. In those moments, a dense screen helps maintain shared context across service, operations, SRE, business, and risk. The advantage appears when everyone discusses the same dashboard, with the same filters and thresholds, not when each area interprets a different slice.

Strengths of the feature

It increases density without requiring export to BI, preserving operational context inside Amazon Connect Customer.
It helps supervisors on small screens, especially 13-inch laptops and remote coordination setups.
It works well with queue, agent, hierarchy, and proficiency filters, as long as filter design is governed.
It reduces the cognitive cost of comparing similar operational rows, which is a real pain in large contact centers.
The announcement identifies no separate charge for the mode itself; relevant cost still comes from Connect Customer usage and associated services.

The risk: density can hide poor metric governance

What concerns me is not the smaller font; it is the temptation to add more numbers without improving decision quality. The documentation allows teams to customize widgets, metrics, columns, filters, groupings, and thresholds. It also states important limits: up to three table groupings, up to 20 conditions in certain routing/proficiency filters, up to three thresholds per metric, and up to 10 custom metrics in a widget. Those limits are reasonable, but they also expose a trap: with enough flexibility, each area can create its own operational semantics.

In financial environments, I would treat contact center metrics as an operations contract. Service level cannot mean one thing for service, another for risk, and another for technology. If the metric excludes transferred contacts, callbacks, or abandonments within a given interval, that definition must live in an ADR or operational runbook. Otherwise, the dashboard becomes attractive, compact, and dangerous: everyone looks at the same number, but each person believes it represents a different reality.

Compact mode also hides widget descriptions and places filters behind an icon, according to the documentation. That is the right trade-off for screen space, but it increases the need for explicit names, dashboard conventions, and periodic review. A dashboard named “Today’s Operation” is not enough. I would prefer names such as “Card Fraud - Real Time - 30s SLA - Critical Queues” and thresholds aligned with documented SLOs.

Compact is not observability: I would not use compact mode as a substitute for CloudWatch alarms, SLOs, structured logs, traces, runbooks, or response automation. Dashboards are good for perception and coordination; they are weak as the primary detection mechanism. In a critical operation, if the first indication of a problem is someone noticing a red row on the dashboard, the observability architecture is incomplete.

How I would design for scale, cost, and reliability

The first decision is to separate the operational dashboard from the control system. Amazon Connect Customer should be the experience and supervision plane; CloudWatch, EventBridge, Lambda, Step Functions, SSM Incident Manager, or tools such as Datadog should compose the detection and response plane. The compact dashboard shows the actionable picture, but alarms and automation must fire even when nobody is watching.

I would configure dashboards by operational domain, not by generic org chart. A fraud desk needs queues, oldest contact, abandonment, active contacts, available agents, agents in error, and adherence with its own thresholds. A collections cell needs campaign, channel, and contact-window cuts. A premium support operation needs priority, customer, language, and proficiency. When the documentation allows proficiency and routing filters with conditions, I would use that to reduce supervisory noise, not to replicate all routing logic on screen.

On cost, the discussion is not “how much compact mode costs,” because the announcement does not identify a separate charge. The discussion is operational volume. The pricing page lists voice, chat, email, and messaging charged by unit of use. More effective dashboards can reveal waste: poorly routed queues, excessive transfers, abandonment that generates repeat contacts, inflated after-call work, and weak self-service that pushes cost to humans. The useful financial metric here is cost per effective resolution, not isolated cost per contact.

Adoption path I would recommend

Define the metric contract before the layout — Document service level, abandonment, adherence, after-call work, and exclusions. Include formula, time window, owner, and expected decision when the metric crosses the threshold.
Create dashboards by operating scenario — Separate real time, intraday, agent performance, campaigns, fraud, and incidents. Compact mode works best when each screen answers a clear operational question.
Apply least privilege to security profiles — Use dashboard and metrics permissions according to role. Performance data, recordings, evaluations, and analytics can be sensitive in regulated environments.
Connect thresholds to runbooks — Colors on the dashboard only matter if they imply action: reallocating agents, pausing a campaign, opening an incident, adjusting routing, or escalating to crisis management.
Validate on real screens — Test on a 13-inch laptop, supervisor monitor, and crisis room display. If the user must zoom the browser or memorize hidden filters, density has gone too far.

Security, audit, and the human side of the screen

Contact center dashboards can expose more than neutral metrics. Individual performance, hierarchy, proficiency, evaluations, conversation categories, campaigns, and recordings can touch privacy, labor relations, supervision conduct, and regulatory evidence. The documentation makes clear that access depends on security profile permissions and that different dashboards require specific permissions, such as flow view permissions for flow data. I would take that seriously from the first design.

My pattern would be to segment profiles: operational supervisor, quality manager, workforce analyst, SRE/operability, and audit. Each profile sees the minimum necessary for its decision. CSV and PDF exports should fall under retention, encryption, and audit-trail policy, especially when used for committees, incident reviews, or compliance evidence. If the company already has a governed lake, Glue Data Catalog, Lake Formation, or S3 trails with KMS, the exported report should not become an informal exception outside that control.

The human factor also matters. Compact mode reduces visual space and can increase fatigue if used during long shifts. I would alternate dense dashboards for active supervision and cleaner panels for later analysis. The best operation is not the one that sees more numbers all the time; it is the one that knows which number must appear when a decision must be made.

Well-Architected reading

security: Security profile permissions should limit who can view metrics, evaluations, recordings, and sensitive data. Exports need retention, encryption, and audit trails compatible with the regulated environment.
reliability: The dashboard should be a coordination layer, not the primary detection mechanism. CloudWatch alarms, automations, and incident integrations must cover the operation when the screen is not being watched.

Compact mode versus executive dashboard

Criterion	Operational compact mode	Executive dashboard
Primary user	Supervisor, workforce, shift operations, service SRE.	Executives, product, finance, governance, and committees.
Decision window	Seconds to minutes; focused on queue, agent, and exception.	Days to weeks; focused on trend, cost, and strategy.
Main risk	Visual noise, fatigue, and action based on poorly defined metrics.	Excessive aggregation hiding local operational problems.

My curator note: I would enable compact mode first on real-time supervision dashboards, not on every panel. My practical lesson is that a dense screen only works when there is a shared operational grammar: good names, few thresholds, and explicit decisions. If I need five minutes to explain the dashboard before a crisis, it is not ready for a crisis yet.

Verified references

Verdict

My recommendation is to adopt compact mode for active Amazon Connect Customer supervision, with explicit metric and permission governance. It is a small product improvement, but practically relevant: less scrolling can mean faster human detection and better coordination during critical windows. I would not sell this as analytics transformation; I would treat it as a serious refinement of the operational cockpit. Rating: 8/10 for real-time supervision; 5/10 if used only to squeeze more widgets onto a screen without rethinking decisions.

Rating: 8/10

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

MWAA with Airflow 3.3.1: state, language, and operational discipline

Fernando Azevedo — Tue, 01 Sep 2026 20:57:10 +0000

The September 1, 2026 announcement bringing Apache Airflow 3.3.1 to Amazon MWAA looks small if read as a version update. I do not read it that way. For financial environments, regulated data platforms, and operations with hundreds of DAGs, the central point is that Airflow is formally acknowledging three realities architects already solved outside the platform: tasks need to remember progress, not every domain implementation should become Python, and retries need to understand why a failure happened. MWAA reduces part of the operational burden of running schedulers, workers, webservers, and the metadata database, but it does not remove the obligation to design idempotency, isolation, cost control, and observability. My assessment: it is worth studying and adopting selectively, especially for incremental workloads and long-running jobs; it is not a reason to turn every DAG into an experimental SDK lab.

Numbers that shape the decision

3.3.1 — Airflow version now supported on MWAA. Apache Airflow 3.3.1 was released on August 12, 2026; MWAA support was announced on September 1, 2026.
25 — Workers per environment as the default quota. The documented MWAA quota lists 25 workers per environment, 5 webservers, and 10 environments per account per Region, all adjustable.
$0.49/h — Public example for a small environment in us-east-1. The pricing page uses $0.49 per hour for a small environment and $0.055 per hour for each additional small worker in a Northern Virginia example.

What actually changed

I split this version into two groups: what changes workflow modeling and what improves daily operations. The first group includes the Task and Asset State Store, expanded asset partitioning, pluggable retry policies, and the experimental Language Task SDK for Java and Go. The second group includes bulk actions for DAG runs and task instances, stability, security, and UI fixes, plus practical improvements for teams that operate many pipelines.

The State Store is the most architectural item. Until now, many teams used XCom, Variables, DynamoDB, S3, or auxiliary tables to keep cursors, external job ids, watermarks, and checkpoints. Some of those solutions were correct; others became technical debt with unclear semantics. Airflow 3.3 creates an explicit place for this state: task state for a task instance and asset state for metadata associated with an asset. That does not remove DynamoDB or external databases when the state is domain-owned, auditable, or shared by systems outside Airflow. But it improves DAG hygiene when the state is operational and belongs to the execution cycle.

On MWAA, this matters because the service already runs schedulers and workers on Fargate and maintains a managed metadata database. The temptation will be to fill the metadatabase with too much state. I would do the opposite: use the store for small, versioned, disposable pointers, and keep business data in S3, DynamoDB, Aurora, or the lakehouse.

Where I see value

External job checkpoints become cleaner: task state can store the job id for EMR Serverless, Glue, Batch, ECS, or SageMaker before polling completes.
Per-asset watermarks reduce coupling: a processed S3 partition or table can carry state without becoming a global Variable.
Pluggable retry policies help separate transient from permanent failures, especially for APIs with throttling, expired credentials, and contract validation.
Java and Go become options for existing domain logic, but I would treat the SDK as experimental until packaging, logs, and operational support are proven.
Bulk actions reduce operational friction in reprocessing, but they also call for stronger change controls and audit trails.

Durable state is not permission to forget idempotency

The main mistake I expect with Airflow 3.3 is turning the state store into an application database. In financial systems, a task that settles files, publishes events, calculates exposure, or reconciles positions cannot rely only on “remembering where it stopped.” It needs idempotency by design. That means natural keys, deduplication, conditional writes, and explicit reprocessing boundaries.

A design I would accept: a DAG receives file-arrival events, materializes metadata in DynamoDB with a partition key such as dataset#business_date and a sort key such as source_file#version, stores raw data in S3 with versioning and SSE-KMS, and uses task state only to keep the identifier of an already submitted Glue or EMR Serverless job. If the worker dies, the next attempt retrieves the job id, queries the job state, and decides whether to follow it, cancel it, or open a new attempt with an idempotency token. Durable state avoids duplicated work; the control table remains the auditable ledger.

I would also apply TTL or short retention to task states whenever possible. For asset cursors, I would use small payloads: last confirmed offset, contract hash, partition window, and producer version. If the value starts looking like a document, event, or snapshot, it probably does not belong in the state store.

How I would design safe State Store usage

The diagram shows my preferred pattern: Airflow keeps minimal operational state; auditable state and data remain in purpose-built services.

🟦 Orquestração MWAA

DAG Python asset-aware (compute)
Task State Store job_id, checkpoint (data)
Asset State Store watermark, partition (data)

🟧 Execução AWS

AWS Glue / EMR long-running job (compute)
Amazon S3 raw + curated data (storage)
DynamoDB control ledger (data)

🛡️ Governança e Operação

IAM + KMS least privilege (security)
CloudWatch logs, metrics, alarms (edge)
Runbook replay window (ci)

Flows

dag -> glue: submits job with idempotency token
dag -> taskstate: persists job_id before polling
glue -> s3: writes partitioned outputs
glue -> ddb: updates control ledger
dag -> assetstate: advances watermark after commit
iam -> dag: assumes restricted role
dag -> cw: emits duration, retry, and lag
cw -> runbook: triggers controlled replay

Java/Go SDKs: useful integration, maturity still under test

I like the direction of the Language Task SDK because it recognizes that enterprise data platforms are rarely 100% Python. There are pricing libraries in Java, contract validators in Go, internal clients with already reviewed security policies, and binaries that platform teams would rather not rewrite. The documented model keeps the DAG in Python and declares stub tasks with @task.stub(queue=...); the worker delegates execution to a coordinator, such as JavaCoordinator for JVM or ExecutableCoordinator for self-contained binaries such as Go.

The opportunity is to reduce fragile wrappers. Instead of calling a script through BashOperator and losing task semantics, retries, pools, and XCom behavior, the non-Python task participates in the graph. That improves readability and ownership: the data team keeps orchestration, the domain team keeps implementation.

But I would be conservative. The documentation itself marks this capability as experimental, and that matters. On MWAA, I would first validate whether binary or JAR packaging fits the plugin, requirements, managed image, and worker-directory flow. I would also test cold start, artifact size, remote logs, secrets through Connections, memory limits of the selected environment class, and rollback behavior. For regulated production, I would start with low-risk tasks, without irreversible side effects, and keep pure Python as the recovery path.

The hidden risk is the metadata database: The Airflow metadata database already carries DAG runs, task instances, XComs, indirect logs, serialization, and scheduler state. When adding the state store, I would monitor growth, cleanup, and cardinality from day one. On MWAA, pricing also includes metadata database storage in GB-months; in a platform with many dynamic mappings, per-partition cursors, and frequent reruns, the small cost may matter less than the operational impact of a bloated database.

MWAA operations: managed version is not outsourced operations

MWAA remains a pragmatic choice when an organization wants Airflow without managing Kubernetes, Celery, the metadata database, and image patching. The documentation confirms that schedulers and workers run on Fargate, with a managed Aurora PostgreSQL metadata database and integrations with CloudWatch, S3, SQS, and KMS. That removes undifferentiated work from the team, but it does not turn Airflow into an invisible service.

I would start any 3.3.1 upgrade with a parallel environment, not a direct change to the critical environment. Since the announcement allows a new 3.3.1 environment and upgrades from 3.2 or later, I would use the parallel path to validate dependencies, constraints, airflow.sdk imports, providers, and DAG serialization. The public MWAA version table also shows the lifecycle discipline: AWS supports at least three minor versions and announces end of support 180 days in advance, but the responsibility to keep environments current remains with the customer.

For capacity, I would not size by DAG count alone. Classes range from mw1.micro, with 3 default concurrent tasks and no autoscaling, to mw1.2xlarge, with 80 default concurrent tasks and much larger resources. Schedulers for Airflow v3 accept 2 to 5 in environments above micro, and workers have default 10, minimum 1, and maximum 25. Those numbers require load testing with real DAGs, not an optimistic spreadsheet.

Where to place state in the design

Criterion	Correct use	Avoid	Practical decision
Task State Store	External job id, retry checkpoint, small progress marker inside the task instance.	Business result, large payload, auditable event, or data contract.	Use it to resume execution; do not use it as the source of truth.
Asset State Store	Watermark, last confirmed partition, schema hash, and operational metadata for the asset.	Enterprise catalog, full lineage, quality rules, or authorization.	Good for DAG coordination; complement it with Glue Data Catalog, Lake Formation, or a lineage tool.
DynamoDB / Aurora	Auditable control, cross-system idempotency, locks, approval, SLA, and reconciliation.	Ephemeral state that only one task needs during a retry.	Use it when auditability, external query, or formal retention are requirements.

Pluggable retries change the failure contract

Retry is one of the places where data platforms lose money silently. A fixed retries=3 with a fixed delay looks harmless, but in transactional APIs it can triple cost, create duplicates, and mask permanent errors. In market, risk, fraud, or regulatory data workloads, I want the DAG to know the difference between ThrottlingException, network failure, AccessDenied, schema validation, and a semantic contract break.

Airflow 3.3 retry policies allow rules by exception type and actions such as retry, fail, or default behavior. That brings orchestration closer to an explicit operational policy. I would apply this around operators that call external APIs, managed jobs, and rate-limited integrations. For AWS SDK calls, I would still let low-level botocore retries handle short transient failures; the Airflow retry should be used for workflow decisions, with a delay compatible with the SLA and processing windows.

A healthy pattern: permission failure should fail fast and trigger a runbook; throttling should use backoff and alarm only after N minutes of backlog; schema error should open a contract incident and block downstream; external job failure should check remote state before resubmission. This detail reduces cost and improves trust in reprocessing.

How I would adopt it in production

Create a parallel 3.3.1 environment — I would clone DAGs, requirements, plugins, and configuration while keeping the current environment intact. I would validate constraints, imports, IAM permissions, Connections, pools, and serialization before any real traffic.
Classify DAGs by risk — I would separate read-only DAGs, idempotent DAGs, DAGs with side effects, and regulatory DAGs. The state store would enter first in incremental workflows with simple rollback.
Define the state contract — Each key needs an owner, JSON format, scope, retention, cleanup policy, and relationship to audit. I would reject free-form keys created inside tasks without convention.
Instrument before migrating — I would track task duration, queue time, scheduler lag, active workers, retries by cause, metadata database size, XCom growth, and state count.
Cut over by data window — Instead of migrating all DAGs, I would move one window or domain, freeze competing reprocessing, and keep a clear runbook for replay and rollback.

Anti-patterns I would avoid

Moving to 3.3.1 just to be on the newest version, without an inventory of DAGs, providers, and transitive dependencies.
Storing business payloads in the Task State Store and later trying to reconstruct audit from an orchestration metadata database.
Using the Java/Go SDK to bypass image governance, secret scanning, dependency review, and observability.
Increasing workers up to the default quota of 25 without checking whether the real bottleneck is scheduler, database, external API, pool, or downstream.
Allowing broad bulk clear or rerun without approval, change annotation, and an explicit reprocessing window.

Well-Architected reading

security: I would keep MWAA on private access when the audience is internal, use IAM scoped by environment, KMS for DAG/log/data buckets, and S3 conditional policies such as aws:SecureTransport and domain-specific prefixes. Connections should reference managed secrets, not credentials embedded in DAGs or bundles.
reliability: State Store improves recovery, but reliability comes from idempotency, controlled replay, pools, cause-aware retries, and domain isolation. For critical workloads, I would separate environments by criticality or domain to prevent experimental DAGs from pressuring shared schedulers and workers.
performance: MWAA classes should be selected by measurements: parse time, scheduler lag, task duration, CPU/memory use, and effective concurrency. Webserver scaling reacts to CPU above 70 or ActiveConnectionCount above 15, but that solves UI/API pressure, not processing bottlenecks.

My technical verdict before the final verdict

This version does not change my basic rule for Airflow: it should orchestrate, not do heavy processing inside the worker. On MWAA, I would still call Glue, EMR Serverless, Batch, ECS, EKS, Lambda, or specialized services for substantial work, leaving the DAG to coordinate dependencies, windows, contracts, execution state, and retry decisions. The State Store makes that coordination more honest, especially when a worker can die after submitting a remote job and before recording the result.

I also see a broader trend in recent AWS signals: managed services receiving finer operational controls. CloudWatch added alarm warm-up, DocumentDB received direct major-version upgrade, Redshift strengthened IAM Identity Center authentication with VPC routing, and now MWAA follows Airflow’s evolution in state and multi-language execution. The direction is clear: less undifferentiated operation, more responsibility for policy, governance, and change design.

For engineering leadership, the right conversation is not “should we enable Airflow 3.3.1?” The question is: which incidents, rework, and manual controls does this version remove without increasing risk? If the answer points to incremental reprocessing, long-running jobs, smarter retries, and clean separation of Python orchestration from Java/Go domain logic, there is concrete value. If the answer is only novelty, I would wait.

Curator note: I would adopt MWAA with Airflow 3.3.1 first in a data domain with frequent reprocessing and controlled impact, not in the company’s most sensitive flow. My practical lesson is that orchestrators fail less because of missing features and more because of poorly defined state, lazy retries, and diffuse ownership. The State Store is useful precisely because it forces the team to name the state; without discipline, it only moves the debt elsewhere. For Java and Go, I would wait for the team to prove packaging, logs, and rollback before calling it a standard.

Verified references

Verdict

My recommendation is selective adoption: strong for incremental pipelines, long-running jobs, recovery after worker failure, asset partitioning, and more explicit retry policies; cautious for Java/Go SDKs until the organization proves packaging, security, observability, and support. On an enterprise architecture scale, I would rate MWAA with Airflow 3.3.1 at 8/10 as a managed orchestration platform, provided the team keeps operational state small, auditable data outside Airflow, and an upgrade plan tested. I would not use this version as an excuse to put heavy logic inside workers or to replace data governance with DAG metadata.

Rating: 8/10

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

DocumentDB 8.0: direct MVU reduces risk, not migration work

Fernando Azevedo — Tue, 01 Sep 2026 20:57:09 +0000

The August 31, 2026 announcement looks small: Amazon DocumentDB now allows in-place major version upgrades from 3.6 and 4.0 clusters directly to version 8.0 while preserving data, configuration, and endpoints. In financial-grade environments, I read it as more than operational convenience. It changes the risk economics of delayed modernization: fewer hops, fewer windows, less temporary inventory, and a more realistic path out of old engines without turning the migration into a multi-month side program. But I would not confuse the upgrade button with an upgrade strategy. Version 8.0 brings mandatory TLS 1.2, a new planner, collation, views, Text Index V2, Zstd compression, and behavior differences that can expose old assumptions in queries, indexes, and drivers. The value is in industrializing the move, not in treating it as routine maintenance.

What I would measure before approval

3.6/4.0 -> 8.0 — Announced path. The What's New page announces direct upgrades to 8.0 from 3.6 and 4.0; I would validate target engine availability per Region through the API before the change.
TLS 1.2+ — Compatibility gate. DocumentDB 8.0 does not accept TLS 1.0/1.1; any old client must be identified before the window.
256 TiB — 8.0 cluster limit. The quotas documentation lists 256 TiB for clusters and databases on version 8.0 and later, compared with 128 TiB on earlier versions.

What really changed

I see this release as a pressure relief mechanism for organizations that stayed on old DocumentDB versions because of application dependencies, downtime concerns, or regulatory backlog. Previously, the typical mental model was sequential modernization: inventory, fix clients, rehearse an intermediate upgrade, stabilize, repeat, and only then reach the desired version. With dozens of clusters across business domains, that choreography multiplies change approvals, evidence collection, regression testing, and periods where several engine versions coexist.

A direct upgrade to 8.0 mainly reduces the number of intermediate states that must be governed. Instead of keeping an intermediate version only as a stepping stone, I can organize an evidence-driven transition: clone, rehearsal, realistic workload validation, cutover window, and post-upgrade observability. The cluster remains the same logical endpoint and storage, which reduces DNS changes, secret rotation, and application reconfiguration.

The critical point is that “in-place” does not mean “no downtime.” The MVU documentation is explicit that the cluster is unavailable during the upgrade and may reboot multiple times. In financial systems, that moves the decision into the territory of SLOs, business calendar, and operational reversibility, not only engine compatibility.

Where 8.0 actually helps

The technical reason to target 8.0 is not only escaping Extended Support. The version brings compatibility with MongoDB 6.0, 7.0, and 8.0 API drivers, Planner Version 3, collation, views, Text Index V2, new aggregation operators, and Zstd compression. AWS has also published performance improvements of up to 7x lower aggregation pipeline latency and up to 5x better compression ratio in specific scenarios. I would treat those numbers as observed ceilings, not universal promises; still, they show where the version can pay for itself.

Read-heavy workloads, recurring aggregations, simple text search, small documents, and well-indexed query patterns are good candidates. In a customer service, fraud detection, or digital onboarding platform, for example, the combination of better compression and a more capable planner can reduce I/O pressure and stabilize p95/p99 when enrichment queries or operational dashboards read large collections.

There are architecture benefits as well. Views help encapsulate canonical projections instead of spreading that logic across services. Collation reduces homegrown handling for textual ordering and comparison. Zstd can improve storage and I/O economics, but it should be enabled with measurement because compression trades CPU for storage efficiency. In an operational database, that trade must show up in metrics, not conviction.

The operating model I would use for the upgrade

The center of the decision is not the upgrade command; it is the evidence loop before, during, and after the window.

🧭 Governance and readiness

Cluster inventory 3.6/4.0, owners, SLO (ci)
Compatibility review TLS, drivers, indexes (security)
Change record rollback and evidence (ci)

🟧 AWS validation lane

DocumentDB clone same instance count (data)
Replay tests queries, jobs, APIs (compute)
CloudWatch + logs CPU, I/O, cursors, p99 (data)

🟦 Production lane

Manual snapshot pre-upgrade restore point (storage)
In-place MVU 3.6/4.0 to 8.0 (data)
Index metadata refresh wait for completion (data)
Applications feature flags, retry budget (compute)

Flows

inventory -> compat: classifies risk
compat -> clone: defines rehearsal
clone -> tests: runs representative load
tests -> metrics: compares baseline
metrics -> cab: produces evidence
cab -> snapshot: authorizes window
snapshot -> mvu: restore point
mvu -> refresh: post-upgrade
refresh -> apps: resumes traffic
apps -> metrics: observes regressions

The limits I would not ignore

The first limit is documentary and operational. While researching on September 1, 2026, I found the What's New page announcing direct upgrades from 3.6 and 4.0 to 8.0, while the MVU documentation page I opened still showed a table with 3.6/4.0 to 5.0 and 5.0 to 8.0, plus a note saying there was no direct path. I would trust the newer announcement as the service direction, but I would not approve production without confirming in the actual environment through describe-db-engine-versions, the target Region console, and AWS Support if needed. That discrepancy is exactly the kind of detail that breaks a well-planned window.

The second limit is compatibility. DocumentDB is not upstream MongoDB; it is API-compatible, with documented functional differences. An engine upgrade can change query planning, text index behavior, default collation for new objects, and TLS requirements. Old applications can “work” in happy-path tests but fail under pool exhaustion, TLS renegotiation, BSON serialization, or aggressive timeouts.

The third limit is post-upgrade state. The documentation describes index metadata refresh after MVU, usually in minutes but potentially up to two hours, with guidance to contact support if it exceeds three hours. I would not mark the window complete when the cluster returns to available; I would close it only after critical queries, batch jobs, and capacity signals stabilize.

Where the feature shines

It reduces the number of change windows for old clusters, which matters when each outage requires communication, regulatory evidence, and an approved rollback plan.
It preserves endpoints, tags, storage, and cluster configuration, reducing changes in applications, secrets, runbooks, and observability dashboards.
It creates a shorter bridge to 8.0 capabilities such as Planner Version 3, Zstd, collation, views, Text Index V2, and newer drivers.
It makes it easier to financially justify leaving DocumentDB 3.6, which entered Extended Support in 2026 and carries an additional cost premium.
It favors fleet modernization programs: inventory, waves, automated prechecks, standardized evidence, and domain-level governance.

The dangerous part is overconfidence: I would not run this upgrade as an isolated database change. The direct path shortens the journey, but the cluster is still unavailable, there is no in-place downgrade, clients must support TLS 1.2 or higher, and global or elastic clusters have their own upgrade restrictions. For critical production, restoring a snapshot to a new cluster is a recovery plan, not a magic rollback button; it changes endpoints, return time, and application reconnection operations.

How I would design the window in a financial environment

My approach would start with criticality classification. A cluster serving an internal catalog does not deserve the same process as one participating in fraud checks, onboarding, Open Finance consent, or transactional experience. For each critical cluster, I would capture owner, RTO, RPO, volume, number of collections, number of indexes, drivers per application, runtime versions, use of change streams, transactions, partial indexes, text search, and integrations with Lambda, Glue, MSK, or nightly jobs.

Then I would create a clone of the cluster, with the same instance count as the target whenever possible, to estimate real duration and catch regressions. Testing must go beyond health checks: representative query replay, explain plans where useful, batch jobs, APIs with realistic connection pools, controlled failover outside the window, and comparison of p95/p99, CPU, I/O, FreeableMemory, DatabaseConnections, DatabaseCursors, and slow query logs.

During the window, I would freeze deployments that could change access patterns, reduce concurrency for non-essential jobs, confirm a manual snapshot, and apply the upgrade with an explicit engine version and parameter group. After the cluster returns, I would keep an observation mode: no scaling, no reboot, and no forced failover until index metadata refresh completes. Final evidence would be both functional and operational: business transactions, latency, errors, queues, jobs, and dashboards.

Practical decision: direct upgrade, two hops, or parallel migration

Criterion	Option	When it fits	Main risk
Direct MVU to 8.0	My preferred option when the Region and cluster confirm the path, drivers have been tested, and the downtime window is acceptable.	It concentrates more change in one window; any incompatibility appears closer to cutover.	—
Two-hop MVU	Useful when documentation, the Region, or Support still indicates an intermediate path, or when the organization wants to reduce functional delta per step.	More windows, more time in transitional state, and higher coordination cost.	—
Parallel migration with DMS or application flow	Good for clusters that cannot tolerate MVU downtime, need data model redesign, or want to validate 8.0 with shadow traffic.	It doubles complexity: synchronization, dual-write or CDC, reconciliation, cutover, and possible data divergence.	—

The adoption path I would approve

Inventory by risk, not by AWS account — Group clusters by criticality, RTO/RPO, Region, version, instance class, use of transactions, change streams, indexes, and application dependencies. The fleet plan must expose who accepts downtime and who needs parallel migration.
Validate path and prerequisites in the target Region — Use the console and aws docdb describe-db-engine-versions to confirm 8.0 as an available target, apply pending OS patches, review old instance families, configure the target parameter group, and handle partial indexes or problematic collection names before the window.
Rehearse on a clone with representative load — Clone the volume, keep instance count close to production, run the upgrade, and execute real queries, batch jobs, and APIs with production-like pools. Compare latency, CPU, I/O, connections, cursors, and slow log baselines.
Prepare an honest rollback — A manual snapshot is mandatory, but restore creates a new cluster. Document DNS, secrets, security groups, parameter groups, estimated restore time, abort criteria, and who has authority to decide during the window.
Operate post-upgrade as stabilization — Wait for index metadata refresh, validate critical transactions, re-enable jobs in waves, watch p95/p99 and error rate by endpoint, and only then consider enabling new capabilities such as Zstd on existing collections or views in applications.

Likely failure modes and how I would mitigate them

The most common database upgrade failure is not the service failing; it is the application revealing an invisible dependency. I would look for old Java, Node.js, Python, or .NET clients, container images without updated CA bundles, tlsAllowInvalidCertificates in legacy environments, rigid timeouts, and pools with aggressive reconnection. From 8.0 onward, TLS 1.2 or higher is required; that needs to become an automated test, not a manual checklist.

On the data side, I would inspect indexes and queries. Indexes inherited from 3.6/4.0 may work but not deliver the expected plan until rebuild or refresh. The documentation recommends considering reIndex after upgrades from 3.6/4.0 for optimal performance, with additional I/O cost. I would not do that blindly in production; I would prioritize collections appearing in top queries, critical dashboards, and jobs with SLAs.

Operationally, I would treat the upgrade event as controlled unavailability. API Gateway, ALB, Lambda, EKS, and workers must fail predictably: circuit breakers, maintenance responses, queues with DLQs where appropriate, retries with jitter, and idempotency for write commands. If the application keeps trying to write without limits during the window, the database returns to an artificial storm. The best architecture here is the one that knows how to stay quiet when a critical dependency is explicitly unavailable.

Anti-patterns I would block

Running the direct MVU without a clone, without duration estimates, and without replaying the queries that support business SLOs.
Using the return to available as the only success criterion while ignoring index metadata refresh, p99 latency, driver errors, and job backlog.
Combining engine upgrade, data model change, driver replacement, new compression policy, and application refactoring in the same window.
Promising fast rollback without rehearsing snapshot restore to a new cluster, secrets update, DNS, or consumer application configuration.
Approving the change only because Extended Support costs more, without quantifying downtime, compatibility risk, and validation cost.

Well-Architected reading

security: The TLS 1.2+ requirement is an opportunity to remove insecure clients, review certificate rotation, validate Secrets Manager, and restrict IAM around docdb:ModifyDBCluster, docdb:CreateDBClusterSnapshot, and ARN/Tag scope where possible.
reliability: The architecture must declare the unavailability: communicated maintenance, tolerant queues, write idempotency, manual snapshot, restore runbook, and objective abort-or-continue criteria.
performance: Planner Version 3, Text Index V2, and Zstd can improve latency and efficiency, but they count only after comparison with query-level baseline, index cardinality, document size, and real aggregation behavior.

My curation note: I would use the direct MVU to 8.0, but only after proving the path on a clone and resolving the temporary discrepancy between the announcement and operational documentation. In a financial environment, fewer steps are good; less evidence is dangerous. The lesson I have learned in database modernization is simple: the risk is rarely in the managed command; it is in the silent dependencies that were never tested under controlled failure. I would treat this release as a chance to retire old technical debt with discipline, not as a shortcut.

Verified references

Verdict

My rating is 8/10. The direct upgrade to DocumentDB 8.0 is a very welcome capability for reducing accumulated risk, Extended Support cost, and modernization program complexity. I recommend adopting it for 3.6 and 4.0 clusters that can tolerate an outage window, provided the path is confirmed in the Region, drivers pass realistic testing, a manual snapshot exists, and post-upgrade monitoring continues until indexes and latency stabilize. For systems without an acceptable window or with poorly understood dependencies, I would still choose parallel migration or smaller waves. The service has improved; the architectural responsibility remains proving that the application can absorb the change.

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

PRM by User Agent: better attribution, governance still required

Fernando Azevedo — Tue, 01 Sep 2026 20:56:37 +0000

The August 31, 2026 announcement expanding Partner Revenue Measurement to more services through User Agent looks, at first glance, like an AWS partner ecosystem detail. I read it differently: it is another step in the convergence of operational telemetry, product monetization, and commercial governance. The mechanism remains simple: partner applications making regular AWS API calls include an identifier in the User Agent using the APN_1.1/pc_<AWS Marketplace product-code>$ format; AWS uses applicable control-plane events logged in CloudTrail to attribute aggregated consumption to partner products. The value is fewer blind spots. The risk is mistaking monthly aggregated attribution for cost accounting, detailed billing, or absolute proof of adoption.

What was confirmed

24 — services listed for User Agent. The current included-services documentation lists 24 service codes, including EC2, S3, RDS, CloudFront, EventBridge, Kinesis, WAF, and Shield.
17 dias — dashboard availability lag. The Attributed Revenue documentation states monthly processing and availability 17 days after the previous month ends.
1 KB — maximum userAgent field size in CloudTrail. CloudTrail documents a 1 KB limit for userAgent, with truncation under event-size conditions.

What actually changed

The change does not require a new library, a new agent, or a heavy integration. Partners that already embedded the required User Agent format automatically benefit from the expanded coverage, according to AWS. That matters because real architectures rarely drive consumption in only one service. A data connector may touch S3, Glue, EventBridge, CloudWatch Logs, and Kinesis; a security platform may call WAF, Shield, CloudFront, Route 53, EC2, and ELB; a DevOps tool may operate CodeBuild, ECS, and DynamoDB. If attribution sees only a slice, the commercial conversation becomes distorted.

The technical point is that AWS is using evidence that already exists in the control plane: API calls recorded in CloudTrail. This favors solutions that automate creation, update, inspection, or operation of AWS resources. It does not equally favor purely passive workloads, data-plane traffic without monthly control-plane activity, or components that run once and disappear. To me, the expansion turns User Agent into a low-friction instrumentation path for ISVs, but it does not remove the need to combine methods: Marketplace Metering when the product is an AMI or ML product sold through Marketplace, Resource Tagging when there are persistent attributable resources, and User Agent when the natural evidence is application action.

How User Agent attribution enters the operating cycle

The diagram shows the right boundary: the application emits instrumented calls; CloudTrail records operational evidence; PRM consolidates attributed revenue; the partner uses it for product and partner management, not to replace granular FinOps.

🏢 Produto do parceiro

Aplicação ISV SDK/CLI AWS (compute)
Application ID APN_1.1/pc_$ (security)



  
  
  ☁️ Conta AWS do cliente



APIs AWS suportadas chamadas de plano de controle (edge)
Serviços medidos EC2, S3, RDS, Kinesis... (data)
AWS CloudTrail campo userAgent (security)



  
  
  📊 AWS Partner Central



Partner Revenue Measurement consolidação mensal (data)
Attributed Revenue Dashboard produto, serviço, mês (frontend)



  
  
  🧭 Gestão



FinOps e produto hipóteses e reconciliação (user)



  
  
  Flows



app -> ua: configured once per runtime
ua -> api: attaches User Agent to regular calls
api -> services: operates supported resources
api -> trail: generates events with userAgent
trail -> prm: evidence for attribution
prm -> dash: aggregates by product, service, and month
dash -> finops: guides commercial analysis



  
  
  Where it shines


The strongest use case is a product that operates AWS environments on behalf of the customer: provisioners, data gateways, observability platforms, security tools, backup automation, landing zone accelerators, integration engines, and modernization products. In these scenarios, placing the identifier in the SDK reduces dependency on tags for resources the partner may not directly control. It also avoids the classic aws-apn-id conflict: a resource can have only one tag with that key, so two partners cannot share the same marker on the same resource without an explicit ownership decision.

In financial-grade environments, that operational detail matters. A governance platform that creates EventBridge rules, updates WAF, writes configuration to S3, and queries CloudWatch Logs needs to demonstrate value in an auditable way without invading the customer's accounting model. User Agent provides a lower-sensitivity signal: it shows that the solution drove consumption of certain services, aggregated by product and period, without exposing workload detail in the dashboard that should remain under the customer's control. That separation is healthy. The partner gets an impact view; the customer keeps control over detailed financial data, Cost Explorer, CUR, internal tags, and cost centers.


  
  
  Strengths of the approach



Low adoption friction: configuration can be applied through Application ID via shared config file, environment variable, or JVM property, depending on the SDK and tool.
Good fit for automation: products making recurring API calls can generate usage evidence without requiring tag mutation on every resource.
Less conflict in multi-partner ecosystems: the User Agent belongs to the call made by the solution, while tags compete on the same resource.
More objective commercial governance: the dashboard consolidates product, service, and month, helping product leadership, alliances, and FinOps discuss trends from a shared base.



  
  
  Where it hurts


The main limitation is semantic: a control-plane call does not automatically equal perfect financial causality. If the application creates a resource once and the customer operates it directly for months, the User Agent evidence may become weak, because the documentation says monthly API operations on resources are required for attribution to occur. The opposite also deserves care: automation may touch an existing resource and, depending on PRM rules, produce attribution that must be interpreted in the right product, contract, and operating context.

Another point is that userAgent was not designed as a financial ledger. In CloudTrail it identifies the agent through which the request was made, has a documented maximum size of 1 KB, and can be truncated under event-size limits. For requests originated by AWS services, the field may reflect the calling service, not necessarily the original SDK client. I would not build a variable compensation process, financial pass-through, or contractual SLA solely on the attributed revenue dashboard. I would use the dashboard as aggregated evidence and trend data; for reconciliation, I would cross-check Marketplace, CUR, contracts, opportunity IDs, internal tags, and customer-consented data.


Do not treat PRM as the partner's Cost Explorer: The Attributed Revenue Dashboard is aggregated by product, service, and billing period, with monthly availability. It is excellent for direction, coverage, and partner conversations; it does not replace CUR, Cost Explorer, chargeback, showback, budgets, cost anomalies, or the customer's internal audit trails.



  
  
  How I would adopt it in a real solution


I would start with the product causality matrix, not the code. For each capability, I would map which AWS calls the solution makes, in which account, in which Region, at what frequency, and with what relationship to the delivered value. In a SaaS platform managing resources in the customer's tenant, I would require a single runtime wrapper for AWS SDK clients: Node.js v3, boto3, Java v2, Go v2, or whatever stack the application uses. That wrapper would set the Application ID once, prevent divergent strings per microservice, and emit structured logs for critical calls: aws.service, aws.operation, aws.region, productCodeConfigured=true, requestId when available, and outcome.

In CI/CD, I would add contract tests validating the presence of AWS_SDK_UA_APP_ID or equivalent configuration in the container, Lambda, ECS job, CodeBuild project, or runner that actually makes AWS calls. In production, I would verify samples in CloudTrail Lake or S3-delivered logs, searching for the APN_1.1/pc_ prefix in the userAgent field. For accounts with SCPs and permission boundaries, I would not grant extra permission because of PRM; instrumentation must follow calls the solution is already authorized to make. Security comes before attribution.


  
  
  Adoption path I would approve



1. Define the responsibility boundary — List only calls made directly by the partner solution. AWS documentation warns not to configure User Agent strings for customer-initiated calls that are independent of the solution.
2. Centralize configuration — Use SDK Application ID, the AWS_SDK_UA_APP_ID variable, the ~/.aws/config file, or JVM property depending on the runtime. Avoid copying strings manually across dozens of clients.
3. Validate in CloudTrail before celebrating the dashboard — Make real test calls and confirm userAgent, eventSource, eventName, awsRegion, and requestID. Then monitor the monthly dashboard window, knowing the documented availability is 17 days after month end.
4. Combine methods when needed — Use tags on persistent resources where appropriate, Marketplace Metering for eligible AMI/ML products, and User Agent for API automation. No single method models every product type.
5. Build monthly reconciliation — Compare PRM trends with contracts, expected consumption, release notes, incidents, customer expansion, and architecture changes. The operating question is: does the signal make sense?



  
  
  User Agent, tags, and Marketplace Metering





Criterion
Best use
Main risk
Recommended control




User Agent
Solutions making regular API/CLI calls in customer or partner accounts.
Confusing operational calls with complete financial causality.
Single SDK wrapper, configuration test, and CloudTrail sampling.


Resource Tagging
Persistent resources clearly associated with the product.

aws-apn-id conflict, IaC drift, and tag removal by any user with suitable access.
IaC-based tagging, tag policy, AWS Config, and explicit customer agreement.


Marketplace Metering
AMI and ML products purchased and consumed through AWS Marketplace.
Coverage limited to the eligible product model.
Validation of listing, product code, and Marketplace commercial flow.





  
  
  The Well-Architected reading


From the operational excellence pillar, I like the expansion because it forces a discipline many companies postpone: instrumenting their own operational footprint. A mature product knows which APIs it calls, why it calls them, which retry policy it applies, which idempotency token it uses, and which cost impact it may induce. If the solution creates S3 buckets, DynamoDB tables, Kinesis streams, or EventBridge rules, PRM through User Agent should be the consequence of an already observable architecture, not a commercial decoration added at the end.

From the security pillar, I would be conservative. The product identifier must not leak secrets, tenant IDs, customer names, environments, or contracts. The published format uses the Marketplace product code, not sensitive data. I would enforce that pattern strictly and block attempts to place commercial context in the User Agent. In regulated organizations, I would also review privacy documentation and DPAs: even if the dashboard is aggregated, the signal originates from control-plane events, and that needs to be described in the customer transparency model.

From the cost pillar, the benefit is indirect. PRM does not reduce the bill, but it helps compare where the product actually moves AWS consumption. That improves roadmap decisions: perhaps the product sells an analytics story, but attributed revenue appears in EC2 and RDS; perhaps a serverless promise produces more CloudWatch Logs than Lambda. Those deviations are architecture speaking to the business.


  
  
  Controls by pillar




security: Do not place customer data in the User Agent. Ensure the string is only the APN format with product code, validate least privilege, and preserve segregation between partner and customer accounts.

reliability: Configure User Agent at the common SDK client creation point to avoid partial coverage. Monitor call failures, throttling, and retries so lack of attribution is not confused with lack of value.



  
  
  Anti-patterns I would avoid



Putting the User Agent in generic customer automation that does not belong to the partner product.
Adding tenant, email, customer name, or commercial opportunity into the User Agent to create inappropriate granularity.
Using the monthly dashboard as the single proof for commission, renewal, or consumption disputes.
Manually tagging resources managed by Terraform, CDK, or CloudFormation, causing drift and loss of operational trust.
Instrumenting only the services currently measured and forgetting that AWS recommends applying PRM to the services and resources the solution interacts with to reduce rework as coverage grows.



My curator note: I would implement User Agent in any serious partner product that operates AWS through SDKs, but I would put it under the same rigor as observability, security, and FinOps. The practical lesson is simple: weak commercial signals become political discussions; well-instrumented operational signals become objective conversations. Still, I would never promise ledger-level precision from a monthly aggregated source. For financial-grade systems, good architecture separates evidence, attribution, billing, and executive decision-making.



  
  
  Verified references



AWS What's New: Partner Revenue Measurement expands service coverage for User Agent string capability
AWS PRM Documentation: Included AWS Services for User Agent string
AWS PRM Documentation: Automated User Agent
AWS PRM Documentation: Troubleshooting Partner Revenue Measurement
AWS Partner Central Documentation: Attributed Revenue
AWS CloudTrail Documentation: Record contents and userAgent field
AWS PRM Documentation: Manual Resource Tagging implementation
fieldnotes: Attributed Revenue Dashboards - almost completing the PRM puzzle



  
  
  Verdict


My recommendation is to adopt it. For AWS partners, especially ISVs operating customer workloads through APIs, the PRM User Agent expansion improves impact visibility with low technical cost and without requiring a new operating surface. The score is not higher because the capability still depends on service coverage, control-plane events, monthly processing, and careful interpretation. I would put it in the immediate backlog for partner platforms, with three conditions: centralized configuration, CloudTrail validation, and monthly reconciliation with FinOps and commercial data. As attribution technology, it is strong; as a true financial system, it still needs surrounding controls.

Rating: 8/10




Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Criterion	Best use	Main risk	Recommended control
User Agent	Solutions making regular API/CLI calls in customer or partner accounts.	Confusing operational calls with complete financial causality.	Single SDK wrapper, configuration test, and CloudTrail sampling.
Resource Tagging	Persistent resources clearly associated with the product.	`aws-apn-id` conflict, IaC drift, and tag removal by any user with suitable access.	IaC-based tagging, tag policy, AWS Config, and explicit customer agreement.
Marketplace Metering	AMI and ML products purchased and consumed through AWS Marketplace.	Coverage limited to the eligible product model.	Validation of listing, product code, and Marketplace commercial flow.

MediaTailor Analytics: console, BI, or lakehouse?

Fernando Azevedo — Tue, 01 Sep 2026 20:56:35 +0000

When AWS adds a native MediaTailor dashboard for fill rate, impressions, video completion, and beacon recovery, the right question is not whether the console became nicer. The architecture question is: which layer should answer each class of question about money, viewer experience, and operational health? In streaming systems with real monetization, especially with contracts, audit, multiple regions, and ad partners, an aggregated metric can accelerate a decision or hide a loss. I would treat the new dashboard as the level-one control room: fast, official, and useful for triage. For financial close, SLOs, and causal investigation, I would still design a more explicit evidence chain.

The real choice: operational speed versus analytical depth

The August 31, 2026 announcement is small, but it touches a recurring issue in media platforms: the gap between available metrics and actionable metrics. MediaTailor already published metrics to CloudWatch; what changes is the curation inside the console, with a global and multi-region view, comparison between region groups, and drill-down by playback configuration. That shortens the time between noticing a monetization drop and locating a suspicious configuration or tracking domain.

I would compare four options. The first is using the native dashboard as the primary operations experience. The second is building custom CloudWatch dashboards and alarms. The third is moving logs, beacons, and business data into a lakehouse on S3, Glue/Athena, or Redshift. The fourth is relying on an ad-tech or QoE SaaS platform that correlates player telemetry, ads, and commercial contracts. None is universally better. They answer different questions, with different latency, cost, governance, and accountability.

The trap is expecting one screen to solve monetization, engineering, and finance. In financial-grade environments, I split the problem into three planes: operational triage in minutes, technical diagnosis in hours, and reconciliation/audit in closed reporting windows. The new dashboard strongly improves the first plane.

What the native dashboard actually buys

The native dashboard’s main advantage is semantics. It already knows which metrics matter for SSAI: weighted fill rate, ad insertion rate, ad impression rate, video completion rate, and ad insertions. The documentation also makes important formulas explicit, such as weighted fill rate calculated as summed filled duration divided by summed avail duration, not as a simple average across regions. That sounds operational, but it avoids a common distortion: a small region with marginal traffic should not carry the same statistical weight as a primary region.

Another gain is the beacon dimension by tracking domain. The August 2026 metrics for fired, retried, and recovered impression and complete beacons, associated with AdTrackingDomain, create a view closer to the money: not only how many ads were inserted, but how many billable confirmations were recovered by server-side retry. Practically, this helps separate ADS issues, tracking vendor issues, network issues, and player issues.

I also value that the dashboard opens on the last 1-day window and supports region comparison. For operations, that is enough for first triage: “is the problem global, regional, configuration-specific, or partner-specific?” Answering that quickly saves long calls across teams.

Comparison of the four approaches

Criterion	Option	Best use	Decision latency	Governance	Main risk
Native MediaTailor dashboard	Daily triage of SSAI monetization and performance by region, configuration, and beacon domain.	Minutes; uses CloudWatch metrics in the console.	Read IAM for MediaTailor and CloudWatch; good for controlled operations.	Becoming the source of truth for financial reconciliation without a separate data trail.	—
Custom CloudWatch	Alarms, SLOs, product dashboards, and correlation with origin, CDN, and applications.	Minutes to tens of minutes, depending on period and alarms.	Excellent for IaC, versioned alarms, tags, and runbooks.	Cardinality, API cost, and alarm sprawl without a clear service model.	—
Lakehouse on S3/Glue/Athena/Redshift	Audit, financial close, historical analysis, and correlation with commercial contracts.	Hours or closed windows; can be near real time with Firehose, but does not need to be.	Strong for retention, lineage, cataloging, KMS, and segregated access.	Building an expensive analytics platform to answer simple operational questions.	—
QoE/ad-tech SaaS	Independent player view, user journey, partner validation, and benchmarking.	Minutes to hours, depending on SDKs and ingestion.	Depends on contracts, DPA, data residency, and privacy controls.	Vendor lock-in without reconciliation against official SSAI provider metrics.	—

Where I would draw the CloudWatch boundary

I would not duplicate the native dashboard in CloudWatch out of platform vanity. I would use the MediaTailor console for exploratory questions and create custom CloudWatch assets only where there is an automatic operational decision: an alarm, an SLO, a runbook, or a capacity change. For example, an ad impression rate drop alarm per playback configuration should exist if it triggers investigation or mitigation. A chart someone looks at once a month may not deserve IaC.

The dashboard’s minimum access requires mediatailor:ListPlaybackConfigurations, cloudwatch:ListMetrics, and cloudwatch:GetMetricData. In a serious operation, I would separate roles: operators with multi-region read access, engineers allowed to change playback configurations through a pipeline, and automation accounts scoped explicitly by region and tag. For MediaTailor, the authorization model includes resources such as playbackConfiguration and conditions through aws:ResourceTag, so tagging configurations by product, environment, criticality, and owner makes sense.

Cost also needs design. The dashboard documentation says tables can make up to 10 ListMetrics calls per selected region on each load, while tiles and trends do not use ListMetrics. That is not a problem in normal operations, but it can become noise in NOCs with screens refreshed all day. I would put CloudWatch API usage into the FinOps review, not as a blocker, but as hygiene.

Decision layers for SSAI analytics

The diagram shows how I would separate triage, diagnosis, and reconciliation without turning the native dashboard into the single source of truth.

🎬 Tráfego de streaming

Players HLS/DASH sessões e beacons (user)
CloudFront/CDN cache e entrega (edge)
Origem de conteúdo manifests com markers (storage)

🟧 SSAI AWS

MediaTailor playback configurations (compute)
ADS externo VAST/VMAP (external)
Retry de beacon fired/retried/recovered (messaging)

📊 Observabilidade

CloudWatch Metrics AWS/MediaTailor (data)
Dashboard nativo triagem multi-região (frontend)
Alarmes/SLOs runbooks versionados (security)

🏦 Evidência de negócio

Logs e exports S3/Firehose (storage)
Lakehouse Glue/Athena/Redshift (data)
Reconciliação contratos e billing (external)

Flows

player -> cdn: requests manifests/segments
cdn -> mt: SSAI personalization
mt -> origin: fetches base manifest
mt -> ads: requests ad decision
mt -> beacon: fires and retries beacons
mt -> cw: publishes metrics
cw -> native: GetMetricData/ListMetrics
cw -> alarms: evaluates SLOs
mt -> logs: ADS/reporting events
logs -> lake: partitions by date/region/config
lake -> finance: closes revenue gaps

Lakehouse: when a metric becomes evidence

The dashboard answers “what is happening now?” well. It should not be the only artifact to answer “how much revenue did we recognize and why?” When there is revenue share, premium campaigns, make-good clauses, or partner audit, I want reprocessable data. The lakehouse layer belongs there: ADS and reporting logs, CSV exports when useful, player data, content catalog, contract tables, and financial events. The minimum partitioning I would use is dt, region, configuration_name, ad_tracking_domain, and event_type, with KMS managed by the media domain key, Lake Formation for role-based access, and S3 lifecycle separating raw, curated, and aggregated zones.

Granularity matters. A CloudWatch metric is excellent for trend, but it loses event context. Logs can explain whether a fill drop came from ADS timeout, invalid VAST, incompatible creative, unstable beacon domain, or origin limit. MediaTailor has relevant quotas for this analysis: 10,000 ad insertion requests per second per region as an adjustable quota, up to 1,000 configurations, a 2 MB manifest, 3-second default ADS timeout, 2-second origin timeout, and session expiration at 10 times the manifest duration. Those numbers define the operational envelope.

I would not ingest infinitely without purpose. I would define business questions before the pipeline: discrepancy between AdsBilled and impressions, recovery by domain, loss by region, affected campaigns, and estimated revenue impact.

Decision matrix

Start with the native dashboard

Pros

Lowest adoption time; no initial data modeling.
Monetization metrics already calculated with official semantics.
Multi-region comparison and configuration drill-down help triage.

Cons

Does not replace financial reconciliation or event-level investigation.
Tables can consume ListMetrics calls per region on repeated loads.

Verdict: My default choice for level-one operations and immediate adoption.

CloudWatch as a custom operations panel

Pros

Enables alarms, SLOs, warm-up periods, and runbooks treated as code.
Correlates MediaTailor with CloudFront, origin, WAF, Lambda, and internal services.

Cons

Requires discipline against cardinality and ownerless dashboards.
Can duplicate the console without creating new operational action.

Verdict: Use it when there is an associated alarm, SLO, or automation.

Monetization lakehouse

Pros

Creates an auditable, reprocessable trail for revenue, contracts, and partners.
Enables historical analysis, content cohorts, and campaign-level discrepancy review.

Cons

More expensive in engineering, governance, and data quality.
Should not become the primary tool for a five-minute incident.

Verdict: Mandatory when the metric affects revenue recognition or audit.

Independent QoE/ad-tech SaaS

Pros

Brings client-side view, player SDK, and external partner validation.
Can accelerate benchmarking and device-level regression detection.

Cons

Needs privacy, data residency, and lock-in assessment.
Does not remove the need to reconcile against official SSAI metrics.

Verdict: Useful as a second opinion, not as the sole authority.

Real failure modes the comparison must cover

In SSAI, failure is rarely binary. The viewer can keep watching while revenue is lost. The ADS can respond within timeout, but with VAST that results in skipped creatives. The origin can deliver a manifest that is too large or too slow. The tracking vendor can accept some beacons and fail in bursts. A secondary region can look healthy as a percentage while representing little volume. That is why I like comparing counts and rates together.

A plausible example: a live sports final with 500,000 viewers and 18 ad breaks. AWS’s own pricing page uses a similar scenario for monetization functions and shows 9.5 million invocations when there is an initialization hook and a hook per break. In that kind of event, a 2 percentage point drop in impression rate for 20 minutes can be relevant enough to trigger a war room, even if video playback is perfect. On the other hand, a fill drop in a region with 1% of traffic may be less urgent than origin latency affecting every manifest.

I would model idempotency and retry around stable identifiers: playback session, avail, ad, beacon type, tracking domain, and region. For operations, the native dashboard shows the surface. For engineering, logs and metrics must confirm whether retry recovered impressions or merely delayed loss. For finance, only reconcilable evidence counts.

The new signal is beacon recovery: For me, the most important point is not the dashboard itself; it is the visibility into fired, retried, and recovered by tracking domain. That turns server-side retry from an invisible implementation detail into an economic resilience indicator. In media platforms, technical availability and monetization availability are not the same thing.

Well-Architected reading

security: Use least privilege for operational read access: mediatailor:ListPlaybackConfigurations, cloudwatch:GetMetricData, and cloudwatch:ListMetrics should be separated from change permissions. Tags by product, environment, and criticality help apply IAM conditions where the service supports taggable resources.
reliability: Compare regions by volume, not only by percentage. Monitor ADS timeouts, origin failures, manifest limits, and beacon drops. Use alarm warm-up periods when creating resources and alarms together to avoid startup noise.

My phased architecture recommendation

I would adopt the native dashboard immediately in any existing MediaTailor operation. I would do it without a large project: review IAM, validate regions, check playback configuration naming, standardize tags, and document three triage questions. First: is the regression global or regional? Second: is it concentrated in one configuration? Third: does it appear on a specific beacon domain? That alone improves response time.

Then I would create custom CloudWatch assets sparingly. Alarms for sustained drops in impression rate, fill rate, and complete rate need product-specific thresholds, not global ones. For live sports, I would accept shorter windows and more noise; for VOD, I would prefer longer windows and baseline comparison. Where alarms are created together with new configurations or new services, the recent CloudWatch warm-up period feature matters: delaying evaluation for 1 to 2,880 minutes, or starting when enough data exists, reduces bootstrap false positives.

Finally, I would add a lakehouse only when there is a reconciliation, audit, or contract-correlation obligation. The pipeline can start simple: periodic export, logs in S3, Glue catalog, Athena queries, and daily aggregates. Redshift enters when analytical cadence, concurrency, and commercial joins justify it. The mature solution is not the most sophisticated one; it is the one that makes clear which number is for operations, which is for engineering, and which is for revenue.

Curator note: I would start with the MediaTailor dashboard and resist the temptation to build a parallel cockpit in the first week. The lesson I learned in critical environments is that good observability is not the number of charts; it is a short chain between signal, owner, and action. When monetization is involved, however, I never let the console be the only evidence. Operations need speed, but finance and audit need reprocessable data.

References

Verdict

My recommendation is to use the new MediaTailor dashboard as the official operational triage layer, custom CloudWatch only for alarms/SLOs with clear action, and a lakehouse for financial reconciliation and historical analysis. If the operation has nothing yet, do not start with a data platform: start with the console, standardize tags and runbooks, then promote only the metrics that truly change a decision. If there is material revenue, complex contracts, or partner dispute, the dashboard remains valuable, but it is no longer sufficient. In that case, the right architecture is deliberately layered: console for speed, CloudWatch for operational reliability, and reprocessable data for financial trust.

Rating: recommended-layered

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

ADR: bringing governed agents into Amazon Quick

Fernando Azevedo — Tue, 01 Sep 2026 20:56:03 +0000

My decision would be to adopt AWS Agent Registry as the governed catalog for agents and MCP servers exposed to Amazon Quick, but only after clearly separating three responsibilities: technical publishing, risk curation, and business-user consumption. The August 31, 2026 announcement looks small because it mentions discovery and pre-populated connection details. In practice, it moves agent architecture away from a handcrafted model, where each team pastes an MCP URL into a tool, and toward a platform model, where the catalog becomes the operational control plane.

Context: coupling moved from code into the catalog

In financial environments, the first wave of agents is usually productive and risky at the same time. One team creates an MCP server for ticket lookup, another publishes a tool for contract search, a third connects an internal credit-risk API, and soon there is a collection of endpoints with unclear owners, scopes, credentials, and versions. MCP itself is not the issue. The issue is the lack of a reliable inventory once those servers become an operational surface.

What Amazon Quick changes is that this inventory becomes directly consumable by users in the same workspace where they chat, build apps, automate flows, and run research. AWS Agent Registry already provided the catalog, hybrid search, approval, and records for MCP servers, agents, skills, and custom resources. The Quick integration adds the last mile: a user can find an approved resource and create a connector with pre-filled details instead of relying on a wiki, an issue, or a Slack message containing the right URL.

The main architectural force is governance without excessive friction. If I let every area connect MCP servers directly in Quick, I gain local speed and lose traceability. If I centralize too aggressively, I create a queue and encourage bypasses. The governed registry is the middle ground: distributed publishing, centralized discovery, explicit approval, and consumption close to the work.

Options considered

Connect MCP servers directly in Quick per team

Pros

Shortest initial time for prototypes and local automations.
Low dependency on a central platform team.

Cons

Weak inventory, unclear ownership, and connector duplication.
Hard to prove who approved an MCP server, which version was active, and which scope was shared.

Verdict: Acceptable for a lab; unsuitable for regulated use.

Block Quick and consume agents only through internal applications

Pros

Maximum control over UX, authentication, telemetry, and approval flow.
Allows specific policies per critical domain.

Cons

Increases internal product cost and reduces business-user adoption.
Duplicates capabilities Quick already provides: chat, apps, flows, research, and sharing.

Verdict: Useful for very high-risk flows, but expensive as the enterprise default.

Use AWS Agent Registry as the approved catalog for Quick

Pros

Maintains one source of truth for MCP servers and agents discovered in Quick.
Uses approval, search, CloudTrail, tags, IaC, and AWS RAM sharing where applicable.
Reduces manual setup without removing the need for per-user authentication and permissioning.

Cons

Requires maturity around curation, versioning, and record deprecation.
Quick supports only certain MCP descriptors in this integration, so A2A and local endpoints stay out.

Verdict: My choice for an enterprise agent platform.

Decision: treat the registry as a control plane, not a link list

I would decide on one registry per relevant operating environment, starting with production rather than dozens of squad-level catalogs. AWS Agent Registry allows registries to be organized by type, stage, team, or business unit, but in a large organization, early fragmentation destroys discovery. For Quick, I would start with a regional registry aligned to the Quick account itself, because the integration documentation requires the same account and same Region, a READY registry, AWS_IAM authorization with SigV4, the agent-registry namespace, and only one active registry per Quick account.

This has an important consequence: if the company has multiple accounts producing agents, the decision is not merely technical. It is a multi-account governance decision. The Agent Registry general availability announcement mentions AWS RAM sharing and automatic detection of AgentCore Runtime and Gateway resources across an organization, but the Quick integration creates a practical consumption boundary in the Quick account and Region. I would design the topology with a platform account for curation and a Quick account per domain only when regulatory or data-segregation reasons justify it.

The registry must not be populated by enthusiasm. Each record needs a stable name, version, owner, data class, authentication policy, runbook, expected SLO, and domain tags. Without that, the catalog becomes a polished storefront for invisible coupling.

Chosen flow: publish, approve, discover, and share

The design centralizes discovery and approval in Agent Registry, while Quick remains the consumption experience for chat, agents, apps, flows, and research.

🛠️ Engenharia — Publicação

Servidor MCP remoto API interna ou AgentCore Gateway (compute)
Registro versionado mcpServer descriptor (data)

🟧 AWS — Catálogo governado

AWS Agent Registry READY, AWS_IAM, SigV4 (ai)
Curadoria aprovar, rejeitar, depreciar (security)
CloudTrail + tags trilha e ownership (security)

💼 Amazon Quick — Consumo

Admin Quick vincula um registry (user)
Custom MCP connector detalhes pré-preenchidos (frontend)
Times de negócio chat, agents, apps, flows, research (user)

Flows

mcp -> record: describes endpoint and tools
record -> registry: publishes version
registry -> approval: submits for approval
approval -> registry: makes discoverable
registry -> audit: logs APIs
admin -> registry: links in same account and Region
registry -> connector: loads approved MCP servers
connector -> team: shares governed use

Forces: identity, scope, and the risk of invisible automation

The most sensitive decision is not semantic search in the catalog; it is the authorization path between the user, Quick, the connector, and the MCP server. The Quick documentation indicates that Agent Registry does not store connector authentication credentials; each user authenticates or provides credentials according to the connector setup. That is the right model for environments where an account lookup, ticket update, or report generation must reflect the real user identity, not a generic automation identity.

I would avoid service connectors for mutable operations in financial domains unless there is an explicit business authorization layer behind them. For reads, I would still require minimal scopes and domain segmentation. For writes, I would make idempotency keys mandatory, validate payloads, enforce per-user rate limits, and require human confirmation for commands that move money, alter critical registration data, change risk parameters, or affect regulated servicing.

The MCP server design must be as rigorous as an internal public API. Tool names should be stable; schemas should reject extra fields; responses should carry a correlation id; timeouts should be lower than the maximum acceptable interaction time in Quick. As an operating reference, I would target p95 under 2 seconds for simple reads, 5 to 10 seconds for aggregations, and asynchronous execution with Step Functions or a queue for anything beyond that.

Operating model: version agents as internal products

An agent catalog only works if the lifecycle is explicit. I would define states similar to draft, approved, deprecated, and revoked, even when part of that already exists as a native registry workflow. Draft is technical publication that is not yet consumable. Approved requires security review, a business owner, test evidence, and data classification. Deprecated preserves compatibility for a defined window. Revoked removes discovery immediately when there is an incident, loss of ownership, or scope change.

The registry version must mean contract, not decorative release notes. If an MCP server changes a tool schema, changes authorization semantics, or starts performing a write where it previously only read, that is a new version and a new approval. I would use names like domain/capability and semantic versions when the contract is consumed by multiple agents. Tags should carry domain, data-classification, environment, owner, cost-center, and criticality. In AWS accounts, that aligns with IAM conditions and governance reporting, even when the end user only sees a connector in Quick.

For observability, I would require three trails. CloudTrail for Registry control APIs. Structured MCP logs with tenant, pseudonymized user subject, tool, registry-record-version, latency, outcome, and correlation id. Tool-level metrics in CloudWatch or Datadog: invocation count, error rate, p95, timeout rate, auth failures, and deny decisions. Without these trails, the platform cannot distinguish healthy adoption from uncontrolled automation.

Architectural consequence: The integration does not make every registered agent automatically safe for business use. The Quick documentation limits support to MCP records with an mcpServer descriptor, remote servers with a URL, and specific requirements for account, Region, namespace, READY state, and AWS_IAM authorization. Skills, custom records, A2A descriptors, local endpoints such as stdio, docker, or npx, and JWT-based registries should not be treated as available through this path. I would also validate the Region in the console before rollout, because same-day public launch pages show a regional-list difference between the general Registry announcement and the Quick-specific announcement.

Reference implementation for a regulated organization

My reference implementation would start small: one production registry in the Region where Quick and AgentCore are available for the domain, linked by an administrator through Manage account, Permissions, AWS Agent Registry. The minimum policy for the Quick managed or customer role needs to allow reading and searching discoverable records, including agent-registry:SearchDiscoverableRegistryRecords and agent-registry:GetDiscoverableRegistryRecord; administrators also need to list and view registries for setup to appear.

MCP servers would sit behind AgentCore Gateway when there is a need to transform APIs, Lambda functions, or existing services into tools with more governed inbound and outbound authentication. For sensitive internal APIs, I would prefer VPC, PrivateLink, or private connectivity where the integration pattern supports it. The Quick MCP documentation also supports private servers reachable through a Quick VPC connection, but requires the OAuth endpoints used by the MCP server to be publicly reachable; that must enter the security review because many companies wrongly assume the entire authentication path can remain private.

In the backend, DynamoDB can store idempotency state with partition key userId#toolName and sort key idempotencyKey, a TTL of 24 to 72 hours, and an attribute_not_exists condition on first execution. Step Functions should orchestrate long operations with explicit retries, backoff, and compensation. S3 with SSE-KMS stores non-sensitive execution evidence. The key is not putting risk logic in the prompt: authorization, validation, and auditability live in code and policy.

Well-Architected reading

security: Use user identity whenever the action represents human privilege. Restrict Quick IAM to the minimum needed for discovery, validate OAuth scopes in the MCP server, separate reads from writes, and log deny decisions. The catalog reduces shadow AI, but it becomes real control only when approval, tags, CloudTrail, and tool logs are mandatory.
reliability: Treat every tool call as a fallible distributed call. Define timeouts, retries with jitter, idempotency, circuit breakers for unstable dependencies, and degraded responses when the source system is unavailable. A shared connector in Quick can multiply traffic quickly.

Anti-patterns I would block

Registering an MCP server without a technical owner and a business owner.
Exposing write operations with shared credentials and no idempotency.
Using tool description as a substitute for authorization policy.
Approving connectors for convenience without classifying data and operational impact.
Keeping old versions discoverable without a deprecation date.

Curator note: I would adopt this integration, but I would start with three read connectors and one low-risk write flow. My experience is that enterprise catalogs fail when they try to catalog everything before proving the operating model. The first goal is not the number of agents; it is proving that publishing, approval, observability, and revocation work without an emergency meeting. After that, scaling becomes engineering, not hope.

References

Verdict

My recommendation is to approve AWS Agent Registry as the governed catalog for resources consumed in Amazon Quick, with a domain-controlled rollout, versioned records, mandatory curation, and per-tool telemetry. I would not approve adoption as a simple MCP setup shortcut. The real value is turning agents and MCP servers into discoverable, auditable, and revocable internal products without pulling business users away from the environment where they work.

Rating: adopt-with-guardrails

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Private Redshift SSO with EVR: a pattern teardown

Fernando Azevedo — Tue, 01 Sep 2026 20:56:02 +0000

On August 31, 2026, AWS announced that Amazon Redshift now supports authentication through AWS IAM Identity Center for provisioned clusters and serverless workgroups configured with enhanced VPC routing. The change looks small in the console, but it closes a recurring gap in regulated analytics platforms: it was common to keep data, COPY/UNLOAD, and S3 integrations on a private path while part of authentication still required an egress exception. My reading is simple: when the warehouse is financial-grade, subject to data residency, network segregation, and operational evidence, identity is production traffic too.

The problem: SSO was not enough when the network was the control

In many data architectures, SSO is presented as a user experience feature, but the real problem is boundary control. An analyst signs in through Query Editor v2 or a JDBC/ODBC driver, receives a token from IAM Identity Center, and presents that token to Redshift. The critical point is that Redshift should not accept that token without validation; it must validate scopes, exchange the token for a service-scoped session, and resolve the user and group memberships. If the cluster uses enhanced VPC routing, that validation call must also follow the private design.

The older pattern created an inconsistency: the organization invested in private subnets, an S3 endpoint, bucket policies, KMS, VPC Flow Logs, and then accepted an identity egress exception because authentication had to reach regional services outside the governed path. In banking, insurance, and payments environments, that exception becomes an audit question: what flow left, through which domain, with which justification, and how do I prove it was only used for authentication? The new support provides a cleaner answer: Redshift talks to sso-oauth and identitystore through interface endpoints, with Private DNS, security groups, and network logs.

Private authentication flow with Redshift EVR

The diagram shows the core idea: the client still obtains corporate identity, but the calls made by Redshift to validate and resolve identity traverse private endpoints inside the VPC.

🧑‍💻 Client access

Query Editor v2 / JDBC presents Identity Center token (frontend)

🟧 AWS VPC — analytics subnet

Amazon Redshift EVR enabled, not public (data)
Interface endpoint sso-oauth, Private DNS (network)
Interface endpoint identitystore, TCP 443 (network)
S3 gateway endpoint COPY / UNLOAD path (storage)

🔐 Regional identity plane

IAM Identity Center OIDC + assignments (security)
Identity store groups and users (security)

📊 Evidence layer

VPC Flow Logs endpoint traffic evidence (network)
CloudTrail identity/API evidence (security)

Flows

analyst -> query: signs in with corporate credentials
query -> redshift: opens connection with token
redshift -> ssoep: validates and exchanges token
ssoep -> idc: PrivateLink
redshift -> idstoreep: resolves user and groups
idstoreep -> groups: PrivateLink
redshift -> s3ep: data through EVR
ssoep -> flowlogs: flow metadata
idc -> cloudtrail: authentication events

Anatomy of the pattern

The practical anatomy has five pieces. First, the provisioned cluster or serverless workgroup must have enhanced VPC routing enabled and must not be publicly accessible. For provisioned clusters, the documentation confirms an important operational detail: changing EVR restarts the cluster, so I would treat it as a maintenance-window change, with communication to BI and ingestion teams.

Second, the VPC must have DNS hostnames and DNS resolution enabled, because the endpoints depend on Private DNS. Without that, the standard service name keeps resolving to public addresses and the observed behavior looks like an IAM problem when it is actually DNS. Third, I create two interface endpoints in the same availability design as Redshift: com.amazonaws.<region>.sso-oauth and com.amazonaws.<region>.identitystore. Both are required; if either is missing or unreachable, IAM Identity Center sign-in fails.

Fourth, the endpoint security group must allow TCP 443 from the subnets or security groups used by Redshift. Fifth, I would not start with an action-restrictive endpoint policy. The documentation itself recommends keeping the default policy or, if corporate policy requires restriction, scoping by principal and validating sign-in. In identity, an elegant policy that blocks token exchange is just a well-written outage.

When to use it: residency, audit, and closed lakehouse design

I would use this pattern when the architecture already has a clear isolation thesis: Redshift in private subnets, S3 accessed through a gateway endpoint, Glue and Lake Formation through interface endpoints when Spectrum or a lakehouse is involved, and internet egress denied by default. This is the common design in financial environments that treat analytics as a production plane, not as a lab. IAM Identity Center authentication improves governance because it centralizes assignments, groups, and sessions; EVR with PrivateLink improves the network posture because it removes the need for NAT or public allow-listing for authentication.

It also makes sense in organizations that need to run Redshift in a different Region from the IAM Identity Center primary Region. IAM Identity Center multi-Region documentation allows replication of identities, permission sets, assignments, sessions, and metadata to additional Regions, with relevant prerequisites: an organization instance, compatible identity source, customer-managed multi-Region KMS key, and support from the applications involved. I would not treat this as a performance button only. It is a resiliency and residency decision: where the data lives, where users are, which Region carries identity authority, and what behavior is acceptable if the primary identity Region is degraded.

Architectural reading of before and after

Criterion	Without this pattern	With Redshift EVR + private IAM Identity Center
Network boundary	Data may be private, but authentication requires egress or firewall exceptions.	Data and identity validation follow private endpoints governed by the VPC.
Operational evidence	Audit depends on proxy, NAT, or external inspection logs, often incomplete.	VPC Flow Logs, CloudTrail, and Redshift logs can be correlated by time window and user.
Failure mode	Failures appear as generic timeouts or SSO errors that are hard to separate from IAM.	Failures tend to concentrate around private DNS, missing endpoint, SG 443, or wrong Region.

Real failures I would test before calling it done

The first test is to deliberately break the identitystore endpoint in a non-production environment. If the user can still authenticate, something is wrong: maybe EVR is not active, maybe private resolution is not being used, or maybe there is an undocumented public route. The second test is to remove Private DNS and verify whether the runbook captures the error. Many teams only notice this detail when a DNS incident turns sign-in into an intermittent failure.

I would also validate Region placement. The documentation says to create the endpoints in the IAM Identity Center Region; when Redshift and Identity Center are in different Regions, the design must use the cross-Region endpoint option or multi-Region replication. This is easy to miss in organizations that standardized identity in us-east-1 but created regional data warehouses for residency. Next, I would review adjacent paths: S3 gateway endpoint for COPY/UNLOAD, Glue and Lake Formation if a data lake is involved, KMS for encryption, and policies that do not break federated access.

Finally, I would alarm on symptoms, not only components. Failed connections, p95 session establishment time, CloudTrail denied authentication events, and VPC Flow Logs with no traffic to the expected endpoints are better signals than an endpoint merely being available.

Well-Architected applied to the pattern

security: The primary gain is reducing egress exceptions and aligning authentication, authorization, and data with the same perimeter. I would combine SSO with Redshift RBAC, IAM Identity Center groups, customer-managed KMS when required, CloudTrail, and endpoint policies only after functional testing.
reliability: The new dependency is not Identity Center itself, but private connectivity to it. Treat DNS, endpoint AZs, security groups, and Region as availability components. For multi-Region, validate IAM Identity Center prerequisites and document failover behavior.

Reference design for a financial-grade environment

My reference design starts with a Redshift Serverless workgroup or RA3 cluster in private subnets across at least two AZs, with no public access and enhanced VPC routing enabled. I would use IAM Identity Center groups as the human authorization boundary, mapped to Redshift roles by business function: regulatory read, data engineering, risk, fraud, audit. Permission to obtain a token should be explicit: redshift:GetIdentityCenterAuthToken for a provisioned cluster or redshift-serverless:GetIdentityCenterAuthToken for a serverless workgroup, limited to the expected ARNs.

On the network side, I would create sso-oauth and identitystore endpoints with Private DNS, a dedicated security group accepting 443 only from Redshift subnets or SGs, and adjacent endpoints for S3, Glue, and Lake Formation where applicable. For S3, I would still keep bucket policies conditioned on expected origin, KMS encryption, and prefixes separated by data domain. For observability, I would correlate four sources: VPC Flow Logs for endpoint ENIs, CloudTrail for identity and administration calls, Redshift logs/audit for sessions and queries, and client-side connection metrics.

The goal is not to build ornamental fortress architecture. It is to make the architecture explainable: who entered, through which identity, which group authorized it, through which network path the token was validated, which data was queried, and what evidence remains afterward.

Anti-patterns this launch does not fix by itself

Enabling EVR in production without a change window for a provisioned cluster, ignoring that the change restarts the cluster.
Creating only one of the two required endpoints and investigating the error as if it were a password, group, or driver issue.
Disabling public access while keeping broad NAT as a silent path to AWS services and SaaS without purpose-level evidence.
Copying restrictive endpoint policies from another service and blocking internal validation or identity resolution calls.
Using SSO as a substitute for authorization modeling in Redshift; centralized identity does not remove RBAC, masking, secure views, and grant reviews.

Minimum acceptance test: I would only call the change ready after proving three things: SSO sign-in works without a public route, a controlled break of each endpoint fails predictably, and a COPY/UNLOAD query still uses the expected private path. This test belongs in the runbook, not only in the memory of whoever deployed it.

My curator note: I would implement this pattern first in warehouses that carry regulated data or executive reporting, not across every environment by impulse. The lesson I learned in financial platforms is that network exceptions age poorly: they start as pragmatism and end as audit debt. When identity, data, and operational evidence follow the same private design, the conversation with security moves from opinion to proof.

Verified references

Verdict

My recommendation is to adopt Redshift with IAM Identity Center over EVR whenever the warehouse operates under residency, network isolation, or strong audit-trail requirements. The pattern does not replace authorization modeling, data governance, or observability, but it removes an important inconsistency: authentication stops being an exception and becomes part of the private perimeter. For less regulated environments, I would weigh endpoint operational cost and team maturity before standardizing; for sensitive financial data, I would treat this design as a baseline.

Rating: recommended-for-regulated-analytics

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Migrating Time Series to Timestream for InfluxDB

Fernando Azevedo — Tue, 01 Sep 2026 20:55:30 +0000

When AWS announces that Amazon Timestream for InfluxDB has reached eight more Regions, including Cape Town, Bangkok, Hong Kong, Hyderabad, Melbourne, Seoul, Zurich, and Tel Aviv, I do not read it only as wider coverage. For a financial, industrial, or telecom operation already running InfluxDB for metrics, device events, capacity, trading telemetry, or SRE, the question becomes different: can I now remove a critical self-managed server stack without losing operational compatibility, data residency, and cost predictability? The answer is yes, but only if the migration is designed as a platform change with cutover criteria, cardinality protection, replication observability, and token governance.

The starting point: InfluxDB that became critical infrastructure

In many environments, InfluxDB started small: a few Telegraf agents, infrastructure dashboards, and near real-time alerts. Then came tags by customer, Region, device, application version, trading desk, and digital channel. What was a metrics database became a dependency for incident response, capacity planning, operational fraud analysis, and event reconciliation. At that stage, the problem is rarely the InfluxDB API; the problem is the operation around it.

I look for three symptoms before recommending a migration. The first is the team treating upgrades, backups, disk, compaction, and tuning as recurring SRE work rather than exceptions. The second is hidden cost: large nodes reserved for ingestion peaks, overprovisioned storage, snapshots without a clear policy, and dashboards competing with writes. The third is regional governance. If telemetry from a subsidiary must remain in a specific geography, operating a centralized cluster elsewhere is no longer just a technical decision; it becomes a risk decision.

The regional expansion reduces that friction. It allows local time-series cells with familiar APIs inside the AWS control plane, without forcing an immediate rewrite of producers and consumers.

The architecture decision is not InfluxDB versus Timestream; it is control versus managed operation

I would split the analysis into two tracks. The first is functional compatibility: line protocol, existing clients, queries, buckets, organizations, dashboards, and collection integrations. Timestream for InfluxDB documentation preserves access through InfluxDB ecosystem APIs and tools, but that does not remove behavior testing. Flux functions, retention, tokens, time precision, and cardinality must go through a query regression suite, especially when executive dashboards and NOC alerts use different windows.

The second track is the operating model. In self-managed InfluxDB, I control the host, operating system, filesystem, process, and maintenance scheduling. In Timestream for InfluxDB, I give up that direct control and receive managed provisioning, backups, patching, and integration with AWS metrics. That trade is healthy when the organization measures availability through service experience, not through the freedom to tune sysctl in production.

The choice becomes stronger for workloads where the value is in the data and queries, not in cluster administration. For financial teams, this often appears in digital-channel telemetry stores, low-latency monitoring, batch and streaming observability, and operational risk metrics. I would keep self-managed only when an extension, plugin, host access, or engine-level control is indispensable and proven.

The migration journey I would use

1. Inventory series, not servers — I would start by measuring real cardinality, write rate in lines per second, average batch size, queries per second, p95/p99 query latency, retention windows, and high-variability tags. AWS sizing guidance uses examples such as 5,000 lines per request and classes from db.influx.medium to db.influx.24xlarge; those numbers only help if the team knows its own profile.
2. Create a minimal regional cell — The first instance should be created in private subnets, with security groups allowing only producers, consumers, and automation. I would create secrets in AWS Secrets Manager, planned rotation for long-lived tokens, account-managed KMS controls, and cost tags by domain, environment, and technical owner.
3. Run dual-write with operational idempotency — During the transition, producers write to the current cluster and to Timestream for InfluxDB. I would add a telemetry envelope with source, schemaVersion, producerId, and eventTime, and measure divergence by window. For metric series, deduplication is usually defined by measurement, tag set, and timestamp; that decision must be documented before cutover.
4. Separate operational reads from analytical reads — Incident dashboards, alerts, and internal APIs should not compete with notebooks and broad reports. When the workload is read-heavy, I would evaluate read replicas or, with InfluxDB 3 Enterprise, a multi-node cluster with separate read and write endpoints. The goal is to protect the writer, not only to add CPU.
5. Cut over by domain, with measurable rollback — I would avoid a global big bang. Cutover should happen by data domain or Region: reads first, then primary writes. Rollback must have objective triggers, such as ingestion error rate above 0.1%, query p99 above the SLO for 15 minutes, replica lag above the agreed limit, or count divergence by window.

The target design: regional cell, Multi-AZ, and a clear scaling path

My target design for a regulated institution is a regional cell per critical domain, not a shared global cluster. Each cell receives local telemetry through the VPC, keeps tokens and secrets inside the same account perimeter, publishes operational metrics to CloudWatch, and exports aggregates or cold data to the analytics platform when needed. This reduces latency, simplifies data residency, and limits blast radius.

For production environments, I would start by evaluating Multi-AZ or a read replica cluster, depending on the read pattern. The documentation confirms that a read replica cluster uses asynchronous replication, with writer and reader in different Availability Zones within the same Region. This improves read capacity and availability, but it introduces an explicit decision: in an unrecoverable writer failure, data not yet replicated can be lost. In financial systems, I would not accept that semantic without classifying the data. Infrastructure metrics may tolerate minimal tip loss; events used for audit, billing, or reconciliation should not depend only on that path.

If the workload evolves to InfluxDB 3 Enterprise, the scaling path shifts from vertical sizing to node composition. AWS documents clusters up to 15 nodes, with up to 4 writer/reader nodes, up to 13 reader-only nodes, and a dedicated compactor for clusters with 3 or more nodes. I would use that flexibility to isolate heavy reads, but keep the architecture honest: all nodes in a cluster use the same instance class, so horizontal scaling does not replace cardinality modeling.

Target architecture for controlled regional migration

The visual shows a regional cell with temporary dual-write, validation before cutover, and separation between operational reads, analytical reads, and governance.

🏢 Origem atual / Current source

Telegraf / apps line protocol (compute)
Self-managed InfluxDB current production (data)
Existing dashboards baseline queries (frontend)

🟧 AWS regional cell / Célula regional AWS

Private VPC ingress SG allowlist (network)
Timestream for InfluxDB Multi-AZ or cluster (data)
Read endpoint / replica query isolation (data)
CloudWatch CPU, memory, disk, ReplicaLag (compute)
Secrets Manager + KMS tokens and rotation (security)

📊 Consumers / Consumidores

Alerts and SLOs incident path (compute)
Analytics export aggregates and cold data (storage)
Dashboards validated queries (frontend)

Flows

agents -> oldinflux: current write
agents -> ingest: temporary dual-write
ingest -> timestream: validated batches
timestream -> reader: async replication or read endpoint
timestream -> cw: operational metrics
secrets -> ingest: tokens and rotation
reader -> newdash: read-heavy queries
reader -> alerts: alert windows
timestream -> analytics: governed aggregates
dashold -> newdash: query regression
ops -> cw: cutover go/no-go

Capacity: the trap is cardinality, not the metric count

For time series, I do not accept sizing based only on average CPU. The real risk is the combination of cardinality, write concurrency, batch size, and queries that scan broad windows. Timestream for InfluxDB documentation lists db.influx classes from 1 vCPU and 8 GiB up to 96 vCPU and 768 GiB, with documented network bandwidth from 10 Gbps on smaller classes to 40 Gbps on db.influx.24xlarge. That gives growth room, but it does not fix a poorly designed tag model.

My process is to measure active series by bucket and hour, identify tags carrying userId, sessionId, requestId, or near-unique values, and block new high-cardinality tags through a schema contract. For critical producers, I would standardize write batches close to the documented 5,000-line guidance when latency allows, lexicographically sorted tags, and the coarsest time precision the use case accepts. Writing in nanoseconds when the sensor produces data every 10 seconds only adds noise.

I also separate ingestion SLO from query SLO. A dashboard scanning weeks with regex may look harmless, but it can compete for cache and CPU with incident ingestion. In production, expensive queries need an owner, a limit, a default window, and, when possible, an isolated read endpoint.

Before and after metrics I would put in the business case

40 — instances per account and Region. Documented quota for Timestream for InfluxDB; enough for domain cells, but it requires creation governance.
15 — nodes in InfluxDB 3 Enterprise. Documented horizontal scale with up to 4 writer/readers, up to 13 reader-only nodes, and a dedicated compactor in 3+ node clusters.
~2x — cost difference in the Multi-AZ example. In the public example, db.influx.2xlarge moves from $737.88/month Single-AZ to $1,476.49/month Multi-AZ with 400 GiB.

Security: a database token is also a privileged credential

The migration is only complete when the access model changes with it. In self-managed InfluxDB, it is common to find tokens shared by squads, dashboards with broad permissions, and manual rotation postponed because nobody wants to break collection. By moving the workload to AWS, I would use the change as a cleanup milestone: one token per producer or producer class, secret stored in Secrets Manager, rotation tested in non-production, and an audit trail for who changed secrets and security groups.

At the network perimeter, I would prefer private subnets, explicit security groups, and no public exposure for internal workloads. The documentation states that Timestream for InfluxDB does not allow direct host access; that is good for reducing operational surface, but it requires runbooks to be adapted. The team will not log into the machine to inspect a process; it will depend on metrics, logs, events, AWS APIs, and support.

In IAM, I would separate provisioning, operational read, and automation roles. Policies should use tag conditions where possible, for example environment and domain, to prevent a development pipeline from modifying the production cell. For regulated institutions, I would also record an ADR: which telemetry data may contain sensitive identifiers, which tags are forbidden, and which retention satisfies security, audit, and privacy.

The hidden risk: availability can improve while tip consistency gets worse: Read replicas and Multi-AZ clusters help keep reads and operations alive during failures, but asynchronous replication must be treated as a business contract. I would add CloudWatch alarms for ReplicaLag, CPUUtilization, MemoryUtilization, and DiskUtilization, and define in the runbook when to prioritize automatic failover and when to protect data not yet replicated. For events supporting financial audit, I would keep a parallel immutable trail, for example Kinesis or MSK to S3 with Object Lock where applicable, before deriving time series for fast queries.

Well-Architected reading of the migration

security: Reducing host access is positive, but it does not replace token governance. I would require Secrets Manager, KMS, minimum security groups, role-based IAM, and an ADR about sensitive tags.
reliability: Multi-AZ, read replicas, and clusters help, provided the system monitors replica lag and has an explicit criterion for tip loss versus write availability.
performance: Optimization starts at the producer: proper batching, sorted tags, coherent time precision, limited query windows, and separate endpoints for heavy reads.

How I would cut over without romanticizing the migration

The cutover I consider healthy starts with mirrored reads, not with definitive writes. First, I would put non-critical dashboards reading from Timestream for InfluxDB, comparing results with the current cluster across 5-minute, 1-hour, and 24-hour windows. Differences must be classified: expected delay, schema divergence, incompatible query function, different time precision, or real point loss.

Then I would move low-criticality alerts and keep the most severe incident alerts on the old path until the team has at least one full operational cycle: daily peak, batch window, maintenance, and simulated incident. In financial environments, I like a dual-write period that crosses an accounting close or relevant settlement cycle, because that is when ad hoc queries and unusual loads appear.

Primary write cutover should have a short schema-change freeze, tested rollback, and a clear decision owner. If query p99 rises, that does not automatically mean rolling back; it may mean moving dashboards to a read-only endpoint, reducing the default window, or correcting cardinality. If there is count divergence in regulated data, I stop. A well-run migration is not one that never finds a problem; it is one that knows which problems are acceptable before starting.

My curator note: I would use this regional expansion to move critical InfluxDB workloads away from hand-maintained servers, but I would not sell the change as automatic savings. In the field, the largest gain is usually returning engineering time to product reliability, while the largest mistake is migrating bad cardinality to a better platform and expecting a miracle. I would first build a small regional cell, with dual-write, query regression, and lag alarms, and only then scale the pattern. The hard-won lesson is simple: a managed database removes undifferentiated work, not architectural responsibility.

Verified references

Verdict

My recommendation is to migrate operational InfluxDB workloads to Amazon Timestream for InfluxDB when the team needs regional coverage, managed operation, Multi-AZ, read scaling, and integration with AWS controls, but to do it by cell and by domain. I would not migrate regulated single-event data without a parallel immutable trail, and I would not approve production without cardinality measurement, query regression, ReplicaLag alarms, and a rollback plan. For most teams using InfluxDB as the backbone for observability and telemetry, the new regional availability makes modernization more defensible. The success criterion is not shutting down servers; it is reducing operational risk without diluting governance, SLOs, and cost discipline.

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Active-active Amazon Connect: the resilience lesson

Fernando Azevedo — Tue, 01 Sep 2026 20:55:29 +0000

When a financial contact center fails, the incident rarely shows up as a clean exception in a log. It shows up as customers stuck in IVR, agents authenticated in the wrong place, queues growing without clear ownership, CRM integrations answering from one region while voice flows expect another. The announcement that Amazon Connect Global Resiliency now supports routing contacts to agents across two active regions targets precisely that problem. To me, the main value is not the existence of a second region. It is making the second region work every day, receive agents, receive contacts, produce metrics, and expose drift before the incident.

What actually changed

On August 31, 2026, AWS announced that Amazon Connect Global Resiliency now supports cross-region routing of contacts to agents across two linked regions. The published example is straightforward: a contact entering US East (N. Virginia) can be offered to the longest-available matching agent whether that agent is in US East or US West (Oregon). The same idea applies to the documented regional pairs: N. Virginia/Oregon, Frankfurt/London, and Osaka/Tokyo.

The change matters operationally because it moves the architecture away from a passive design. Previously, many contact-center DR designs relied on a replica, phone numbers associated with a Traffic Distribution Group, and runbooks to move traffic. Those parts still matter, but the new point is that both regions stop being a promise. They exercise configuration, authentication, routing, integrations, metrics, and contact search as part of normal operation.

I would treat this capability as a maturity step, not as permission to relax engineering discipline. Active-active increases confidence, but it also increases failure surface. Everything that used to remain hidden in the secondary region now appears in production: asymmetric quotas, Lambda functions with different names, flows with fixed ARNs, incomplete IAM permissions, SaaS integrations without regional allowlists, and dashboards that aggregate without explaining origin.

Timeline of an incident that now becomes more visible

T-30 days: the replica exists but is not exercised — The team creates the second instance, replicates the primary configuration, and believes the runbook covers the switch. Quotas, integrations, and authentication paths have not yet seen the same real volume.
T-5 days: a small change creates drift — A flow starts calling a new Lambda function, but the equivalent function does not exist with the same name in the other region. In passive operation, this can remain invisible until a DR test.
T-0: contact surge forces redistribution — Operations tries to move new contacts and agents to absorb load. If both regions already carry traffic, the likely failure becomes controlled degradation. If one region never carried real traffic, the switch becomes a production experiment.
T+15 minutes: the problem is no longer just telephony — The queue may be healthy, but CRM, IdP, recording, Lex bot, API Gateway enrichment, or analytical writes may not be. Investigation must inspect contact, agent, active region, integration, and quota in the same view.

Typical root cause: The root cause is not 'the region went down.' In many continuity incidents, the root cause is the untested difference between the environment that serves traffic every day and the environment that exists only to save the day. Active-active routing reduces that difference, but only when configuration, quotas, IAM, identity, integrations, alarms, and runbooks are treated as a product, not as a DR artifact.

Active-active operating model for Amazon Connect Global Resiliency

The core idea is to separate traffic distribution, agent distribution, and operational isolation. The diagram below shows where I would place the controls that prevent a regional change from becoming an incident.

👤 Clientes e agentes

Cliente voz/chat (user)
Agente global sign-in (user)

🟧 Plano global do Connect

Traffic Distribution Group 10% increments (network)
Cross-region routing longest-available match (compute)
Isolamento controlado UpdateCrossRegionRouting (security)

🟦 Região A

Amazon Connect A fluxos, filas, CCP (compute)
Integrações A Lambda/API/CRM (compute)

🟩 Região B

Amazon Connect B config equivalente (compute)
Integrações B mesmos nomes e políticas (compute)

📊 Operação

Métricas e busca visão consolidada (data)
Service Quotas por região e instância (security)

Flows

customer -> tdg: enters through global number
tdg -> connect-a: traffic share
tdg -> connect-b: traffic share
agent -> routing: agent availability
routing -> connect-a: regional offer
routing -> connect-b: cross-region offer
connect-a -> int-a: invokes flows and APIs
connect-b -> int-b: invokes flows and APIs
connect-a -> obs: events and contacts
connect-b -> obs: events and contacts
quotas -> isolation: operational threshold
isolation -> routing: disables cross-region routing

Remediation starts before failover

The first change I would make in a critical operation is to turn the secondary region into an ordinary path, even with a small initial share. Global Resiliency allows traffic and agents to be distributed across regions in 10% increments, or moved all at once. I would start with a small and deliberate slice, for example 10% of new contacts in a lower-risk queue and an agent group trained to operate on both sides. The goal on day one is not perfect balancing; it is discovering drift with controlled impact.

That choice has cost and discipline attached to it. If both regions carry traffic, both need equivalent quotas, useful alarms, configuration pipelines, flow tests, and identity integration. The documentation is explicit about points I would not treat as details: the instance must be in supported regions, access to the feature requires engagement with AWS, and AWS recommends monthly failover testing. There is also a strong practical requirement: for flows that call Lambda, function names should be consistent across regions, and hardcoded ARNs need to be removed wherever the regional parameter is supported.

In a financial environment, I would also add a post-deploy validation pipeline. After any change to queue, routing profile, security profile, flow, Lex bot, Lambda, or external integration, a canary should execute a minimal journey in both regions: entry, routing, enrichment, recording, after-contact work, and analytical event.

The weak point is no longer just voice

The biggest trap in this architecture is assuming contact-center resilience ends when the contact finds an agent. In practice, service depends on a chain that includes SAML, CCP/Agent Workspace, CRM, internal APIs, profile stores, recording, transcription, classification, fraud controls, audit, and sometimes a conversational journey with Lex or AI agents. Amazon Connect may route the contact correctly, but the experience fails if the agent receives an empty screen or if an eligibility lookup gets stuck behind a regional API.

That is why I would design integrations with explicit regional contracts. A Lambda called by a flow should have the same name in both regions when required by the Global Resiliency pattern. IAM permissions should constrain by aws:RequestedRegion, connect:InstanceId where applicable, and environment-scoped resources rather than improvised names. For internal APIs behind API Gateway, I would use idempotency keys based on contactId, initialContactId, and logical step, stored in DynamoDB with TTL to avoid duplicate charges during retries or re-offers.

I would also separate operational state from analytical state. What is required to serve the customer must be available on the hot path with predictable latency. What is required for audit can move through EventBridge, Kinesis, or Firehose into S3 with replication and later reconciliation. Mixing both paths increases coupling exactly during failure.

Observability for the moment when everything looks green

The unified analytics and contact-search visibility across regions is relevant, but I would not let operations depend only on aggregated dashboards. Aggregated dashboards are useful for management; incident investigation needs slicing by origin, active region, queue, routing profile, flow, integration, and quota. The Global Resiliency model itself exposes metadata such as origin region, active region, and Traffic Distribution Group in contact structures, and I would use that as a telemetry dimension from the start.

My minimum signals would be: rate of contacts routed across regions, queue time by active region, error rate by flow block, p95/p99 latency of Lambdas invoked by flows, SAML authentication failures, contacts in after-contact work during a regional change, Amazon Connect API throttling, quota utilization by instance, and applied-capacity differences between regions. In CloudWatch, quota alarms should not wait for 100%; I usually start at 70% for trend and 80% for action, adjusted by criticality and seasonality.

I also like creating explicit operational events for distribution changes: who changed it, from what to what, ticket, reason, window, expected rollback, and observed result after 15 and 60 minutes. That can go to EventBridge and to an immutable S3 bucket with Object Lock when the organization needs a stronger audit trail. In serious incidents, the question is not only 'is it back?'. It is 'do we know exactly what was shifted, by whom, and with what effect?'.

Capacity: the bill that appears later

In a contact center, capacity is not only concurrent calls. It is limits for users, queues, profiles, flows, numbers, APIs per second, integrations, recording, storage, and third parties. A poorly calibrated active-active design can fail in a counterintuitive way: region B receives only 10% of traffic for months, but one morning it must absorb 100%. If B's quotas were not raised at the same pace as A's, failover works in the routing plane and fails in the capacity plane.

The Global Resiliency requirements documentation advises requesting that all quotas in the replica match the source. A recent AWS article on quotas reinforces a point I consider critical: Service Quotas requests are regional, and after initial replication, increases must be kept in sync by region. This is a good example of invisible operational debt. The team increases capacity on the side under daily pressure and forgets the other side because it looks quiet.

I would solve this with declarative automation: daily inventory of applied quotas by instance and region, automatic diff, alert when the difference becomes non-zero for critical resources, and change blocking when a planned distribution exceeds estimated capacity. The practical rule is simple: if region B must receive 100% tomorrow, it must be measured as capable of receiving 100% today.

Well-Architected reading

security: Global sign-in and regional integrations need least-privilege IAM, well-governed SAML, consistent KMS for recordings and analytical data, and an auditable trail for distribution or isolation changes.
reliability: The real gain comes from continuously exercising both regions, validating failover monthly, removing hardcoded configuration, and keeping quotas symmetric. I would define RTO/RPO per service journey, not only per Connect instance.

Anti-patterns I would remove from the design

Regional replica created once and never validated again with real traffic.
Flows with fixed ARNs for Lambda, Lex, or regional resources without a substitution strategy.
Quotas increased only in the primary region because that is where pain appears first.
Global dashboards without drill-down by active region, origin region, and Traffic Distribution Group.
Failover runbook that depends on a single administrator, single console session, or single regional endpoint.

My curator note: I would not move a regulated contact center to active-active in the first weekend. I would start with controlled queues, measure errors by region, and increase distribution only after quotas, identity, flows, and integrations survived repeated tests. The lesson I learned in critical environments is that DR does not fail in the diagram; it fails in the detail nobody exercised under load. This launch is valuable precisely because it forces that detail to appear earlier.

References

Verdict

My recommendation is to treat Amazon Connect Global Resiliency cross-region routing as an operating program, not as a DR switch. Use real traffic in both regions, start small, keep quotas and integrations in parity, record every distribution change, and test isolation before you need it. For financial organizations, that is the difference between a failover that looks good in a presentation and an operation that keeps serving customers when infrastructure stops cooperating.

Rating: recommended-with-operational-discipline

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.

Redshift and Iceberg v3: field notes for mutable lakes

Fernando Azevedo — Tue, 01 Sep 2026 20:54:57 +0000

When I look at Amazon Redshift support for Apache Iceberg v3 tables, the interesting part is not simply "it now reads and writes v3". The architectural point is that three old pains in analytical lakes start becoming table contracts: schema evolution with explicit defaults, stable row identity for incremental pipelines, and compact deletes for workloads with frequent update/delete operations. In financial-grade environments, that directly touches reconciliation, erasure workflows, audit trails, scan cost, and coupling between engines.

The situation: the lake is no longer only append-only

For years, a relevant share of enterprise data lakes worked well because the implicit contract was simple: immutable Parquet files, predictable partitions, compaction jobs, and mostly append-only consumption. That model still holds for raw events, logs, and regulatory snapshots. It starts to creak when the platform becomes a shared data product across risk, fraud, customer operations, Open Finance, BI, credit models, and audit.

In those scenarios, the operational question is not "can I query S3 with SQL?". The question is whether I can modify a row without destroying performance, add a column without reprocessing terabytes, propagate only changes to downstream consumers, and prove what changed between snapshots. The Redshift announcement on August 31, 2026 matters because it brings Iceberg v3 features into a SQL engine many organizations already use as a consumption, transformation, and governance layer.

I would not read this as permission to turn every lake table into an OLTP database. Iceberg is still an analytical table format. The gain is reducing patches: CDC mirror tables, inconsistent technical columns, accumulated delete files, and pipelines that scan everything because there is no trustworthy change marker.

What actually changed

Redshift can now create or upgrade Iceberg v3 tables with the 'format-version' = '3' property, while keeping SQL DML syntax similar to v2.
Default column values reduce reprocessing during schema evolution, but require governance over the historical meaning of the default.
Row lineage exposes _row_id and _last_updated_sequence_number for incrementality, audit, and reconciliation, as long as consumers understand the contract.
Deletion vectors replace new positional delete files in v3 and help workloads with many deletes, updates, and MERGEs, especially when table maintenance is disciplined.
Compatibility is not universal: complex types and some Iceberg v3 types are not currently supported by Redshift, and downgrade from v3 to v2 is not supported.

Default values: schema evolution without rewriting the past

The most pragmatic use of default values appears when a large table needs a new operational column: source_system, risk_bucket, consent_status, schema_version, retention_class, or something similar. In Iceberg v2, many teams solved this with expensive backfills, views with coalesce, or different rules per consumer. In Iceberg v3 on Redshift, a column added with a default can return the initial value for older files without rewriting data, and new INSERTs can write the default when the column is omitted.

The trap is semantic. A default does not mean the historical data truly had that attribute at the original event time. In regulated environments, I separate technical defaults from business assertions. schema_version DEFAULT 3 is relatively safe. customer_consent DEFAULT true is dangerous if there is no historical basis for it. It is also worth noting that defaults are literals, not dynamic expressions; I would not rely on them for processing timestamps or contextual rules.

In practice, I would record each change in a short ADR, with effective date, reason, domain owner, and consumer impact. If the default changes metric interpretation, the publication needs a contract version and a coexistence window.

Row lineage: analytical CDC with less guessing

The feature I would watch most closely is row lineage. Redshift exposes _row_id and _last_updated_sequence_number pseudo-columns, which must be selected explicitly and are not included in SELECT *. That is a good decision: it avoids accidental leakage into dashboards, but lets technical pipelines create sequence-based checkpoints. For an incremental pipeline, the basic contract becomes simple: persist the last processed sequence number, read rows with a sequence greater than or equal to the planned watermark, write downstream results idempotently, and advance the checkpoint only after the downstream commit.

I would still not replace every CDC mechanism with this. In integrations with transactional systems, I still want source logs, partition ordering, and business events. Row lineage is strongest inside the lakehouse: deriving silver/gold tables, recalculating affected aggregates, feeding feature stores, rebuilding search indexes, and assembling analytical audit trails.

One operational detail matters in a v2-to-v3 upgrade: pre-upgrade data does not immediately get useful lineage; Redshift documentation says the values return null until the first write after the upgrade, when values are generated for the table. So the migration needs an explicit cutover line. I would treat that line as a platform event, not an invisible detail.

Iceberg v3 table pattern for controlled mutations

The design shows how I would isolate ingestion, table contract, incremental queries, and governance when adopting Iceberg v3 with Redshift.

🟦 Domínios produtores

Eventos de negócio Kafka/MSK ou batch (messaging)
Glue/EMR jobs validação e merge (compute)

🟧 Contrato lakehouse

AWS Glue Data Catalog metadados Iceberg (data)
Tabela Iceberg v3 format-version=3 (storage)
Deletion vectors Puffin + bitmap (storage)
Row lineage _row_id + sequence (data)

🟥 Governança

Lake Formation FGAC e grants (security)
KMS + S3 policies criptografia e perímetro (security)

🟩 Consumo e operação

Amazon Redshift Graviton provisioned/serverless (compute)
Checkpoint incremental última sequência processada (data)
CloudWatch/SLOs scan, latency, failures (compute)

Flows

events -> etl: normalizes and validates
etl -> table: INSERT/UPDATE/MERGE
catalog -> table: schema and snapshots
table -> dv: marks removed rows
table -> lineage: identifies changes
lf -> catalog: authorizes tables/columns/rows
kms -> table: protects S3 objects
redshift -> catalog: discovers table
redshift -> lineage: queries by sequence
redshift -> checkpoint: advances after commit
redshift -> obs: emits operational signals

Deletion vectors: fewer delete files, not zero maintenance

Deletion vectors address a familiar problem: updates and deletes on immutable files tend to generate auxiliary structures that hurt reads, planning, and compaction. In Iceberg v3, Redshift records removed positions in compact bitmaps stored in Puffin files, with at most one deletion vector per data file in a snapshot. That reduces positional delete file proliferation and makes regulatory deletes, customer-data corrections, and CDC MERGE workloads less costly to read and write.

But I would not sell this internally as free maintenance. If a table receives random deletes all day, the data files still physically contain invalidated records. Reads are better than with many small delete files, but there is still additional work. For high-churn tables, I would keep explicit policies for compaction, file rewrite, and snapshot expiration, using low-demand windows for maintenance.

A practical number I often use as an initial trigger is not absolute: when the share of logically deleted rows in hot partitions crosses 5% to 10%, or when P95 latency on queries over those partitions grows consistently, I review compaction. I am not presenting this as an AWS limit; it is an operational heuristic to start measuring before users feel it.

The architecture point: interoperability with clear boundaries

The value of Iceberg is allowing multiple engines to share a table without turning the lake into accidental coupling. Redshift, Athena, EMR, Glue, and other tools can participate, but I would only allow multiple writers where ownership is explicit. For critical tables, I choose one primary writer per domain and treat the others as readers or transformers with very specific permissions.

In Redshift, some details shape the design. The documentation states support for Lake Formation fine-grained access control on Iceberg tables, the use of Glue-generated column statistics for better performance, and different cost behavior depending on compute type: RG and Serverless use their own compute for S3 lake queries, while DC2 or RA3 use Redshift Spectrum. The pricing page confirms that Spectrum charges by bytes scanned, rounded to the next megabyte with a 10 MB minimum per query; in Serverless, external queries are part of the workgroup's RPU-hour consumption.

I would configure this as a platform product: S3 bucket with SSE-KMS, policies conditioned on aws:PrincipalArn and aws:SecureTransport, Lake Formation as the authorization plane, static roles for writes, sensitivity tags, and workload-specific budgets. The common mistake is treating an open format as absence of governance. It is the opposite: the more open the format, the clearer the contract must be.

The adoption playbook I would use

Choose a table with real mutation, not the most critical one — Look for a table with frequent MERGE or DELETE activity, known consumers, and an operational rollback path. Avoid starting with accounting ledgers, intraday risk, or tables with multiple writers and no clear owner.
Audit engines and types before the upgrade — Confirm that all relevant readers and writers understand Iceberg v3. In Redshift, do not plan v3 for tables that depend on struct, list, map, variant, geometry, geography, binary, uuid, time, or nanosecond timestamps, because the documentation lists those limitations.
Define the incrementality contract — Standardize checkpointing by _last_updated_sequence_number, a reread window to tolerate retries, an idempotency key in the target, and a metric comparing rows read with rows actually applied.
Run the upgrade as a platform change — The format ALTER is metadata-only, but the consequence is not trivial: there is no downgrade to v2, pre-upgrade data has specific lineage behavior, and old positional deletes remain valid until later writes merge them into deletion vectors.
Measure before and after with the same query set — Collect P50/P95/P99, bytes scanned, file count, MERGE duration, Lake Formation permission failures, planning time, and cost by domain. Without a baseline, any improvement becomes a story.

Start with the contract, not with ALTER TABLE: Before running ALTER TABLE ... SET TABLE PROPERTIES ('format-version' = '3'), write down three things: who may write, how incremental consumers advance checkpoints, and which metric triggers compaction. The command is short; the governance around it is what prevents incidents.

Anti-patterns I would avoid

Upgrading to v3 because it is new, without inventorying the engines that read the table. In a lakehouse, silent incompatibility across tools costs more than the upgrade.
Using default values to hide lack of data governance. A technical default helps evolution; a poorly defined business default creates historical error that looks like complete data.
Treating _last_updated_sequence_number as a universal substitute for domain events. It is excellent for analytical change, but it does not carry business causality by itself.
Allowing multiple writers without an ownership, retry, and idempotency protocol. The format supports interoperability; it does not solve organizational contention.
Ignoring column statistics and file maintenance. Redshift can optimize better when statistics exist; deletion vectors reduce pain, but they do not remove table hygiene.

Observability, security, and cost in production

I would put Iceberg v3 under the same observability discipline as any critical service. For each candidate table, I keep a dashboard with hourly write volume, rows updated/deleted, age of the latest snapshot, small-file count, MERGE duration, read P95 by query class, and scanned-bytes variation. In Redshift Serverless, I would also use RPU-hour usage limits by day, week, or month; the documentation allows actions such as logging, alerting, or turning off user queries when the limit is reached.

On security, I avoid relying only on IAM at the bucket. Lake Formation should express permissions by table, column, and where applicable row/cell; S3 and KMS should reinforce perimeter, encryption, and access by expected roles. Writing to Iceberg through Redshift should not use federated identity with SESSION, according to the documented consideration; I prefer a static platform role with least privilege to specific table paths and corresponding KMS keys.

For engineering leadership, the most important metric may be recovery time. If a faulty MERGE applies deletion vectors or bad defaults, how do I return to a previous snapshot, who approves it, how long does it take, and which consumers need reprocessing? Without that answer, adoption is not ready.

Questions I would ask in a design review

Should I convert all Iceberg v2 tables to v3?

No. I would prioritize tables with frequent schema evolution, recurring MERGE/DELETE activity, or incremental consumers. Stable append-only tables can remain on v2 until there is clear benefit and validated compatibility.

Does row lineage replace regulatory audit?

Not by itself. It improves analytical traceability and incrementality, but regulatory audit still needs decision trails, actor identity, approval, source evidence, and controlled retention.

Do deletion vectors eliminate compaction?

No. They reduce the operational cost of deletes compared with many positional delete files, but high-mutation tables still need maintenance policy, file rewrite, and snapshot expiration.

What is the largest upgrade risk?

Engine compatibility and data semantics. Redshift does not support downgrade from v3 to v2, and there are documented type limitations. I would validate readers, writers, Lake Formation grants, and critical queries on a copy before the change.

My curator note: I would adopt Iceberg v3 on Redshift first where there is measurable pain: analytical CDC, regulatory deletions, and contract evolution on large tables. The practical lesson is that open formats do not reduce the need for architecture; they move the discipline into metadata, ownership, and operations. If I cannot explain who writes, who compacts, who authorizes, and who reprocesses, I do not yet have a production design.

References

Verdict

My recommendation is to adopt Redshift with Iceberg v3 selectively and with discipline: use it for mutable analytical tables where default values, row lineage, and deletion vectors solve a real operational problem. Do not mass-upgrade. Inventory engines, validate supported types, define write ownership, monitor cost and performance, and treat the migration as a data contract change. For financial-grade platforms, this launch matters because it moves the lakehouse closer to an auditable and incremental model without abandoning S3, Glue Catalog, Lake Formation, and SQL; the value appears when the team combines the technical feature with production governance.

Rating: adopt selectively

Originally published at fernando.moretes.com. By Fernando F. Azevedo — Senior Solutions Architect.