Most production systems don’t fail because they lack metrics.
They fail because the metrics they do have flatten reality.
Over time, I kept seeing the same pattern across teams and architectures: engineers had plenty of dashboards, yet struggled to answer simple questions during incidents.
Not because the data wasn’t there but because it was aggregated in ways that hid meaningful differences.
This is the problem that led to the addition of multiple dimension sets in AWS Lambda Powertools for Python.
The Real Problem: Aggregation, Not Instrumentation
CloudWatch’s Embedded Metric Format (EMF) has long supported dimensional metrics.
In theory, this allows teams to slice metrics by environment, region, customer type, or deployment shape.
In practice, most teams are forced to choose one aggregation view per metric emission.
You can measure latency by:
• service + region, or
• service + environment, or
• service + customer_type
But not all of them at once unless you emit the same metric repeatedly with different dimension combinations.
That trade-off shows up quickly in real systems:
• Metrics get duplicated
• Code becomes verbose and fragile
• CloudWatch costs increase
• Important aggregation paths are missing when you need them most
The result isn’t just inefficiency it’s lost confidence during incidents.
The Feature Request That Captured the Pattern
This limitation wasn’t theoretical.
In early 2025, a community member opened a feature request in the AWS Lambda Powertools repository:
“Add support for multiple dimension sets to the same Metrics instance”
(Issue #6198)
The use case was clear:
• A Lambda deployed across multiple regions and environments
• Metrics that needed to be aggregated by environment, region, and both
• One metric value, many meaningful views
The request also highlighted an important fact:
The EMF specification already supports this.
The Dimensions field in EMF is defined as an array of arrays each inner array representing a different aggregation view.
Other Powertools runtimes (TypeScript, Java, .NET) already exposed this capability.
Python didn’t.
From Feature Request to Production-Ready Implementation
After maintainers aligned on the approach, I picked up the work to implement this feature for the Python runtime.
The goal wasn’t to invent something new.
It was to:
• Align Python with the EMF specification
• Reach feature parity with other Powertools runtimes
• Deliver a clean, intuitive API that felt natural to existing users
Design principles
Before touching code, a few constraints guided the implementation:
• Backward compatibility - existing add_dimension() behavior must remain unchanged
• Clear mental model - no hidden side effects or ambiguous APIs
• Spec-aligned output - serialized EMF must match CloudWatch expectation
• Production safety - strict validation and cleanup between invocations
The Resulting API
The final design mirrors the proven pattern from the TypeScript implementation:
• add_dimension() → adds to the primary dimension set
• add_dimensions() → creates a new aggregation view
Example usage
from aws_lambda_powertools import Metrics
from aws_lambda_powertools.metrics import MetricUnit
metrics = Metrics(namespace="ServerlessAirline", service="booking")
metrics.add_dimensions({"environment": "prod", "region": "us-east-1"})
metrics.add_dimensions({"environment": "prod"})
metrics.add_dimensions({"region": "us-east-1"})
metrics.add_metric(
name="SuccessfulRequests",
unit=MetricUnit.Count,
value=100
)
With a single metric emission, CloudWatch can now aggregate across:
• environment + region
• environment only
• region only
No duplicate metrics.
No parallel pipelines.
No guesswork.
What Changed Under the Hood
The implementation introduced a few key changes:
• Tracked multiple dimension sets internally
• Updated EMF serialization to emit all dimension arrays
• Ensured default dimensions are automatically included
• Enforced CloudWatch’s 30-dimension limit
• Handled duplicate keys deterministically (“last value wins”)
• Cleared dimension state safely between invocations
To ensure reliability, the change shipped with 13 new tests, covering:
• Multiple dimension set creation
• Validation and edge cases
• Integration with existing metrics features
• High-resolution metrics compatibility
All existing tests passed, code quality checks succeeded, and maintainers approved the change for merge.
Why This Matters in Production
This feature doesn’t add more metrics.
It makes existing metrics more truthful.
When teams can express multiple aggregation views at the point of emission:
• Incident response becomes faster
• Dashboards become simpler
• Alerting becomes more precise
• Engineers trust what they see
Metrics are contracts.
If they can’t reflect how users actually experience the system, they quietly fail.
Multiple dimension sets don’t eliminate operational problems but they remove a blind spot that many teams didn’t realize they had.
The full implementation, tests, and maintainer review can be found in the merged pull request:
https://github.com/aws-powertools/powertools-lambda-python/pull/7848
Open Source as Shared Problem-Solving
What made this contribution meaningful wasn’t just the code.
It was the process:
• A well-documented community feature request
• Maintainer collaboration across runtimes
• Alignment with existing specifications
• A solution designed for long-term maintainability
This is open source at its best: turning recurring operational pain into shared infrastructure improvements.
Measuring What Actually Matters
Reliability isn’t about collecting more data.
It’s about choosing the signals that deserve to exist.
This change helps teams measure systems the way users experience them — not just the way dashboards prefer.
And that difference matters.
If you’re duplicating EMF emissions just to get different aggregation views, this should make your metrics simpler, clearer, and more reliable.
And if you run into edge cases, open an issue.
That’s how this ecosystem keeps improving.
Top comments (0)