Lambda Powertools for Python: logs, traces, metrics

Lambda Powertools for Python gives your functions structured logs, traces and metrics without you having to build a small logging framework of your own. Still, in Python projects on Lambda I keep seeing the same picture: a few print() calls, a logging.basicConfig() somewhere, and when things go wrong, a hunt for text fragments in CloudWatch. That works fine until the first error shows up at 2 a.m. and nobody knows which line belongs to which request.

In this article you will build a small Python function for a fictional bike rental, step by step. It logs as JSON, sends traces to AWS X-Ray and reports business metrics to CloudWatch. You deploy it with the Serverless Framework, manage dependencies locally with uv, and run the tests with pytest without any AWS account. I already showed the same for TypeScript in Lambda Powertools TypeScript; this one covers the Python side and its own pitfalls.

In short

Powertools for AWS Lambda (Python) ships Logger, Tracer and Metrics as three decorators that turn a silent function into an observable one. The easiest way to add the library is the official Lambda layer, whose ARN you read from Parameter Store. The most common pitfall is metric dimensions with too many distinct values, because those cost real money.

What you will learn in this article

• How to combine Logger, Tracer and Metrics correctly in a Python handler
• How to add the Powertools layer to the Serverless Framework via an SSM parameter
• How to test logs, EMF metrics and cold starts with pytest, no AWS account needed
• How to avoid costly mistakes with dimensions, log levels and sensitive data

What is Lambda Powertools for Python?

Powertools for AWS Lambda is an open-source library that AWS maintains for Python, TypeScript, Java and .NET. It packages proven practices for serverless functions into small building blocks. For Python, version 3 is the current major release; I tested the examples with version 3.39. Three building blocks form the core, and those are what this article is about.

The Logger writes every log line as a JSON object. This is called structured logging: instead of “rental 42 started” you get an object with fields such as level, message, service and your own keys. You can filter and aggregate those fields later instead of searching free text with regular expressions.

The Tracer creates traces for AWS X-Ray. A trace shows how long a request spent in each part of your system, for example 4 ms in your price calculation and 18 ms in the call to DynamoDB. Tracer is a thin layer on top of the X-Ray SDK and automatically patches supported libraries such as boto3.

Metrics creates custom CloudWatch metrics in the Embedded Metric Format, EMF for short. Your function does not call any API for this. It writes a specially structured JSON blob to the logs, and CloudWatch turns it into a metric asynchronously. That is fast and saves you API calls inside the handler.

Logger and Metrics both write to stdout. CloudWatch extracts the EMF blobs as metrics automatically.

Prerequisites

• Python 3.13 locally and uv as package manager (pip works too, the commands are listed alongside)
• Node.js and Serverless Framework v4 (npm install -g serverless)
• An AWS account with permissions for Lambda, API Gateway, DynamoDB, CloudFormation and X-Ray – only for deployment, the tests run locally
• Basic knowledge of Lambda handlers in Python. That is all you need.

Set up Lambda Powertools for Python step by step

The example is deliberately small: an HTTP endpoint POST /rentals accepts a rental, calculates the price and stores the record in DynamoDB. In the end, the project structure looks like this:

bike-rentals/
├── pyproject.toml
├── serverless.yml
├── src/
│   └── handler.py
└── tests/
    ├── conftest.py
    └── test_handler.py

Step 1: Create the project with uv

Create the folder and a pyproject.toml. Notice that Powertools only appears in the dev group. At runtime the library comes from the official layer, so it does not need to be in your deployment package. You still need it locally so your editor and tests work. boto3 is local only as well, because the Lambda runtime already ships the AWS SDK.

# pyproject.toml
[project]
name = "bike-rentals"
version = "0.1.0"
requires-python = ">=3.13"
dependencies = []

[dependency-groups]
dev = [
  "aws-lambda-powertools[tracer]>=3.39",
  "boto3>=1.40",
  "pytest>=8.4",
]

[tool.pytest.ini_options]
testpaths = ["tests"]

mkdir -p bike-rentals/src bike-rentals/tests && cd bike-rentals
# create pyproject.toml as shown above, then:
uv sync
# alternative without uv:
# pip install "aws-lambda-powertools[tracer]" boto3 pytest

The [tracer] extra installs the X-Ray SDK as well. Without it your code still starts locally, but importing the Tracer fails. The layer already contains everything.

Step 2: Write the handler

Now for the actual code. Logger, Tracer and Metrics are created once at module level, outside the handler. That way they survive between two invocations in the same execution environment, and only the cold start pays for initialization. Service name and namespace come from environment variables, so the constructors stay empty.

# src/handler.py
import json
import os
import uuid
from datetime import datetime, timezone

import boto3
from aws_lambda_powertools import Logger, Metrics, Tracer
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.metrics import MetricUnit
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
tracer = Tracer()
metrics = Metrics()

TABLE_NAME = os.environ.get("RENTALS_TABLE", "bike-rentals-local")
PRICE_PER_HOUR_CENTS = {"city": 300, "ebike": 650, "cargo": 900}

_table = None

def rentals_table():
    """Creates the DynamoDB table on first access (fast cold start, replaceable in tests)."""
    global _table
    if _table is None:
        _table = boto3.resource("dynamodb").Table(TABLE_NAME)
    return _table

class InvalidRequest(Exception):
    pass

def parse_request(event: dict) -> dict:
    try:
        body = json.loads(event.get("body") or "{}")
    except json.JSONDecodeError as err:
        raise InvalidRequest("Body is not valid JSON") from err

    bike_type = body.get("bikeType")
    station = body.get("station")
    hours = body.get("hours")

    if bike_type not in PRICE_PER_HOUR_CENTS:
        raise InvalidRequest(f"Unknown bike type: {bike_type}")
    if not isinstance(station, str) or not station:
        raise InvalidRequest("Station is missing")
    if not isinstance(hours, int) or not 1 <= hours <= 24:
        raise InvalidRequest("hours must be between 1 and 24")
    return {"bike_type": bike_type, "station": station, "hours": hours}

@tracer.capture_method
def calculate_price(bike_type: str, hours: int) -> int:
    price = PRICE_PER_HOUR_CENTS[bike_type] * hours
    if hours >= 8:
        price = int(price * 0.8)  # day discount
    return price

@tracer.capture_method
def save_rental(rental: dict) -> None:
    rentals_table().put_item(Item=rental)

def response(status: int, body: dict) -> dict:
    return {
        "statusCode": status,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(body),
    }

@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_HTTP)
@tracer.capture_lambda_handler
@metrics.log_metrics(capture_cold_start_metric=True)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    try:
        request = parse_request(event)
    except InvalidRequest as err:
        logger.warning("Invalid request", reason=str(err))
        metrics.add_metric(name="InvalidRequest", unit=MetricUnit.Count, value=1)
        return response(400, {"message": str(err)})

    rental_id = str(uuid.uuid4())
    logger.append_keys(rental_id=rental_id, station=request["station"])
    tracer.put_annotation(key="station", value=request["station"])

    price_cents = calculate_price(request["bike_type"], request["hours"])
    rental = {
        "rentalId": rental_id,
        "bikeType": request["bike_type"],
        "station": request["station"],
        "hours": request["hours"],
        "priceCents": price_cents,
        "startedAt": datetime.now(timezone.utc).isoformat(),
    }

    try:
        save_rental(rental)
    except Exception:
        logger.exception("Could not save rental")
        metrics.add_metric(name="RentalSaveFailed", unit=MetricUnit.Count, value=1)
        return response(500, {"message": "Please try again later"})

    metrics.add_dimension(name="bike_type", value=request["bike_type"])
    metrics.add_metric(name="RentalStarted", unit=MetricUnit.Count, value=1)
    metrics.add_metric(name="RentalRevenue", unit=MetricUnit.Count, value=price_cents)
    logger.info("Rental started", price_cents=price_cents)
    return response(201, {"rentalId": rental_id, "priceCents": price_cents})

A few spots deserve a second look. The three decorators are stacked in a deliberate order: on the outside, the Logger sets the Lambda context and the correlation ID; below it, the Tracer opens the segment; on the inside, Metrics collects the values and flushes them together at the end of the invocation. The Logger reads the correlation ID via JMESPath from requestContext.requestId; the constant correlation_paths.API_GATEWAY_HTTP saves you from typing the path.

With logger.append_keys() you attach the rental and the station to every following log line of this invocation. Later you can find every line for a rental, including those from deeper functions. tracer.put_annotation() makes the station searchable in X-Ray. Annotations are indexed, metadata is not. So for values you want to filter on, use annotations.

In the metrics, the bike_type dimension stands out. It only has three possible values, which is harmless. A dimension such as rental_id, on the other hand, would be an expensive mistake – more on that below. Also, RentalRevenue counts revenue in cents. The Count unit is not a perfect fit, but it works fine for a sum in CloudWatch.

Step 3: serverless.yml with the official layer

Instead of hard-coding a layer version, the configuration reads the current ARN from a public SSM parameter. AWS publishes the latest Powertools layer there for every architecture and Python version. The Serverless Framework resolves ${ssm:…} during deployment.

# serverless.yml
service: bike-rentals
frameworkVersion: "4"

provider:
  name: aws
  runtime: python3.13
  architecture: arm64
  region: eu-central-1
  memorySize: 256
  timeout: 10
  tracing:
    lambda: true
  environment:
    POWERTOOLS_SERVICE_NAME: bike-rentals
    POWERTOOLS_METRICS_NAMESPACE: BikeRental
    POWERTOOLS_LOG_LEVEL: INFO
    RENTALS_TABLE: ${self:service}-${sls:stage}-rentals
  iam:
    role:
      statements:
        - Effect: Allow
          Action:
            - dynamodb:PutItem
          Resource: !GetAtt RentalsTable.Arn
        - Effect: Allow
          Action:
            - xray:PutTraceSegments
            - xray:PutTelemetryRecords
          Resource: "*"

package:
  patterns:
    - "!**"
    - "src/**"

functions:
  startRental:
    handler: src/handler.lambda_handler
    layers:
      - ${ssm:/aws/service/powertools/python/arm64/python3.13/latest}
    events:
      - httpApi:
          path: /rentals
          method: post

resources:
  Resources:
    RentalsTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.environment.RENTALS_TABLE}
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: rentalId
            AttributeType: S
        KeySchema:
          - AttributeName: rentalId
            KeyType: HASH

Three points matter to me here. First, the function architecture and the layer must match: arm64 in the provider, arm64 in the parameter path. Second, tracing.lambda: true turns on active tracing; the two X-Ray permissions are listed explicitly in the role so nobody has to guess. Third, package.patterns only packages the src folder. Tests, virtual environment and lock file stay out, which keeps the package at a few kilobytes.

A word on pinning: latest is convenient, but a later deployment can quietly pull in a new layer version. For production stacks I prefer to pin the concrete ARN and update it on purpose, for example through an automated update pull request or with every release. For getting started, the parameter is exactly right.

Step 4: Tests with pytest – no AWS required

Lambda Powertools for Python is easy to test once you know three things. Tracer disables itself automatically outside Lambda. Metrics needs a namespace, otherwise it raises a SchemaValidationError when flushing. And Metrics keeps data in memory between invocations, so you have to reset it between tests. The conftest.py handles all of this and provides a fake Lambda context.

# tests/conftest.py
import os
import sys
from dataclasses import dataclass

import pytest

os.environ.setdefault("POWERTOOLS_SERVICE_NAME", "bike-rentals")
os.environ.setdefault("POWERTOOLS_METRICS_NAMESPACE", "BikeRental")
os.environ.setdefault("AWS_DEFAULT_REGION", "eu-central-1")
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))

@dataclass
class FakeContext:
    function_name: str = "start-rental"
    memory_limit_in_mb: int = 256
    invoked_function_arn: str = "arn:aws:lambda:eu-central-1:123456789012:function:start-rental"
    aws_request_id: str = "11111111-2222-3333-4444-555555555555"

@pytest.fixture
def context():
    return FakeContext()

@pytest.fixture(autouse=True)
def reset_metrics():
    import handler
    from aws_lambda_powertools.metrics.provider import cold_start

    handler.metrics.clear_metrics()
    handler.metrics.clear_default_dimensions()
    cold_start.is_cold_start = True
    # The logger is created at import time; point its output to the real stdout so capfd sees it
    handler.logger.registered_handler.setStream(sys.__stdout__)
    yield

The setStream line is the least obvious pitfall: the Logger remembers whatever sys.stdout was at import time. pytest, however, swaps stdout for each test, so log lines end up nowhere. By pointing the logger at the real stdout and using the capfd fixture, logs and EMF blobs reliably reach your test.

# tests/test_handler.py
import json

import handler

class FakeTable:
    def __init__(self, fail: bool = False):
        self.items = []
        self.fail = fail

    def put_item(self, Item):
        if self.fail:
            raise RuntimeError("DynamoDB unreachable")
        self.items.append(Item)

def http_event(body: dict) -> dict:
    return {"body": json.dumps(body), "requestContext": {"requestId": "req-42"}}

def output_lines(capfd) -> list[dict]:
    return [json.loads(line) for line in capfd.readouterr().out.splitlines() if line.strip()]

def metric_names(lines: list[dict]) -> set[str]:
    names = set()
    for line in lines:
        for block in line.get("_aws", {}).get("CloudWatchMetrics", []):
            names.update(m["Name"] for m in block["Metrics"])
    return names

def test_rental_is_saved_and_metrics_are_emitted(monkeypatch, capfd, context):
    table = FakeTable()
    monkeypatch.setattr(handler, "rentals_table", lambda: table)

    result = handler.lambda_handler(http_event({"bikeType": "ebike", "station": "Central Station", "hours": 2}), context)

    assert result["statusCode"] == 201
    assert table.items[0]["priceCents"] == 1300
    lines = output_lines(capfd)
    assert {"ColdStart", "RentalStarted", "RentalRevenue"} <= metric_names(lines)
    log = next(line for line in lines if line.get("message") == "Rental started")
    assert log["correlation_id"] == "req-42"
    assert log["station"] == "Central Station"
    assert log["service"] == "bike-rentals"

def test_day_discount_applies_from_eight_hours():
    assert handler.calculate_price("city", 8) == 1920

def test_invalid_request_returns_400(monkeypatch, capfd, context):
    monkeypatch.setattr(handler, "rentals_table", lambda: FakeTable())

    result = handler.lambda_handler(http_event({"bikeType": "tandem", "station": "Harbour", "hours": 1}), context)

    assert result["statusCode"] == 400
    assert "InvalidRequest" in metric_names(output_lines(capfd))

def test_storage_error_is_logged_with_stack_trace(monkeypatch, capfd, context):
    monkeypatch.setattr(handler, "rentals_table", lambda: FakeTable(fail=True))

    result = handler.lambda_handler(http_event({"bikeType": "city", "station": "Harbour", "hours": 1}), context)

    assert result["statusCode"] == 500
    lines = output_lines(capfd)
    error = next(line for line in lines if line.get("level") == "ERROR")
    assert error["exception_name"] == "RuntimeError"
    assert "RentalSaveFailed" in metric_names(lines)

uv run pytest -q

Expected result:

....                                                                     [100%]
4 passed in 0.59s

The test replaces the DynamoDB table with a small fake object via monkeypatch. Because the handler only creates the real table on first access, the test needs neither credentials nor network. If you also want to validate events against a schema, see my article on validating Lambda events with the Powertools Parser.

Step 5: Deploy and invoke

serverless deploy --stage dev

curl -s -X POST "https://<your-api-id>.execute-api.eu-central-1.amazonaws.com/rentals" \
  -H "Content-Type: application/json" \
  -d '{"bikeType":"ebike","station":"Central Station","hours":2}'

The response contains a new rental ID and the price of 1300 cents. For comparison, send an invalid request too, for example with "bikeType":"tandem". You get status 400 and, shortly afterwards, an InvalidRequest metric in CloudWatch.

{"rentalId": "1c5ddae2-314f-409d-bac8-ba64f4d8a92e", "priceCents": 1300}

Where do you find logs, traces and metrics?

🎬 Request flow with Lambda Powertools for Python – this animated diagram only runs on my site: watch it on sascha-ms.de

Logs in CloudWatch Logs Insights

A log line of the function looks like this (shortened). The fields cold_start, function_request_id and correlation_id come from the decorator, rental_id and station from append_keys:

{
  "level": "INFO",
  "location": "lambda_handler:110",
  "message": "Rental started",
  "service": "bike-rentals",
  "cold_start": true,
  "function_name": "bike-rentals-dev-startRental",
  "function_request_id": "11111111-2222-3333-4444-555555555555",
  "correlation_id": "req-42",
  "rental_id": "1c5ddae2-314f-409d-bac8-ba64f4d8a92e",
  "station": "Central Station",
  "price_cents": 1300
}

In Logs Insights you can now filter on these fields directly. The following query lists all errors of the service with station and correlation ID:

fields @timestamp, message, station, correlation_id, exception_name
| filter service = "bike-rentals" and level = "ERROR"
| sort @timestamp desc
| limit 20

Traces in X-Ray

In the CloudWatch trace view, every invocation shows a segment for the function, with subsegments for calculate_price, save_rental and the DynamoDB call below it. Tracer also adds the ColdStart and Service annotations automatically. With the filter annotation.station = "Central Station" you can find all rentals for one station.

Metrics in CloudWatch

After one or two minutes, the metrics RentalStarted, RentalRevenue, InvalidRequest and ColdStart show up under the BikeRental namespace. Powertools writes the cold start metric into a separate EMF blob with the dimensions function_name and service. How much cold starts hurt in Python and how to reduce them is covered in SnapStart for Python: measure and reduce Lambda cold starts.

Common mistakes with Lambda Powertools for Python

The namespace is missing

Without POWERTOOLS_METRICS_NAMESPACE or the namespace parameter, Metrics raises a SchemaValidationError when flushing. In production you notice this as a status 500 from your API. So set the variable in the provider and in your tests as well; then the error surfaces before you deploy.

The layer does not match the architecture

An x86 layer on an arm64 function deploys without complaint. It only breaks when a module with compiled code is loaded, for example the Parser via pydantic-core. Make sure the architecture and the parameter path use the same value. Even a Serverless Framework example in the official docs combines arm64 with an x86 layer.

High-cardinality dimensions

Every unique combination of metric name and dimension values is a separate metric in CloudWatch, and custom metrics are billed individually. A rental_id dimension therefore creates a new metric with every rental. For such values there is metrics.add_metadata(): they end up in the log, but not as a dimension. My rule of thumb: a dimension should only have values you could list on one hand or in a short list.

The log level seems to be ignored

If you enabled Lambda’s advanced logging controls, Lambda sets the AWS_LAMBDA_LOG_LEVEL variable. It takes precedence over POWERTOOLS_LOG_LEVEL. If the Lambda level is WARN, you will not see INFO lines, no matter what your serverless.yml says. Keep both values in sync; otherwise Powertools warns you with a message in the log.

Sensitive data in logs and traces

By default, Tracer stores return values and exceptions as metadata in the trace. For responses containing personal data, or responses larger than about 64 KB, set capture_response=False on the decorator. The same goes for log_event=True on the Logger: handy for debugging, but in production a direct way to write complete requests, tokens included, into your logs.

Cost and security

Powertools itself is free, but the destinations are not. JSON logs are longer than free text and increase the data volume in CloudWatch Logs. Each custom metric has a fixed monthly price, and X-Ray charges for recorded traces above the free tier. With high traffic it pays to look at X-Ray sampling and at POWERTOOLS_LOGGER_SAMPLE_RATE, which enables debug logs for only a share of invocations. You can explore how memory and duration affect the Lambda bill itself with my Lambda cost calculator.

For security, apply least privilege: the role in the example may call exactly dynamodb:PutItem on exactly one table. Logs are among your most sensitive data, so set retention periods and restrict access to log groups.

My take

To me, Powertools belongs in every production Python function on Lambda. The effort is small, and the difference shows during the first real incident: a correlation ID that finds every line of a request saves you more time than any runtime optimization.

Still, I would not switch everything on at once. Start with the Logger, because it brings the biggest benefit. Add metrics for two or three business values that someone actually looks at. Bring in the Tracer once several services are involved. If your team is already committed to OpenTelemetry, check whether the AWS Distro for OpenTelemetry is a better fit; Powertools deliberately relies on the X-Ray SDK to keep cold starts small. And the Python flavour is nicer than the TypeScript one in some details, especially when it comes to decorators.

FAQ

Do I need Powertools in my deployment package if I use the layer?

No. With the official layer, the library and the X-Ray SDK are available at runtime. You only install it locally for your editor and tests, for example as a dev dependency with uv.

Does Lambda Powertools for Python work without X-Ray?

Yes. Logger and Metrics do not need X-Ray. You can leave out the Tracer or disable it with POWERTOOLS_TRACE_DISABLED=true.

How do I test Powertools metrics with pytest?

Set the namespace and service name as environment variables, read the output with capfd or capsys, and parse each line as JSON. Between tests, reset the state with clear_metrics() and the cold start flag.

Which Python versions does Powertools v3 support?

According to the installation page, there are layers for Python 3.10 to 3.14, each for x86_64 and arm64. Python 3.9 layers stopped at version 27.

Your next step

Take an existing Python function and add only the Logger with a correlation ID. It takes half an hour and pays off the next time something breaks. After that, pick a single business value as a metric. Which value would that be for your most important function?

How to package your Python functions’ dependencies reproducibly and for the right Lambda architecture is covered in uv for AWS Lambda: Packaging Python Dependencies.

Sources

• Powertools for AWS Lambda (Python): documentation and environment variables
• Powertools (Python): installation, layer ARNs and SSM parameters
• Powertools (Python): Logger
• Powertools (Python): Tracer
• Powertools (Python): Metrics
• Amazon CloudWatch: Embedded Metric Format
• Serverless Framework: AWS Lambda functions (tracing, architecture)

This post was created with AI assistance.

---

👉 Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator – it runs right in your browser.

📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en