DEV Community

Cover image for AWS Lambda in Python with Datadog Integration
AWS Community Builders profile image Srinivasulu Paranduru
Srinivasulu Paranduru for AWS Community Builders

Posted on

AWS Lambda in Python with Datadog Integration

#aws #monitoring #python #serverless

Introduction
AWS Lambda is a serverless compute service that allows running code without provisioning or managing servers. It automatically scales applications by running code in response to events. Monitoring and observability are crucial for maintaining performance and reliability, and Datadog provides powerful tools to monitor Lambda functions, collect metrics, and visualize logs.

This guide explains how to integrate AWS Lambda written in Python with Datadog, including setup steps and sample Python code.

Prerequisites

  • AWS account with permissions to create and manage Lambda functions
  • Datadog account with API key
  • AWS CLI and Datadog CLI installed
  • Python 3.8 or later

Step 1: Create a Python AWS Lambda Function

  1. Open the AWS Management Console.
  2. Navigate to Lambda > Create function.
  3. Choose Author from scratch.
  4. Enter a function name (e.g., lambda-datadog-demo).
  5. Select Python 3.9 as the runtime.
  6. Choose or create an execution role with permissions for CloudWatch Logs.
  7. Click Create function.

Step 2:

  • Install Datadog Lambda Library

  • The Datadog Lambda Library enables tracing, metrics, and log forwarding.

  • Install the library locally:

pip install datadog-lambda pip install ddtrace

Package dependencies into a deployment zip:

mkdir package cd package pip install datadog-lambda -t . pip install ddtrace -t . zip -r ../lambda_function.zip . cd .. zip -g lambda_function.zip lambda_function.py

Step 3: Configure Datadog Integration

  1. In the Datadog dashboard, navigate to Integrations > AWS.
  2. Connect the AWS account by following the setup wizard.
  3. Enable Lambda integration.
  4. Copy the Datadog API key from Integrations > APIs.

Step 4: Add Environment Variables
In the Lambda configuration, add the following environment variables:

Key Value
DD_API_KEY your_datadog_api_key
DD_SITE datadoghq.com (or datadoghq.eu for EU region)
DD_LOGS_ENABLED true
DD_TRACE_ENABLED true
DD_FLUSH_TO_LOG true

Step 5: Instrument the Lambda Function
Create a file named lambda_function.py:

from datadog_lambda.wrapper 
import datadog_lambda_wrapper from datadog_lambda.metric 
import lambda_metric 
from ddtrace import tracer 

@datadog_lambda_wrapper 
def lambda_handler(event, context): # Custom metric   lambda_metric("custom.lambda.invocations", 1, tags=["env:dev", "function:lambda-datadog-demo"]) 

# Example trace with 
tracer.trace("lambda.process_event") as span:  span.set_tag("event.source", event.get("source", "unknown")) result = process_event(event) return {"statusCode": 200, "body": result} 

def process_event(event): # Simulated processing logic 
  message = event.get("message", "Hello from AWS Lambda!") 
  return f"Processed message: {message}"
Enter fullscreen mode Exit fullscreen mode

This code:

  • Wraps the handler with datadog_lambda_wrapper for automatic instrumentation.
  • Sends a custom metric to Datadog.
  • Creates a trace span for event processing.

Step 6: Deploy the Lambda Function
Upload the deployment package:

aws lambda update-function-code \ --function-name lambda-datadog-demo \ --zip-file fileb://lambda_function.zip

Step 7: Enable Log Forwarding

Datadog can collect logs from CloudWatch using the Datadog Forwarder Lambda.

  1. Deploy the Datadog Forwarder from the AWS Serverless Application Repository.
  2. Set the environment variable DD_API_KEY in the forwarder.
  3. Subscribe the forwarder to the CloudWatch log group of the Lambda function.

Step 8: Verify in Datadog
After invoking the Lambda function, open the Datadog dashboard:

  • Navigate to APM > Traces to view traces.
  • Go to Metrics Explorer to see custom metrics.

- Check Logs > Live Tail for Lambda logs.

Conclusion
Integrating AWS Lambda with Datadog provides deep visibility into serverless applications. With metrics, traces, and logs unified in one platform, teams can monitor performance, troubleshoot issues, and optimize costs effectively. Using the Datadog Lambda Library in Python simplifies instrumentation and ensures observability from the start.

💬 If you enjoyed reading this blog post and found it informative, please take a moment to share your thoughts by leaving a review and liking it 😀 and follow me in dev.to , linkedin, github

Top comments (0)