Jaya Ganesh

Posted on Jun 1

Don’t log it... Just Trace it! OpenTelemetry in AWS Lambda - Part 3

In the Previous article, we have set up OpenTelemetry in an AWS Lambda function and saw the traces in Honeycomb. In this article, let's see how to customise the OTel data by sending custom application-related metadata.

Step 1: Update Lambda Code

We can add a few business logics to our Lambda and also make external API calls.

Update the lambda function with the below code. It makes an API call to get a list of products and returns.

💡For the requests to work, you need to add the requests package via lambda layers. You either package your function with requests, or you can get lambda layer ARNs from the below URL.

import json
import requests

def get_products():
    try:
        url = "https://fakerapi.it/api/v2/products?_quantity=1&_taxes=12&_categories_type=uuid"
        response = requests.get(url)
        response.raise_for_status()
        data = response.json()["data"]
        return data
    except Exception as e:
        print(e)
        return None

def lambda_handler(event, context):
    try:
        print(event)

        products = get_products()

        print(len(products))

        return {"statusCode": 200, "body": json.dumps(products)}
    except Exception as e:
        return {
            "statusCode": 500,
            "body": json.dumps({"message": "There was some error"}),
        }

Step 2: Invoke and see the trace

Load the URL in browser and we will be able to see the data.

In Honeycomb, open the trace and see it.

if you notice, we are able to see the GET call that was made, time taken and also few metadata like URL and status..

We achieved all of these without make any changes to code. By just adding few more lines of code we would be able to enrich the telemetry data.

💡You get this level of detail without writing any additional instrumentation code, thanks to OpenTelemetry’s auto-instrumentation!

Step 3: Create custom spans

In Trace, we are able to see the API call that was made, but we don't have info about what was the function name which invoked the GET call, get_products function.

And, we have added a print statement in lambda handler for the incoming event. But that is not visible in Trace.

Let’s update the lambda function with the code below to add Span Attributes and Span Events.

import json
import requests

from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode

tracer = trace.get_tracer(__name__)

@tracer.start_as_current_span("get_products")
def get_products():
    try:
        url = "https://fakerapi.it/api/v2/products?_quantity=1&_taxes=12&_categories_type=uuid"
        response = requests.get(url)
        response.raise_for_status()
        data = response.json()["data"]
        return data
    except Exception as e:
        print(e)
        return None

def lambda_handler(event, context):
    with tracer.start_as_current_span("process_data") as span:
        try:
            span.add_event(
                "log",
                {"event": str(event)},
            )

            products = get_products()


            span.set_attribute("products length",  str(len(products)))

            return {"statusCode": 200, "body": json.dumps(products)}
        except Exception as e:

            span.record_exception(e)
            span.set_status(Status(StatusCode.ERROR))

            return {
                "statusCode": 500,
                "body": json.dumps({"message": "There was some error"}),
            }

After invoking the API again, head to Honeycomb and inspect the trace.

You would be able to see the span in a hierarchical format. We can see that the span event with the name log was added to the process_data span.

You should now see:

A custom span for get_products
A parent span named process_data
An event named log with the Lambda input event
A span attribute called products.length

We have also added a span attribute with the name “product length”. We can see it in the fields section.

Code Explanation:

Starting a new span

We can start a custom span in our code and wrap it around our function or part of a function which we want to track separately in OTel.

@tracer.start_as_current_span("get_products")

with tracer.start_as_current_span("process_data") as span:

Add custom span attribute

We can add our custom data in a key-value pair to Span as a span attribute, which would help us in querying the data efficiently and know the system status better

span.set_attribute("products length", str(len(products)))

Add Span Events

Use events to log contextual information like errors or input payloads:

span.add_event("log", {"event": str(event)})

Conclusion:

In this article, we learned how to:

Create custom spans in AWS Lambda
Add span events to track log-like info
Enrich spans with custom attributes

These enhancements help us better understand our application's behaviour, performance, and internal state using distributed tracing tools like Honeycomb.

DEV Community