DEV Community

Cover image for CloudWatch Insights - how to find the context of multiple requests?
Rafal Hofman
Rafal Hofman

Posted on • Originally published at brightinventions.pl

1 1

CloudWatch Insights - how to find the context of multiple requests?

Recently I was searching through our application logs. The task was to extract extra context for a group of requests (ex. errors in the external provider system with the original request). For our app, we are using CloudWatch to store the logs. I have used CloudWatch Insights as out of the box tool to analyze them.

Our logs have a format like below, with each console output in a separate line:

2021-02-06T13:38:31.730Z info [some request id 1; some user id 1] Some external provider error message
2021-02-06T14:21:00.000Z info [some request id 2; some user id 2] Some external provider error message
Enter fullscreen mode Exit fullscreen mode

We can use Cloudwatch Insights to extract all the information related to that requests:

filter @message like "Some context to error message log"
| parse @message "* * [* *] *" as timestamp,type,requestId, user, textMessage
| filter requestId in ["some request id 1;", "some request id 2"]
| sort @ingestionTime desc
Enter fullscreen mode Exit fullscreen mode

If the field you are searching for is a JSON array, you can search it like:

filter @message like "Some context to error message log {
    "someInfo": [
        some1,
        some2
    ]
}"
| parse @message "* * [* *] *" as timestamp,type,requestId, user, textMessage
| parse textMessage '"someInfo":[*]' as someInfo
| filter requestId in ["some request id 1;", "some request id 2"]
| sort @ingestionTime desc
Enter fullscreen mode Exit fullscreen mode

You can then export the data that you need or build some stats around it.

Let me know in the comments if you found CloudWatch Insights useful too and how you are using them.

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Engage with a sea of insights in this enlightening article, highly esteemed within the encouraging DEV Community. Programmers of every skill level are invited to participate and enrich our shared knowledge.

A simple "thank you" can uplift someone's spirits. Express your appreciation in the comments section!

On DEV, sharing knowledge smooths our journey and strengthens our community bonds. Found this useful? A brief thank you to the author can mean a lot.

Okay