Facing Your Fears in AWS AgentCore Observability - Tracking Malicious Behavior (and Poor Performance)
We have so much to consider when we face the risks to our Generative AI workloads. Some of the current categories of threats include varieties of attacks on our data and on our users. The best case scenario is to prevent these from occurring, of course. The reality is that we are not always going to succeed, especially since these threats are evolving so rapidly. My breakdown of some existing attacks on Gen AI is underscoring the importance of being able to look at and analyze logs.
Observability is already an incredibly important part of assessing your AI workloads’ performance and functionality, but we are also going to need it for security. I have been building agents lately and one of the aspects I am liking the most with AWS Bedrock AgentCore is the built-in observability options. We could look at a number of aspects about Bedrock model invocation in the past, but AgentCore observability adds much more detail.
Agents View
The Agents view tab adds metrics on sessions, traces, throttling, and errors.
There is additional information on this tab for runtime metrics:
Additionally, you will be able to see a table of instrumented agents that you are able to drill down into, and you have the ability to navigate to data protection and Logs Insights settings from here:
Sessions View
The Sessions view tab is a drill down enabled menu to list your sessions and metrics related to those:
Traces View
Traces — I have been spending quite a bit of time in the Traces tab. There is a summary of our traces:
Clicking on each trace will give you detailed information about the spans, including a trajectory flow map:
We can toggle into specific events in the logs including some of the items that will give us indicators that an attack may be occurring:
The information, errors, and red indicators in the logging are definitely helping me understand some of the agentic design issues that I need to tackle. All in all, incredibly useful information for both performance and security.
If you haven’t played with AgentCore yet, there are a number of great features to help with security issues- https://aws.amazon.com/bedrock/agentcore/. AgentCore has the easy button for agentic observability, but also session isolation and identity management, among other features. I’ve been building some agentic workflows I can share in my next article. Thanks for reading!
Top comments (0)