DEV Community

Rocky
Rocky

Posted on

You Enabled CloudTrail. That Doesn't Mean You'd See Someone Emptying the Bucket

An incident responder gets pulled in on a Friday afternoon because a customer's data turned up somewhere it shouldn't have. The AWS account in question has CloudTrail enabled, has had it enabled since the account was created, checked the box, ticked the compliance requirement, moved on. So the first move is obvious: pull the trail, filter to the S3 bucket in question, and look for the download. Nothing. Hours of digging, and the trail shows role assumptions, a few ListBuckets calls, nothing that accounts for gigabytes of objects leaving that bucket. The data is gone. The evidence isn't there. And the reason it isn't there has nothing to do with retention policy or log deletion. It's because CloudTrail was never logging the thing they needed it to log.

CloudTrail splits what it records into two categories, and the split matters more than most teams realize until they're in exactly this situation. Management events are control-plane operations, creating a bucket, changing an IAM policy, launching an instance, and a default trail captures those account-wide, in every region, at no extra cost. Data events are the actual object-level and record-level operations against a resource: GetObject and PutObject calls against S3, Invoke calls against Lambda, item-level reads against DynamoDB. Data events are not on by default. They have to be explicitly configured, per resource or per bucket, and AWS bills for them separately because the volume is enormous compared to management events. An account can have had CloudTrail switched on since day one and still have logged not one single GetObject call, ever, because nobody went back and turned on the layer that actually watches what happens to the objects inside the bucket.

Which means the failure mode isn't a misconfigured trail. It's a trail configured exactly to spec, doing exactly what a default CloudTrail setup is documented to do, while an attacker with valid, unremarkable credentials pulls object after object out of a bucket for as long as they want, leaving a record that looks, to anyone who hasn't gone looking for data events specifically, like a perfectly quiet account. The gap isn't a bug in CloudTrail. It's a decision AWS leaves to you, on purpose, because logging every object read at scale is expensive, and most accounts never come back to make that decision deliberately. They inherit whatever the default was on day one and call it "we have logging."

The same shape of gap shows up past AWS too: every major cloud splits control-plane activity logging from data-plane activity logging, and in every one of them the data-plane layer is the opt-in, costlier tier that's easy to skip when nobody's specifically auditing for it. The fix isn't complicated once you know to look for it: turn on data event logging for anything that holds data worth stealing, decide deliberately what that costs against what a silent exfiltration costs, and build the detection logic that actually watches the layer where the real damage happens instead of the layer that's free to log by default.

That's the exact gap Codelivly's Cloud Detection and Response Book: AWS, Azure, GCP is written to close: detection-as-code and incident response built around what each provider actually logs by default versus what you have to turn on yourself, across AWS, Azure, GCP and Kubernetes, not just the control-plane events every default dashboard already shows you.

Top comments (0)