Introduction
CloudTrail lets us log all API calls in our AWS cloud. In this post, we shall see how to create a CloudTrail, see where the logs are stored in S3, delete log, digest files and perform log file validation.
Create a CloudTrail
Search for CloudTrail on the AWS console and create a trail.
S3 bucket
A bucket should be automatically created and associated with the CloudTrail. A folder with the name CloudTrail should appear on the bucket where all the cloud trail logs should get saved.
Generate logs
Now let's do an activity and see if it gets logged. Create a lambda function with name helloWorld and all other settings as default. You can do any other activity on AWS cloud as well, instead of creating a function.
We should see some files on S3 for this activity.
Delete log file
I am deleting one of the log files.
Log files are not suppose to be modified/deleted, as they can hold important auditing information, so now we need to find if our log files are modified or deleted(as in this case).
We try to validate now from the AWS CLI, it should say the digest file doesn't exit.
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<accoount-id>:trail/management-events --start-time 2022-08-29
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and 2022-08-29T06:26:38Z
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T06:26:38Z
No digests found
This is because we have not enabled log file validation for the cloud trail.
Enable Log file validation
We can enable log file validation, by editing the cloud trail.
Digest
As the log file validation is enabled, we should see a new folder CloudTrail-Digest in S3.
And digest files should get added each hour.
Validate
As we enabled Log file validation, we can now check the integirty of the logs.
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and 2022-08-29T07:00:20Z
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T07:00:20Z
Results found for 2022-08-29T05:55:08Z to 2022-08-29T06:55:08Z:
1/1 digest files valid
Though we deleted a log file earlier, it shows the digest is valid, because we did not enable log file validation then.
After an hour, we should see two digest files.
The log file validation seems good for now.
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and 2022-08-29T08:17:57Z
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T08:17:57Z
Results found for 2022-08-29T05:55:08Z to 2022-08-29T07:55:08Z:
2/2 digest files valid
10/10 log files valid
Delete log file with validation check
We can try deleting log file that was created after enabling log file validation.
As expected the log file validations fails for one file. However the digests are still valid.
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and 2022-08-29T08:22:42Z
Log file s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail/ap-south-1/2022/08/29/<account-id>_CloudTrail_ap-south-1_20220829T0755Z_7rDSVFC6Icgi9Z8V.json.gz INVALID: not found
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T08:22:42Z
Results found for 2022-08-29T05:55:08Z to 2022-08-29T07:55:08Z:
2/2 digest files valid
9/10 log files valid, 1/10 log files INVALID
It also clearly says the validation failed because it can't find a file that we deleted.
Delete digest
This time we can try deleting a digest file.
Hence digest validation should also fail.
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events --start-time 2022-08-29
Validating log files for trail arn:aws:cloudtrail:ap-south-1:<account-id>:trail/management-events between 2022-08-29T00:00:00Z and 2022-08-29T10:09:35Z
Digest file s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail-Digest/ap-south-1/2022/08/29/<account-id>_CloudTrail-Digest_ap-south-1_management-events_ap-south-1_20220829T085508Z.json.gz INVALID: not found
Log file s3://aws-cloudtrail-logs-<account-id>-4a8dcb98/AWSLogs/<account-id>/CloudTrail/ap-south-1/2022/08/29/<account-id>_CloudTrail_ap-south-1_20220829T0755Z_7rDSVFC6Icgi9Z8V.json.gz INVALID: not found
Results requested for 2022-08-29T00:00:00Z to 2022-08-29T10:09:35Z
Results found for 2022-08-29T05:55:08Z to 2022-08-29T09:55:08Z:
3/4 digest files valid, 1/4 digest files INVALID
20/21 log files valid, 1/21 log files INVALID
Note that we can enable versioning on S3 buckets to restore files.
Summary
So we saw how the log file validation feature in CloudTrail helps us find if there were any manual modifications to the log files or digest files. Thank you for reading !!!
Top comments (0)