This topic is the subject of my presentation at the AWS User Group Mega Manila Meetup last February 20, 2020
Anyone who has dabbled in AWS will know that one of the reasons that it leads the cloud computing market is because of the breadth of its services. It has hundreds of services across multiple use cases. But with this kind of scale, keeping track of the AWS resources you use and enforcing best practice on them becomes difficult. This is an even more daunting challenge for an AWS account with many users (with varying degrees of experience) making changes all at once.
What usually happens is that teams hold periodic audits of their AWS environment. They go through each and every AWS resource, look through its configuration, recommending ways to improve its security posture and make it compliant with standards. These audits take a lot of effort; that's why it is done just every quarter or every year. But what if we can continuously assess our resources against the best practice?
1 | Enter AWS Config.
Keeps track of your current and historical configuration.
This service makes it easy to keep track of your AWS resources and their configurations, and how that configuration changes over time. You also get to see how your AWS resources relate to one another.
AWS Config displays the configuration of your AWS resource as a timeline. For this specific EC2 instance, we can see that its configuration has been changed only once since AWS Config was turned on for this account.
You can also see the various components related to the EC2 instance such as EBS Volumes, Security Groups, Subnets, etc.
Seeing the current and historical configuration of each AWS resource allows us to be reactive about security. When an attack happens, we can look through AWS Config and see what changes made the attack possible. But what if we could become more proactive about security?
Check your AWS environment against predefined rules
AWS Config also allows us to be more proactive. You can choose from dozens of predefined rules to check your environment against. AWS Config will run these checks once after the setup, and then it runs them periodically (i.e once a week) or whenever the configuration of a particular AWS resource changes. After the checking process, you will be able to see a list of resources that were not compliant with the rules you chose. With this list, you can be proactive about possible security vulnerabilities brought about by non-compliant resources. Why wait for a security incident to fix a non-compliant resource? Be proactive now so the incident doesn't have to happen at all.
Here are some examples of the predefined rules I found useful:
- rds-storage-encrypted - looks at each of your RDS instances and checks which of them don't have their storage encrypted. It is best practice to encrypt your RDS instances to make it harder for hackers to get something useful from your data (should they be able to actually hack into the instance hosting your database in RDS).
- alb-http-to-https-redirection-check - looks at each of your Application Load Balancers (ALB) and checks which of them don't redirect HTTP traffic to HTTPS. It is always best practice to use HTTPS (instead of HTTP) for your applications. Using HTTP lets the traffic between you and your users to be sent as plain text. This leaves your users vulnerable to anyone who could be listening to the flow of traffic. HTTPS makes the traffic encrypted, so if there's someone else listening to your user's traffic, they wouldn't be able to make sense of it.
- eip-attached - looks at each of your Elastic IP Addresses and checks which of them are not attached to an EC2 instance (or an in-use ENIs / this could be thought of as any AWS resource that needs an IP address). It is always best practice to have your EIPs attached because unattached EIPs incur a fee of around 5 USD per month.
As of the writing of this post, there are 116 predefined rules in AWS Config. Enough to be able to enforce basic best practices, if you ask me. But there are times when you'd want to create your own custom rule to enforce certain standards that might be fitted to your use case. You may want, for example, to enforce that each EC2 instance to have the tag "department" so you can keep track of the spending of each department in your company. You can take it up a notch and check if the department tag contains a valid department (to prevent sneaky developers from adding a filler department like "test-dev" just for the rule to pass).
Set up SNS notifications whenever an AWS Config rule gets violated.
You can also take it to the next level and set up an SNS topic to send the event to an SQS queue. The events in the SQS queue can be consumed by a Lambda function or an EC2 instance. The consumer can undo the change made and notify you about it.
For example, you are afraid that someone in your team will create a security group with all ports open to the world. You can set up an AWS Config rule to check for this kind of insecure security group. Then, once AWS Config detects this, it can send an SNS notification to an SQS queue. With an AWS Lambda function configured to consume tasks from the SQS queue, you can automatically fix this insecure security group by letting the lambda function change the insecure security group by removing the rule that makes all ports open to the world. Then, Lambda will send you an email of the event, and what it has done to fix it. If removing the rule seemed too restrictive of a response, you can just manually resolve it when you get to the office. But at least you don't leave your AWS environment at any given moment.
With this approach, you don't even have to be the one to fix non-compliant AWS resources, you can just pre-program a response to fix the issue.
2 | Let's get our hands dirty with AWS Config
I hope I was able to convince you on how useful AWS Config is for enforcing best practices. In this section, I will show you how easy it is to create and configure AWS Config is for your environment.
(2.1) Go to the AWS Management Console. Since AWS Config is a regional service, you have to choose the region you want to monitor and set up AWS Config there. Then, under the Services menu, search for Config. Click the first result that comes out.
(2.2) If you haven't set up AWS Config before in your region of choice, you will see this splash screen. Just click Get started.
(2.3) On the first page, you will configure your Configuration Recorder. This records all information about AWS resources in the region and their configuration.
Resource types to record - If you uncheck Record all resources supported in the region, you can choose the AWS services you want the CR to record (i.e just record all RDS DB Instance and EC2 Instance). I highly recommend just selecting to record all the resources in the region.
Amazon S3 Bucket - The changes in the configuration of your AWS resources are tracked by AWS Config, compiled into a configuration history file and sent to S3 every six hours. You can choose to create a bucket, use an existing bucket, or use an S3 bucket on another account. The third option is usually used by big companies to isolate the logging account from the actual account where the logs come from.
(2.4) Scroll down and you will see the section for the SNS topic and the Config role. Skip the SNS topic for now (make sure it is not enabled). For the AWS Config Role, select Create AWS Config service-linked role or Use an existing AWS Config service-linked role. This will create an IAM Role with read-only permissions (or use a role that's already been created by Config). This role will authorize AWS Config to look through all your AWS resources and compare them against the rules we will set in the next section. It is also authorized to add files to your chosen S3 bucket.
When you're done, click "Next".
(2.5) In the next screen, you will be able to choose what AWS Config rules you want to check your AWS resources against. As of the writing of this post, there are 116 AWS-managed rules (only 83 can be configured during the creation of the configuration recorder). I'm sure this number is just going to get bigger as time goes by.
The rules are actually interesting to read. You can take time browsing the rules you'd want to add to your environment. But just make sure to add the ec2-instance-no-public-ip rule. If this rule isn't one you'd like for your AWS environment in the long run, we can still remove it later on. We are adding it for demo purposes.
When you're satisfied, click next.
(2.6) Review the configuration for AWS Config. When you're satisfied with it, Click "Confirm".
You will see this screen for a few seconds. That means it is still setting up.
(2.7) Then, you will see this screen. You have to wait a few minutes as AWS Config scans your AWS environment for the first time, looking at each AWS resource in the region
(2.8) After a few minutes, you will be able to see this dashboard.
(2.9) Because I have no EC2 instance and I only have a rule that validates whether an EC2 instance has a public IP, the pane on the right looks as if there are no rules. To remedy this, let us create an EC2 instance with the following specs. If you need a guide on how to create EC2 instance you can check out this post
- Instance type: t2.micro
- VPC and Subnet: Depends on you (public subnet)
- Auto-assign Public IP: Enabled (make sure this is enabled so that we can demonstrate that this EC2 instance validates the AWS Config rule we made earlier)
- Key-Pair, Security Group, Storage - leave this as default. We don't be accessing this EC2 instance anyway.
After creating the EC2 instance, refresh the AWS Config dashboard. After a few minutes, you will see one non-compliant resource. It will correspond to the EC2 instance with a public IP address that we just created.
This demonstrates that the AWS Config rule has recognized that the EC2 instance we made wasn't compliant with the ec2-instance-no-public-ip rule. This is because we made sure that the EC2 we created in step 2.9 had a public IP address.
You can add other rules to monitor other resources in your environment. In the next post, we will set up a custom AWS Config rule with AWS Lambda.
That's it for this post. I hope you learned a lot. If you have any comments or suggestions, you can comment on my post below, or you can send me a private message!.
Top comments (4)
It is possible to create a custom rules? For example I need instances with public IP, but AWS config rule doesn't include any exceptions .
Hi monsad, yup, it is very possible to create custom rules. I actually have a blog post for that in my pipeline. If you want, I can publish it by end of week and update you in this thread when it is available.
Did you update for the exception
Yes It will be great.