Cut AWS Costs by 90%: How We Saved Big and Gained Flexibility

Take a moment to reconsider!

That’s correct, no typo involved. After activating AWS Config across five of our AWS accounts, we made the surprising decision to disable all but two of our Config rules. But what prompted this move?

Let’s take a step back. If you’re not familiar with AWS Config, it’s a service offered by AWS that evaluates the configuration settings of your AWS resources. This is achieved by enabling AWS Config rules in one or multiple of your AWS accounts to check your configuration settings against best practices or your desired/approved settings.

At the time of writing, there are over 80 managed rules available. These rules provide a solid foundation for a configuration audit. However, not all rules are applicable, depending on your environments and use cases. Internally at LifeOmic, we selected 21 out of those and enabled them last year.

Now, let’s dive into the reasons behind disabling most of them last week.

90% Cost Reduction of AWS Config: A Game-Changer for Lean Startups

For the five AWS accounts on which we’ve enabled Config, the service is costing us over $3,300 per month, or around $40,000 a year. This was significantly higher than anticipated, especially for a lean startup. Enabling across all ten AWS accounts, we would likely have doubled that.

Luckily, we’ve started using JupiterOne. With the J1QL queries and the latest alerts capability, we decided to configure the equivalent of AWS Config evaluations using JupiterOne instead. Out of the 21 Config rules we’ve previously enabled, 19 of those can already be replaced.

AWS Config Rule	Supported by J1 query/alert
acm-certificate-expiration-check	Yes
ec2-instances-in-vpc	Yes
ec2-volume-inuse-check	Yes
encrypted-volumes	Yes
restricted-ssh	Yes
iam-root-access-key-check	Yes
iam-password-policy	Yes
iam-user-no-policies-check	Yes
lambda-function-settings-check	Yes
db-instance-backup-enabled	Yes
rds-snapshots-public-prohibited	Yes
rds-storage-encrypted	Yes
dynamodb-throughput-limit-check	No
s3-bucket-public-read-prohibited	Yes
s3-bucket-public-write-prohibited	Yes
s3-bucket-replication-enabled	Yes
s3-bucket-server-side-encryption-enabled	Yes
s3-bucket-ssl-requests-only	No
s3-bucket-logging-enabled	Yes
s3-bucket-versioning-enabled	Yes
cloudtrail-enabled	Yes

Read more about how we saved 90% on costs and gained flexibility by ditching AWS Config here.

For further guidance, please consult our documentation page and the alerts rule pack on GitHub.

We preserved two rules that are currently incompatible with JupiterOne J1QL queries and alerts. However, given JupiterOne's existing integration with AWS Config, we developed a "fallback" J1QL alert rule to detect non-compliant Config rule evaluations:

JupiterOne Alert Rule
name	config-rule-noncompliant
description	AWS Config rule evaluation identified non-compliant resource configurations.
query	Find aws_config_rule with complianceState='NON_COMPLIANT'
condition	When query result count > 0

This approach allowed us to minimize our dependence on AWS Config rules, limiting them to only those essential, while maintaining consistent and centralized configuration alerts within JupiterOne. As a result, we achieved a remarkable 90% reduction in our AWS bill for the Config service.

Was this effort solely driven by cost savings? While that's certainly a welcome benefit, it's not the only motivation.

Augmented Agility in Customization

Previously, with AWS Config rules, it was difficult to incorporate additional contextual filters into the rule configuration to minimize false positives.

For example, the “s3-bucket-public-read-prohibited” rule from AWS Config does not accept any additional parameters.

What if I have certain S3 buckets hosting public resources and, therefore, are intended to be publicly readable?

With JupiterOne's query, it's effortless to fine-tune those out by adding the classification property/tag filter to the J1QL query:

Find aws_s3_bucket with classification != 'public'
  that ALLOWS as grant Everyone where grant.permission='READ'

Similarly, if we want to take into account additional contexts such as production status and classification label for “s3-bucket-replication-enabled” and “s3-bucket-versioning-enabled,” we can add those filters easily to the J1QL query:

// s3-bucket-replication-enabled
// find production buckets that do not have replication enabled
Find aws_s3_bucket with tag.Production = true
  and (replicationEnabled != true or destinationBuckets = undefined)

// s3-bucket-versioning-enabled
// find production buckets classified as critical without versioning or
// mfaDelete enabled
Find aws_s3_bucket
  with tag.Production = true and classification = ’critical’
  and (versioningEnabled != true or mfaDelete != true)

We can apply the same pattern to easily tune any rule to reduce false positives. We did the same for the “fallback” config-rule-noncompliantrule:

Find aws_config_rule with compliant = false
  that evaluates * with
    tag.Production = true or
    classification = ‘critical’ or
    criticality >= 9

Additionally, we leverage the power of relationships from the JupiterOne knowledge graph to create more precise targets. For example, we can correlate users who have been assigned access to our production AWS accounts via SAML single sign-on, the user’s endpoint devices, and the compliance status on those devices to create a query/alert only when those privileged users have endpoints that fall out of compliance:

Find HostAgent with compliant!=true
  that monitors (Host|Device)
  that owns Person
  that is User
  that (assigned|has)
    (aws_iam_role|aws_iam_policy|aws_iam_user_policy|aws_iam_group)
    with tag.Production=true

As an aside, for those curious about the visual representation, here's a glimpse of the graph from the preceding query:

Streamlined Sophistication

The inherent abstract class labeling within the JupiterOne data model empowers us to harness the potential of abstract rules, thereby minimizing the complexity of rule management and alert analysis. This is illustrated in the following example:

For instance, to verify whether encryption is enabled across all production data stores, we might otherwise require multiple AWS Config rules, each tailored to a specific data store type:

• S3 buckets
• RDS instances
• DynamoDB tables
• EBS volumes

However, within JupiterOne, these entities are unified under the abstract class label: DataStore. This enables the application of a single, comprehensive alert rule, eliminating the need for multiple rules. Furthermore, we can refine this rule by applying tag/property filters to minimize false positives.

Find DataStore with 
  tag.Production = true and 
  classification = ’critical’ and
  encrypted != true

Consider another scenario, where thousands of security findings from diverse security scanners, agents, and monitoring tools necessitate the setup of alerts for the most critical issues. By leveraging the abstract entity labeling and graph relationships within JupiterOne, we can achieve a streamlined approach, such as:

Find (Device|Host|Database|Application|CodeRepo) with
  tag.Production = true or classification = ’critical’
that has Finding with severity = ‘critical’ or numericSeverity > 7

This single query encompasses real-time threat detection from AWS Inspector, GuardDuty, application code scanning solutions like Veracode and WhiteHat, and endpoint monitoring tools like Carbon Black and SentinelOne. Moreover, JupiterOne boasts seamless, out-of-the-box integrations with all of these platforms.

Centralized Alert and Notification Hub

In the past, receiving alerts from AWS Config evaluations required configuring CloudWatch to capture the findings, setting up alarms, and then configuring SNS and/or SES to dispatch notifications. This involved coordinating multiple services. And this was just for alerts and notifications in one environment; we would have had to replicate this process for half a dozen other security controls we had implemented. Furthermore, when the alerts were triggered, we would have had to visit the dashboard/console of each individual system to view them.

Instead, we have consolidated all alerts within JupiterOne. In the Alerts app, we have access to alerts, findings, and vulnerabilities from all integrated sources. We can set up notifications for them and perform in-depth analysis on them all in one place. This is still in its early stages of development. More to come, but here is a sneak preview.

That’s it. We’ve moved on from AWS Config and have no regrets whatsoever.