Personally, I think cutting corners is inevitably linked to how we function as human beings. It's all about following the desired path. In my head, I often go over a problem and its solutions and in the meantime, I consider the tradeoffs of all options. Usually, this process is about finding the right balance between value and effort. For example, for a one-time job it's often not worth to put tremendous effort into automation.
However, from time to time, you stumble upon something which you thought you'd well considered. A few weeks later, giving your solution a second thought, you could feel like you're being hit by lightning.
Lets come to the point
One of the first things you learn on AWS is to follow the standard security advice of granting least privilege. It's a simple rule, but that doesn't mean it's easy to implement. When we started creating AWS CodePipelines a while ago we came up with the following CloudFormation code describing our infrastructure:
DeliveryPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtefactStores:
- Region: eu-west-1
ArtifactStore:
Location: !Ref ArtefactBucketName
Type: S3
Name: some-project-pipeline
RoleArn: !GetAtt CodePipelineServicenRole.Arn
Stages:
- Name: Source
Actions:
- Name: GitSource
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: "1"
Configuration:
Owner: !Ref GitHubOwner
Repo: !Ref GitRepo
Branch: !Ref GitBranch
PollForSourceChanges: False
OAuthToken: !Ref GitHubOAuthToken
OutputArtifacts:
- Name: SourceZip
...
- Name: Deploy
Actions:
- Name: DeployPersistantStack
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: "1"
Configuration:
ActionMode: CREATE_UPDATE
Capabilities: CAPABILITY_NAMED_IAM
RoleArn: !GetAtt CloudFormationExecutionRole.Arn
StackName: some-stack-cfn
TemplatePath: BuildArtifactAsZip::cfn-template.yaml
TemplateConfiguration: BuildArtifactAsZip::dist/config/cloudformation/stack-config.json
InputArtifacts:
- Name: BuildArtifactAsZip
RunOrder: 1
...
CloudFormationExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- cloudformation.amazonaws.com
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AdministratorAccess
I think the corner that has been cut in the code above is easy to find. A CloudFormationExecutionRole having AdministratorAccess isn't compliant the least privilege rule. The reason to take a shortcut -in this case- was the fact that CloudFormation stacks often do a lot of different things. Applying the least privilege rule in this context would mean a long and well-thought-out policy. So for simplicity and speed, we decided to cut a corner. The trade-off? Allowing the pipeline a tad more access then it should, no big deal, I could live with that.
Defence lines
When it comes to security, I think we made a pretty good fortress of our AWS environment: a good IDP, AWS WAF, AWS Security Hub, AWS Inspector and an endless list of other tools to close the gates. We made AWS a safe place to go to. A Git repository, however, often lives outside the AWS ecosystem. GitHub, GitLab and Bitbucket are the most common places to put your code. Next question: is your Git repository set up like a fortress as well? If the answer is no, then allow me to create some horror.
If someone could gain access to your Git repository, he could do one of the following if a pipeline has been granted AdministratorAccess for CloudFormation:
- Change a DNS Name to re-route traffic (to a phishing site)
- Change your DeletionPolicy and throw away all your resources
- Change an instance profile or a security group
- I think this already showed you enough horror
If I now repeat my question: can I live with that? The answer is the complete opposite. It's a big deal, no way I can accept this!
So this is also a note to myself. Always think twice when cutting corners, especially if security might be at risk. Sometimes it's not easy to understand the blast radius of decisions at first. To make things even worse: in many cases, it will be very hard or even impossible for AWS Security tools to define the correct severity level. In this particular case, AWS will give you a warning but it cannot describe all the underlying risks.
I'm feeling lucky that I got this epiphany before this could turn into a real issue.
Extra warning: Self-maintaining Pipelines
Although I have never been a fan of pipelines that maintain themselves, in this case, I need to give an extra warning. For clarity: by self-maintaining pipelines I mean pipelines that have a stage to update themselves. Having such a pipeline listening to Git means that the Pipeline policy itself can be changed by pushing a commit to Git. So if your self-maintaining pipeline doesn't have AdministratorAccess, this is easily changed by having access to its Git repository.
For that reason, I would remove self-updating logic from a pipeline. We use makefiles that need a client to authenticate in order to make Pipelines changes.
The solution
In the end, the best way to close this security hole is to apply the least privilege rule to your Pipeline. The downside of this approach is the introduction of extra work. Whenever someone will make changes to the CloudFormation stack that incur security context changes he will need to update the Pipeline's policy first. Be prepared for people who get really upset by this.
If you are in a situation where your current pipeline policy is too relaxed, you could already consider starting with an intermediate solution. You could begin by denying all IAM and Route 53 access and/or deny delete access for all resources. Although this is already more secure, this approach has some downsides. So it's just to buy you a bit more time before applying the least privilege rule.
Sharing is caring ;-)
Until next time.
Note: the same security hole can be created on AWS CodeBuild.
Top comments (0)