DEV Community

Dom Derrien
Dom Derrien

Posted on • Edited on

From Chaos to Control: Why We Need Secure Deployment Pipelines

#aws #cdk #githubactions #githubrulesets

This is the first article in a 5-part series about implementing GitHub Rule Sets and secure workflows for AWS deployments. We'll go from a speed-first approach to a validation-first pipeline that maintains developer velocity while ensuring security and reliability.

Series Overview

Over the coming articles, we'll build a complete secure deployment pipeline:

  1. From Chaos to Control: Understanding the problem and solution architecture (this article)
  2. GitHub Rule Sets: Implementing parallel validation with status checks
  3. Secure Code Review: Branch protection and automated security scanning
  4. The Trust Challenge: Safe infrastructure previews in forked workflows
  5. Lessons Learned: Real-world insights and best practices

Let's start with why we needed this transformation in the first place.

From Speed to Security: Rethinking Our AWS Pipeline

The Starting Point: A Fast but Fragile System

Over the past few months, I've been building and refining a monorepo with NPM Workspaces that hosts different parts of my AWS-based application.

my-aws-app/
├── 📁 iac/           ← CDK Infrastructure
├── 📁 serverless/    ← Lambda Functions + Tests
├── 📁 webapp/        ← Vite+Lit SPA
└── 📁 cdn/           ← Static Assets → S3
Enter fullscreen mode Exit fullscreen mode

Initially, our pipeline prioritized speed over everything else. The workflow was simple.

 "Move Fast" Approach
┌───────────────┐  ┌─────────────┐  ┌───────┐  ┌───────────────┐  ┌─────────────┐  ┌───────┐
│ PR -> develop │─▶│ Auto Deploy │─▶│Reviews│─▶│   Manual PR   │─▶│ Auto Deploy │─▶│Reviews│
└───────────────┘  │ to develop  │  └───────┘  │develop -> main│  │   to main   │  └───────┘
                   └─────────────┘             └───────────────┘  └─────────────┘
Enter fullscreen mode Exit fullscreen mode

Merging a pull request into develop would instantly deploy a new Develop stack—great for rapid feedback and previews. Production deployments were gated behind manual pull requests from develop to main. This informal control worked well initially, but it relied entirely on our team's discipline rather than automated validation.

⚠️ What went wrong with our speed-first approach:

  • Dependencies with unchecked vulnerabilities
  • Infrastructure changes without proper review
  • No audit trail for changes

The final straw came when Dependabot started flooding us with dependency updates. While helpful for staying current, each update triggered automatic deployments without proper validation. We needed structure, not just speed.

The Vision: Control Without Bottlenecks

I decided to implement GitHub Rule Sets and re-architect the GitHub Actions workflows. The goal was to build confidence in every deployment.

"Validate First" Approach
┌─────────────┐  ┌─────────┐  ┌───────┐  ┌───────────┐  ┌───────────────┐  ┌─────────┐  ┌───────┐  ┌─────────┐
│PR -> develop│─▶│CI Checks│─▶│Reviews│─▶│ Approved  │─▶│   Manual PR   │─▶│CI Checks│─▶│Reviews│─▶│Approved │
└─────────────┘  └─────────┘  └───────┘  │  Deploy   │  │develop -> main│  └─────────┘  └───────┘  │ deploy  │
                 ┌──────────────┐        │to develop │  └───────────────┘                          │ to main │
                 │ Quality Scan │        └───────────┘                                             └─────────┘
                 └──────────────┘
                 ┌────────────────┐
                 │ AWS Infra Diff │
                 └────────────────┘
Enter fullscreen mode Exit fullscreen mode

The new system enforces four key validation gates:

Gate What It Checks Why It Matters
✅ Build & Test Code compiles, tests pass Prevents broken deployments
🔍 Security Scan npm audit, ESLint rules Catches vulnerabilities early
🧠 Infrastructure Diff CDK changes preview Transparency before infrastructure changes
🛑 Review Gate Human approval required Maintains code quality and knowledge sharing

The Implementation Challenge

Setting up this secure pipeline wasn't trivial—it took ome intensive week to get right. The complexity came from several technical challenges:

  • Status Checks: Configuring GitHub Rule Sets to recognize parallel validation jobs
  • Security Scanning: Balancing thoroughness with cost (avoiding GitHub's $53/user Advanced Security)
  • Infrastructure Diffing: Safely running cdk diff on untrusted pull request code
  • Performance: Optimizing workflows to avoid slowing down development

What's Ahead

In the following sections, I'll walk you through each component of this hardened pipeline:

  • Parallel Validation: How status checks and matrix strategies streamline CI
  • Review Requirements: Setting up code owners and approval gates
  • Security Scanning: Cost-effective tools for vulnerability detection
  • Infrastructure Safety: The dual-checkout pattern for secure CDK diffs
  • Lessons Learned: What worked, what didn't, and what I'd do differently

The result is a pipeline that's both safer and more trustworthy, without sacrificing the development velocity that made us productive in the first place.

What's Coming Next

In the next article, we'll dive into the technical implementation of GitHub Rule Sets and status checks. You'll learn how to set up parallel validation pipelines that can handle multiple projects simultaneously, using matrix strategies to optimize both performance and clarity.

We'll cover:

  • Creating Rule Sets with JSON configuration
  • Setting up status checks that actually work
  • Implementing matrix strategies for parallel CI
  • Optimizing caching and cleanup

The goal is to transform your pull requests from simple code reviews into comprehensive validation checkpoints.

Continue with Part 2: GitHub Rule Sets - Enforcing Quality Through Status Checks →

Top comments (0)