CI/CD Pipelines: The Automated Path from Code to Production
A practical guide to CI/CD — the discipline and pipeline design behind automatically building, testing, and deploying applications — covering the difference between CI and CD, pipeline stages, testing strategy, deployment strategies, pipeline design principles, and how the tool-specific guides in this series fit into the bigger picture.
Table of Contents
- Introduction
- Continuous Integration vs. Continuous Delivery vs. Continuous Deployment
- Anatomy of a Pipeline
- The Testing Pyramid Inside a Pipeline
- Build Artifacts: Build Once, Deploy Many
- Deployment Strategies
- Pipeline Design Principles
- Quality and Security Gates
- Environment Promotion
- Observability of the Pipeline Itself
- Common Anti-Patterns
- Choosing and Combining Tools
- Quick Reference Table
- Conclusion
Introduction
CI/CD — Continuous Integration and Continuous Delivery/Deployment — is the automated pipeline that takes a code change from a developer's commit through building, testing, and ultimately into a running production environment, with as little manual intervention as reliably possible. It's less a single tool and more a discipline: a set of practices and a pipeline architecture that this series has already touched on concretely through GitHub Actions and Azure DevOps. This guide steps back to the underlying principles — what actually makes a pipeline good, independent of which specific product implements it.
Commit → Build → Unit Tests → Integration Tests → Package → Deploy to Staging → Smoke Tests → Deploy to Production
That chain, automated end-to-end, is what CI/CD is actually for: turning "ship a change safely" from a manual, error-prone, human-memory-dependent process into a repeatable, fast, and trustworthy one.
1. Continuous Integration vs. Continuous Delivery vs. Continuous Deployment
These three terms are related but distinct, and the differences matter for how a pipeline is actually designed.
Continuous Integration (CI)
CI means every developer merges their changes into a shared branch frequently (multiple times a day, ideally), with an automated build and test suite running on every merge to catch integration problems immediately rather than during a painful, infrequent "merge day."
Developer pushes → automated build + tests run → pass/fail feedback within minutes
The core promise of CI is fast feedback: a broken build or failing test is caught within minutes of the change that caused it, while the change is still fresh in the developer's mind, rather than discovered days or weeks later when many other changes have piled on top of it.
Continuous Delivery
Continuous Delivery extends CI one step further: every change that passes the pipeline is automatically built into a deployable artifact and is always in a state ready to be released to production — but the actual production deployment still requires an explicit, deliberate trigger (a button click, an approval).
CI pipeline passes → artifact built and verified → ready to deploy → [human decides when to actually deploy]
Continuous Deployment
Continuous Deployment goes further still: every change that passes the full pipeline (including automated tests and any automated quality gates) deploys to production automatically, with no human approval step at all.
CI pipeline passes → automatically deployed to production, no human gate
Why the distinction matters for pipeline design
| Continuous Integration | Continuous Delivery | Continuous Deployment | |
|---|---|---|---|
| Automated build/test on every change | Yes | Yes | Yes |
| Automatically produces a deployable artifact | Not necessarily | Yes | Yes |
| Production deployment | Manual, separate step | Manual trigger, but pipeline-ready | Fully automatic |
| Human approval gate before production | N/A | Yes | No |
Most organizations land somewhere between Continuous Delivery and Continuous Deployment in practice — a high level of automated confidence (comprehensive automated tests, quality gates) combined with a deliberate choice about where a human approval gate still makes sense (often for production specifically, while lower environments deploy fully automatically), rather than treating it as an all-or-nothing choice.
2. Anatomy of a Pipeline
Regardless of which specific tool implements it (GitHub Actions, Azure DevOps, Jenkins, GitLab CI, and others all share this same basic shape), a mature CI/CD pipeline typically moves through the following stages:
1. Source — triggered by a commit, PR, or scheduled/manual event
2. Build — compile the code, resolve dependencies
3. Test — unit tests, then progressively broader test types
4. Package — produce a versioned, immutable artifact (a container image, a NuGet package, a zip)
5. Deploy (lower env) — deploy the artifact to a test/staging environment
6. Verify — smoke tests, integration tests against the real deployed environment
7. Deploy (production) — promote the *same* artifact to production, often behind an approval gate
8. Post-deploy verification — health checks, synthetic monitoring, error-rate watching
Build: compiling and resolving dependencies
dotnet restore
dotnet build --configuration Release
The build stage should fail fast and cheaply — catching a compilation error here, in seconds, is far preferable to discovering it later in a more expensive stage (a failed deployment, or worse, a runtime error in production).
Test: layered, from fast to slow
Covered in depth in Section 3 — the general principle is running the fastest, cheapest tests first, so a pipeline fails as early and as cheaply as possible when something's genuinely broken.
Package: producing the artifact that will actually be deployed
Covered in depth in Section 4 — the artifact produced here is what should flow, unchanged, through every subsequent environment.
Deploy and verify: progressively higher-stakes environments
A pipeline typically deploys to progressively more production-like (and higher-stakes) environments — a short-lived ephemeral/PR environment, then a shared staging/QA environment, then production — with verification steps at each stage designed to catch problems before they reach the next, more consequential one.
3. The Testing Pyramid Inside a Pipeline
The shape of a healthy test suite
/\
/ \ E2E / UI tests (few, slow, high-fidelity, expensive to maintain)
/----\
/ \ Integration tests (moderate count, moderate speed)
/--------\
/ \ Unit tests (many, fast, cheap, isolated)
/------------\
The "testing pyramid" describes a healthy test suite's shape: many fast, cheap, narrowly-scoped unit tests at the base; a moderate number of integration tests verifying that components actually work together (against a real database, a real message queue); and a small number of slow, expensive, broad end-to-end tests at the top, reserved for the handful of critical user journeys that genuinely need that level of fidelity.
Why order and speed matter in a pipeline specifically
jobs:
unit-tests:
runs-on: ubuntu-latest
steps: [ /* fast — seconds to a couple minutes */ ]
integration-tests:
needs: unit-tests # only run if unit tests already passed
runs-on: ubuntu-latest
steps: [ /* slower — spins up a real database, message queue, etc. */ ]
e2e-tests:
needs: integration-tests
runs-on: ubuntu-latest
steps: [ /* slowest — full application + browser automation */ ]
Running unit tests first and gating slower test stages on their success means a pipeline fails fast on the most common category of bug (caught by a fast unit test) without first paying the cost of spinning up a full integration/E2E environment — a meaningful difference in both pipeline duration and infrastructure cost across hundreds of pipeline runs per week.
Test flakiness: the silent trust-killer
A test suite with intermittently failing ("flaky") tests — tests that sometimes pass and sometimes fail with no underlying code change — is one of the most corrosive problems a CI/CD pipeline can have: once developers learn that a red pipeline might just be "the flaky test again," they start ignoring genuine failures too, and the entire safety net the pipeline exists to provide quietly stops functioning. Treating flaky tests as a priority to fix or quarantine (rather than re-running until they pass) is essential to keeping a pipeline's signal trustworthy.
Where each testing guide in this series fits
The layered testing structure described here draws directly on where things run and what they run against — unit tests need no external dependencies at all, integration tests commonly spin up the real databases covered in this series' SQL Server, PostgreSQL, Cosmos DB/MongoDB, and Redis guides (often via Testcontainers, as mentioned in the EF Core guide's testing section), and end-to-end tests exercise a fully deployed environment running the actual containers described in the Docker and Kubernetes/Helm guides.
4. Build Artifacts: Build Once, Deploy Many
The principle
A build should happen exactly once per change, producing a single immutable, versioned artifact — and that same artifact should be promoted through every subsequent environment (staging, then production) without being rebuilt.
Build → artifact:my-api:a1b2c3d (tagged with the commit SHA)
↓
Deploy a1b2c3d to staging
↓ (verified)
Deploy the *same* a1b2c3d to production
Why rebuilding per environment is a real risk, not just inefficiency
# ❌ Risky: separate builds for staging and production
- job: build-and-deploy-staging
steps: [restore, build, test, deploy-to-staging]
- job: build-and-deploy-production
steps: [restore, build, test, deploy-to-production] # a SEPARATE build
If staging and production each trigger their own independent build, there's no actual guarantee the two environments are running identical code — a different dependency version resolved at a slightly different time, a non-deterministic build step, or a subtle environment difference in the build agent itself could all produce a meaningfully different artifact despite starting from "the same" source commit. This directly undermines the entire point of testing in staging: the thing you tested and the thing that goes to production need to be the exact same bits, not just "built from the same source."
# ✅ Build once, deploy the same artifact everywhere
- job: build
steps: [restore, build, test, package]
# produces: my-api:a1b2c3d
- job: deploy-staging
needs: build
steps: [deploy my-api:a1b2c3d to staging]
- job: deploy-production
needs: deploy-staging
steps: [deploy my-api:a1b2c3d to production] # the SAME artifact
Artifacts and version tags
docker tag my-api:latest myregistry.azurecr.io/my-api:a1b2c3d
Tagging build artifacts with something traceable back to the exact source (a commit SHA, or a build number tied to one) — rather than a mutable tag like latest — is what makes "which exact code is running in production right now" an answerable question at any point, not just something inferred from deployment timestamps and hoping nothing else changed in between.
5. Deployment Strategies
This series has covered specific mechanics for several of these already (App Service deployment slots, ECS blue/green via CodeDeploy, Kubernetes rolling updates/canaries) — this section frames them as a shared set of general strategies, independent of the specific platform implementing them.
Rolling deployment
Old: [v1] [v1] [v1] [v1]
↓ replace one at a time
[v2] [v1] [v1] [v1] → [v2] [v2] [v1] [v1] → ... → [v2] [v2] [v2] [v2]
Instances are replaced incrementally — the default strategy across most platforms (App Service, ECS, Kubernetes Deployments) since it requires no extra infrastructure capacity beyond a small surge, at the cost of a brief window where both old and new versions serve traffic simultaneously (which is exactly why the expand/contract migration pattern from this series' Database Migrations guide matters).
Blue/green deployment
Blue (current, live): [v1] [v1] [v1] [v1] ← 100% of traffic
Green (new, staged): [v2] [v2] [v2] [v2] ← 0% of traffic, fully warmed and verified
[traffic cutover — atomic switch]
Blue (now idle): [v1] [v1] [v1] [v1] ← 0% of traffic, kept briefly for instant rollback
Green (now live): [v2] [v2] [v2] [v2] ← 100% of traffic
An entirely separate, fully-scaled copy of the new version runs alongside the old one, verified completely before an atomic traffic cutover — the old version stays available briefly afterward, making rollback as simple as switching traffic back, at the cost of temporarily running double the infrastructure.
Canary deployment
v1: 95% of traffic
v2: 5% of traffic ← watch error rates, latency, business metrics closely
if healthy: gradually increase v2's share (5% → 25% → 50% → 100%)
if unhealthy: cut v2's traffic back to 0%, no full-scale impact occurred
A small percentage of real traffic is routed to the new version first, with the rollout's pace governed by observed health — the strategy that limits blast radius most precisely among the three, at the cost of the added complexity of traffic-splitting infrastructure (a service mesh, a sufficiently capable load balancer/Ingress controller) and needing solid real-time metrics to actually judge "is the canary healthy" reliably.
Feature flags: decoupling deployment from release
if (featureFlags.IsEnabled("new-checkout-flow", user))
{
return NewCheckoutFlow();
}
return LegacyCheckoutFlow();
Feature flags let new code deploy to production dormant, then be released (activated) independently — for a subset of users, gradually, or instantly reverted with a flag flip rather than a full redeploy/rollback. This decoupling is a powerful complement to any of the deployment strategies above: the deployment itself becomes lower-risk (the new code path isn't even active yet), and the "is this feature actually good" decision becomes separate from and much faster than "is this deployment mechanically successful."
Choosing a strategy
| Strategy | Infrastructure cost | Rollback speed | Blast radius control |
|---|---|---|---|
| Rolling | Low (small surge only) | Moderate (redeploy previous version) | Moderate |
| Blue/green | High (double capacity, temporarily) | Fastest (traffic switch) | Coarse (all-or-nothing cutover) |
| Canary | Moderate | Fast (cut canary traffic) | Finest-grained |
| + Feature flags | Minimal additional cost | Instant (flag flip) | Very fine, per-user/segment |
6. Pipeline Design Principles
Fail fast
Order pipeline stages from cheapest/fastest to most expensive/slowest (Section 3's testing pyramid ordering is a specific instance of this general principle) — the goal is surfacing a real problem with the least wasted time and infrastructure cost possible.
Idempotency
A pipeline stage, re-run with the same inputs, should produce the same result — a deployment step that behaves differently depending on prior partial-failure state, or a migration that can't safely be re-applied (covered in this series' Database Migrations guide), makes a pipeline fragile and unpredictable exactly when you need it to be reliable: during an incident recovery.
Determinism
# ❌ Non-deterministic: floating version ranges resolve differently over time
dotnet add package Newtonsoft.Json --version "13.*"
# ✅ Deterministic: pinned, exact versions, with a lock file committed
dotnet add package Newtonsoft.Json --version "13.0.3"
A build should produce the same artifact given the same source commit, regardless of when it's run — floating dependency versions, unpinned base images (as touched on in this series' Docker guide), or build steps that depend on external, unversioned state all undermine this, and directly threaten the "build once, deploy many" principle from Section 4.
Fast feedback loops
The time between "developer pushes a change" and "developer knows whether it's good" should be minimized wherever the cost/benefit makes sense — parallelizing independent test suites (as shown in Section 3's job structure), caching dependencies (covered in this series' GitHub Actions guide), and running the fastest checks first are all in service of this single goal.
Pipelines as code
Every tool covered in this series' GitHub Actions and Azure DevOps guides treats the pipeline definition itself as version-controlled, reviewable YAML — a pipeline change goes through the same review process as an application code change, rather than being a manually-configured setting living only in a CI tool's UI, invisible to code review and easy to silently drift out of sync with what the team actually intends.
7. Quality and Security Gates
Static analysis and linting
- run: dotnet format --verify-no-changes
- run: dotnet build -warnaserror
Catching style violations, common bug patterns, and compiler warnings automatically, as an early pipeline stage, keeps this class of feedback fast and consistent — not dependent on a human reviewer happening to notice the same issue during code review.
Code coverage thresholds
- run: dotnet test --collect:"XPlat Code Coverage"
- run: reportgenerator -reports:coverage.xml -targetdir:coveragereport
A coverage threshold gate (failing the pipeline if coverage drops below some agreed floor) is a reasonable guardrail against test coverage silently eroding over time — though it's worth treating as a floor to prevent regression, not a target to chase for its own sake; 100% coverage doesn't guarantee good tests, and a lower but well-targeted coverage number covering genuinely critical paths is often more valuable than a higher number achieved by testing trivial code.
Dependency vulnerability scanning
- run: dotnet list package --vulnerable
Scanning dependencies for known vulnerabilities (CVEs) as a pipeline stage — the same principle covered for container images in this series' Docker guide — catches a real, ongoing risk category that has nothing to do with whether your own code is correct; a perfectly correct application built on top of a vulnerable dependency is still a vulnerable application.
Container image scanning
As covered in this series' Docker guide, scanning the actual built container image (not just the application's direct dependencies, but the OS packages and everything else baked into the image) closes a gap that source-level dependency scanning alone doesn't cover.
Infrastructure change review
As covered in this series' Terraform/Bicep guide, running terraform plan/az deployment ... what-if and posting the diff for review in a pull request extends the same "review before it runs" discipline from application code to infrastructure changes flowing through the same pipeline.
8. Environment Promotion
The progressive-environments model
Feature branch → ephemeral PR environment (optional) → Dev → Staging/QA → Production
Each environment along this chain serves a distinct purpose: an ephemeral PR environment lets a reviewer or the PR author interact with the actual running change before merge; a shared dev/staging environment catches integration issues against realistic, shared infrastructure; and production is where real users are actually affected, warranting the most caution.
Configuration differences per environment, structure kept identical
appsettings.Production.json — different values, same schema as appsettings.Development.json
production-values.yaml — different Helm values, same chart as staging-values.yaml
production.bicepparam — different parameters, same Bicep template as staging
A recurring theme across this series (ASP.NET Core's configuration layering, Terraform/Bicep's per-environment parameter files, Helm's per-environment values files) — environments should differ in configuration values, not in the underlying structure of what's deployed. An environment that's structurally different from production (different Kubernetes manifests entirely, a fundamentally different Terraform configuration) undermines the entire purpose of testing in a lower environment: catching problems before they reach production only works if the lower environment is genuinely representative of it.
Not every change needs every environment
A trivial documentation or comment-only change arguably doesn't need a full staging deployment and manual QA cycle before merging — path-based trigger filtering (covered in this series' GitHub Actions guide) and risk-based judgment about which changes warrant the full pipeline vs. a lighter one are both reasonable, deliberate parts of pipeline design, not a compromise of rigor.
9. Observability of the Pipeline Itself
Pipeline metrics worth tracking
- Lead time — how long from a commit landing to it being live in production; a core measure of how much friction the pipeline itself adds to shipping a change.
- Deployment frequency — how often changes actually reach production; frequent, small deployments are generally lower-risk than infrequent, large ones (a smaller change is easier to reason about and easier to roll back or fix forward).
- Change failure rate — what percentage of deployments cause a production incident or require a rollback/hotfix.
- Mean time to recovery (MTTR) — how quickly the team can detect and resolve a production issue once one occurs.
These four ("DORA metrics," from the DevOps Research and Assessment program) are widely used as a shared vocabulary for how healthy a delivery pipeline actually is — not as a target to game, but as a useful lens for noticing whether pipeline changes are actually making delivery faster and safer, or just faster.
Alerting on pipeline health, not just application health
A pipeline that's been silently failing for a week (perhaps a scheduled nightly job nobody's watching) is its own kind of production incident — treating sustained pipeline failures with the same alerting rigor as application errors (rather than only noticing when someone happens to check) keeps the "automated safety net" actually functioning as one.
10. Common Anti-Patterns
| Anti-pattern | Why it hurts | Better approach |
|---|---|---|
| Rebuilding the artifact separately for each environment | No real guarantee staging and production run identical code | Build once, promote the same immutable artifact through every environment |
| Long-lived feature branches merged infrequently | Large, high-risk merges; defeats the purpose of continuous integration | Merge small, frequent changes to a shared branch, behind feature flags if needed |
| No fast feedback — slow tests run first, or everything runs in one giant serial stage | Wastes time and infrastructure cost discovering a simple bug the slow way | Order stages fast-to-slow; parallelize independent test suites |
| Manual, undocumented production deployment steps alongside the "automated" pipeline | The pipeline isn't actually the full story of how production gets updated, undermining trust in it | Every production-affecting step goes through the pipeline, no manual exceptions |
| Ignoring flaky tests (re-running until green) | Silently erodes trust in the pipeline's signal over time | Treat flaky tests as a priority bug; quarantine and fix, don't just re-run |
| No rollback plan tested until an actual incident | Discovering the rollback mechanism doesn't actually work, during an outage, is the worst possible time | Regularly exercise rollback (game days, or simply as part of normal low-stakes deployments) |
| Environments structurally different from production ("works in staging" doesn't mean much) | Lower environments fail to catch the exact class of problem they exist to catch | Keep structure identical across environments; vary only configuration values |
| Treating 100% test coverage as the goal | Encourages testing trivial code for the metric's sake, not genuine risk reduction | Use coverage as a regression floor; prioritize testing genuinely critical, complex, or historically bug-prone code |
11. Choosing and Combining Tools
This series has already covered two complete, capable CI/CD platforms in depth — GitHub Actions and Azure DevOps — and the principles in this guide apply almost identically regardless of which (or which combination) a team uses. A few points worth restating at this more general level:
- The platform matters less than the discipline. A team with excellent testing practices, build-once-deploy-many artifact discipline, and well-chosen deployment strategies will ship more safely on either platform than a team with poor practices on the "better" tool.
- Most real organizations' pipelines span multiple tools. A typical pipeline might use GitHub Actions or Azure Pipelines for orchestration, Docker for packaging (this series' Docker guide), Terraform/Bicep for infrastructure changes (reviewed via plan/what-if as part of the same pipeline), and Helm for deploying to Kubernetes (this series' Kubernetes/Helm guide) — the CI/CD platform is the conductor, not the entire orchestra.
- Start simple, add sophistication where the risk profile justifies it. A small internal tool with low traffic and forgiving users doesn't need canary deployments, feature-flag-gated rollouts, and a four-environment promotion chain — that sophistication earns its complexity for systems where the cost of a bad deployment is genuinely high, not as a default applied uniformly regardless of actual risk.
Quick Reference Table
| Concept | Purpose |
|---|---|
| Continuous Integration | Frequent merges, fast automated build/test feedback |
| Continuous Delivery | Every passing change is deployment-ready; human decides when |
| Continuous Deployment | Every passing change deploys to production automatically |
| Testing pyramid | Many fast unit tests, fewer integration tests, fewest E2E tests |
| Build once, deploy many | The exact same artifact flows through every environment |
| Rolling / blue-green / canary | Deployment strategies trading infrastructure cost for blast-radius control and rollback speed |
| Feature flags | Decouples code deployment from feature release |
| DORA metrics | Lead time, deployment frequency, change failure rate, MTTR |
| Fail fast | Cheapest/fastest checks run first in the pipeline |
| Pipelines as code | Version-controlled, reviewable pipeline definitions |
Conclusion
CI/CD is ultimately about trust: trusting that a green pipeline genuinely means the change is safe to ship, and trusting the pipeline enough to let it actually run the show rather than falling back on manual, undocumented steps "just this once." That trust is built from a consistent set of underlying disciplines — fast, layered testing that fails cheaply and quickly; a single immutable artifact promoted unchanged through every environment; deployment strategies matched deliberately to a system's actual risk tolerance; and quality/security gates that catch entire categories of problems automatically rather than depending on a reviewer's memory.
Every tool-specific guide in this series — GitHub Actions, Azure DevOps, Docker, Kubernetes/Helm, Terraform/Bicep, Database Migrations — is, in a real sense, one piece of this larger picture: the mechanics of how to build, package, provision, and deploy. This guide is the why and the what shape it should take underneath all of them — the principles that make the difference between a pipeline that technically runs and one a team can actually depend on when it matters most.
Found this useful? Feel free to star the repo, open an issue with corrections, or share the deployment strategy that turned your team's scariest release day into a non-event.
Top comments (0)