A deep dive into the security scanning pipeline that runs on every pull request before a single line of code reaches staging or production
In my last article, I described how I designed a smart CI pipeline trigger for a .NET monorepo containing 11 APIs. The core idea was to make the pipeline aware of what actually changed before deciding how much of the test suite to run. That article focused on efficiency. This one focuses on security.
Here, I will walk through what actually happens once the pipeline is triggered: how the application is built, how it is attacked, how its dependencies are interrogated, and how its code quality is evaluated before a single line of code is allowed to reach the staging or production environment.
The pipeline is designed to be PCI-DSS compliant, which means security is not an afterthought bolted onto the end of the process. It is embedded into every stage.
The Pipeline Trigger
The CI pipeline is triggered by a pull request to either the staging or main branch. This is a deliberate design decision. The goal is to ensure that before any merge to these critical branches, the incoming commit passes the full suite of security and quality checks. At no point should a vulnerability or a failing test be allowed to land on a protected branch undetected. The main and staging branches are kept clean at all times.
For this article, I will focus on the Layer 2 scenario from the smart detection step: a change is detected in a shared dependency directory such as Core or Persistence. This is the most exhaustive scenario because it triggers the pipeline to build and test the entire suite of 11 APIs.
Stage 1: Building the Application
Once the detect-changes job determines that the full suite needs to run, the pipeline proceeds to the build-app job. This job depends on detect-changes completing successfully before it can start.
In this stage, all 11 APIs are built in parallel using the GitHub Actions matrix strategy, along with the Core and Persistence projects. The build process follows four steps for each application: it restores all NuGet dependencies, compiles and builds the application, publishes the build output, and finally uploads the compiled artifact to GitHub Actions artifact storage. Each artifact is tagged with the name of the application it belongs to, which serves as the identifier the matrix uses to route subsequent jobs to the correct artifact.
The reason for uploading build artifacts at this stage rather than rebuilding at each subsequent step is significant. Without this, every security scan job would need to compile the entire application from scratch before it could run its tests. Across 11 APIs and multiple scan types, that would add a considerable amount of redundant build time to every pipeline run. By building once and sharing the artifact, all downstream jobs can skip straight to their core function.
If any one of the 11 applications fails to build, the entire pipeline fails immediately. There is no point running security scans on code that does not compile.
Stage 2: Dynamic Application Security Testing (DAST) with OWASP ZAP
Once the build artifacts are available, the DAST job runs. This job depends on both detect-changes and build-app completing successfully.
DAST is a category of security testing that evaluates a running application from the outside, simulating the behaviour of a real attacker. Unlike static analysis, which reads source code, DAST probes the application while it is actually executing and looks for vulnerabilities that only manifest at runtime. The types of issues it targets include SQL Injection, Cross-Site Scripting (XSS), broken authentication flows, security misconfigurations, and sensitive data exposure.
For this pipeline, I use OWASP ZAP (Zed Attack Proxy) to carry out the DAST scan. ZAP is an open source tool maintained by the Open Worldwide Application Security Project and is widely used in both manual penetration testing and automated security pipelines.
The job works as follows. It first downloads the build artifact for each API from the matrix. It extracts the compiled .dll file and starts the application directly within the GitHub Actions runner VM. A separate step then verifies that the application is actually accessible and responding before ZAP begins its scan, because running a scan against an unreachable application would produce meaningless results.
The scan that runs within the pipeline is a ZAP baseline scan rather than a full active scan. The reason for this is time. A full active scan on 11 APIs within a CI pipeline would be prohibitively slow and would block engineers from merging for an unacceptable amount of time. The baseline scan is fast, catches the most critical runtime vulnerabilities, and is appropriate for a PR gate.
The full deep scan is handled separately by a scheduled cron job that runs routinely against the codebase outside of the normal PR flow. This deep scan generates a detailed report and triggers an alert if any application fails.
After each baseline scan, ZAP produces a report that is uploaded as an artifact to GitHub Actions for review. The pipeline evaluates the results using ZAP's risk code system. A risk code of 2 (medium severity) or 3 (high severity) causes the pipeline to return an exit code of 1, which fails the job. Low severity findings are logged but do not block the merge.
Stage 3: Software Composition Analysis (SCA) with Trivy
The SCA job also depends on both detect-changes and build-app. It runs concurrently with the DAST job rather than waiting for it to finish.
Software Composition Analysis is the practice of examining an application's third-party dependencies for known vulnerabilities. Modern applications rarely consist entirely of first-party code. They rely on dozens, sometimes hundreds, of external packages and libraries. Each of those packages is a potential attack surface. A vulnerability in a single dependency can compromise the security of the entire application that uses it.
Developers working day to day on feature delivery cannot reasonably be expected to track every CVE published against every package in their dependency tree. SCA automates this responsibility and moves it into the pipeline where it runs on every commit.
For this pipeline, I use Trivy, an open source vulnerability scanner developed by Aqua Security. Trivy scans the project's dependency manifest and checks every package against multiple vulnerability databases simultaneously, including the National Vulnerability Database (NVD), the GitHub Advisory Database, the OSS Index, and the broader CVE database ecosystem.
The job downloads the build artifact produced by build-app, runs Trivy against it, and evaluates the results. Any dependency with a vulnerability rated High or Critical severity causes the pipeline to fail. Medium and low severity findings are surfaced in the scan output but do not block the merge on their own, giving the engineering team visibility without creating unnecessary friction for lower-risk issues.
Stage 4: Static Application Security Testing (SAST) with SonarQube
The final security stage is the SAST scan, which runs using SonarQube. This job is slightly different in its design compared to the DAST and SCA jobs because SonarQube does not consume the pre-built artifact from the build-app stage.
The reason for this is architectural. SonarQube performs its analysis by instrumenting the build process itself. It needs to observe the compilation as it happens in order to collect the full range of metrics it analyses. It cannot do this by inspecting a finished artifact after the fact. So for SonarQube, the application is built again as part of this job, and SonarQube hooks into that build to perform its analysis.
SAST tests the application at rest, meaning it analyses the source code and compiled output without running the application. This is complementary to DAST rather than a replacement for it. Where DAST catches vulnerabilities that appear at runtime, SAST catches issues that are visible in the code itself before the application is ever started.
SonarQube tests for a broad range of concerns: bugs, technical debt, security hotspots and exposed secrets, code smells, code coverage, code duplication, reliability ratings, security ratings, and maintainability ratings.
A key part of the SonarQube setup is the quality gate. Before the pipeline was deployed, the DevOps team and the software engineering team agreed on a set of coding standards and quality thresholds. These standards were encoded as a quality gate within SonarQube. Any commit that falls short of this agreed standard causes the pipeline to fail.
The SAST scan runs across all 11 APIs. Every developer on the team has access to the SonarQube dashboard, where they can see exactly which checks their commit failed and what aspect of the quality gate was not met. This creates a feedback loop that goes beyond simply failing a pipeline: it tells the developer precisely what needs to be fixed and why.
Failure Handling: Why fail-fast Is Set to False
One important design decision in this pipeline is that fail-fast is set to false for all jobs that run concurrently, specifically the DAST, SCA, and SAST stages.
By default, GitHub Actions matrix jobs will cancel all in-progress jobs the moment one of them fails. This is efficient in terms of minutes but produces incomplete information. If the DAST scan fails on API number 3, you want to know whether APIs 4 through 11 would also have failed, or whether the issue is isolated. Cancelling everything at the first failure obscures that picture.
With fail-fast set to false, all jobs run to completion even if one fails. The pipeline still fails at the end if any job returned a failure, but the team receives the full diagnostic picture from every scan across every API. This makes root cause analysis significantly faster and reduces the number of subsequent pipeline runs needed to surface all issues.
Monitoring and Alerting: Slack Integration
Every job in the pipeline is connected to a Slack notification step. When any job fails, a notification is sent to the team's Slack channel immediately. The notification identifies which job failed and which repository it belongs to, so the responsible team can investigate without needing to manually check the GitHub Actions dashboard.
A successful pipeline run also sends a notification, which serves as a positive signal that the commit is clean and the merge can proceed. This keeps the whole team informed of pipeline health without requiring anyone to actively monitor it.
The CD Pipeline: Security as a Deployment Gateway
The Continuous Deployment pipeline is triggered by a push to either the main or staging branch, which occurs after a pull request is merged.
Before any deployment begins, the full CI pipeline is retriggered. This acts as a final gateway. Even though the code passed all checks during the PR, this second run ensures that nothing has changed in the environment or the dependencies between the PR being raised and the actual merge. Only after the CI pipeline passes in full does the deployment proceed.
Both successful deployments and failed deployments trigger a Slack notification. In the event of a failed deployment, a rollback is initiated and that too triggers a notification. Every deployment event is logged through Slack for audit purposes, which is a requirement under PCI-DSS compliance.
A Note on Pipeline Maintainability
This is a long pipeline. The full script currently exceeds 500 lines of YAML. At that length it becomes difficult to read, review, and maintain as a single file.
To address this, my team is in the process of refactoring the pipeline to split each job into its own separate reusable workflow file. The build job, DAST job, SCA job, and SAST job will each live in their own file and be referenced from the main pipeline file. This makes the overall structure significantly easier to navigate and allows individual jobs to be updated, tested, and reviewed in isolation without touching the rest of the pipeline.
Summary: The Full Pipeline Flow
To bring it all together, here is how a pull request to staging or main moves through the pipeline from start to finish.
The detect-changes job runs first and determines the scope of the test suite based on what changed. The build-app job runs next, compiling all affected APIs and uploading their artifacts. The DAST, SCA, and SAST jobs then run concurrently against those artifacts, each applying a different category of security and quality analysis. All three must pass before the pipeline is considered successful. If any one of them fails, the full diagnostic output from all three is preserved and a Slack notification is sent to the team. On a merge, the CI pipeline runs again as a final deployment gate before the CD pipeline takes over and pushes to the environment.
Security is not a checkpoint at the end of this process. It is the process.
Top comments (0)