When your primary developers are AI agents operating on background loops, you cannot rely on trust. You need a paranoid, uncompromising, automated guillotine.My Jenkinsfile for Plënka has grown into 43 blocking quality gates. If an agent hallucinates, ignores an ADR, leaks memory, or attempts to comment out a failing test to pass CI, this pipeline crushes the build, collects the diagnostic logs, and sends the agent back to fix its mess.Here is the exact blueprint of all 43 gates.
Group 1: Preparation (2 Gates)
- Checkout: Workspace cleanup, repository clone, git submodules initialization, and report directory creation.
- Secrets Scan (BLOCKING): Three-tier secret detection using an inline scripts/secret-scan.sh, gitleaks with SARIF reporting/redaction, and detect-secrets with a baseline. A single leaked key immediately fails the build.
Group 2: Read-Only Rules & Schema Validators (9 Parallel Gates)
- Validate Schemas: Compiles all events/v1/*.json event schemas using ajv (draft2020).
- Validate Glossary: Verifies that both code and documentation strictly adhere to terms defined in docs/GLOSSARY.md (and blocks anti-glossary terms).
- Validate Features: Validates features/// directory structures, Gherkin syntax linting, and BDD coverage anchor alignment.
- ADR Compliance: Ensures every feature.md points to a valid Architecture Decision Record (ADR), and every ADR contains Quality Drivers and Compliance sections.
- No-Skip Check: Absolute ban on GTEST_SKIP(), DISABLED_*, or commented-out tests without explicit architectural justification (ADR-022 §A3). No sweeping bugs under the rug.
- Naming Lint: Enforces test naming convention matching TEST(_...).
- Validate Structure: Confirms the repository tree strictly matches the STRUCTURE.md manifest.
- RTM Uniqueness: Enforces single-use Test Case IDs across acceptance.md to prevent duplicate requirements.11. Validate Image Pins: Requires all Docker base images to be pinned by immutable SHA256 hashes (@sha256:...).
Group 3: Static Analysis & Code Quality (7 Parallel Gates)
- shellcheck: Static analysis for all Bash scripts in scripts/, tools/, and ci/.
- hadolint: Dockerfile linting configured at warning severity.
- yamllint: YAML validation against .yamllint.
- markdownlint: Documentation and ADR markdown linting.
- clang-format / clang-tidy: C++23 code formatting and static analysis integrated with vcpkg compilation databases.
- cppcheck (BLOCKING): Executed with --error-exitcode=1 covering warnings, style, performance, and portability. No legacy grandfathering allowed—tech debt must be fixed or explicitly suppressed.
- ESLint: Frontend linting where all warnings are treated as hard errors.
Group 4: Security Scanners (2 Gates)
- SCA (Trivy - BLOCKING): Filesystem dependency vulnerability scanning. Any HIGH or CRITICAL CVE fails the build and outputs SARIF reports.
- SAST (Semgrep - BLOCKING): Runs OWASP Top 10, CWE Top 25, and custom ci/semgrep.yml rulesets with --error enforcement.
Group 5: Build, Test & Sanitizer Matrix (4 Gates)
- Build Frontend: Production Vite + Preact + TailwindCSS compilation.
- Frontend Tests: Vitest unit tests and Playwright E2E suites running against a live backend (no API mocking).
- DB Reset & Migrate: Fresh Postgres + Redis spin-up with database migrations applied in a single transactional block (ON_ERROR_STOP=1 --single-transaction).
- Build & Test Matrix (4 Parallel Branches):Coverage: Unit, property, and integration tests (Enforced thresholds: Line $\ge 80\%$, Branch $\ge 70\%$).ASan: AddressSanitizer + LeakSanitizer for memory corruption and leak detection.UBSan: UndefinedBehaviorSanitizer targeting undefined C++ behaviors.TSan: ThreadSanitizer for concurrency and data-race detection under load.Catch2 Unit Tests: Isolated core logic tests (Money, TraceId, Result types).
Group 6: Architecture & Defense-in-Depth (6 Gates)
- Archcheck: Detects circular dependencies, god-headers, and god-classes (strict threshold: 150 lines per class).
- Binary Hardening (checksec): Verifies compiled binaries enforce PIE, RELRO, Stack Canaries, NX bit, and Fortify Source.
- SBOM & SCA (syft + grype): Generates Software Bill of Materials (SBOM) and performs vulnerability scanning directly against the SBOM.
- Secret Verification (Trufflehog): Scans for live credentials, validating active API keys against vendor endpoints.
- License Audit: Dependency license verification via trivy --license-full.
- Container Image Scan: Vulnerability scanning of fully assembled production container images.
Group 7: Fuzzing, Chaos & Load (4 Gates)
- Fuzz (libFuzzer): In-process fuzzing of core C++ parsing and business logic functions (20-minute run).
- Fuzz (AFL++): External fork-server fuzzing against protocol endpoints (20-minute run).
- Chaos Engineering (Toxiproxy): Injects latency, packet loss, and connection dropouts to test backend resilience.
- Load Test (k6): Automated performance benchmarks enforced against SLAs in ci/k6-thresholds.json.
Group 8: Supply Chain Integrity (3 Gates)
- Artifact Signing (cosign): Container image signing via Sigstore.
- Provenance (in-toto): Attestation generation for SLSA supply-chain compliance.
- Reproducible Builds: Verifies bit-identical output across isolated compilation passes.
Group 9: Infrastructure & Cloud Scanners (3 Gates)
- Snyk (SCA + SAST): Advanced vulnerability intelligence scanning (Master/Release branches).
- IaC Scan (Checkov): Misconfiguration analysis for Infrastructure-as-Code (Terraform, Dockerfiles).
- Terraform Lint (TFLint): Static analysis for Terraform infrastructure configs.
Group 10: Final Verification & Quality Gate (3 Gates)
- Traffic Replay (GoReplay): Replays recorded production HTTP traffic against the local backend to catch edge-case regressions missed by unit tests.
- Native Quality Gate: Custom aggregator replacing SonarQube (per ADR-041). Consolidates findings from cppcheck, clang-tidy, jscpd (code duplication threshold $<3\%$), semgrep, trivy, grype, and checkov. Hard rule: NEW_ISSUES <= 1.
- DAST Baseline (OWASP ZAP): Staging environment vulnerability scan targeting XSS, CSRF, SQL Injection, and Open Redirects (configured via ci/zap-baseline.conf).
How do you handle AI-generated code quality in your own projects? Do you let agents push directly, or do you run them through a similar meat grinder? Let me know your thoughts, feedback, or constructive criticism!
Top comments (0)