This article was originally published on AI Study Room. For the full version with working code examples and related articles, visit the original post.

CI/CD Observability: Build Metrics, Test Analytics, Deployment Tracking, and DORA Metrics

Introduction

CI/CD pipelines are critical infrastructure, yet they often lack the observability applied to production systems. Without pipeline observability, teams cannot measure deployment frequency, identify build bottlenecks, track flaky tests, or correlate deployments with incidents. CI/CD observability applies monitoring, analytics, and alerting principles to the software delivery process itself.

This article covers build metrics collection, test analytics, deployment tracking, DORA metrics, and tooling recommendations.

Build Metrics Collection

Build pipelines generate rich telemetry data: duration, resource utilization (CPU, memory, disk), cache hit rates, dependency download times, and stage-level timing. Collecting and analyzing these metrics identifies optimization opportunities.

Key build metrics include:

• Pipeline duration (total and per-stage).
• Queue time (time waiting for runner availability).
• Cache restore and save times.
• Dependency resolution and download times.
• Artifact upload and download times.

• Success rate and failure distribution by stage.

  GitLab CI with metrics collection

  build:

  script:

  - ./build.sh
  

  after_script:

  - curl -X POST https://metrics.internal/api/v1/build \
  
      -H "Content-Type: application/json" \
  
      -d '{"duration": "'$CI_JOB_DURATION'", "status": "'$CI_JOB_STATUS'"}'
  

Build metrics should be stored in a time-series database (Prometheus, InfluxDB) and visualized in dashboards. Historical trends reveal performance degradation from incremental changes — such as growing dependency trees, larger artifacts, or slower test suites.

Test Analytics

Test analytics provides visibility into test suite health: pass/fail rates, execution times, flakiness, and coverage trends. The goal is maintaining fast, reliable test suites that provide rapid feedback.

Flaky tests — tests that pass and fail without code changes — erode trust in the test suite. Analytics identify flaky tests by tracking test results across multiple runs on the same commit. A test that passes and fails on the same SHA is flaky.

# Flaky test detection algorithm

def is_flaky(test_results):

    results_per_commit = group_by(test_results, "commit_sha")

    for commit, results in results_per_commit.items():

        statuses = set(r.status for r in results)

        if "passed" in statuses and "failed" in statuses:

            return True

    return False

Test duration tracking identifies slow tests that dominate pipeline time. The Pareto principle applies — 20% of tests often account for 80% of execution time. Identifying and optimizing these slow tests directly improves pipeline speed.

Test coverage trends reveal degradation over time. Coverage thresholds in CI pipelines prevent merging code that reduces coverage below the team's standard.

Deployment Tracking

Deployment tracking correlates releases with production behavior. Every deployment should be recorded with metadata: commit SHA, image tag, configuration changes, deployer identity, deployment time, and promotion path (dev to staging to production).

# Deployment event schema

deployment:

  service: api-gateway

  version: v2.14.3

  commit: a1b2c3d4e5

  environment: production

  timestamp: 2026-05-12T10:30:00Z

  deployer: github-actions

  duration: 145s

  rollout_strategy: canary

Deployment markers enable powerful analysis. Superimposing deployment events on monitoring dashboards reveals which changes caused performance shifts, error spikes, or traffic changes. Automated rollback detection flags deployments followed by increased error rates within a configurable window.

DORA Metrics

The DORA (DevOps Research and Assessment) metrics are the industry standard for measuring software delivery performance:

Deployment Frequency: How often an organization deploys to production. Elite performers deploy on demand (multiple times per day), while low performers deploy once per month or less.

Lead Time for Changes: The time from commit to production. Elite performers achieve less than one hour. Low performers take weeks.

Change Failure Rate: The percentage of deployments causing a failure in production. Elite performers have under 5% failure rate. Low performers exceed 45%.

Time to Restore Service: The time from incident detection to recovery (MTTR). Elite performers restore in under one hour.

-- Query for deployment frequency

SELECT date_trunc('day', deployed_at) AS day,

       COUNT(*) AS deployments

FROM deployments

WHERE deployed_at > NOW() - INTERVAL '30 days'

GROUP BY day

ORDER BY day;

Implementing DORA metrics requires instrumenting CI/CD pipelines to emit deployment events, monitoring tools to track incidents and recovery times, and dashboarding to vi

---

Read the full article on AI Study Room for complete code examples, comparison tables, and related resources.

Found this useful? Check out more developer guides and tool comparisons on AI Study Room.