Top 6 Change Failure Rate Tools

Quick answer: GitDailies is the best tool for GitHub teams who want to start measuring change failure rate without a lengthy rollout. It uses GitHub Actions or webhooks for deployment data and accepts incidents through webhooks, so you can get started without changing your incident management workflow. If you need deeper deployment analytics, Sleuth stands out with native PagerDuty integration, rollback detection, and per-deployment attribution. LinearB and Swarmia bundle the metric into larger platforms, while Middleware and Apache DevLake offer open-source alternatives.

A release goes out at 4:40 p.m. on a Friday. It includes a configuration change and a version bump. Both have been reviewed, and every check in the pipeline goes green. At 5:55 p.m., the on-call engineer gets an alert. A downstream service is still looking for a field that the configuration change renamed. The team quickly rolls the release back, and by 6:30 p.m. everything is working again. Everyone heads home. On Monday, nobody logs the issue. It lasted only 35 minutes, the rollback fixed it, and there's a common feeling that if a problem is resolved before the end of the day, it wasn't really an incident.

Three weeks later, someone asks for the team's change failure rate. Nobody knows, because half the failed changes were never recorded anywhere for a reporting tool to find them. The dashboard shows a 4% change failure rate, but that number doesn't reflect reality. Meanwhile, the postmortem template has listed "we deployed on a Friday" as the root cause for three consecutive Friday incidents, even though the day of the week wasn't the real problem.

Change failure rate is the DORA metric that depends less on your deployment pipeline than on consistently recording what went wrong. While Top 6 Deployment Frequency Tools focuses on what was deployed, this article looks at what had to be rolled back or fixed afterward. If you're comparing all four DORA metrics, Top 9 DORA Metrics Tools ties them together.

The focus here is simple: which tools can reliably identify a failed deployment, and how much additional infrastructure do you need before they can?

We compare six options, starting with GitDailies, the tool that requires the least additional setup.

GitDailies

It's worth acknowledging this upfront: Sleuth offers deeper change failure rate analysis than GitDailies. It has native PagerDuty integration, rollback detection, and can attribute incidents to individual deployments. If those capabilities are your top priority, it's a strong choice, and the next section explains why.

GitDailies makes a different case, and it is the case that decides most purchases. It's aimed at GitHub teams that want to start tracking change failure rate quickly, using the delivery process they already have, without adopting a dedicated incident management platform first.

The mechanics are simple.

• GitDailies captures deployment events directly from GitHub Actions by monitoring your deployment workflow, or from external CI/CD platforms (such as Jenkins, CircleCI, Octopus Deploy, and Travis CI) through an incoming webhook.
• Incident events are handled the same way: a webhook posts an "Incident started" event, followed later by an "Incident resolved" event. Those webhooks can come from tools such as PagerDuty, Datadog, or Squadcast.

GitDailies then computes Change Failure Rate as what percentage of deployments experienced one or more incidents. The same deployment and incident data is also used to calculate Time to Restore Service (TRS).

You don't have to define an environment model or map out your service topology before you can start. The GitHub App is read-only, accessing metadata rather than source code. Results can be grouped by repository or time period and exported as CSV files. Full DORA metrics are available on the Pro plan, which starts at $49 per month for up to 250 pull requests. Metering is per pull request rather than per seat, and users are unlimited on every tier, including the free one.

Honest take: The best metrics are the ones you can start collecting right away. GitDailies uses the deployment pipeline you already have and incident events from a simple webhook to calculate Change Failure Rate and Time to Restore Service. There's no need to roll out another platform first. Unlimited users, no seat floor, metered on the pull requests you ship. For a team whose delivery lives on GitHub and whose incident process lives in a Slack channel and somebody's head, this is the change failure rate tool that fits your model.

Sleuth

Sleuth is the most capable tool in this list when it comes to measuring deployment failures, and pretending otherwise would be a waste of time.

Its integration with PagerDuty is native and deeply built into the product. PagerDuty is an "impact source" because:

• You choose which services and urgency levels count as failures
• Sleuth imports up to 30 days of incident history
• It recalculates Change Failure Rate and Mean Time to Restore (MTTR) from that data

That means you have reliable metrics on day one, based on incidents that have already happened, instead of waiting weeks for enough new data to accumulate.

The underlying model is also more detailed than most tools offer. Sleuth calculates change failure rate by dividing failed deployments with the total number of deployments in a given period. It classifies failures into four types: Incidents, Rolled Back, Unhealthy, and Ailing. "Rolled back" means a deployment Sleuth detected had been rolled back, such as a Friday evening deployment that is quickly reverted and forgotten by Monday.

The costs are real. To get accurate metrics, you need to define your deployments, and environments upfront. Sleuth DORA offers a free tier of three team members, one project, and two environments. Paid plans start with Standard at $38 per user per month when billed monthly, or $35 per user per month when billed annually, with support for unlimited projects and up to 50 contributors. Enterprise pricing is available through sales.

It's also worth noting that Sleuth now has two product lines. Its homepage focuses primarily on AI agent governance, while DORA is the second line — live, still sold, no deprecation notice, but no longer the company’s main message. For teams with mature delivery pipelines, multiple environments, and PagerDuty already in place, Sleuth provides some of the richest change failure rate data available.

LinearB

LinearB treats change failure rate as part of a broader engineering management platform rather than a standalone metric. It doesn't just report what happened. Its WorkerB automation can step in to address delivery issues, making it the only tool on this list that aims to improve delivery workflows. This is a genuinely different posture: most tools here hand you a percentage and trust you to hold a meeting about it.

Another strength is the context it wraps around the numbers. DORA and SPACE-aligned indicators are benchmarked against a vendor dataset of more than 8.1 million pull requests from over 4,800 organizations. That makes a change failure rate of 9% much easier to interpret because you can see how it compares with similar engineering teams instead of looking at the number in isolation. Teams arguing about whether their number is bad usually want that comparison more than they want another dashboard.

Pricing is where the decision often gets made. The Essentials plan costs $29 per user per month, billed annually, but requires a minimum of 30 billable users. 30 x 29 x 12 = $10,440 a year, whatever the team's actual headcount. The Enterprise plan is $59 per user per month with a 50-user minimum. There's no free tier, only a 45-day trial, and the Essentials plan supports GitHub Cloud only.

For organizations with 30 or more developers, and with the budget for a full software delivery platform, LinearB is a strong purchase and WorkerB's automation can provide real value. Below 30, though, the minimum seat requirement is often the deciding factor long before the feature list comes into play.

Swarmia

Swarmia integrates directly with PagerDuty and Datadog, bringing incident data together with Git, Jira or Linear, and Slack in a single engineering platform. That native integration is a real advantage over webhook-based setups because the tools your team already relies on become first-class data sources instead of simply sending events to an endpoint.

What sets Swarmia apart is what happens after the metric is calculated. Its ‘Working Agreements’ let teams turn a metric into a shared commitment. That could mean agreeing to keep deployments small, avoiding late-Friday releases, or making sure someone stays available after every production deployment. Swarmia tracks whether those agreements are being followed instead of only showing the resulting change failure rate. It's a practical way to address a common problem with DORA metrics: teams review the numbers every month but never change the habits behind them.

Because Swarmia combines Git, Jira or Linear, Slack, and incident management tools in one place, change failure rate becomes part of a broader view of software delivery instead of another isolated dashboard.

Swarmia is free for companies with fewer than ten software developers, and priced per developer per month above that. For teams that already manage incidents with PagerDuty or Datadog and want their DORA metrics to drive accountability, Swarmia is one of the strongest options in this list.

Middleware

Middleware stands out as the open-source engineering-intelligence option on this list. It's released under the Apache 2.0 license, is actively maintained, and isn't a source-available product marketed as open source. The self-hosted Community Edition is free and includes DORA metrics, including change failure rate.

It's appealing for more than just the price.

• Middleware benchmarks your DORA metrics against industry standards, giving you context instead of just a number.
• Because you can self-host it, your deployment and incident data stays within your own infrastructure. For organizations with strict security or compliance requirements, that's a necessity rather than a nice-to-have.
• Middleware also offers SOC 2 Type 2 compliance and an on-premises deployment option for customers that need those assurances.

If you prefer a managed service, the cloud tier starts at $39 per user per month, billed annually. That's more expensive than some lightweight alternatives, and because pricing is based on users rather than pull requests, costs increase as your engineering team grows. The self-hosted tier is free and gets low-priority support, which is a fair tradeoff if your team is comfortable managing the infrastructure itself.

For organizations with platform engineering expertise and clear data residency requirements, Middleware provides a practical way to track DORA metrics without handing data over to a third-party service.

Apache DevLake

Apache DevLake (incubating) is free, Apache-licensed, and self-hosted. More importantly, it lets you decide what counts as a failed deployment. Depending on your team, that flexibility is either its biggest advantage or its biggest challenge.

The appeal is control.

• DevLake pulls data from far more than just GitHub
• It includes prebuilt Grafana dashboards for DORA metrics
• It supports custom SQL for any metric you can define as a query

If your team has a clear definition of a failed change (for example, counting a Friday rollback but ignoring an alert triggered by a healthy canary deployment), DevLake lets you encode those rules instead of relying on a vendor's default logic. There are also no per-user or per-repository licensing limits, so it costs the same across four repositories or four hundred.

The tradeoff is the operational overhead. While the software is free, running it isn't. You'll need to deploy and maintain Docker Compose or Kubernetes, a MySQL database, and Grafana yourself, and there isn't a managed cloud service to fall back on. The project is also still in incubation at the Apache Software Foundation. Another limitation is that DevLake doesn't provide native Slack digest reports, so change failure rate lives in a dashboard that someone has to remember to check instead of arriving automatically in the team's daily workflow.

For organizations that already operate this kind of infrastructure and want complete control over how DORA metrics are defined and calculated, Apache DevLake is one of the most flexible options available.

FAQs

As an SRE, what counts as a failed change?

The answer depends on how you define it, and different tools define it differently.

• Sleuth counts a deployment as failed if it falls within a change failure window and categorizes the outcome as Incident, Rolled Back, Unhealthy, or Ailing.
• GitDailies counts a deployment as failed when one or more incidents are linked to it through a webhook.
• Apache DevLake lets you define the criteria yourself through custom queries.

Apply those three definitions to the same week's deployments, and you'll likely get three different change failure rates. Pick one definition, write it down, and apply it consistently. And stop comparing your number to another team's, because their number is measuring something else.

As an EM, is a change failure rate of zero a good thing?

No. It means one of two things, and neither means your software is perfect.

• The first is that you are shipping infrequently. A team deploying once a quarter has ample time to catch problems before production, and has traded speed for a flattering statistic.
• The second, and much more common, is that failures simply aren't being recorded. The 35-minute revert on a Friday evening never became a ticket, so the tool never saw it, and the dashboard says zero because nobody told it otherwise.

A rate that never changes is not a signal of health. It is a signal that the measurement is broken.

As a CTO, do I need an incident tool before I can measure change failure rate?

No. In fact, the answer to that question often determines which tool makes the most sense.

• If you already run PagerDuty, Sleuth ingests its incidents natively and recalculates Change Failure Rate and Mean Time to Restore (MTTR) based on that data. Swarmia connects to PagerDuty and Datadog directly. The result is less manual work and more consistent reporting.
• If you don't already have an incident platform, there's no need to purchase one just to start tracking a DORA metric. GitDailies accepts incident events from an incoming webhook — "Incident started", "Incident resolved". Whether those events come from PagerDuty, Datadog, Squadcast, a custom script, or even a simple internal tool, GitDailies uses them to calculate Change Failure Rate and Time to Restore Service (TRS).

Start measuring with the tools you already have. If you outgrow that setup, add a dedicated incident management platform accordingly.

Which tool should you choose?

The right choice depends on how your team works today.

• If you already have an incident management platform and a well-defined, multi-environment deployment pipeline, Sleuth offers the deepest change failure rate analysis of the tools in this list.
• If you have 30 or more engineers and a budget for automation, LinearB is a strong option. It doesn't just measure delivery performance; it helps teams improve it through workflow automation.
• If you want to own the metric definitions and the infrastructure, Middleware and Apache DevLake are the open-source routes.
• If your team already works out of GitHub, has a deployment workflow, and handles most incidents through Slack, you can instantly start measuring change failure rate. Install GitDailies on your repositories, connect it to your deployment workflow, and send a webhook when an incident starts and another when it's resolved. By the end of the month, you'll have a change failure rate based on your team's actual deployments and incidents. The full set of DORA metrics is available on the Pro plan for $49 per month, so budget for it rather than for a platform.