Weibull, Crow-AMSAA & Duane for Reliability Growth

#testing #platform

When to use Weibull, Crow-AMSAA and Duane in your program
How to perform Weibull analysis to separate and fix failure modes
How to build Crow-AMSAA and Duane curves for growth tracking
How to interpret MTBF, make forecasts, and calculate confidence intervals
Practical Application: checklists, protocols and code for implementation

Reliability growth lives or dies on the numbers: findable, attributable, and statistically defensible. Use per-failure-mode weibull analysis to expose the mechanism; use a system-level crow-amsaa (power-law NHPP) or the empirical duane model to prove MTBF growth and to make forecasts with quantified uncertainty.

The Challenge: Programs confuse levels of analysis and lose control of reliability budgets. Tests produce time-stamped failures but teams treat every failure as the same kind of data: some failures are one‑shot lifetime events, others are repairable recurrence events; the lab hands over aggregated MTBFs to the program office and the program manager demands a projection with 90% confidence — but the model used is wrong or assumptions are unstated. The consequence: wasted test hours, missed FRACAS closures, unrealistic contractual claims, and a growth curve that looks pretty on paper but cannot be defended under audit.

When to use Weibull, Crow-AMSAA and Duane in your program

Pick the model that answers the question you actually have — not the one that feels familiar.

Use Weibull analysis when you have time‑to‑failure for a component or failure mode where a single failure removes the article from the tested sample (non‑repairable data) or where you want to characterize life distribution by mode. The Weibull shape (β) separates infant mortality (β<1), random failures (β≈1), and wear‑out (β>1), and the scale (η) gives characteristic life; parameter estimation, MTTF and confidence bounds come from standard life‑data methods.
Use Crow‑AMSAA (power‑law / NHPP) to track reliability growth for repairable systems undergoing test‑analyze‑fix cycles. Model the failure process as a Non‑Homogeneous Poisson Process with cumulative intensity Λ(t)=λ t^β and instantaneous intensity ρ(t)=λ β t^{β-1}; the parameters track whether failure intensity is falling (β<1) or rising (β>1). This is the defense/aerospace workhorse for growth planning and projection.
Use Duane for quick, empirical trend checks in early test phases. Plot the Duane relation (log cumulative MTBF vs log cumulative test time) to eyeball a learning slope and compare against baseline expectations — but treat Duane as exploratory/graphical, not a substitute for NHPP MLE when you need formal confidence intervals or to handle censoring.

Model	Best fit question	Data required	Assumptions	Key outputs
Weibull analysis	What is the lifetime distribution of a failure mode?	Time‑to‑failure (censoring allowed)	Independent failure times, per‑mode homogeneity	`β`, `η`, `MTTF = η Γ(1+1/β)`, hazard `h(t)`
Crow‑AMSAA (PLP / NHPP)	Is system failure intensity decreasing with fixes? How many failures next phase?	Time‑stamped repairable events (can be multiple per unit)	Minimal repair model, NHPP / power‑law intensity	`β`, `λ`, `Λ(t)`, predicted failures `Λ(t2)-Λ(t1)`
Duane plot	Is there a visible learning slope?	Cumulative MTBF vs cumulative time	Empirical smoothing of cumulative averages	Duane slope (graphical), fast diagnostics

Important: Treat Weibull as a per‑mode diagnostic tool and Crow‑AMSAA as a system‑level growth model. Conflating them (e.g., plugging Weibull MTTFs into a Crow projection without careful aggregation) is a common source of false confidence.

How to perform Weibull analysis to separate and fix failure modes

A practical, defensible weibull analysis protocol that fits defense programs.

Data discipline first
- Record time_on_test or usage metric, event_flag (failure vs right‑censor), FRACAS id, assembly/lot/firmware, environmental conditions, and corrective action reference. No analysis survives poor data collection.
Exploratory diagnostics
- Plot histograms, PP/QQ/Weibull probability plots, and the empirical hazard (nonparametric kernel) to detect mixtures or time‑dependent changes. A curving probability plot often signals mixed failure modes.
Choose parameterization
- Start with the 2‑parameter Weibull (β, η) unless there is a compelling physical reason for a third parameter (γ) shift. For many A&D datasets the two‑parameter model suffices.
Estimate parameters
- Use Maximum Likelihood Estimation (MLE) when possible — it's asymptotically efficient and handles censoring cleanly. For small numbers of events, apply bias corrections or bootstrap to quantify uncertainty.

MTTF formula (two‑parameter Weibull):

MTTF = η * Gamma(1 + 1/β).

Diagnostic checks
- Check residuals on probability plots, perform goodness‑of‑fit tests available in NIST/SEMATECH resources, and look for distinct clusters (submodes). If modes are mixed, split and re‑analyze.
Produce actionable FRACAS inputs
- For each mode produce: β with 95% CI, η with 95% CI, MTTF with CI, recommended FMEA criticality change, and suggested fix verification test (design‑of‑experiments for root cause if hardware).
Small sample and censoring cautions
- With very small event counts (n<10) MLEs are unstable; use median‑rank regression for a sanity check, bootstrap for CI, and flag high uncertainty in reports.

Python example: Weibull MLE (two‑parameter, loc=0)

import numpy as np
from scipy.stats import weibull_min
# data: times (failures only or include censored separately)
times = np.array([120, 305, 450, 810])
# fit shape c and scale
c, loc, scale = weibull_min.fit(times, floc=0)
beta_hat = c
eta_hat = scale
mttf = eta_hat * np.math.gamma(1 + 1/beta_hat)
print("beta:", beta_hat, "eta:", eta_hat, "MTTF:", mttf)

R example: Weibull + bootstrap CI

library(fitdistrplus)
data <- c(120,305,450,810) # failures
fit <- fitdist(data, "weibull")
beta_hat <- fit$estimate["shape"]
eta_hat  <- fit$estimate["scale"]
mttf <- eta_hat * gamma(1 + 1/beta_hat)
boot <- boot::boot(data, function(d,i){
  f <- fitdistrplus::fitdist(d[i], "weibull")
  c(f$estimate["shape"], f$estimate["scale"])
}, R=2000)

Citations and comprehensive diagnostics follow Meeker & Escobar's methods and the NIST e‑Handbook recommendations.

How to build Crow-AMSAA and Duane curves for growth tracking

A stepwise approach to credible system‑level growth curves and defensible projections.

The model
- Crow‑AMSAA is a power‑law NHPP with cumulative mean function Λ(t) = λ t^β and intensity ρ(t) = λ β t^{β-1}. Estimate parameters with MLE and use the model to forecast failures and instantaneous intensity.
Closed‑form MLE (single test phase, failures at times t_i, observation end T)
- Let n be number of failures, S = Σ ln(t_i) and T the total test time on test.
- MLE for beta (common textbook form):
- β̂ = n / (n * ln(T) - Σ ln(t_i))
- λ̂ = n / T^{β̂}
- These closed forms arise directly from the power‑law NHPP likelihood and give quick, exact MLEs for the standard parameterization.
Duane plot vs Crow
- The Duane model graphs log cumulative MTBF (or cumulative TTF per failure) vs log cumulative test time; the slope is the Duane learning exponent. Use Duane as a graphical summary and sanity check; do not treat it as a full inferential engine when you need confidence bounds or to handle censoring. Switch to the Crow NHPP for formal inference.
Piecewise and change‑point handling
- When fixes are implemented the process often becomes piecewise (different β, λ per phase). Fit segmentwise PLP or use change‑point detection (likelihood‑ratio tests or Bayesian online detection) and treat each segment as its own PLP for projection. MIL‑HDBK‑189 describes planning/tracking/projection variants for this use.

Crow‑AMSAA (PLP) fitting — short Python example (MLE + parametric bootstrap for CI)

import numpy as np
import math
def fit_crow_amsaa(failure_times, T):
    n = len(failure_times)
    S = sum(math.log(t) for t in failure_times)
    beta_hat = n / (n * math.log(T) - S)
    lambda_hat = n / (T ** beta_hat)
    return beta_hat, lambda_hat

def parametric_bootstrap(failure_times, T, B=2000):
    beta_hat, lambda_hat = fit_crow_amsaa(failure_times, T)
    lamT = lambda_hat * (T**beta_hat)
    boot_params = []
    for _ in range(B):
        # simulate N ~ Poisson(lambda*T^beta)
        N = np.random.poisson(lamT)
        if N == 0:
            boot_params.append((0.0, 0.0))
            continue
        # simulate failure times: t = T * U^(1/beta)
        U = np.random.rand(N)
        sim_times = T * (U ** (1.0/beta_hat))
        # refit
        b_sim, l_sim = fit_crow_amsaa(sim_times, T)
        boot_params.append((b_sim, l_sim))
    return boot_params

# Example
t = [50,120,210,380,700]  # failure timestamps (hours)
T = 1000  # total test hours
beta, lam = fit_crow_amsaa(t, T)

Use the bootstrap sample distribution to form percentile CIs for β, λ, predicted failures, or ρ(t) at a chosen time.

How to interpret MTBF, make forecasts, and calculate confidence intervals

Translate model outputs into program decisions — with quantified uncertainty.

From Weibull to MTBF and mission reliability
- MTTF = η * Γ(1 + 1/β) for the two‑parameter Weibull; reliability at mission time t0 is R(t0) = exp( - (t0/η)^β ). Use parametric bootstrap to propagate uncertainty from (β̂, η̂) to MTTF and R(t0).
From Crow‑AMSAA to forecasts and instant MTBF
- Expected cumulative failures by future time T2 given test history through T1:
- E[ N(T2) - N(T1) ] = λ (T2^β - T1^β).
- Instantaneous failure intensity at time t: ρ(t) = λ β t^{β-1} — approximate instantaneous MTBF is 1/ρ(t) (use with caution; MTBF is an engineering shorthand in repairable contexts). Use bootstrap to get CIs for ρ(t) and the reciprocal MTBF.
Projecting test time to reach a target instantaneous MTBF
- For target MTBF_target, solve 1 / (λ β t^{β-1}) ≥ MTBF_target for t (special case when β ≠ 1). Because λ and β are estimated, compute the distribution of the required t by sampling (β, λ) through parametric bootstrap and solving for t in each draw — the empirical percentiles become the CI for required test hours.
Use the delta method where appropriate but prefer parametric bootstrap when models are non‑linear and sample sizes are modest; bootstrap preserves skew in interval estimates and is straightforward to implement for both Weibull and PLP models.

Concrete projection example (conceptual):

Fit PLP and obtain β̂ = 0.6, λ̂ = 2e-6. Compute expected failures for next phase T2 and use bootstrap to give 90% upper bound on expected failures for schedule risk assessments.

Important: When β is very close to 1 the algebra for required time becomes numerically sensitive; report both the point estimate and a bootstrap interval and flag the sensitivity in test reports.

Practical Application: checklists, protocols and code for implementation

A compact field checklist and protocol you can adopt immediately.

Weibull per‑mode checklist

Export a validated CSV from FRACAS: test_id, time_hours, event_flag, mode, env, lot, FRACAS_id.
For each failure mode:
- Make probability plot and kernel hazard plot.
- Fit 2‑parameter Weibull by MLE (floc=0), get β̂, η̂.
- Compute MTTF and 95% CI via parametric bootstrap (≥2000 resamples for stable tails).
- Prepare FRACAS action: link failure to fix, assign verification test built on accelerated or repeatable test plans.

Crow‑AMSAA / Duane protocol

Consolidate repairable event stream (time‑stamped) and verify minimal‑repair assumption (i.e., repairs don't return unit to 'as new' state).
Fit PLP (β̂, λ̂) using closed‑form MLE shown earlier.
Run parametric bootstrap to produce:
- CI for β, λ
- Predicted number of failures in next test phase with 90% bound
- CI for instantaneous ρ(t) at key milestones (e.g., OT start)
If design fixes occur, re‑segment the data and re‑estimate parameters per segment (piecewise PLP).
Report: growth curve, Duane plot, list of FRACAS fixes closed with verified effect, required remaining test hours for contractual reliability.

Reporting template (minimum)

Figure: Weibull probability plot per critical mode with bootstrap CI.
Figure: Crow‑AMSAA growth curve (Λ(t)) with 90% projection band.
Table: β̂, λ̂ (Crow), β̂, η̂, MTTF (Weibull) with 90% CI.
Table: "Test hours remaining to reach contract MTBF at 90% confidence" (method: bootstrap).
FRACAS summary: number of corrective actions, effectiveness rating, repeat occurrence.

Parametric bootstrap code sketch (Crow → forecast failures in next dt hours)

# assuming beta_hat, lambda_hat, T (current time)
# bootstrap_params = parametric_bootstrap(failure_times, T, B=2000)
# For each (beta_i, lambda_i) compute expected failures from T to T+dt:
expected_fails = [lm*( (T+dt)**b - T**b ) for (b,lm) in bootstrap_params if b>0]
# take percentiles for CI
lower = np.percentile(expected_fails, 5)
upper = np.percentile(expected_fails, 95)
median = np.percentile(expected_fails, 50)

Operational notes from hard‑won experience

Always document what counts as a failure in your FRACAS ground rules; inconsistent definitions destroy growth curve credibility.
Treat high uncertainty as a program risk: quantify it, put it on the risk register, and require engineering closure evidence before counting a fix as effective.
Don’t present point estimates without intervals; auditors and program offices will ask for the 90% or 95% confidence band.

Sources:
Statistical Methods for Reliability Data (Meeker & Escobar, 2nd ed.) - Core methods for Weibull parameter estimation, MLE and bootstrap techniques used throughout life data analysis.

Statistical Methods for the Reliability of Repairable Systems (Rigdon & Basu) - Foundation for NHPP / power‑law (Weibull process) modeling and MLE for repairable systems.

Reliability Growth: Enhancing Defense System Reliability (National Academies Press) - Historical context for Duane and Crow modelling; interpretation of growth parameters at program level.

Crow‑AMSAA (JMP documentation) - Practical description of the Crow‑AMSAA (power‑law) NHPP parameterization and intensity function used in tool chains.

Reliability Growth (DAU Acquipedia) - DoD practice, references to MIL‑HDBK‑189 and the role of growth planning/tracking.

NIST/SEMATECH e‑Handbook of Statistical Methods - Weibull distribution properties, graphical methods, and goodness‑of‑fit guidance.

MIL‑HDBK‑189 Revision C: Reliability Growth Management (document reference) - Program‑level handbook describing planning, tracking and projection methodologies used by defense acquisition programs.

Apply these methods inside your TAFT cycles and FRACAS governance: demand per‑mode Weibull evidence for root cause, use Crow‑AMSAA for system‑level growth and formal forecasting, and always report intervals so program decisions rest on defensible statistics.