Hi everyone!
As AI adds more layers of abstraction to software development, understanding how systems work and debugging them are increasingly important skills. I’d like to explore QA insights with all of you, starting with the lessons we can learn from major software failures.
A test can pass while the assumption that makes it useful remains untested.
A content template expects 21 fields, but the running system supplies 20. A release reaches seven servers and leaves one behind. Two components exchange a number while assigning it different units.
These are the details worth carrying into a test review. A green result tells us what happened under the conditions we exercised. The next question is which conditions we quietly assumed.
The eight investigations below show where those assumptions broke down. Each visual explains a failure mechanism; each test to add turns the lesson into an observable check. These are proposed engineering controls, not reconstructions of the original test suites or guarantees that QA alone could have prevented the incidents. Prevention also depends on design, deployment, operational limits, and recovery.
As you read, pick one assumption that sounds familiar in your own system.
1. CrowdStrike: exercise every field
A mismatch between the content template and the real sensor inputs escaped tests that used wildcard matching.
On 19 July 2024, a CrowdStrike content update exposed a mismatch: a template expected 21 inputs, while the sensor supplied 20. Earlier wildcard tests had not required reading the missing input. An explicit match did, causing an out-of-bounds read. Microsoft estimated that 8.5 million Windows devices were affected. Sources: CrowdStrike technical RCA; Microsoft impact estimate.
Test to add — reject a broken contract:
Given a template declaring 21 fields and a consumer supplying 20, when content is validated with either wildcard or explicit criteria, then reject the mismatch before evaluating the fields.
Also exercise every supported field with an explicit match against the actual consumer. A wildcard can let a test pass without touching the value you intended to test. Stage content deployment and monitor crashes to limit the impact of anything that still escapes.
QA lesson: Configuration and content changes need executable validation against the system that consumes them.
2. Knight Capital: test the deployment
Seven servers received the new release; one stale server activated defective legacy code when a flag was reused.
On 1 August 2012, a deployment reached seven of Knight Capital's eight servers. On the remaining server, a repurposed flag activated defective legacy Power Peg code. Runaway orders produced losses exceeding $460 million in roughly 45 minutes. The SEC identified deployment and risk-control deficiencies. Sources: SEC investigation.
Test to add — leave one server behind on purpose:
Given a test cluster with seven updated servers and one stale binary, when the release verification runs, then it identifies the stale server and blocks promotion of the release.
In an isolated trading simulation, send the repurposed flag through that mixed cluster and check execution accounting. Separately prove that order-volume and loss limits actually halt routing. A deployment script reporting success cannot establish what every server is running.
QA lesson: A passing build is only one part of a verified production deployment.
3. Cloudflare: make performance an assertion
The WAF rule passed functional checks, but excessive regular-expression backtracking exhausted CPU.
Cloudflare's 2 July 2019 outage lasted 27 minutes. A WAF rule caused excessive regular-expression backtracking. Existing tests checked blocking and allowing requests without detecting runaway CPU use. A CPU safeguard had also been removed, and the rule could reach the global network without a staged rollout. Sources: Cloudflare postmortem.
Test to add — make a near-match expensive:
Given increasingly long inputs that nearly match a rule, when the regex is evaluated, then it completes within a declared CPU/time budget or a tested safeguard stops evaluation while unrelated requests remain within their service limits.
Choose the budgets before running the test. Check the safeguard as well as the regex, and use a small initial rollout to expose production effects before expansion. An assertion about the answer should sit beside an assertion about its cost.
QA lesson: Correct output must arrive within an acceptable resource budget.
4. Ariane 5: retest inherited assumptions
Software reused from Ariane 4 encountered values outside its assumptions under Ariane 5's flight conditions.
Ariane 5 Flight 501 was lost on 4 June 1996. Reused inertial-reference software converted a 64-bit floating-point value into a 16-bit signed integer. The new flight conditions exceeded the assumed range. Both reference units encountered the same defect, and testing had not adequately represented the new trajectory. Sources: Ariane 501 Inquiry Board report; ESA inquiry summary.
Test to add — replay the new environment:
Given representative trajectories for the new vehicle, when both reference units process the same inputs, then an unhandled numeric-conversion exception must not disable both channels. Exercise values just inside and outside the conversion limits separately.
Review input ranges, timing, operating modes, and fault responses whenever a component is reused. Two copies of the same software can reproduce the same defect; the test needs to examine that shared failure.
QA lesson: Previous success supplies evidence for the previous environment. New operating conditions require new evidence.
5. Mars Climate Orbiter: verify the meaning of data
The interface required metric units, but ground software delivered impulse values in pound-force seconds.
Mars Climate Orbiter was lost in September 1999. Ground software delivered thruster-impulse values in pound-force seconds; navigation software expected newton-seconds. The interface specification already required metric units. NASA found inadequate verification of the relevant ground-system interfaces. Sources: NASA investigation, including the Phase I report.
Test to add — follow one physical quantity end to end:
Given a known impulse equivalent to one pound-force second, when it passes through the producer, the interface file, and the consumer, then the consumer interprets approximately 4.448 newton-seconds, within a documented tolerance.
Calculate the expected result independently of the production conversion code. Otherwise the test can repeat the same mistake. Matching field names and numeric types establishes only part of the contract.
QA lesson: An interface contract includes units, interpretation, and physical meaning.
6. Therac-25: test sequences and timing
This diagram focuses on the Tyler race condition, one of several defects documented in the Therac-25 investigations.
Therac-25 overdoses caused deaths and serious injuries during 1985–1987. The illustration focuses on the Tyler accidents: rapid edits interacted incorrectly with concurrent machine-setup tasks. The investigation also documented other defects and serious software QA weaknesses. Sources: Leveson and Turner investigation, Part III.
Test to add — vary the moment of the edit:
Given machine setup in progress in a controlled simulator, when a mode edit occurs before, during, or just after each relevant state transition, then beam activation remains inhibited whenever the commanded mode and physical configuration disagree.
Repeat across task execution orders and edit speeds. Verify independent interlocks separately, including when software reports inconsistent state. A test that always types at the same speed explores only one schedule.
QA lesson: The same inputs can produce different outcomes when their order and timing change.
7. Patriot: treat uptime as an input
Numerical error in time conversion increased with continuous operating time and displaced the predicted search area.
On 25 February 1991, a Patriot system at Dhahran failed to track and intercept a missile that killed 28 Americans. Limited precision in time conversion produced an error that grew with uptime. The system had been operating continuously for more than 100 hours. Sources: GAO report IMTEC-92-26.
Test to add — run the same case at different system ages:
Given the same numerical test fixture at startup and after simulated continuous operation beyond 100 hours, when time-dependent calculations run, then their error stays within a predefined tolerance against an independent, higher-precision reference.
Cross elapsed time with the other input dimensions. Use simulated time to explore duration efficiently and real long-running tests to check integration behavior. Restarting between every test can erase the condition you need to expose.
QA lesson: Duration belongs beside values, states, and sequences in the test input space.
8. NATS: test the failures a backup can repeat
The same unusual flight-plan data caused primary and backup systems to enter maintenance mode to preserve safety.
On 28 August 2023, a valid-format flight plan containing a rare six-attribute combination disrupted NATS processing. Primary and backup systems applied the same logic and entered maintenance mode to preserve safety. NATS identified omitted intended logic, while judging the original testing reasonable. Sources: NATS final investigation.
Test to add — make the backup face the same fault:
Given a controlled exercise in which the same input disables automated processing on both primary and backup, when fallback procedures begin, then manual processing meets its declared capacity and service is restored within the defined recovery target, while safety constraints remain enforced.
Add regression fixtures for ambiguous identifiers and trace the intended processing logic to executable checks. The report does not justify certainty that a tester should have predicted the exact six-attribute combination. The broader lesson is to rehearse a shared software failure.
QA lesson: Redundancy needs validation against shared software failures, as well as individual hardware failures.
Before your next release, name the assumption
Pick one consequential change and bring these questions to its test review:
- Which fields, boundaries, and optional branches have never been exercised?
- What happens when deployment is incomplete or versions differ?
- Which resource limits are asserted under difficult inputs?
- Which assumptions came from a different environment or older product?
- Who independently verifies the meaning of data crossing an interface?
- What changes when events arrive faster, slower, or in another order?
- What changes after hours or days of operation?
- Which failure could the primary and backup systems repeat together?
Take one answer and turn it into a small piece of evidence: a test, a release gate, a measurable limit, or a recovery exercise. Name the owner and define what failure would look like before you run it.
A useful sentence to add to a test plan is:
We assume [assumption]. This test would show us that assumption is wrong: [test].
That sentence connects these incidents to everyday engineering. A missing field, a stale server, an old numeric limit, or a shared failure can hide behind a perfectly green result until someone deliberately challenges the assumption.
Which assumption in your system has never been tested? Share the assumption and the assertion you would use to challenge it.
Illustrations created with AI-assisted code using Rough.js and the Virgil font inspired by EDA Visuals by David Boyne, and reviewed for wording and causal structure. Historical findings are linked above; the QA controls are proposed engineering lessons.
Top comments (0)