A Go pricing example show my green tests and high coverage can still miss a business rule defect.
Imagine a checkout service that gives customers a 10% discount on orders of $200 or more. The change has passed code review. The CI pipeline is green and the unit tests report 100% statement coverage.
Then a customer places a $150 order and recieves the discount. The service is working as written but it is not following the business rule.
How could the tests pass ?
The answer is simple
a test can run the code without checking whether the result is correct.
To understand this problem we need to seperate test execution, passing tests, code coverage, assertion quality and software correctness.
A small Go service with a real defect
Our business rule is
Apply a 10% discount only when the order subtotal is at least $200, Otherwise return the original subtotal.
The example stores money as integer cents. It assumes a non negative subtotal and rounds the discount down to a thole cent through integer division. We will focus on the discount threshold, not input validation.
Create a go module
mkdir pricing-demo && cd pricing-demo
go mod init example.com/pricing
Add pricing.go
package pricing
// FinalPrice returns the price in cents after any eligible discount.
func FinalPrice(subtotalCents int) int {
if subtotalCents >= 10_000 { // BUG: the rule says 20_000.
return subtotalCents - subtotalCents/10
}
return subtotalCents
}
The code gives the discount starting at $100 instead of $200 This matters for any subtotal from 10,000 to 19,999 cents.
For example, FinalPrice(15_000) return 13_500. The correct result is 15_000 because a $150 order should not recived a discount.
Test execution is not the same as verification
Now add pricing_test.go
package pricing
import "testing"
func TestFinalPrice_Weak(t *testing.T) {
for _, subtotal := range []int{9_000, 20_000} {
got := FinalPrice(subtotal)
if got < 0 {
t.Fatalf(
"FinalPrice(%d) = %d, want a non-negative amount",
subtotal, got,
)
}
}
}
This test execute both path in FinalPrice one order without a discount and one with a discount. It also has an assertion, which is a check that compares an observed result with an expected condition.
But the assertion is too weak. It only checks that result is not negative. A wrong price of 13_500 would satisfy that condition, just as the correct price of 15_000 would.
Run
go test -count=1 -cover ./...
For this isolated example the test passes and Go report
coverage: 100.0% of statements
Both branches have run. Every statement in this small function has been exercised. The business rule defect is still present.
Figure 1. Passing tests confirm only the checks the suite actually makes
Test Execution
Running tests confirms that the test code was executed. However, it does not mean the tests checked important behavior.Passing Tests
Passing tests mean all executed checks succeeded. However they do not prove that every requirement was tested.Code coverage
Code coverage shows which parts of the source code were executed duting testing. However, high coverage does not mean the results were correct.Assertion Quality, Can the tests detect incorrect behavior ?
Good assertions help tests catch wrong results. However strong assertions do not guarantee that every possible scenario was tested.Software Correctness, Does the software work as expected ?
Software correctness means the system meets it requirements under relevant conditions. Passing tests provide edividence of correctness, but they cannot prove it on thier own.
Running tests, passings test and achieving high code coverage are not the same as proving software correctness. What matters is whether the tests can detect meaningful defects.
Thesese concepts support each other, but they are not interchangeable.
Go coverage tooling measures executed statement. It does not judge the quality of your expectations. The official
explains how instrumentation redcords which parts of the source code are exercised.
Stornger tests use the business rule as expeceted result
A better test starts from the rule, not from what the implementation happens to return.
Add pricing_rule_test.go
package pricing
import "testing"
func TestFinalPrice_BusinessRule(t *testing.T) {
tests := []struct {
name string
subtotal int
want int
}{
{"ordinary order", 9_000, 9_000},
{"below threshold", 15_000, 15_000},
{"just below threshold", 19_999, 19_999},
{"at threshold", 20_000, 18_000},
{"above threshold", 25_000, 22_500},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := FinalPrice(tt.subtotal)
if got != tt.want {
t.Errorf(
"FinalPrice(%d) = %d, want %d",
tt.subtotal, got, tt.want,
)
}
})
}
}
This test use a test oracle, meaning a source of expected results. Here the oracle is the written business rule, expressed as exact amounts in the test cases.
go test -count=1 -cover ./...
The stronger suite fails on two cases
FinalPrice(15000) = 13500, want 15000
FinalPrice(19999) = 18000, want 19999
Notice what changed. We did not need more statement coverage to find this defect. We needed test inputs near the rule boundary and assertions that checked the actual price.
Fix the condition in pricing.go
if subtotalCents >= 20_000 {
Now both test files pass. The coverage number remaings 100% in this small example, but the tests provide stronger evidence about the discount rule.
Why 100% coverage is not 100% correctness
Figure 2. Equal statement coverage does not mean equal defect-detection ability.
Coverage answer an execution question
Which parts of this code ran while tests were running ?
it does not answer a requirements question
Did the program produce the right result ?
Coverage is useful when it points to a path that no test exercises. It is much less useful as a score for test quality. Martin Fowler makes this distinction in his article on
Even 100% statement coverage leaves important gaps. A test might check only that a function returns without an error. It might use the wrong expected value. It might miss an input boundary. And a rule missing from the implementation cannot be discovered merely by measuring execution of the code that exists.
Go's coverage report is also about instrumented source statements, not a guarantee that every possible input, condition or execution order has been explored. The Go command documentation describe the available coverage options and test behavior.
Use coverage to find places that need attention. Do not treat its percentage as proof that the service meets its contract.
A pasing test it evidence, not a proof
A test has a limited claim for these inputs, in this environments, the observed result met these expectations.
That is valuable edividence, it can prevent regressions, document an example of expected behavior and detect defects early. But its strength depends on the selected cases, the correctness of the expected results and the reliability of the test environment.
A services has many more possible situations than a small test suite can usally exercise. Inputs can change dependencies can fail and concurrent requestes can arrive in difference orders. Passing tests cannot by themselves, establish that every possible behavior is correct.
This does not make testing pointless. It means we should be precise about what our tests demonstrate. Formal verification can esablish specific properties under stated assumptions, but an orinary green go test result its not that kind of mathematical proof.
How to make test result more meaningful
When reviewing a unit test, begin with the requirement. Ask what result should occur and which mistake the test should catch. In the pricing example, the key condition is the threshild at 20,000 cents, so the values just below and at that boundary are more useful than two arbitary positive amounts.
Prefer assertions about observable behavior. Checking got != want for the final amount is more informative than checking that the amount is non-negative. When a function returns an error, check both the error and relevant result, according to its contract.
Next, use coverage reports to find the code that has not been exercised. in Go, these command produce and inspect a statement coverage profile
go test -count=1 -coverprofile=coverage.out ./...
go tool cover -func=coverage.out
go tool cover -html=coverage.out
The HTML view makes it easier to see which statements ran. It does not mark wak assertions or missing business requirements.
Finally review failure cases as carefully as success cases. For services, that may include invalid input dependency timeouts, retries and concurrent updates. Choose these cases based on the service's actual contract not on a target coverage percentage.
Conclusion
Passing tests are useful, but thier meaning is narrower than the green status suggests. Coverage tells us what ran Assertions tell us what was checked. Correctness depends on whether those checks reflect the real requirements.
The important question is not only whether we have tests, but whether those tests whould fail when the behavior becomes wrong. Mutation testing explores this by deliberately changing code and checking whether the tests notice.
Further reading
- Go: Add a test, an introduction to the built-in test workflow.
- Go: The cover story, an explanation of statement coverage and reports.
-
Go command documentation, including
go testand coverage flags. - Martin Fowler: Test Coverage, on using coverage to find gaps rather than treating it as a quality score.
Top comments (1)
The $150-order example lands the point well: 100% statement coverage with an assertion that only checks the result isn't negative is really just testing that the code runs, not that it's right. Starting the test from the business rule — the oracle — instead of from what the implementation happens to return is the habit that would have caught the $100 vs $200 threshold immediately. Coverage is still useful as a map of untested paths, just not as a score for correctness.