Ten Tests, Written Before the Code. The One That Failed Taught Me the Most

There is a number in solid mechanics that is exactly 3.

Take an infinite plate with a circular hole, pull it in one direction, and the stress right at the edge of the hole is exactly three times the stress far away. Kirsch worked it out analytically in 1898. It is not approximately 3. It is 3.

I wrote a finite element solver from scratch and never told it that number. It came back with 3.00002.

What "from scratch" means here

No FEA library. No scipy. The Delaunay mesher, the isoparametric elements, the global assembly and the conjugate-gradient solver are all my own code.

That constraint was the point. Calling a library and getting 3 proves the library works.

Configuration                                  K_t        error
Annulus, P2 curved, finest mesh (21,220 dof)   2.99981    0.0063 %
Annulus, extrapolated to h = 0                 3.00002    0.0005 %
Finite plate, extrapolated                     2.99970    0.0101 %
Conservative Richardson variant                2.99937    0.0211 %

Hitting the number is the weak evidence

This is the part I want to argue for, because it applies well beyond numerical code.

Landing on 3 could be luck. A bug that happens to cancel, a mesh that happens to flatter you, a fudge factor someone tuned until the output looked right. One number matching one expected value is the weakest test you can run — and it is the test most of us stop at.

The strong evidence is the rate. Finite element theory predicts exactly how fast the error must shrink as the mesh gets finer, and the rate is different for each element type. You cannot fake that by accident.

Element        L2 rate  predicted   H1 rate  predicted   energy  predicted
P1 (linear)      2.026      2         1.019      1        2.000      2
P2 (quadratic)   3.249      3         2.013      2        3.988      4

Six numbers, six predictions, all recovered. That is what says the implementation is right rather than coincidentally close.

Writing the tests before the code

I registered ten acceptance gates in a literature audit before any code existed. Quadrature exactness, patch tests, mesh validity, checking the iterative solver against a direct one, scale invariance, convergence orders for both element types.

The reason is not discipline for its own sake. It is that I know myself. If I write the criteria after seeing the output, I will unconsciously write criteria the output passes. Registering them first means the standard cannot move.

Nine gates passed. One did not.

Gate I failed, and it is published

Gate I concerned a variational crime — an approximation in how the curved boundary is handled. It failed. I tested two hypotheses for why and both were refuted.

What is published is an arithmetic account consistent with every number I measured, plus a plain statement that the discrepancy cannot be settled from this experiment and is not settled. The literature figure I had written the gate against turned out to come from a search summary rather than a paper I had actually opened. That is in the record too.

I could have deleted the gate. Nobody would have known it existed, because I wrote it myself, before the code, in a document nobody had read.

The gate that passed for the wrong reason

This one I am more pleased with than the failure.

Gate H passed. Then I went back and criticised it: the acceptance band I had written was the wrong shape for a one-sided theorem. It should have been a one-sided bound, and I had written a two-sided one. The result was fine, but the test would have passed things it should have caught.

No reviewer forced that. A passing test that passes for the wrong reason is a bug in your test suite that no failure will ever reveal to you.

Making the central claim checkable instead of asserted

The whole thing rests on "the target was never an input." That is easy to say and easy to lie about.

So the README documents it as something you can verify: grep the source for the target value, and the only hits you find are ones that never feed the computation. You do not have to believe me. You can run grep in about four seconds.

The full run reproduces with python fem_003.py in roughly fifty seconds, with a --smoke mode for a quick shake-out, and it ships its own evidence — gate report, convergence CSVs, hoop-stress CSV, mesh and convergence plots.

The honest framing

This is reproduction and validation, not discovery. Kirsch published the analytic solution in 1898 and the convergence theory is in every textbook. Nothing here is new.

The contribution is building the thing and demonstrating that it lands on a number it was never given, with the rates it was supposed to have, against criteria written before the first line of code.

If you take one thing from this: write your acceptance criteria before you write the thing, publish the ones that fail, and go back and check whether the ones that passed deserved to.

Full write-up: Ten tests, written before the code

Code, data and the full gate report: 10.5281/zenodo.21892064