Your Tests Passed. That Is Not the Same as Correct.

#testing #ai #career #programming

A green test suite does not mean your code is correct; it means you have not found the failure yet. Dijkstra's warning is a reminder to write tests that try to break your code, not tests that confirm it. That discipline matters more now that AI produces code which passes the easy tests while hiding the case that will fail in the field.

In 1970 Edsger Dijkstra wrote a sentence every engineer should remember: "Program testing can be used to show the presence of bugs, but never to show their absence!" It is short, and it is exact. A test that fails tells you something certain: there is a defect here. A test that passes tells you far less than it seems to. It says only that this input, on this path, did not fail this time.

A passing test proves you have not found the defect yet. Nothing more.

Most engineers read that line, agree with it, and then work as if the opposite were true. The suite goes green and the task feels done. But green is not a proof of correctness. It only means no failure was caught this time. The gap between those two things is where most production bugs come from — the input you did not think of, the ordering you did not try, the board that behaves unlike the one on your desk.

Green is the absence of a caught failure, not the presence of correctness.

Why this matters more now

AI has made the gap wider. A model can produce a function and a set of tests that pass together in seconds, and both can be wrong in the same direction. The code handles the cases the tests check; the tests check the cases the code handles; neither touches the edge that fails in the field. It all looks finished. The tool can pass your tests without understanding your intent, because it was working towards a green result, not a correct system.

The tool can make the code and the test agree, and both can still be wrong.

So the discipline behind Dijkstra's line is now a core engineering skill, not a nicety. When code and tests are both cheap to generate, the scarce ability is the one that goes looking for what is still broken. That is a scientific habit: you do not try to confirm your idea, you try to break it, and you trust it a little more only after it survives an honest attempt to fail.

Do not try to confirm your code. Try to break it.

How to test like you mean it

The shift is from writing tests that agree with your code to writing tests that try to break it. A few concrete habits:

For every function, ask what input would break it, and write that test first. If you cannot think of one, you do not yet understand the function.
Test the edges, not the middle: empty input, the maximum, the off-by-one, concurrent access, the error path nobody runs.
State the invariant the code must always hold, then write a test whose only job is to violate it.
When AI hands you code with passing tests, treat that as the start of review, not the end. Add the case the model did not think to check.
When a bug reaches production, write the test that would have caught it before you write the fix. That test is worth more than the fix.

None of this means tests are not worth writing; they are one of the best tools we have. It means you should know exactly what a passing test does and does not tell you. A test suite is a record of the failures you went looking for, no wider than your own sense of what could go wrong. Widen that, and the tests get stronger. Skip it, and green becomes a false reassurance that hides the defect until a user finds it for you.