Every real bug I found came from running the code, never from reading it

Mohamed Essam — Wed, 16 Sep 2026 09:08:02 +0000

I spent months building a tool that converts Oracle ADF applications into Spring Boot projects. By the time I thought it was solid, it had a test suite I was proud of:

192 unit tests
263 real applications that each generated a project which compiled
a sample of those projects started against a live Oracle database with every JPA mapping validated against the real schema
3,311 of 3,312 ADF attributes either mapped or explained by a diagnostic

That last one matters more than it sounds. Hibernate's ddl-auto: validate refuses to start if a single mapping disagrees with the database. Every column, every type, every key — checked against the real thing, at startup, or the application doesn't come up.

So: it compiles, it boots, and the database agrees with every mapping. What could still be wrong?

Then someone asked me a question I didn't have a good answer to.

"Did you actually call the endpoints?"

I hadn't. I had been treating "the application started" as "the application works."

I wrote a script to call every endpoint of one real application.

Roughly half of them returned HTTP 500.

Bug one: the queries that were never given their parameters

ADF view objects can carry SQL with named bind variables:

select * from orders where region_id = :regionId and status = :status

The generator carried that SQL across faithfully into a native query. It just never passed the parameters.

Every one of those endpoints threw QueryParameterException the moment it was called. Roughly half the API of a real application, dead on arrival.

Here is why nothing caught it:

Compilation passed. The query is a string. Strings compile.
Startup passed. Hibernate validates mappings against the schema. A native query isn't a mapping.
The unit tests passed. They asserted the generated code contained the right SQL — which it did.

Every check I had was looking at the shape of the thing. None of them looked at what happened when you used it.

Bug two: two entities, one name, wrong answers

This one is worse, because it didn't throw anything.

// wrong: two entities answer to "Customer"
entitiesBySimpleName.get("Customer");

// right
entitiesByFullyQualifiedName.get("com.example.billing.Customer");

One application had several pairs of entities that shared a simple class name across different packages — com.example.billing.Customer and com.example.crm.Customer. My code resolved entities by simple name.

So a query written for one table silently ran against the other.

The endpoint returned 200 OK. It returned well-formed JSON. It returned rows. They were rows from the wrong table.

Nothing anywhere in my test suite could have caught that, because every check I had would have been perfectly satisfied. The status code was right. The shape was right. The content was wrong, and only a person who knew the data would have known.

I found it by calling an endpoint and looking at what came back.

Bug three: the filter that quietly disappeared

ADF view objects can narrow their own query — a WHERE clause attached to the view, separate from the entity behind it.

In one case that narrowing wasn't carried over. The generated endpoint returned every row in the table. The original returned a filtered subset.

Again: 200 OK. Again: valid JSON. Again: more rows than the application it replaced was ever willing to show, which in a system with row-level access rules is not a bug, it's a data leak.

The pattern

Three bugs. All three passed:

check	result
Compilation	✅
Hibernate schema validation at startup	✅
192 unit tests	✅
45 acceptance tests against a real database	✅

Every one of my checks was testing structure. Does it compile, do the types line up, does the schema match, does the generated file contain the expected text.

None of them tested meaning. Does this endpoint return the rows it is supposed to return.

And structure is the easy half. Structure is what a compiler and a schema validator are for — they're very good at it, which is exactly why passing them feels like more assurance than it is.

What "tested" actually has to mean

I added a command that starts the generated application and calls every single endpoint it publishes, then reports three numbers: what served, what was correctly denied, and what failed.

The first honest run reported a large number of failures, a large number of endpoints correctly denied, and a minority actually serving data.

That was uncomfortable to read. It was also the first number I had produced that meant anything.

The "correctly denied" column matters too. In ADF, a REST resource with no grant in jazn-data.xml is unreachable. So the generated application denies it as well — because publishing it open would turn a closed door into an open one, and that is a security change disguised as a migration. A 403 there isn't a failure. But you only learn which 403s are correct by calling them and checking against the original's policy.

Why I'm writing this down

There's a comfortable kind of testing that verifies your code is shaped the way you intended. It's fast, it runs in CI, it goes green, and it tells you almost nothing about whether the software does the right thing.

Then there's the slow kind: run it, call it, and compare the output against the truth. For me that meant seeding a database, starting a generated application, calling an endpoint, and reading the rows that came back next to the rows the table actually held.

Every real defect this project has ever had came from the second kind. Not one came from reading code, or from a type error, or from a test I wrote in advance.

I now assume that anything which has only passed structural checks is probably wrong in some way I haven't looked for yet.

One more thing worth saying: when a result looks wrong, suspect your test data before your code. Twice I chased an empty response that turned out to be a failed INSERT in the seed script, not a broken query. The tool was right and the fixture was lying.

The tool is adfmig. The assessment half is MIT licensed and complete — point it at an ADF application and it tells you what a migration to Spring Boot would actually involve: what carries across, what has to be rewritten by hand, and where the source doesn't say enough to migrate safely. It runs entirely on your machine; nothing is uploaded.

If it reads your application wrong, there is an issue template called exactly that. Those reports are worth more to me than any number in this article.