CIF (Chronological Input Failure) has so far rested on two things: a simulation, and a retrospective mapping against public Apache Fineract defects. Both are useful, but neither involved running a fresh check on real transaction data myself. This post is that attempt, and it includes the mistakes I made along the way, because those mistakes matter as much as the result.
The dataset
PaySim is a mobile money transaction dataset, built to model the transaction patterns of a real mobile money service. It is not raw production data, and I want to be upfront about that limitation from the start. It is a simulation of a real service, not a live extract from one. I worked with roughly 77,000 real, unmodified rows sampled from the public dataset.
The test
Since PaySim does not contain separate validation and commit events, I could not test the original CIF mechanism directly. Instead I defined a narrower, honest question the data could actually answer: does the recorded balance change for each transaction reconcile with the transaction that supposedly caused it, and if it does not, is anything flagging that.
Mistake one
My first pass found that 75.92 percent of transactions showed a balance mismatch. That number should have been a red flag immediately, and it was. A finding that large, on a first pass, usually means the check itself is wrong, not that a massive failure has been discovered.
Investigating it turned up two real explanations. First, PaySim does not track merchant account balances at all. Every transaction with a merchant destination shows a balance of zero before and after, by design, not because anything failed. Every single one of those rows was an automatic, guaranteed mismatch under my original check, for no meaningful reason. Second, when a withdrawal amount exceeds the available balance, PaySim clips the resulting balance to zero rather than allowing a negative number. My check treated that clipping as a mismatch too, which was also wrong.
Mistake two
After excluding both of those, the mismatch rate was still high, 36.8 percent, which was still too high to be a real finding. The remaining problem was mine. My check assumed money always leaves the origin account, which is true for payments, transfers, and withdrawals, but not for cash-in transactions, where money enters the origin account. I had the direction of the arithmetic backwards for an entire transaction type.
The corrected result
After fixing both dataset artifacts and my own formula error, the genuine mismatch rate across roughly 35,000 cleanly checkable transactions was 0.07 percent, 26 transactions.
Every one of those 26 shares the same specific shape: a transaction is recorded, with an amount, a type, and a timestamp, but the account's balance shows no change at all, as if the transaction never touched it. 23 of the 26 carry no fraud flag or error indicator of any kind. The remaining 3 are flagged as fraudulent, and involve unusually large transfers.
What this does and does not establish
This is a real, specific, reproducible pattern found in real recorded data, and it matches the shape CIF describes: a system recording that something happened while the underlying state silently does not reflect it, mostly without any alert.
It does not establish that this reflects a genuine class of bug in real production payment systems. PaySim is a simulation. I cannot rule out that these 26 rows are an artifact of how PaySim itself was generated, rather than evidence of anything that happens in a live system. That distinction matters, and I am not going to blur it to make the finding sound stronger than it is.
What is next
This still is not a live case study. It is a real check on real, if simulated, data, with an honest and stated limitation. The next real milestone remains the same one from every previous post: a live system, a real team, a genuine finding that was not already known.
Reproducibility
All analysis code, sampling procedures, and filtering logic used in this check are available in the repository. The reported mismatch counts can be reproduced from the same PaySim sample by applying the documented reconciliation rules.
The code and full results are available at https://github.com/oludeleoluwapelumi/cif-simulation/blob/main/validation/PAYSIM_RECONCILIATION_CHECK.md
Top comments (0)