Originally published at migrationparity.com.
The client had done everything right. The maintenance run kept dying three quarters
of the way through, complaining about a folder that wasn't there; he found the
folder's name in the error, deleted the leftovers by hand, rebooted for good
measure. The error came back identical. His email ended: "I've never had an agency
by that name. I don't know where that file came from."
He was arguing with a record that had been sitting in a table since before his
current computer existed. The folder was gone. The row was not — and the
maintenance engine walks rows, not folders.
A deletion flag is not a deletion
Every record in a DBF file starts with one byte. Space means alive; an asterisk
means deleted. That is the entire deletion mechanism of the xBase family —
dBase, Clipper, FoxPro, and every vertical-market package built on them. When a
1990s application says "record deleted", it writes 0x2A into that byte and moves
on. The data is all still there, every field of it, until someone runs PACK —
the operation that physically rewrites the file without the flagged rows.
Most applications never run PACK on their own. It needs exclusive access, it
rewrites the whole file, and on the hardware of 1994 that was a lunch-break
operation. So it got tucked away in a maintenance menu, and the deleted rows
accumulated. I have opened tables where a third of the file was asterisks —
invoices voided during the Clinton administration, faithfully copied to every
backup since.
The app disagrees with itself about what exists
The flag would be harmless if every part of the application read it the same way.
They don't. xBase has a global switch, SET DELETED, that controls whether flagged
rows are visible — and different modules, different utilities, sometimes different
screens of the same program were compiled with different assumptions.
That's the mechanism behind my client's impossible error. The menu he used honored
the flag: the entry vanished, so for him it stopped existing. The maintenance
engine read the same table with the filter off — or followed rows the menu never
showed — walked into the stale entry, tried to change into its directory, and
died. Two parts of one program, two different realities, one very patient ghost.
Once you know to look for it, you see the pattern everywhere: the report that
still totals a "removed" cost center, the lookup that offers a vendor nobody
remembers creating, the export that has more rows than the screen. None of it is
corruption. The file is doing exactly what it was told in 1997.
Looking at the file instead of the screen
The diagnostic move is always the same: stop trusting the application's view and
read the table raw, deleted rows included. Any DBF dump tool with a
"show deleted" switch works; our browser-based
DBF Analyzer shows the deleted-record count in the
header summary precisely because this number — screen says 4, file says 6 —
is the fastest tell there is. Nothing uploads anywhere, which matters when the
table is your customer list.
In this client's case the whole investigation ran over email: a small diagnostic
utility on his side, a log file back to me, one stale row visible in it within
minutes. The full story of that engagement is in the
case study.
Removing a ghost without making new ones
The repair itself is a few lines of any xBase tool: open exclusively, delete the
stale row properly, PACK. The discipline around it is what keeps you out of
trouble. Copy the DBF and its index file first — PACK renumbers records, and a
production index (.cdx/.mdx) that doesn't get rebuilt in the same operation now
points at the wrong rows, which looks far scarier than the ghost did. Do it with
the application closed; half these packages hold tables open for the whole
session. And write down what you removed — the row you're deleting is usually the
only remaining record of whatever 1998 experiment created it.
Then run the application's own maintenance once more and let it finish. For one
of my clients this fall, that was the first complete pack-and-reindex his data
had received in years — the ghost had been aborting the run long before he
started noticing.
The asterisk byte is still in every DBF written today. So are the ghosts.
If your business still runs on DBF tables and odd errors are starting to
surface, that's what our dBase/DBF migration service
assesses first — and sometimes the honest first deliverable is an afternoon of
ghost hunting, not a migration.
Top comments (0)