The tape survived, the problem came later.
In 1999, an independent hardware store in Westbank near Kelowna lost its lumber yard to a fire, and I learned an important lesson about data security.
The kiln-dried lumber burned hot and fast. With little chance of saving the yard, firefighters concentrated on keeping the fire from spreading to the surrounding buildings. Within a short time, the entire lumber yard was gone.
The equipment room was close to the lumber yard. Heat and water destroyed the servers and the store’s records.
When the owner filed the insurance claim, the insurer had one question.
How much lumber had been destroyed?
The store consulted its staff and submitted an estimate based on what they remembered had been in the yard.
The insurance company wanted proof.
It hired an expert to examine the ashes and estimate how many square feet of lumber had been stored there. His number came in well below the store’s claim. The owner knew it was far too low.
From the insurer’s position, the request was reasonable. The physical inventory was gone. The store was asking to be compensated for material that could no longer be counted.
Then someone remembered the backup.
We had installed the store’s accounting system and trained the staff to use a three-generation backup rotation. At the time, we called it grandfather-father-son. One weekly archive had to be kept off-site. Every backup also had to be verified.
I would not work on a customer’s server unless someone could produce a printout showing that a current backup had completed successfully.
If they could not, I made one first and charged them an extra hour.
The procedure was strict because those systems held accounting records, payroll, inventory, and audit history. A failed repair could not be allowed to become a failed business.
The company produced the off-site tape. The printout stored with it contained the most reassuring word in backup administration:
Verified.
They had done everything correctly.
The backup was in their hands. A small tape that contained everything that was needed.
All they had to do was restore it.
So they called me.
The inventory system ran on SCO Unix. At the time, I was one of the few people in the area who could install it and make the attached hardware work reliably.
We knew the customer’s system because we had built and configured it.
The backup had been written on an HP SureStore Tape 2000 using the original DDS format, later referred to as DDS-1.
By 1999, I had newer DDS drives available, including DDS-3 units. They were good drives, but backward compatibility was not as simple as the name suggested. The later drives I had could not read the early DDS media used for this backup.
The data was still on the tape. The machines I had could not reach it.
That sent me looking for the same model of drive that had written it.
Eventually, I found one on eBay, which was still new enough that buying obsolete hardware from a stranger felt like a gamble. I didn’t care about the cost at that point. I needed the drive, so I paid a lot for a discontinued product and fast shipping.
The replacement drive arrived. I verified the model and confirmed that it supported the correct tape format. We had the original backup, the verification printout, and hardware from the same generation.
I started the restore.
It failed.
The tape was not behaving like a modern file copied from one disk to another. It was a physical recording made by a particular machine, on a particular piece of magnetic media, at a particular moment.
Two drives could be built to the same specification and still differ slightly in head alignment. Those differences were usually small enough to remain invisible. In this case, they were enough to turn a verified backup into unreadable tape.
I opened the drive and the service manual.
Inside, the problem became mechanical rather than digital.
I spent several hours with precision screwdrivers making tiny adjustments to the tape heads. Adjust, test, fail. Adjust again. Test again. Each change risked tangling the tape in the spindles or making the backup permanently unreadable.
Eventually, the tape began to read without failure and the restore completed.
The inventory records confirmed the store’s estimate.
I provided a statement describing the tape’s chain of custody and how the backup had been restored. My role was over.
Until this time, I had treated verification as the end of the backup process.
The tape had been written. It had been read back. It had been rotated. It had been stored off-site. By every standard I enforced at the time, the store had done what it was supposed to do.
The fire taught me that verification on the original machine proved only that the original machine could read its own work. Recovery means more than this. It also means being able to restore the data after the original system has disappeared.
Afterward, we kept a spare drive for every tape model we installed.
The data was never the problem. The problem was everything required to make it legible: medium, hardware, software, and someone who understood. Break any one of them, and the medium might as well be a rock.
We had preserved the data. We had almost failed to preserve the means of access.
The tape had survived the fire. It had been stored off-site. It had been verified. It had done everything expected of it.
What had nearly vanished was the machine that could make it speak.
Top comments (0)