DEV Community

Discussion on: 👻 Do you have any horror stories to share? Spooky bugs, scary data leaks, horrifying code, etc. 🎃

Collapse
 
oscherler profile image
Olivier “Ölbaum” Scherler

In the third week at my new job, we had to import a multi-gigabyte database into MySQL on our development server, and the /var partition was too small to hold it. “Fortunately,” the machine was setup with LLVM, so we could resize it (and /home, to make room). To avoid making silly mistakes with the CLI, we chose to use the GUI tool. After we downsized /home and upsized /var, everything was corrupt. The GUI had resized the partitions, but not the volumes, so /var was overlapping /home.

(Here you have to wonder about the point of having a GUI if it makes even sillier mistakes than those you’d have made in the CLI.)

Fortunately, we could retrieve the exact previous block counts of both partitions from the logs, and resize everything back to how it was before. But wait, it gets better: this time, the GUI chose to resize partition AND volume, so everything was still corrupt, /home was now smaller than the data it contained. So we had to re-re-resize the volumes, and fortunately everything was back to normal and no interesting data was lost.

Did I mention it was my third week on the job that I almost nuked the team’s sole development machine?

From this day on, every time Linux suggests I partition and set up LLVM, I smash the Nope button.