A Really Bad Day

I was out of town for work one day, literally on the other side of the country, in a car bound for the remote office when my cell phone rang. I had flown into LaGuardia Airport in New York City and needed to drive to Connecticut, to an office just outside of Hartford. The company was paying, and it was more convenient for me to fly into NYC and then take a chartered car to CT.

I glance at caller ID and it’s a client of mine back in Boulder, CO who I do some occasional SQL Server work for. I answer, thinking that he needs help with some SQL command and I can score some brownie points here without charging him.

“Steve, I have a problem”

The words that anyone in technology is loath to hear. Even if you’re being paid by the hour, clients that cause issues are often going to be more trouble than the payment is sometimes worth. I much prefer to hear “Steve, we’d like to do x. How can we do that?”

I ask him to tell me what’s wrong and he said that RAID card was throwing errors on the development server, so he performed a SQL Server backup, copied the backup to a file server and then replaced the RAID card, rebuilding the array. When he went to restore the server, he got an error.

I am the backup and restore champ, at least of my little world. “Read me the error, “ I say, confident I can solve this for him before we arrive at the office..

“The media set has 2 media families but only one are provided”

Uh, oh, I’m thinking, and not because of the poor grammar. This is not good and I’m not going to be able to fix this. Breaking bad news to a client is never fun, even if it’s not your fault. I’ve lost a couple because they were so upset they let me go at some bad news. That’s actually OK with me because if they can’t tell the difference, they are likely to leave me when I least expect it anyway and I’d rather know sooner than later.

I explain that he must have used two backup files and he needs to add the second one in there. The guy is insistent that he didn’t pick two files when he made the backup and he copied the right one off. He even included the date in the filename (like I showed him) and reads it back to me.

I know what’s happened. He’s made an inadvertent striped backup, and since the RAID array was blown away, he’s likely lost the other file. I try to keep from laughing as I explain patiently what happened. He’s upset, then he’s confused, then he’s unhappy. I can picture the sad panda face on him from 2,000 miles away as he realizes that he’s not getting this restore to work. The driver must hear it in my voice as well since he’s smirking in the rear view mirror.

To add misery here, he wasn’t copying backups off this development server to save space on the file servers. He’s lost a few weeks of work now, and he has to break the news to the developers, who likely don’t have backups of their own.

I sign off, not sure what to do as I get to my destination. It’s a good example of why SQL Server still needs a DBA, or someone with decent DBA skills. Despite the ease with which you can get many things done, not understanding why you do them cause big problems.

About way0utwest

Editor, SQLServerCentral
This entry was posted in Blog and tagged , , . Bookmark the permalink.

1 Response to A Really Bad Day

  1. Mark says:

    Developers should *always* have backups of their own. Knowing this saved me once years ago after a virus attack. Everyone else was running around screaming, but not me. (I’d been burned before.) I don’t care if you’re told that your work is mirrored and backed up to 15 different places, make your own backup.


Comments are closed.