Voice of the DBA

Tag: disaster recovery

  • Data Worms

    The FLAME malware
    Are we going to see more and more targeted cyberwarfare attacks in corporations? I wouldn’t be surprised.

    Effective data administration include backups. I would argue this is the most important part of managing your data since no matter what happens, backups give you the ability to recover your system. Whether it’s an accidental data modification by a user, a hardware failure, a natural disaster or some other event, having a copy of your data (or log) on some other system or media can be the difference between getting back to work and having to go find other work.

    For the most part we’ve been concerned over disasters that are random, and while they may be destructive, their effect is usually limited in scope. SQL injection attacks often affect a single system, and with a good backup, you can usually recover your data quickly. However that may be changing as the world grows closer together.

    The Shamoon malware has been causing problems lately and making companies rethink their incident response. The worm has wiped workstations and destroyed data. Destructive software is nothing new, but as more and more cyber warfare takes places at the government level, it’s likely that retaliation might target civilian targets, especially successful companies. I’d hate to think that destructive malware might target databases, but I’m sure it will. The SQL Slammer worm was a complete disruption of our database services for days at JD Edwards almost a decade ago, but it could have been much worse if that malware had been written to be destructive.

    We don’t have default names and passwords in SQL Server, but we should make sure all accounts have strong passwords. There’s also some value in not configuring all systems identically. It makes management easier, but it also makes a successful attack that much more successful.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Prepare for When Disaster Strikes

    Abstract:

    A talk on disasters and what you can do to prepare for them. I look for four types of disasters, and I briefly talk about the immediate response. However we then go back and look at the ways in which we can respond quicker or perhaps mitigate the disaster. We mostly talk about RTO and how we can reduce it as much as possible.

     

    Length: 60 minutes

    Slides: Prepare for When Disaster Strikes

    Code: No code

    Presentations

    You can view my complete speaking schedule here:  http://wp.me/P14wgJ-1tV

     

  • Prepare for When Disaster Strikes

    When disaster strikes you have to respond and get things fixed ASAP. However if you could turn back the clock, and think about the potential problems and possible issues, can you decrease the time it takes to get things back to normal?

    This presentation looks at reducing your RTO by examining four common disasters, of which most disasters are a variation. We examine the components of RTO and then how you can reduce each one of them with a little forethought and preparation. The disasters are

    • Hardware Failure
    • User (or DBA) error
    • Corruption
    • Bad Deployment

    The talk includes a few demos of how you can respond quicker using Red Gate tools.

    Level: 100

    Length: 60 minutes

    Slides: Available after the SQL in the City 2012 tour.

  • 13 Disasters

    Hopefully you don’t have any server disasters because of coffee spilled on a machine.

    This editorial that was originally published on Aug 15, 2007. It is being rerun as Steve is traveling.

    This is an interesting list of disasters that can befall a production system. It comprises more than database servers, but certainly can happen to them as well.

    The list is pretty good and of these I’ve had more than a few of them happen to me. There were a few that I’ve never seen and it makes me worry a bit that this guy doesn’t have a good staff or set of vendors working with him. But one of these is very interesting.

    The second item on the list, a controller going bad and corrupting disks, is an interesting one in today’s world. What would you do if this happened on your SAN? Actually I know what you’d do. First you’d be in denial, and I don’t mean the river in Africa.

    Then you’d tell the SAN guys. They wouldn’t believe you. You’d argue and they’d check things, then they’d wonder how it could happen and why? You’d scream at them to fix it, probably using 4 letter, not 3 letter words. They’d start looking for ways to recover data and your hair would be slowly thinning as upper managers started calling down looking for answers.

    It shouldn’t happen, but it could. Some marketing VP is showing off that nice piece of SAN equipment with it’s high speed switches, dozens of drives, and lots of colored wires. He sloshes some coffee onto the system, freezes, but when the flashing red lights and siren from the Enterprise don’t go off (Red Alert!), he continues his tour while the controller scribbles on your disks.

    Restoring from backup might be easy. Of course if you have a system like some I’ve seen that shares physical disks among multiple LUNs, it might not.

    Disaster Recovery is rarely the hurricane Katrina type of issue. Usually it’s something like a disk drive, raid controller, cut wire, etc. that you have to deal with. So think about all of the minor disasters that you want to be sure you can handle and get some practice in on those. Make sure you have spare parts, you can rebuild a server (QA machines are handy for this), you know how to perform a restore.

    And most of all, be sure that you know where the backup files are stored. Preferably on different disks than the production data.

    Steve Jones