Author: way0utwest

  • T-SQL Tuesday #19 – Disaster Recovery

    TSQL2sDay150x150This month Allen Kinsel hosts T-SQL Tuesday #19 with a disaster recovery theme. This is the blog party idea started by Adam Machanic where everyone that participates writes on a single theme.

    If you want to host, contact Adam. If you want to participate, read Allen’s post and write your own post and link it in.

    What’s a Disaster

    It’s interesting that this is the theme since I already had two posts on DR scheduled this week and am working on modifying my “Preparation for Disaster” presentation for SQL in the City.

    It’s hurricane season for Allen on the Gulf, and while hurricanes give you advance notice when they are coming, you have to be prepared. I’ve lived on the East coast, and we always were worried about getting hit in Virginia Beach. We never did get hit, but that didn’t stop us from being prepared and sometimes even initiating some of our disaster protocols.

    As a side note, we had an office in High Point, NC, and it actually got hit by a hurricane, or the remnants of one and lost power. I was in a downtown hotel at the time and woke up without power.

    The thing is large disasters are rare. You have very little chance of your data center getting hit by a hurricane, a tsunami as Japan recently had, a tornado such as the Northeast have experienced this year,  They happen regularly, but the chance of them hitting your data center is low. The chance of you “losing” a data center is low. It just doesn’t happen that often, and the complete loss of a data center might mean you have bigger things to worry about.

    However there are plenty of disasters that you are likely to experience. It’s possible that you could have a fire in your data center that affects the database server. Not likely, but it could happen. It’s more likely that you could have one of these:

    • an operator spill coffee on your server
    • someone trips over a power cord or fiber cable and brings down your system
    • a memory chip go bad and get in memory corruption.
    • it’s possible power goes out for some reason, even with a UPS.
    • It’s highly likely that a drive will fail that contains data you need.

    These are the types of disasters that you really have to prepare for, since they are possible, and even likely.

    What’s more likely are the “Whoops” disasters as I like to call them. It’s very likely that someone will import the wrong file, update all prices instead of one, or even more likely, a developer or DBA will run the wrong code on the wrong server. The most likely disaster is probably this one:

    drwhoops

     

    If the DBA hits “Execute” here, it’s a disaster. Not a big one, but if this is a critical table in a critical system, you might have an even bigger reaction from manangement than if a hurricane hit.

    You have to prepare for disasters, but don’t get caught up in worrying about the data center being destroyed. Those disasters are rare. Most of your preparation, your practice, your checks, have to be focused on the more likely disasters, which are often smaller in scale and focused on your database.

  • What’s a Disaster?

    In working on my Preparation for Disaster presentation and doing research, one of the questions that came up early for me was “what is a disaster?” After all, if you can’t answer that question, how do you prepare for it.

    I’ve got a list of stuff here, which ones of these qualify as disasters?

    • Hard Drive crash
    • Hurricane wrecks data center
    • Fire in the server room
    • Corruption in a clustered index
    • DBA deletes a table by running DELETE without a WHERE clause
    • User clicks submit on a web page and the database crashed before they get the acknowledgement
    • Admin uses Access to update and lock a table, and forgets to save their work before leaving for lunch
    • DBA deploys code to production instead of development by accident.

    The answer is all of these are disasters. Most people seem to plan for the first 3, and forget to plan for the rest.

    Anything that interrupts your system’s ability to serve clients and get work done is a disaster. It’s worth keeping that in mind and planning for all sorts of potential problems. It doesn’t mean you have to be ultra paranoid, but you ought to have thought about the various potential problems and have some ideas about how to deal with them.

  • Cloud Outages

    Cound vendors need to make an effort to make their service "smarter"

    It seems that almost every few weeks there’s some announcement of an outage for a large service provider on the Internet. We’ve had Amazon Web Services go down, numerous Salesforce.com issues, and Microsoft’s Office 365 recently had problems. Most of us that work in IT are probably painfully aware of our own outages and how difficult it is to keep servers up across a long period of time.

    There are plenty of companies that have had outages, and I expect we’ll regularly see more announcements as companies look to engage cloud computing services. This isn’t to say that cloud services are unreliable, or even less reliable than in-house services. As is the case with many choices, your mileage may vary (YMMV) no matter which way you choose to provision your IT infrastructure.

    In my mind, we need to ensure that cloud computing vendors take these outages seriously, inform customers what went wrong, what was fixed, and what is being done to reduce the chances of outages in the future. Google has a dashboard, which is actually nice, albeit a little simple. It ignores some large Google sites, like Blogger, which had an issue a few weeks ago that was very annoying for me as I tried to publish a few entries.

    One would hope that cloud services would take their responsibility to offer a service seriously, and have less downtime than any random IT department. However the people running many of the daily operations at cloud service providers aren’t the best of the best IT people. They’re the IT people just like you and me that have a job to do, and do it with varying degrees of professionalism, and success. I would hope that they have better procedures, more consistent monitoring, and quicker response times, and some will, but not all of them.

    Moving to the cloud solves some problems in an IT infrastructure, but at the expense of less oversight, potentially less security, and certainly less control over how the daily operations are handled. It should result in better reliability for the systems over time, but as I mentioned, YMMV.

    Steve Jones


    The Voice of the DBA Podcasts

  • SQL Dream Cars

    A mini debate on Twitter the other day got me thinking about cars. People that know me know I like cars. I’ve owned (or my wife has owned with me) 26 cars since I started driving. I used to write a car update, but found that there are a lot of passionate people out there about their vehicles, and too many people got upset during the debates, so I stopped.

    However this is my blog, and I decided to talk about dream cars, or any vehicle that you’d like to own. I’ve got my list of the cars that I’d like to own one day, but here is a good list of five:

    1929gazellembreplica071106

    1929 Mercedes Gazelle – Not a real one, but a kit car. The technology for running a car from that era isn’t great, but give me this body with a modern fuel infected engine, brakes, and a good transmission.

    magnum-pi-ferrari

    Ferrari 308 GTS – The Magnum PI car. This one always caught my eye, and I love the targa tops (my 911 has one), so this would be one that I’d like to drive. Once again, I’d consider a kit car because it’s hard to drive this one around town on a regular basis.

    vw-thing

    VW Thing – At the same time I was first shopping for a 911, I was also looking at Things, in case I couldn’t find a 911 in my budget range. Something about these cars just seems fun to me.

    CJ7_for_sale

    CJ-7 – I had a friend in high school who got one of these. He put a 308 8 cyl engine in it and I went with him to get 33” tires. I actually helped bolt on headers, and love cruising around with the doors off and a bikini top on it in the summer. One of these days…

    1965 lincoln Continental

    1965 Lincoln – Not necessarily this model, but a big old boat convertible with suicide doors. That would be cool.

    There are more I’d like, but these are some fun ones that I came up with today.

    I’ll tag Brent Ozar (blog | @brento), Allan Hirt ( blog | @sqlha), and Aaron Bertrand (blog | @aaronbertrand) to talk about what they’d like. Hopefully a few more of you will post your own dream cars.