Voice of the DBA

Category: Editorial

  • Too Many Bricks, Too Much Data

    Bricks
    Too many bricks are hard to keep straight

    At one point in my career I was working on a new application and we were debating about storing certain metrics related to the application usage and customer’s behavior. My boss asked for my opinion and I said that more data was better than less and we could always delete the data if we found it was not being used.

    However we rarely delete data. We seem loathe to remove old data, even when we find it’s slowing down our systems. Worse still, there aren’t great methods for even stripping out, and preserving older data other than custom work in each system. I ran across a piece that talks about the tremendous amount of data that we are constantly acquiring, a deluge that overwhelms us in so many scientific areas. There are some endeavors collecting so much data that they must restore to storing data in networked systems, making the data sets available only through software that can combine the information from various databases.

    In the corporate world of data we usually don’t deal with such large amounts of data, but we are often dealing with hardware that constrains our ability to work effectively with the data we have. We find that our disparate systems are spread across so many places that it can become hard to aggregate the bits together and extract information.

    I do think that we will start to feel the stresses of dealing with so much data and finding the meaningful information from it. The people that learn to do this well, and filter their bits effectively will become very valuable to their companies in the future.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Skipping April Fools

    April Fools
    Sorry, no joke: no April Fools stories this year.

    Over the last decade, we’ve had some good April Fool’s Day stories at SQLServerCentral. We’ve had Decoding the SQL Server Index Structure, Open Source SQL Server, Database Wars Coming for the XBOX 360, and more. I think my favorites was SQL Server on Linux, which still gets hits today. The best jokes seem to go over on a Monday, but today I’ve decided to skip April Fool’s Day at SQLServerCentral.

    Coming up with something plausible, exciting, but unrealistic is a good challenge for me every year. The people on the team will exchange some ideas throughout the year, and I’ll usually put together in February or early March, trying to contain my worry that the joke will not go over well. I think I’ve done a good job over the years, and I have a few good ideas for next year.

    However April Fools isn’t on a weekday this year. It was yesterday, Sunday, and the idea of a joke the next day didn’t sit well with me. “It’s April Fools” as a response to something run on April 2nd just doesn’t have the same zing, the same excitement. It seems too contrived for me, so I decided to sit out this year.

    Next year April Fools is on a Monday, and with all the changes we’re likely to see over the next year as SQL Server 2012 matures, SQL Azure changes, and Windows 8 comes out, I’ll be ready with a few new stories that will have you anxiously wishing for an RTM date as you read. Until you hit the “April Fools” message near the end.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Backup and Restore

    Rainbow Six
    The team at Rainbow Six has to be prepared for a disaster anytime.

    Today’s editorial was originally published on May 13, 2007. It is being re-run as Steve is at SQL Server Connections.

    I think that it’s important to practice restoring and rebuilding your systems. You never know when you’ll encounter problems and need to restore your data onto another piece of hardware. It’s definitely something that people don’t often practice and with documentation not usually being up to par, it’s something that people should be comfortable with performing.

    I saw this article for CIOs that seemed to suggest that IT people aren’t testing their systems enough. The statistic in the article was that 89% only test once a year. Also that 67% were only minimally confident that their disaster recovery system would work as planned.

    I think it’s pretty good that 89% test their systems. I’m guessing that 11% do it more than once a year, but I’d be concerned if 89% hadn’t tested it at all! I think once a year is a pretty good plan given the hectic scheduled, heavy workloads, and the lack of buy-in by many executives for a DR plan. It’s not like we’re the guys from Rain bow Six that could be called on multiple times to perform this year. In all likelihood most of us will never encounter a disaster in our careers.

    And if you do, it won’t go as planned. I’ve had disasters and disaster tests in my career and none of them goes as planned. It’s like I’ve seen quoted: “No battle plan ever survives contact with the enemy.” We could easily paraphrase this for IT and note that no DR plan will every survive a disaster. You need people that can adapt, think in the heat of the moment, and get things working when the plan doesn’t cover every contingency.

    As DBAs I think most of us tend to practice the DR stuff on a more regular basis as part of our jobs. Many of us are constantly restoring databases for QA and development, getting practice that sysadmins don’t usually get. We also have the advantage of being separated from the hardware and we can install SQL Server and restore databases on any Windows server host.

    I think you should practice your DR skills in little bits. Maybe grab the tail of the log and do a point in time restore for your next QA cycle. But practicing a full plan more than once a year isn’t practical for most of us.

  • The Mechanical Turk

    The original Mechanical Turk
    The original Mechanical Turk

    Today’s editorial was originally published on Apr 17, 2007. It is being re-run as Steve is at SQL Server Connections.

    I stumbled across Amazon’sMechanical Turk while helping search for Jim Gray. As a side note, Jim is still missing and presumed dead after an exhaustive search by the Coast Guard and many of Jim’s friends.

    The Mechanical Turk is based on the mechanical chess playing machine created in the late 1800s. It turned out to be a fake, with a human inside, and that is what Amazon has built. They have a way for you to distribute a workload and let humans work on the problem, or a very small piece of the problem. Similar to a super computer, but this process uses humans for things that artificial intelligence can’t handle.

    The basic idea is you create some task, for example one of the items I looked at was reviewing a list of vendor names associated with a contract for the city of Tulsa. It’s a data task that might not be easy to implement in something like SSIS, but a human can easily complete it. And get paid!

    Maybe not much, in this case $0.02 per item, but it’s a good way to distribute the workload.

    If you can find someone to work on the task. I guess if you wanted to fund a book habit at Amazon, it might be worth taking a few minutes here and there to look through data tasks and earn some money.

    I completed two and earned myself $.07 🙂