Voice of the DBA

Category: Editorial

  • Fun Data Analysis

    The first part of this article is very true: “… correlation doesn’t always equal causality.” It’s a fun look at statistics and there’s a short video from a news station that contains this other great quote: “… numbers can be tricky.”

    As data professionals, we are often tasked with helping business people understand the information hidden in data. We look to identify patterns and relationships between the data that our applications collect or our companies purchase. However it’s not as simple as it might appear.

    This week, I’m wondering if you have some fun stories, or perhaps some scary stories, on conclusions that you or others have reached with data.

    When has a correlation been mistaken for causation by you?

    I know that most people in business don’t have extensive statistics backgrounds, and I’ve often seen people fooled by finding simple or obvious patterns in data that didn’t necessarily match reality. I’ve seen plenty of money lost because of bad decisions made by incorrect data interpretations.

    I once worked in a restaurant where a new chef decided to use sales data from the previous year to predict how many meals would be served. Using a growth factor from sales over the previous year, he increased his purchasing as the summer season approached. A month into summer, we realized that we were wasting almost twice as much food as we had in the past. I had a similar story where purchasing managers used a straight average of the previous three months that left is woefully short on imported wood.

    Let us know this week if you’ve seen mistakes made from naive interpretations of statistics.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and LibSyn. 

  • Continuous Delivery for Windows?

    I read a bit about the next version of Windows, which is coming in 2015 as Windows 10. I’m not sure how much I care about some of the changes coming in the OS, though having CTRL+V working in a command prompt is very welcome. However I did read this piece about the people testing the Technical Previews and was a little intrigued by one quote: “… Threshold testers … will have those features and fixes pushed automatically to them…”

    Does that mean that Microsoft has re-engineered Windows to be integrated with a Continuous Delivery process? If so, then I think this is a good move. We’ve already seen SQL Server move to a pace that releases new versions every two years and bimonthly patches to fix issues. Imagine that we could get patches even more often, as bugs are fixed.

    Also imagine that we could get those bugs quickly rolled back and patches pulled if there are issues.

    I think that’s one of the interesting things for me. There have been patches in the past which caused, issues and were sometimes hard to remove. If new changes can be pushed out quickly, I’d hope they could be removed quickly. And with all the feedback that Microsoft gets from existing installations, I could even start to see custom patches built that are deployed to only certain configurations that are compatible with the patch.

    Of course, that’s an ideal view. I suspect that we’ll still see overworked developers releasing patches that not only fix issues, but cause other problems, and at times, can’t be removed. At least we’ll probably get the patch to fix the patch, a little faster than in the past.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • A T-SQL Code Testing Guide

    Do we need a guide for code like this one? The piece linked is from a programmer at Google who gives some code review items that should be tested for. I’m wondering we we might want to have something similar for T-SQL code?

    I’ve given a few talks on how I to get started with tqslt, which is a great framework for unit testing T-SQL code. As I’ve learned more about it, and experimented, I’ve been amazed by how flexible it is and how it makes it easy to setup and run tests on T-SQL code. However I can only experiment with the way I code and the tests I think of, and I’m sure there are many other ways in which we can better verify that our queries work as expected.

    Testing has changed since I developed software in VB and C++. It seems that much thought has been given to building better unit testing that can not only catch bugs, but also ensure the code performs as we expect it to. There’s also the idea that we can run these tests in an automated fashion. That helps prevent some bug when we first write code, but it more often ensures code continues to work as we re-factor our work and enhance our applications. More and more I think that the constant and continuous regression testing is at least as important as initial testing, if not more so.

    I really hope that we improve our development processes and testing methodologies for SQL code. It seems that it’s an area where we can also increase the speed at we build better database applications, and improve the quality at the same time. If any of you are formalizing the testing of your T-SQL code, we would be interested in publishing your thoughts and results for others to learn from. Please feel free to submit an article.

    Steve Jones

     

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn. 

  • The Next Five Years

    On one hand, the next five years don’t seem too far away. Five years, that’s a year more than most of us spent in high school or college. On the the other hand, 5 years ago was a long time in the computer world. Azure first gave us the ability to create a database, as opposed to just a key value store. SQL Server 2008 was the current version of the platform. SQL Saturday #26 was held the first weekend in October in Redmond. 8GB was the large size for flash drives and the iPad hadn’t started the current tablet revolution.

    The world of data has grown dramatically in 5 years. Our cell phones and other devices are drastically changed the amount of data that is collected and consumed. Advances in bandwidth have almost removed the need for us to move all but the largest sizes of data without any physical media. If we look to the next five years, does anyone think they can accurately predict how much storage we’ll maintain in our pockets or how fast we’ll expect to move data on a daily basis?

    I ran across a prediction of surprising things in the next five years, and while I’m not sure I think they will all come true, I do think that similar things will come to pass because of two things: amazing increases in computational power available to any individual and tremendous amounts of data. That’s no great surprise, but I am excited. I can’t help but think that there will be so many opportunities for data professionals that most of us will be gainfully employed for decades.

    I do think, however, that the mundane, easy jobs of administering individual instances, checking logs and backups, and setting security will be few and far between. Unless you know how to do those things for thousands of instances at a time.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn.