Author: way0utwest

  • Pop Tarts and Hurricanes

    The second most dold item at Wal-Mart before a hurricane.

    What do people buy most of before a hurricane? Wal-Mart determined that batteries are the number ones sales item before a hurricane, but the second most sold item was surprising: pop-tarts. That’s the type of intelligence that can come from analysts asking the right questions about their customers and then IT delivering systems that can help them get an answer.

    More and more companies are starting to deal with big data. Big data is usually seen as extremely large sets of data. Those data sets  often exceed the capability of what a single database server can handle, and place a strain on existing IT infrastructures, especially when there is an explosion of new datab types. Hadoop is an open source framework that’s designed to deal with large sets of data and help with the processing and analysis of these large data sets.

    Microsoft recently added support for Hadoop to SQL Sever. Granted it’s only in the Parallel Data Warehouse, which is a highly specialized version of SQL Server integrated with specific hardware. However I am sure this will eventually make its way to other editions over time.

    Update: I made a mistake, there is a connector for non PDW SQL Server 2008 R2 instances.

    I don’t know if I would recommend learning Hadoop, but the techniques of processing large sets, and performning analysis on big data is important., Even more important might be the writing of well performing T-SQL code that will be used. I’d recommend you read about ways to write better code, and learn to integrate those skills into your daily work. You don’t always need them, but you often don’t know that when you are writing the code.

    It doesn’t take much more effort to write better code the first time, but it does take more effort to learn to do so. It’s an investment in your career that you ought to be regularly working on to become one of those talented people that is in demand in the future.

    Steve Jones


    The Voice of the DBA Podcasts

  • Come Shake My Hand at PASS

    Last year Andy Warren thought about trying to help first timer attendees at the PASS Summit with a little networking event. Two years ago he brought Don Gabor in for a networking pre-con that was very popular, and wanted to try something new. So we rented Lowell’s out down at Pikes Place Market and invited people down. We weren’t sure how it would turn out, but we decided to risk some of our money and see.

    It was a smashing success, with dozens, maybe even a hundred people there. I got to meet quite a few people for the first time, which always makes me smile. I am sure the same thing happened to many other people and the networking was great.

    We decided to do it again, and we’ve reserved Lowell’s again from 6pm-9pm on Monday, October 10, 2011. Anyone is welcome, and it’s a short walk from the convention center down to the famous Pikes Place Market.

    Register today if you’re coming

    We ask that you register so we can plan the staff with the restaurant. It doesn’t cost anything, but you are responsible for your own food and beverage bill.

    Come say hi, meet some new people, and shake my hand. I’d be happy to meet you.

    We are sponsoring this as part of The Mentoring Experiment.

  • Computed Columns and CASE

    I wrote about computed columns recently, but I didn’t realize that a logical expression could be used here, such as a case statement, and misspoke in this thread on SSC. Fortunately someone corrected me and I decided to blog and test this a bit.

    Why an expression?

    Suppose you had a table like this:

    CREATE TABLE OrderDetail
    ( OrderID INT
    , ProductID INT
    , Qty INT
    , OrderDate DATETIME
    , ShipDate DATETIME
    , STATUS INT
    )
    

    I record orders, but I don’t necessarily have a ship date when the order is placed. There are many cases where a user enters an order and some back end system later calculates a shipdate based on the supply situation, or even waits to update this field when the order ships. It’s entirely possible I have data in this table like this:

    computecol2

    Suppose I then have a status table with these values:

    computecol3

    Here is my logic. If the shipdate is null, then the order has not shipped and is in the status of ordered. If the shipdate is filled in, the order is shipped. If the shipdate is null, and the order date is more than 2 weeks old, the order is late. In our example, if the date is 11/5/2011, then the order #4 is late.

    We can easily join to the Status table to display this for the client, but we are depending on some process running every day to check for late orders. Otherwise how does row 4 get marked as late? When the order is entered, it’s not late, and we certainly don’t want to wait until the client checks on their order status to mark it as late.

    If we had a computed column, we could easily handle this, and display late orders for a report, or even for a other queries. I could change the definition of the status column to be this:

    CREATE TABLE OrderDetail
    ( OrderID INT
    , ProductID INT
    , Qty INT
    , OrderDate DATETIME
    , ShipDate DATETIME
    , STATUS AS CASE
           WHEN shipdate is NULL AND orderdate < DATEADD( dd, -7, GETDATE()) THEN 3 
           WHEN shipdate is NOT NULL THEN 2 
           ELSE 1
       end
     )
    GO

    Now the status I get is based on the other values in the row and is correct, regardless of my application logic.

    NOTE: This is a contrived example, and I don’t like this since I am assuming the status logic doesn’t change. In a real system, I would probably prefer to use the actual status in the column if I did this, as shown here:

    CREATE TABLE OrderDetail
    ( OrderID INT
    , ProductID INT
    , Qty INT
    , OrderDate DATETIME
    , ShipDate DATETIME
    , STATUS AS CASE
           WHEN shipdate is NULL AND orderdate < DATEADD( dd, -7, GETDATE()) THEN 'Late' 
           WHEN shipdate is NOT NULL THEN 'Shipped'
           ELSE 'Ordered'
       end
     )
    

    A Better Example

    A better example of where this might be used would be in the case where I might have a variable calculation. For example, in many Internet businesses in the US, you do not collect sales tax if your product is not being sold inside the state in which your company is located.

    If I sold horse products from my ranch, located in CO, I would have to collect sales tax for sales shipped to CO, but not those shipped elsewhere. So perhaps I’d have a table like this:

    CREATE TABLE OrderDetail
    ( OrderID INT
    , ProductID INT
    , Qty INT
    , Price NUMERIC( 10, 2)
    , St CHAR(2)
    , TaxRate NUMERIC( 12, 4)
    , LineTotal AS CASE
           WHEN st = 'CO' THEN (Qty * price) + (Qty * price * TaxRate) 
           ELSE (Qty * price)
       end
     )
    GO
    INSERT dbo.OrderDetail
            ( OrderID ,
              ProductID ,
              Qty ,
              Price ,
              St ,
              TaxRate
            )
    VALUES  ( 1 , -- OrderID - int
              1 , -- ProductID - int
              10 , -- Qty - int
              10 , -- Price - numeric
              'AZ' , -- St - char(2)
              .1  -- TaxRate - numeric
            )
    INSERT dbo.OrderDetail
            ( OrderID ,
              ProductID ,
              Qty ,
              Price ,
              St ,
              TaxRate
            )
    VALUES  ( 1 , -- OrderID - int
              1 , -- ProductID - int
              10 , -- Qty - int
              10 , -- Price - numeric
              'CO' , -- St - char(2)
              .1  -- TaxRate - numeric
            )
    
    
    SELECT OrderID, qty, Price, St, TaxRate, LineTotal
     FROM dbo.OrderDetail
     

    Useful?

    I’m not sure if computed columns are terribly useful. To me they strike of hard coding logic into a schema that I’m not sure belongs, but if you find them useful, you do have the option of doing so, and including logical expressions with CASE.

  • Help with testing Denali, SQL Server 11

    Earlier this week, Dan Jones, one of the Microsoft employees on the SQL Server team, wrote a blog post asking about testing the next version of SQL Server, Denali. Dan is trying to ensure that there are not surprises from customers because of a bug that wasn’t caught in testing.

    I know, I know, how can Microsoft, with billions of dollars being spent on this product miss a bug. It’s easy, coordination is hard, and despite the hundreds, or thousands of machines testing SQL Server, it’s not possible to hit every combination. People stress the software in all kinds of unusual ways, and it’s good to get some testing.

    So, if you have some free time at work, and can space some hardware, even a VM, get get the CTP and install it. Upgrade one of your databases, and run a workload. Even a profiler trace and see if things error out. Run the new SSMS tools and report any issues on Connect. As Dan notes, Connect sends bugs right into the Team Foundation Server that the SQL Server developers use.

    There’s even a Feedback Challenge, where you can win a prize if you get moving now.

    If you’re worried about installing Denali, there’s a nice post from Andrew Fryer on installation.