Author: way0utwest

  • T-SQL Tuesday #027 – Invitation to The Big Data Valentine’s Edition

    tsqltuesdayIt’s T-SQL Tuesday time again, and I’m honored to be able to host the party for February 2012. This is my second time hosting, with the first being #013.

    T-SQL Tuesday is the brainchild of Adam Machanic, who hosted the very first blog party. If you’d like to be a host, contact Adam on his blog. The complete list of older topics is here: T-SQL Tuesday Topics – February 2012, so you can pick a new one if you’d like to host.

    Note, this is advance notice. The party is NOT today, it’s next week, February 14, 2012.

    The Rules

    Here are the rules your post must follow.

    1. Your post must go live between 00:00:00 GMT on Tuesday February 14, 2012 and 00:00:00 GMT on Wednesday February 15, 2012.
    2. Your post must link back to this post, and the link must be anchored from the logo (found above) which must also appear at the top of your post.
    3. Leave a comment or a trackback here on this blog regarding your post.
    4. ‘T-SQL Tuesday #027’ should be included in the title of the post.
    5. If you’re on Twitter, tweet about your post using the hash tag #TSQL2sDay

     

    Big Data

    godivaThe party falls on Valentine’s Day this month, and I thought a little notice would let you get your post done early, and then spend the day with your loved ones

    Since the media has a love affair with Big Data right now, I thought this would make an interesting topic. Data is the lifeblood of our careers and of many organizations. Slowly governments, companies, and individuals are becoming aware of just how important data is to us, and as they do, they want more of it. More storage, more access, and more analysis.

    That creates challenges for us as data professionals. We will have to learn to better manipulate, aggregate, summarize, and handle larger volumes of data in the future. I think this means we will have lots of employment in this area, and it means tremendous opportunity for those that learn to work with data well.

    This month I want to hear what big data problems you’ve solved, or interesting ways of working with big data, challenges that you struggle with, or cool hardware tricks that mask the problems in your code.

    Think big, and let us know how you work with Big Data.

  • Computed Columns and Divide by Zero

    Edit: This was a poor example of using the divide by zero handling. I was trying to alter something I’d done in the past and this didn’t work well. I will rewrite this soon with a better example.

    I have a few posts on computer columns, the basics of computed columns, using CASE in a computed column, and UDFs in computed columns, but there was another use that someone pointed out to me recently: catching divide by zero errors.

    Suppose you have a column that is determining a percentage of profit for some sales. I’ll create a table and include some values:

    CREATE TABLE MySales
    ( salesid int , Product VARCHAR(20) , Cost numeric(10,4) , Price numeric(10,4) , profit AS (price - cost)/ cost
    ) GO INSERT MySales SELECT 1, 'Bike', 150.45, 180.99
    INSERT MySales SELECT 1, 'Shoes', 23.55, 45.99
    INSERT MySales SELECT 1, 'Soda', 0.25, .99

    If I look at the values in the table, I get back these items:

    /*------------------------
    SELECT Product
    , Cost
    , Profit
     FROM MySales
    ------------------------*/
    Product              Cost                                    Profit
    -------------------- --------------------------------------- ---------------------------------------
    Bike                 150.4500                                0.202991026919242
    Shoes                23.5500                                 0.952866242038216
    Soda                 0.2500                                  2.960000000000000

    Things work great, and everything seems to be fine. However, what if we get some free products that we can sell, say something we made out of scraps, or were given to us as a gift with essentially no cost?

    INSERT MySales SELECT 1, 'Keychain', 0, .75

    It works. However if I select data:

    SELECT Product
    , Cost
    , Profit
     FROM MySales

    I get this:

    Product              Cost                                    Profit
    -------------------- --------------------------------------- ---------------------------------------
    Bike                 150.4500                                0.202991026919242
    Shoes                23.5500                                 0.952866242038216
    Soda                 0.2500                                  2.960000000000000
    Msg 8134, Level 16, State 1, Line 2
    Divide by zero error encountered.

    The computation occurs on query, and it doesn’t work well.

    However we can fix this with a function in our computed column. There are a couple choices here. I can use ISNULL and a CASE to build a formula, I could use COALESCE, or I could use NULLIF.

    NULLIF returns null if two values are equal. If a NULL is acceptable in my table, I could use that. I could end up with:

    CREATE TABLE MySales
    ( salesid int , Product VARCHAR(20) , Cost numeric(10,4) , Price numeric(10,4) , profit AS (price - cost) / NULLIF(cost ,0) ) GO INSERT MySales SELECT 1, 'Bike', 150.45, 180.99
    INSERT MySales SELECT 1, 'Shoes', 23.55, 45.99
    INSERT MySales SELECT 1, 'Soda', 0.25, .99
    INSERT MySales SELECT 1, 'Keychain', 0, .75
    GO SELECT Product
    , Profit
     FROM MySales

    and get back:

    Product              Profit

    ——————– —————————————

    Bike                 0.202991026919242

    Shoes                0.952866242038216

    Soda                 2.960000000000000

    Keychain             NULL

    NULL is a good marker here, but your application needs to handle this and let the user know there is an issue.

    You could use COALESCE, which returns the first non-null value. I could see any of these as being a valid formula:

    , profit AS (price - cost) / COALESCE(cost ,0) 

    which uses “0” profit margin as a marker, or even something like:

    , profit AS (price - cost) / COALESCE(cost , 999)

    which uses 999. Lots of times we’ve used a large, nonsense number as a marker that lets someone know there is a strange value here.

    If we converted this to a varchar, it’s possible that we could even use words in there, but I wouldn’t recommend that as downstream uses of this column might involve other calculations.

    That’s a short look at how a computed column can solve an easy, common issue: divide by zero.

    Credit to Atif Shehzad, whose article I found while researching this.

  • T-SQL Tuesday Topics – Febuary 2012

    Updated again.

    I was looking for a static list of all the T-SQL Tuesday topics and couldn’t find one. The domain moves around to the current hosting blog, so when it was my turn to host it, I made a list just so I could reference what was out there.

    After a few more months, I decided to update my list with the latest topics. That will give me the chance to plan for my own future hosting of the party if Adam will have me again.

    I’ll keep updating, mainly so I can keep track. And since I’ve got #027, look for a new topic soon.

  • Insider Security Threats

    Monitoring is the key to really good security.

    Are you worried about internal users at your company compromising your data security? I’d hope that you are at least a little worried, after all, we find out regularly that people we thought we knew acted in a surprising manner, or did something inappropriate that we hadn’t expected. It’s not always malicious or intentional, but even when it’s accidental, our security gets compromised and we receive some of the blame.

    Security is a hard process to implement, especially over time. Too much security implies too little trust, and as humans, we want to trust each other. As we work together, and build trust, we tend to let security lapse a bit. As organizations grow, evolve, and change people around, we introduce security loopholes from mis-configurations, poor architectural foundations, or simple mistakes like failing to remove someone from a security role.

    This piece talks a bit about the internal security threats you face, while ranting a bit about the term “insider threat”. The threats you face from external attackers are different from those you face from internal employees. However in each case, there’s one thing that’s important for getting close to a secure environment: monitoring.

    We can’t determine every type of attack vector, protect every system or database completely, but we can monitor for issues and be prepared to react when a problem occurs. The auditing capabilities of SQL Server have grown tremendously with the eventing enhancements to the platform, and I urge you to spend some time learning about Extended Events, which give you even more of an insight into what is happening on your server.

    Steve Jones


    The Voice of the DBA Podcasts

    We are having some technical issues with our hosting provider and are working to get the podcasts back online soon. Our apologies for the delays.