Category: Blog

  • The Tally Table and Expanding Rows

    Suppose you had some data like this:

    Class           Limit
    ————— ———–
    Calculus        5
    History         4
    Physics         2

    But what you really want is this:

    Class           Student
    ————— ———–
    Calculus       
    Calculus       
    Calculus       
    Calculus       
    Calculus       
    History        
    History        
    History        
    History        
    Physics        

    Physics        

    Where you have a placeholder for each student. This is a little contrived, but for the sake of the scenario, how do you actually expand the data in the first set to the second?

    An easy way is a cursor, but suppose you had a large school environment and you were doing this regularly, you might not want the performance hit of a cursor. Suppose you have some similar scenario, like slotting inventory or holding places for some large report? You would want a better solution.

    Setup

    Let’s first get a table ready for this.

    declare @Table table (
      Class varchar(15)
    , Limit int
    )
    ;
    
    insert into @Table 
      values ('Calculas',5)
           , ('History', 6)
           , ('Physics', 2)
    ;       
    

    A Tally Table

    I first saw this when Jeff Moden talked about this in his “The Numbers or Tally Table” article. It seemed like a neartidea, and he improved upon is in his next piece, finding a more efficient way to generate the data. An even more flexible idea that I like came from his Test Data Generator article, where he uses sys.columns to build a list as large as he would like. I found it to be a great idea, and this problem is a great way to apply it.

    Here’s a short look at the code that makes this work.

    SELECT TOP (4) RowNum = ROW_NUMBER() OVER(ORDER BY a.[Name])
     FROM sys.columns a, sys.columns b

    In this code, a cross join occurs between sys.columns and itself. With the addition of the ROW_NUMBER() function, and an order by, you can easily get a list of numbers. If I run this. I get:

    RowNum

    ——————–

    1

    2

    3

    4

    If I change my TOP value, I can get different rows:

    SELECT TOP (6) RowNum = ROW_NUMBER() OVER(ORDER BY a.[Name])
     FROM sys.columns a, sys.columns b

    RowNum

    ——————–

    1

    2

    3

    4

    5

    6

    To make this work for me in the original problem, I will use the limit value for my TOP in a subquery. However I can’t do a straight join, I need a CROSS APPLY, that will execute the right side of the CROSS apply for each input row from the left table.. It’s intended to be for table valued functions, but in this case, we’ll just return a table from the right side.

    The code looks like this:

    SELECT
      c.Class
    , 'Student' = ' '
    FROM (
    SELECT Class,Limit
    FROM @Table
    ) c
    CROSS APPLY (
    SELECT TOP (Limit) RowNum = ROW_NUMBER() OVER(ORDER BY a.[name])
    FROM sys.columns a, sys.columns b
    ) n

    What happens is that for each row, the derived table on the right side of the CROSS APPLY, returns a set of numbers which are then linked to the row on the left side of the CROSS APPLY.

    For the first row in @table, we have “Calculus” and “5”. When we apply this row to the subquery on the right side (SELECT TOP()…), we get these results:

    1

    2

    3

    4

    5

    When this is joined with the row and the c.Class returned, we end up with the same value being returned 5 times. That results in

    Calculus

    Calculus

    Calculus

    Calculus

    Calculus

    If this is repeated for each of the other rows, we get 4 and 2 rows from the right side of the CROSS APPLY returned, respectively.

    The final results are what we get at the top of this post.

  • From the Labs of SQL Prompt

    I love SQL Prompt as an add-in for SSMS. The intellisense is very handy for me and I’ve gotten used to certain shortcut combinations that make it easy for me to write T-SQL quickly and get information on parameters without opening Books Online. It’s works better than the native intellisense for me, though perhaps I’ve just gotten used to it. When it’s not installed on an instance in one of my VMs, writing code is a chore.

    When I was in Cambridge recently, I had the chance to sit down with one of the developers of SQL Prompt and he showed me a few things I had never seen.

    For the most part when I install SQL Prompt, I leave it with the defaults. There are a few snippets that I change quickly, like the ssf snippet. This normally produces a “SELECT * FROM” and I add a “TOP 10” to it in order to reduce the amount of data I bring back.

    However there are a few features in SQL Prompt that are “experimental” in nature. They are complete, but not deployed into the product by default. You can access them from the SQL Prompt menu in Management Studio.

    prompt3

    This brings up the Experimental Features tab in the options dialog, which you can see below. There aren’t a lot of features, but these are ideas that have been suggested, or are working, but they developers aren’t sure if they are completely spec’d out.

    prompt4

    You can enable a few of these to see if you really want to see how they work. For example, I’ve enabled “Automatic Refresh Suggestions”. Since I tend to work in one database at a time and create lots of objects, I want this to happen. I can ALT+S, Enter for this, but I’d like to tool to do it for me.

    These items change periodically, and some link to related tools (like SQL Tab Magic), and they give you a chance to test the way the feature works and provide feedback. If you are a SQL Prompt user, you might check out this tab.

    If you’d like to see what SQL Prompt can do for you, download a free trial and give it a try:

    14-day-free-trial

  • Rocky Mountain Tech Trifecta 2012

    My fourth visit to the Rocky Mountain Tech Trifecta will be tomorrow where I’ll deliver two talks:

    Branding Yourself for a Dream Job – 9:30am, Rm 1130

    Unstructured Data in SQL Server – 3:15, Rm 1402

    There’s still time to register if you’re in the Denver area, and you will get a full day of free training on SQL Server, .NET Mobile, Office, Kinect, Windows, and more. There are quite a few other professional development talks you can attend as well.

    Pass the word and visit the site for more information or to register.

  • Moving from NoSQL to an RDBMS

    I saw this link about a company that moved (Goodbye, CouchDB) from a NoSQL database (CouchDB) to a relational one (MySQL) because of some problems they experienced. Sauce Labs provides testing services in the cloud for developers using the Selenium framework.

    The first link is a blog post that talks about some of the things they initially liked about CouchDB and then the problems they experienced. Their move to MySQL showed better performance and stability over the CouchDB they were using. It’s an interesting read, not too long, and it seems well thought out.

    I think that this isn’t a knock on NoSQL databases, and I do think there are problem sets they are well suited to solve. I don’t know that I think because your developers don’t like SQL is a good reason, but there are problems with scale and size that are better handled with some NoSQL solutions. I’m an RDBMS person, but I don’t expect everyone to be that way.

    However this does say a few things about NoSQL maturity. Many of these products have gotten popular quickly, with some success stories, but that doesn’t meant they are necessarily ready for your application unless you have staff that have lots of experience with the tools. These are young platforms, they have problems inside the code, especially if you deviate from the way the authors used the platform, and you may have issues.

    Experience matters. Just as many people have issues with SQL Server scaling when they don’t know how to build a good data model or write efficient queries, I suspect that picking a NoSQL database because it’s cool, or seems easy to implement, or it worked well at some other company. I would almost always suggest that you stick with platforms that your staff is very experienced with, since they are likely going to have issues at scale on every platform, and their experience can make a difference.

    Note that this assumes you have good people. Just because you have a DBA with 10 years of experience doesn’t mean he or she is necessarily good. They could easily have 1 year of experience 10 times. You want people that have worked on a variety of systems in different areas.