Author: way0utwest

  • The Tally Table and Expanding Rows

    Suppose you had some data like this:

    Class           Limit
    ————— ———–
    Calculus        5
    History         4
    Physics         2

    But what you really want is this:

    Class           Student
    ————— ———–
    Calculus       
    Calculus       
    Calculus       
    Calculus       
    Calculus       
    History        
    History        
    History        
    History        
    Physics        

    Physics        

    Where you have a placeholder for each student. This is a little contrived, but for the sake of the scenario, how do you actually expand the data in the first set to the second?

    An easy way is a cursor, but suppose you had a large school environment and you were doing this regularly, you might not want the performance hit of a cursor. Suppose you have some similar scenario, like slotting inventory or holding places for some large report? You would want a better solution.

    Setup

    Let’s first get a table ready for this.

    declare @Table table (
      Class varchar(15)
    , Limit int
    )
    ;
    
    insert into @Table 
      values ('Calculas',5)
           , ('History', 6)
           , ('Physics', 2)
    ;       
    

    A Tally Table

    I first saw this when Jeff Moden talked about this in his “The Numbers or Tally Table” article. It seemed like a neartidea, and he improved upon is in his next piece, finding a more efficient way to generate the data. An even more flexible idea that I like came from his Test Data Generator article, where he uses sys.columns to build a list as large as he would like. I found it to be a great idea, and this problem is a great way to apply it.

    Here’s a short look at the code that makes this work.

    SELECT TOP (4) RowNum = ROW_NUMBER() OVER(ORDER BY a.[Name])
     FROM sys.columns a, sys.columns b

    In this code, a cross join occurs between sys.columns and itself. With the addition of the ROW_NUMBER() function, and an order by, you can easily get a list of numbers. If I run this. I get:

    RowNum

    ——————–

    1

    2

    3

    4

    If I change my TOP value, I can get different rows:

    SELECT TOP (6) RowNum = ROW_NUMBER() OVER(ORDER BY a.[Name])
     FROM sys.columns a, sys.columns b

    RowNum

    ——————–

    1

    2

    3

    4

    5

    6

    To make this work for me in the original problem, I will use the limit value for my TOP in a subquery. However I can’t do a straight join, I need a CROSS APPLY, that will execute the right side of the CROSS apply for each input row from the left table.. It’s intended to be for table valued functions, but in this case, we’ll just return a table from the right side.

    The code looks like this:

    SELECT
      c.Class
    , 'Student' = ' '
    FROM (
    SELECT Class,Limit
    FROM @Table
    ) c
    CROSS APPLY (
    SELECT TOP (Limit) RowNum = ROW_NUMBER() OVER(ORDER BY a.[name])
    FROM sys.columns a, sys.columns b
    ) n

    What happens is that for each row, the derived table on the right side of the CROSS APPLY, returns a set of numbers which are then linked to the row on the left side of the CROSS APPLY.

    For the first row in @table, we have “Calculus” and “5”. When we apply this row to the subquery on the right side (SELECT TOP()…), we get these results:

    1

    2

    3

    4

    5

    When this is joined with the row and the c.Class returned, we end up with the same value being returned 5 times. That results in

    Calculus

    Calculus

    Calculus

    Calculus

    Calculus

    If this is repeated for each of the other rows, we get 4 and 2 rows from the right side of the CROSS APPLY returned, respectively.

    The final results are what we get at the top of this post.

  • The TCO of the Cloud

    Cloud computing is growing, and it makes sense for some applications.

    I know that many of you don’t want to put your databases in the cloud. There are definitely concerns for many applications that might have identity, financial, medical, or other data. However there are many other types of applications that might make sense for cloud computing, and I am sure that many of you will encounter pressure to move to the cloud at some point in the future.

    When you do, how do you determine if the cloud makes sense? Amazon has a nice whitepaper for NoSQL databases, which examines a scenario of varying traffic for a system whose usage peaks and then declines. I don’t know how many of us have applications that follow this pattern of traffic, but I think there are plenty of our systems that are over-provisioned and have hardware sitting idle over time. Virtualization helps recover some efficiency from hardware, but the study was interesting to me in showing that the majority of the costs were in administration, redundancy, and support costs (power, space, cooling).

    Whether or not you feel the assumptions made in the whitepaper are valid, there are some interesting ideas here that show where scale can be a huge benefit. Redundancy is very expensive for single systems, but when it is spread out over the large number of systems a cloud provider maintains, it’s a minor cost. The reduction in administration costs can also be a significant savings, especially when you consider how hard it can be for a company to scale up, or down, it’s staff to meet its needs.

    I still prefer the idea of a private cloud inside a company where database services are provided, not database instances. I hope that SQL Server moves in this direction, giving us the chance to choose to manage data and consume a database service, leaving the management of the underlying instances to a specialized group that prefers managing the hardware and platform.

    Steve Jones


    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • From the Labs of SQL Prompt

    I love SQL Prompt as an add-in for SSMS. The intellisense is very handy for me and I’ve gotten used to certain shortcut combinations that make it easy for me to write T-SQL quickly and get information on parameters without opening Books Online. It’s works better than the native intellisense for me, though perhaps I’ve just gotten used to it. When it’s not installed on an instance in one of my VMs, writing code is a chore.

    When I was in Cambridge recently, I had the chance to sit down with one of the developers of SQL Prompt and he showed me a few things I had never seen.

    For the most part when I install SQL Prompt, I leave it with the defaults. There are a few snippets that I change quickly, like the ssf snippet. This normally produces a “SELECT * FROM” and I add a “TOP 10” to it in order to reduce the amount of data I bring back.

    However there are a few features in SQL Prompt that are “experimental” in nature. They are complete, but not deployed into the product by default. You can access them from the SQL Prompt menu in Management Studio.

    prompt3

    This brings up the Experimental Features tab in the options dialog, which you can see below. There aren’t a lot of features, but these are ideas that have been suggested, or are working, but they developers aren’t sure if they are completely spec’d out.

    prompt4

    You can enable a few of these to see if you really want to see how they work. For example, I’ve enabled “Automatic Refresh Suggestions”. Since I tend to work in one database at a time and create lots of objects, I want this to happen. I can ALT+S, Enter for this, but I’d like to tool to do it for me.

    These items change periodically, and some link to related tools (like SQL Tab Magic), and they give you a chance to test the way the feature works and provide feedback. If you are a SQL Prompt user, you might check out this tab.

    If you’d like to see what SQL Prompt can do for you, download a free trial and give it a try:

    14-day-free-trial

  • Continuous Releases

    An actual graph of the features complete, but not released.

    One of the interesting things that I find with my iOS app is that they are updated fairly regularly. I have 20-30 apps, and I would guess that 3-4 of them are updated on any given week. Some of those are bug fixes, some of those are enhancements, but they are regularly changing. I don’t update them all every week, but I try not to wait too long between updates.

    Plenty of other software works like this. Windows Update provides fixed to various software on my desktop, I get a list of patches I can download when VideoStudio starts, and SQL Prompt lets me know about updates when I start SSMS. Those are all channels that allow a company to easily deliver patches to customers that improve the software. Usually these are patches, but sometimes there are improvements, like Experimental Features. In SQL Server, we have usually frowned on feature enhancements in patches, but is that what we really want?

    I noticed an interesting metric being tracked when I was in the UK. Development teams were looking at the time between when the completion of a feature (development and testing) and its release. It’s an interesting metric as there are substantial delays, which can be problematic. Users don’t get the chance to test the feature and give feedback. Developers move on to something new and may have trouble going back to enhance or patch the feature later. If there are helpful new features, and turn them on or off, wouldn’t you want to see them as soon as you can?

    Continuous Integration (CI) is a technique to help smooth the process of merging code from multiple developers, and the next step in smoothing the software life cycle is continuous release of software to get your changes in the hands of customers quicker. It’s not suited for everyone, testing needs to be more thorough, and there needs to be a smooth process that allows customers to pick and choose which updates they apply. However this can be a way to get feedback from customers, and even decide on future priorities as customers provide feedback.

    Steve Jones


    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.