Category: Blog

  • Off to DevOps East

    The travel countdown is down to 2 trips this year. The second to the last is this week, as I’m off to DevOps East, a conference in Orlando, FL that tries to bring together lots of DevOps experts and advocates to share ideas and learn more. This is my first time at this event, and I’m looking forward to talking DevOps for a few days.

    I’ll be in Orlando all week, and then I have a month off before one last trip for SQL in the City.

    If you’re at the conference, say hi. And certainly talk databases with everyone that you see.

  • ROLLing UP Totals–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

    I was editing an article recently that talked about ROLLUP, and I wanted to play with this a bit more. I hadn’t used this much in my career, but it’s a handy operator that’s worth including in your toolkit.

    ROLLUP is used to provide additional totals for your aggregates while using GROUP BY. Here’s a little example. I’ve got some data for American Football quaterbacks. In this case, I’ve extracted some stats for a few noteworthy players today. Here’s a sample:

    2017-10-16 13_05_05-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    I want to aggregate some data together. For example, let’s say that we want to look at the total touchdowns scored per year by these players in their first 5 years. That’s a simple query:

    SELECT
       Season = qs.CareerYear
       , TDs = SUM(qs.Touchdowns)
      FROM dbo.QBStats AS qs
      WHERE qs.CareerYear <= 5
    GROUP BY qs.CareerYear

    This gives me data like this:

    2017-10-16 13_06_28-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    However, when I look at this, I don’t have a total across all years. In my set, there are players that have played up to 18 seasons, and I want to know the aggregate number of TDs. I can get that by adding WITH ROLLUP. This is added

    2017-10-16 13_07_25-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    after the group by. However, I can also change this to GROUP BY ROLLUP and include the columns in parenthesis.

    SELECT
       Season = qs.CareerYear
       , TDs = SUM(qs.Touchdowns)
      FROM dbo.QBStats AS qs
      WHERE qs.CareerYear <= 5
      GROUP BY ROLLUP (qs.CareerYear)

    This is OK, but the NULL isn’t great. What can I do here? I can add an ISNULL or COALESCE to my query and get this:

    2017-10-16 13_10_04-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    Useful, but what about if I’m aggregating by individual players? I can add the player to the column list, but I also need to add the player to the ROLLUP (or GROUP BY) list as well. If I do that, I get this:

    2017-10-16 13_11_40-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    That’s quite a bit more data, and I now see NULL values in the Player name. What’s this?

    These are the totals for that subgroup. In this case, the first NULL above, next to the 1, is the total for all players for season 1. I can use another trick to clean this up:

    2017-10-16 13_14_33-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    This now shows some data in an easier to understand fashion, with some totals. We can see for season 1 that 3 players really started their careers in a strong fashion and 3 didn’t.

    At the end, however, we get a different result.

    2017-10-16 13_17_09-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    This is because we have handled the player name as a NULL for the second column, but not the season as a NULL. For this last row, out “Total” text comes from an ISNULL of the first column. Here we’d want to do this:

    2017-10-16 13_18_30-SQLQuery1.sql - (local)_SQL2016.NFLAnalysis (PLATO_Steve (74))_ - Microsoft SQL

    This is where COALESCE  comes in handy, allowing us to account for multiple NULL values.

    ROLLUP is a nice way to get totals for each of your GROUP BY columns. This can make some of your reporting easier, and perhaps even faster by having the totals calculated in the result set.

    SQLNewBlogger

    This was a fairly easy piece to write. It took longer to get data together and find a set that had two grouping elements and was interesting than write the queries. I also had to play with ROLLUP a bit to clean up the NULLs, which always made this seem like a useless operator.

    I think I’d look to use this more if I had the chance for many reports, as summing totals in some client tools is a pain.

  • DevOps Basics – Ignoring Files in Git

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. This is also a part of a basic series on git and how to use it.

    One of the things you’ll run into at times is the need to keep some scratch files, or extra files, in your Git repository, but not track them. One common type of file for me when working with SQL is a .zip file. I may zip up code to share or copy to a friend (without giving repo access).

    Ignoring files is easy in Git. We just add a .gitignore file. This is a list of files that the git repository will not track or show in status. Essentially, we see them in our file system, but git doesn’t.

    Creating .gitignore

    To create a .gitignore file, the easiest method for me is to just create a text file. I can do it like this:

    2017-10-09 16_36_08-cmd

    This gives me a new file. Certainly VS Code, Sublime, etc. will make this easy as well.  The format is simple, with a list of files and/or patterns to ignore. For example, I’ve got a .zip file in my repo.

    2017-10-09 16_34_24-GitTests

    I don’t want to see this, but I do right now:

    2017-10-09 16_37_31-cmd

    If I want to ignore this file, I’ll enter this in my .gitignore file:

    GitTests.zip

    If I want to ignore all zips, I’ll do this:

    2017-10-09 16_38_09-cmd

    This is a part of my repo, so I need to commit it.

    2017-10-09 16_38_32-cmd

    Now my status is clean.

    2017-10-09 16_39_14-cmd

    Generated .gitignore

    Some applications will generate a .gitignore. For example, my C# project gets this file from Visual Studio.

    2017-10-09 16_40_33-.gitignore — Visual Studio Code

    That’s a subset of files that are often in a VS project, but we don’t want to track in a VCS. Images, archives, executables, etc.

    You can customize this as you need, and it’s easy to just edit the text file and commit the changes.

    Hopefully this helps you understand how to best work with git and keep your repo clean. This also means your  git add –all is easy to use without adding unnecessary files.

  • Prepping for Summit 2017

    It’s about time for the PASS 2017 Summit. The event essentially starts on Monday with pre-cons and the unofficial networking dinner. Be sure you RSVP and come to the dinner if you don’t have plans. The official event is

    Many people are already traveling and packing for the event. I feel a bit behind as I won’t leave until late Monday afternoon and arrive late Monday night. I’m sure I’m not alone, but it seems like everyone’s ready and I’ve still got a day of work Monday.

    Redgate Software has a booth and a few presentations next week on Wednesday. We’d love to chat with you about ways to make database development easier, especially if you’re thinking DevOps. We also have some contests, swag and prizes.

    I’ll be around Tues, Wed, and Friday, and of course, moving from Game Night to the Redgate Party Thursday night. I’ve got other commitments Thursday day, but hope to see lots of you there.