Voice of the DBA

Author: way0utwest

  • Using multiple CTEs in one query

    This is a fairly simple thing to do, but I had to look it up the other day and thought it would make a good blog. This is a good example of a blog that everyone should write to show they know a particular skill.

    TL;DR: Use a comma between CTE definitions.

    Let’s imagine that you need to combine a few queries together and the code is complex. You decide to use a CTE to make things easier. Suppose I have this query:

    SELECT p.title
      , p.firstname
      , p.lastname
    FROM Person.BusinessEntity be
        INNER JOIN Person.Person p
        ON p.BusinessEntityID = be.BusinessEntityID
    WHERE ( SELECT COUNT(*)
                FROM Person.BusinessEntityAddress bea
                WHERE bea.BusinessEntityID = be.BusinessEntityID
          ) > 1
        AND ( SELECT COUNT(*)
                FROM Person.PersonPhone pp
                WHERE pp.BusinessEntityID = be.BusinessEntityID
            ) > 1

    That’s ugly, and I want to move the subqueries. I know this is slightly contrived, but imagine that what you really want is a couple views like: 

    

CREATE VIEW BEACount
AS
    SELECT bea.BusinessEntityID
          , cnt = COUNT(*)
        FROM Person.BusinessEntityAddress bea
        WHERE bea.BusinessEntityID = be.BusinessEntityID


CREATE VIEW PersonPhoneCount
AS
    SELECT bea.BusinessEntityID
          , cnt = COUNT(*)
        FROM Person.PersonPhone pp
        WHERE pp.BusinessEntityID = be.BusinessEntityID

    However you don’t want to create views, what you really want is to make both of these queries CTEs and then call them from the original query.

    I wasn’t sure how to do this, so I Googled. The first result was my friend, Pinal Dave’s blog. It took me about 10 seconds to read his explanation and apply it to my issue.

    WITH BEACountCTE
AS
(    SELECT bea.BusinessEntityID
          , cnt = COUNT(*)
        FROM Person.BusinessEntityAddress bea
     GROUP BY bea.BusinessEntityID
)
,
PersonPhoneCountCTE 
AS
(    SELECT pp.BusinessEntityID
          , cnt = COUNT(*)
        FROM Person.PersonPhone pp
     GROUP BY pp.BusinessEntityID
)
SELECT p.title
      , p.firstname
      , p.lastname
    FROM Person.BusinessEntity be
        INNER JOIN Person.Person p
        ON p.BusinessEntityID = be.BusinessEntityID
        INNER JOIN PersonPhoneCountCTE pp
        ON pp.BusinessEntityID = p.BusinessEntityID
        INNER JOIN BEACountCTE bea
        ON bea.BusinessEntityID = p.BusinessEntityID
    WHERE BEA.cnt > 1
        AND PP.cnt > 1

    Note that at the top I have one WITH statement and both of my CTEs are separated by commas.

    Easy enough, a quick thing to look up, and a handy item to know. If you are breaking up queries and using CTEs to make things easier to read, use a comma between your CTEs.

  • Software in 2014

    What’s the state of software in 2014? Here’s one set of thoughts from Tim Bray. He works for Google, and understandably has a bit of a slant towards the Google view of the world. Or maybe that’s his view and that’s why he works at Google. I am never quite sure how many people choose jobs or employers that fit them and how many get sucked into thinking a certain way because they fell into the job.

    In any case, I think that the state of software in 2014 hasn’t dramatically changed from the last decade. We’ve seen a push towards the idea of “apps” in mobile/tablet spaces, and now that’s coming into the desktop/laptop space as well. Even Google, with the Chromebooks and apps in their own browser store, has been pushing the idea of a store with a curated distribution of software. However I think it’s still a fad, with most people preferring desktop versions, or browser based versions, of the software they need. As users mature and become more savvy, I do think the distinction between how we get our software matters less and less.

    Are we writing better software? I see lots more experimentation, which is good. Some people love Javascript, I mean really love it. Others hate it, as Mr. Bray does. Flash seems to be fading, along the same way that Cold Fusion, Foxpro, and VB6 have gone away. I rarely hear about Java these days, though it seems that C, C++, Java, and Obj-C are still very popular. They don’t seem to have the media attention or excitement that I hear about with Python and Ruby, but there are lots of people still using them.

    Interestingly enough, T-SQL is one of the most popular languages. That’s good for us, though I suspect if we included PL/SQL and other variants, SQL might be one of the most used languages. It seems like most developers need to use some SQL, though surprisingly most don’t work on those skills. As Mr. Bray mentioned, I do think that relational databases are here to stay. They work well, and they will continue to be used, but we’ll also see other types of databases being incorporated more and more into applications. If for no other reasons than because developers get excited by them and want to try out new ideas.

    I can appreciate the view from a programmer used to having the power of thousands (or tens of thousands) of machines available that mobile hardware sucks. In comparison to what we have in a laptop, yes. However the ability to play a game, or edit a spreadsheet, or work with a visualization while walking down the sidewalk is stunning. We’ve just started to scratch the value of mobile, and while there are constraints (not to mention extra work to get your software onto all platforms), it’s also stunning to see how much computing can happen in the palm of my hand. I predict mobile is still exploding from its Big Bang. It’s up to technologists to work on solutions to the constraints in mobile, not complain about the frustrations of building software for mobile devices.

    I think software is continuing to evolve and change, and while it hasn’t changed dramatically (to me) in the last few years, it’s still one of the more exciting industries to be a part of.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • Uncontrolled Code

    Wow, Excel really sucks. To be fair, that isn’t just Excel, but any spreadsheet that’s used to make decisions in business (or government) and hasn’t had proper auditing or checking. Spreadsheets are amazingly powerful and useful, and so ubiquitous that Microsoft gave up on a separate BI tool and built one into Excel.

    I ran across this story on a serious Excel issue that cost a lot of money in the London Whale case. There are a lot of comments with the post as well as a discussion on Hacker News as well. I also found this piece on the amount of spreadsheet errors that exist in the world, based on a number of studies. It’s dry reading, and also scary.

    As I read through the thoughts and posts, I’m somewhat stunned that we don’t have many, many more business problems because of a spreadsheet being used as an application without any auditing. Perhaps we actually do and just don’t know about it. Perhaps the reason you didn’t get a bonus last year at your company was because of a spreadsheet error. There could have been either an error that caused a poor business decision, or one that incorrectly calculated bonus payouts.

    It’s a mess, and I honestly don’t know what to do. A more rigid structure in building applications that are checked, rechecked, audited that can prevent the miscopying of formulas is a great idea, but in the real world, we know it takes too long. Perhaps more time is a good thing and businesses should slow down, especially financial businesses, but I can’t see that happening either. Perhaps we need more tools that handle precedents and dependents. Personally, I’d like to see some VCS hooks built into Excel as well.

    The entire process of building applications with spreadsheets reminds me of a race to the bottom, where companies take more and more shortcuts and chances, just because they think other companies are doing the same thing. Ultimately I don’t think we can fix this, but we can try to make a difference by producing software quicker, and pointing out the errors in spreadsheets that become too important. We can meet somewhere in the middle between a full application development lifecycle and total ad hoc spreadsheet based tracking and processes.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

     

  • Powershell in a Month Day 11 – Filtering

    This is part of my Powershell Challenge, to learn more about PowerShell (PoSh) using the Learn Windows Powershell 3 in a Month of Lunches book by Don Jones.

    I’ve learned that the dash (-) is going to be my friend. Filtering and comparisons are the backbone of SQL. We use them all the time in our WHERE clauses. In Powershell, we have similar comparisons as I’ve seen, but almost always as I use the eq, ge, le, ne, etc. comparisons, there is the need to include a dash before them. Same thing for -and and -like (see I’m doing it).

    This chapter looks at filtering, and how you can reduce the information returned from cmdlets to just those items that you need. The idea of “filtering left”, meaning including the filter as soon as possible in the command. Since commands are interpreted and executed left to right, filtering left means less processing where possible. However since not all cmdlets include a -filter parameter, you must use the WHERE-OBJECT cmdlet.

    Overall, the idea of filtering and using comparisons is very much core to what we do in SQL and what happens in many programming algorithms as we look to make decisions in our applications. This chapter helps to understand this.

    I also learned about the “$_.” syntax, which is really  a way to specify a generic object and then specify a property for all instances of that object. So if I’m looking at the name for something, I want to use “$_.Name” syntax. I had wondered what that was for in the last chapter, but now this makes sense.

    The lab, however, was challenging. I want to use the WHERE syntax when I can do things easier. For example, getting a list of services whose names are “conhost” or “svchost”. I entered:

    PS C:\Windows\System32\WindowsPowerShell\v1.0> get-process | where {$_.Name -match "Conhost" -or $_.Name -match "Svchost"}

    Way too much overkill. This would do it.

    get-process -name conhost,svchost

    That’s an area where I need more practice and need to work on things. I suspect much of my initial Powershell work will be more complex, without filtering occurring in the most left point it could unless I manage to understand the details of various commands.