Author: way0utwest

  • tSQLt Tests for Advent of Code 2017 Day 4

    This is day 4 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1. As I worked through the puzzles, I decided that I should be testing using their test sets and solving the issues that way. This lets me use the sample data, but also add in my own sets to cover strange situations.

    Here are the tests that I used for each part of day 4.

    Part I

    This is a fairly simple test. I’m returning a result set since the solution is a single query, but this is really a scalar. In this cas,e I’ll create a one row, one column expected table and then get the results from my solution (inside a proc) and insert into Actual.

    The rest is standard tSQLt testing framework. Fake a table, enter data.

    CREATE OR ALTER PROCEDURE tDay4.[test Day4a sample data]
    AS
    BEGIN
         -- Assemble
         EXEC tsqlt.FakeTable @TableName = N'Day4';
         
         INSERT dbo.Day4
    (
         passphrase
    )
    -- SQL Prompt formatting off
    VALUES
        ('aa bb cc dd ee' )
      , ('aa bb cc dd aa')
      , ('aa bb cc dd aaa')
    
    -- SQL Prompt formatting on
        CREATE TABLE #Expected (valid INT);
        INSERT #Expected
        ( valid)
        VALUES
        (1  );
        SELECT *
         INTO #actual
         FROM #Expected AS e
         WHERE 1 = 0;
    
    
         -- Act
       INSERT #actual
        EXEC dbo.Day4_a;
    
        -- Assert
         EXEC tsqlt.AssertEqualsTable @Expected = N'#Expected' ,
                                      @Actual = N'#Actual' ,
                                      @Message = N'Incorrect number of valid passphrases';
         
         
    END
    GO
    
    EXEC tsqlt.run 'tDay4.[test Day4a sample data]';

    Part II

    This is the same as part I, but I change the inputs and results.

    CREATE OR ALTER PROCEDURE tDay4.[test Day4b sample data]
     AS
     BEGIN
     -- Assemble
     EXEC tsqlt.FakeTable @TableName = N'Day4';
    
    INSERT dbo.Day4
     (
     passphrase
     )
     -- SQL Prompt formatting off
     VALUES
     ('abcde fghij' )
     , ('abcde xyz ecdab')
     , ('a ab abc abd abf abj')
     , ('iiii oiii ooii oooi oooo')
     , ('oiii ioii iioi iiio')
    
    -- SQL Prompt formatting on
     CREATE TABLE #Expected (valid INT);
     INSERT #Expected
     ( valid)
     VALUES
     (3  );
     SELECT *
     INTO #actual
     FROM #Expected AS e
     WHERE 1 = 0;
    
    -- Act
     INSERT #actual
     EXEC dbo.Day4_b;
    
    -- Assert
     EXEC tsqlt.AssertEqualsTable @Expected = N'#Expected' ,
     @Actual = N'#Actual' ,
     @Message = N'Incorrect number of valid passphrases';
    
    END
     GO
    
    EXEC tsqlt.run 'tDay4.[test Day4b sample data]';
  • Finding the Default Path with dbatools

    I really like the dbatools project. This is a series of PowerShell cmdlets that are built by the community and incredibly useful for migrations between SQL Servers, but also for various administrative actions. I have a short series on these items.

    It’s been awhile since I worked on any dbatools learning with holidays and travel. I find these cmdlets to be really handy, and if I had to manage a large estate of instances, they would be invaluable.

    I ran across Get-DbaDefaultPath on another blog, and thought this would be a handy little item to have. It is.

    I know the project changes, so I ran an update-module to get the latest items first. Then I tried the cmdlet. I ran a simple query against a local instance, and I quickly get the details, my instance and the Data, Log, Backup, and ErrorLog locations.

    2018-01-15 11_54_46-cmd - powershell (Admin)

    This is handy information, especially as I often have multiple instances (same or different machines) and I may want to make sure I don’t put a database on a small drive, or I need to find out where a backup is (or errorlog).

    The normal way of getting this information for me has been to right click the instance in SSMS, possibly connect first, get the properties, and look at the panels in the dialog. It works, but it’s slow.

    This is a much quicker way for me to find out paths, which  just makes admin easier. With tab completion, this will be the new way I find paths.

    The advantages of using this to gather paths, check sizes, and do some scripting to find files, copy them, make decisions about where to create databases, etc. are many. I can see this would be a great way to build scripts that include some decision making that adapts a simple process to new environments.

    If you haven’t tried dbatools, do it today. It’s a fantastic administration tool for your toolbelt.

  • How to Grow In Your Career

    This is a great thread on Twitter from Jim Burger, a long time software developer. The thread is a distillation of how he keeps a passion for learning across decades. I think it’s a good list of items, seven of them in total. I’ll give a quick summary, but he has a bit more depth in each tweet. The seven are:

    • Invest in your brain
    • Embrace exploration
    • Find the fun in fundamentals
    • Mentors from from all kinds of places
    • Use the community as an opportunity
    • You don’t have to finish learning
    • Ask people to ask you questions

    This is a great set of rules to help you learn. I certainly believe in the first one and that’s one reason I preach for you to keep learning. You can always get a little better and grow your skills. Apart from the fun, it helps train your brain to keep growing and stay flexible. I also think that finding the fun in fundamentals is great. I appreciate the simple things I accomplish, and I like writing about those. Part of the reason I keep doing #sqlnewblogger posts.

    I also think the community is a huge help. Someone asks a question, and even if it gets answered before you can, use this as an opportunity to solve the problem. Answer a question in our forums. Lots of people learn by helping others solve problems. In fact, plenty of my learning has come from solving problems for others. I get blog fodder, test myself, and solidify my own understanding of how things work.

    If you’ve found a way to continue to have fun learning, let us know today. I know that I tend to do a little but of everything, but certainly I take pride in what works, and even what doesn’t. The latter is always an opportunity for me to learn a bit more.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Advent of Code 2017 Day 4–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

    This is day 4 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1.

    This is Day 4, which deals with checking a series of passphrases for duplicates. The idea is that for some input, each of the “words” in the string must be unique. This means that if I have a phrase of “dog cat horse”, it’s valid. If I have “dog cat bird cat”, it’s invalid.

    This feels like an easy thing to do in SQL. Let’s begin.

    First, I’ll build a table and load some data.

    CREATE TABLE Day4
    (
         passphrase VARCHAR(100)
    )
    GO
    INSERT dbo.Day4
    (
         passphrase
    )
    VALUES
        ('aa bb cc dd ee' )
      , ('aa bb cc dd aa')
      , ('aa bb cc dd aaa')

    This is the test set. In this set, there are two values that are valid and one that isn’t. I need to figure out which is which.

    This seems like a perfect place to use String_Split(). Since this is a small set (512 rows), performance isn’t a big concern. If it were, I’d be moving to use Jeff Moden’s string splitter.

    For my tests, I decided to try something quick and dirty. I built a quick inline TVF to return string_split values. This is the code:

    CREATE FUNCTION dbo.words
         (@input AS VARCHAR(100))
    RETURNS TABLE
    AS
    RETURN ( SELECT value FROM STRING_SPLIT(@input, ' ') AS ss )

    This made things easier for me to read. I like seeing the code clean, and this way I could easily most to a custom version of Jeff’s code if I wanted.

    I quickly tried a short CTE with a CROSS APPLY to get the sets of rows that didn’t have unique passphrases. I thought this was easier since I know these are incorrect. If I look for corrects, it’s a harder issue as I’ll have counts of 1 in both correct and incorrect strings.

    WITH ctePass
    AS
    (
    SELECT d.passphrase 
            , ss.value, 
            cnt = COUNT(*)
    FROM day4 AS d
         CROSS APPLY dbo.words(d.passphrase) AS ss
    --WHERE d.passphrase = 'aa bb cc dd ee'
    GROUP BY d.passphrase, ss.value 
    HAVING COUNT(*) > 1
    )

    I also added a CTE that just gets me the total count.

    , cteCount
    AS
    (SELECT total = COUNT(*) FROM day4)

    Next, I added a CTE that would get my unique passphrases from the first CTE.

    , cteUnique
    AS
    (
    SELECT ctePass.passphrase
    , cnt = COUNT(cnt) 
      FROM ctePass
      GROUP BY ctePass.passphrase
      )

    Finally, I subtract the invalids from the total to get the vlaids. This gave me the answer that solved the puzzle.

    SELECT total - COUNT(*) FROM cteUnique, cteCount
    GROUP BY cteCount.total

    Now let’s load the actual data. I cut and pasted data into a file. Let’s load that.

    TRUNCATE TABLE day4
    GO
    BULK INSERT dbo.Day4 FROM 'e:\Documents\GitHub\AdventofCode\2017\Day4\Day4_Input.txt'
    GO
    SELECT 
      *
      FROM dbo.Day4 AS d

    I can see I have 512 rows in my table, so my result must be between 1 and 512.  When I ran this, I got the correct result.

    A slight admission here. I misread at first, so I didn’t have the total and just got the 57. When that didn’t work, I re-read things and realized this. I tried to check for the valids, but realized that it was simpler if I just subtracted.

    Part II

    Part II is tricky, and I made a mistake the first time I tried to solve this. It took a bit, but I eventually figured out the issue.

    In this part, we have to avoid anagrams. That means if the phrase as ‘car’ and ‘rac’, it’s invalid. Getting anagrams is tricky, and I looked around to find a post on SO that covers this. I adapted that code from Pawel Dyl to search for anagrams.

    My first task was to put this into a function:

    CREATE OR ALTER FUNCTION dbo.CheckAnagram
         (@w1 VARCHAR(50)
         , @w2 VARCHAR(50)
    )
    RETURNS int
    AS
    begin
    
    declare @r INT;
    
    WITH Src
    AS
    (
    SELECT T.W1 ,
            T.W2
    FROM
    (
         VALUES
             (@w1, @w2)
    ) AS T (W1, W2)
    ) ,
          Numbered
    AS
    (
    SELECT Src.W1 ,
            Src.W2 ,
            Num = ROW_NUMBER() OVER (ORDER BY (SELECT 1))
    FROM Src
    ) ,
          Splitted
    AS
    (
    SELECT Num ,
            Word1 = W1 ,
            Word2 = W2 ,
            L1 = LEFT(W1, 1) ,
            L2 = LEFT(W2, 1) ,
            W1 = SUBSTRING(W1, 2, LEN(W1)) ,
            W2 = SUBSTRING(W2, 2, LEN(W2))
    FROM Numbered
    UNION ALL
    SELECT Num ,
            Word1 ,
            Word2 ,
            L1 = LEFT(W1, 1) ,
            L2 = LEFT(W2, 1) ,
            W1 = SUBSTRING(W1, 2, LEN(W1)) ,
            W2 = SUBSTRING(W2, 2, LEN(W2))
    FROM Splitted
    WHERE LEN(W1) > 0
           AND LEN(W2) > 0
    ) ,
          SplitOrdered
    AS
    (
    SELECT Splitted.Num ,
            Splitted.Word1 ,
            Splitted.Word2 ,
            Splitted.L1 ,
            Splitted.L2 ,
            Splitted.W1 ,
            Splitted.W2 ,
            LNum1 = ROW_NUMBER() OVER (PARTITION BY Num ORDER BY L1) ,
            LNum2 = ROW_NUMBER() OVER (PARTITION BY Num ORDER BY L2)
    FROM Splitted
    )
    SELECT @r =  CASE
                       WHEN COUNT(*) = LEN(S1.Word1)
                            AND COUNT(*) = LEN(S1.Word2) THEN
                           1
                       ELSE
                           0
                   END
    FROM SplitOrdered AS S1
         JOIN
         SplitOrdered AS S2
             ON S1.L1 = S2.L2
                AND S1.Num = S2.Num
                AND S1.LNum1 = S2.LNum2
    GROUP BY S1.Num ,
              S1.Word1 ,
              S1.Word2
    
    IF @r IS NULL 
      SET @r = 0
    RETURN @r
    end
    GO
    

    Next, I wanted to get a list of items to check. Again, STRING_SPLIT is what I used, but a Jeff’s code works as well. I decided to use another function, which will split the string and then do a cross join to run both combinations of works against each other to check for anagrams.

    CREATE OR ALTER FUNCTION dbo.SplitandCheck
    (@string AS VARCHAR(100))
    RETURNS int
    AS
    BEGIN
    DECLARE @r INT = 0;
    
    WITH mycte
    AS
    (
    SELECT checks = CASE WHEN a.value = b.value then 0
    ELSE dbo.CheckAnagram(a.value, b.value)
    end
    FROM STRING_SPLIT(@string, ' ') AS a CROSS JOIN STRING_SPLIT(@string, ' ') AS b
    )
    SELECT @r = SUM(a.checks)
    FROM mycte a
    

    This seemed to work, but when I got the final result with this code, it was wrong. Too high.

    WITH mycte
    AS
    (
    SELECT d.passphrase, IsAnagram = dbo.SplitandCheck(d.passphrase)
    FROM dbo.Day4 AS d
    )
    SELECT mycte.IsAnagram, COUNT(*)
    FROM mycte
    --  WHERE mycte.IsAnagram = 0
    GROUP BY mycte.IsAnagram
    

    I had checked for 0s, but also included other results to try and debug my code. As I went through here, I realized that some fo the input data had a string like “oicgs rrol zvnbna rrol”. Clearly “rrol is an anagram of “rrol”. Initially I was knocking those out as a cross join includes those anyway.

    As a result, I added this code to my function.

    IF @r = 0
    AND EXISTS(
    SELECT COUNT(*)
    FROM STRING_SPLIT(@string, ' ') AS ss
    GROUP BY ss.value
    HAVING COUNT(*) > 1
    )
    SET @r = 1
    

    This will check for duplicate values. If there are dups, then clearly we have an issue already.

    This give me a set of items with 1 dup as well as those with anagrams. The 0 result is the count of valid passphrases.