Tag: syndicated

  • Advent of Code 2017 Day 4–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

    This is day 4 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1.

    This is Day 4, which deals with checking a series of passphrases for duplicates. The idea is that for some input, each of the “words” in the string must be unique. This means that if I have a phrase of “dog cat horse”, it’s valid. If I have “dog cat bird cat”, it’s invalid.

    This feels like an easy thing to do in SQL. Let’s begin.

    First, I’ll build a table and load some data.

    CREATE TABLE Day4
    (
         passphrase VARCHAR(100)
    )
    GO
    INSERT dbo.Day4
    (
         passphrase
    )
    VALUES
        ('aa bb cc dd ee' )
      , ('aa bb cc dd aa')
      , ('aa bb cc dd aaa')

    This is the test set. In this set, there are two values that are valid and one that isn’t. I need to figure out which is which.

    This seems like a perfect place to use String_Split(). Since this is a small set (512 rows), performance isn’t a big concern. If it were, I’d be moving to use Jeff Moden’s string splitter.

    For my tests, I decided to try something quick and dirty. I built a quick inline TVF to return string_split values. This is the code:

    CREATE FUNCTION dbo.words
         (@input AS VARCHAR(100))
    RETURNS TABLE
    AS
    RETURN ( SELECT value FROM STRING_SPLIT(@input, ' ') AS ss )

    This made things easier for me to read. I like seeing the code clean, and this way I could easily most to a custom version of Jeff’s code if I wanted.

    I quickly tried a short CTE with a CROSS APPLY to get the sets of rows that didn’t have unique passphrases. I thought this was easier since I know these are incorrect. If I look for corrects, it’s a harder issue as I’ll have counts of 1 in both correct and incorrect strings.

    WITH ctePass
    AS
    (
    SELECT d.passphrase 
            , ss.value, 
            cnt = COUNT(*)
    FROM day4 AS d
         CROSS APPLY dbo.words(d.passphrase) AS ss
    --WHERE d.passphrase = 'aa bb cc dd ee'
    GROUP BY d.passphrase, ss.value 
    HAVING COUNT(*) > 1
    )

    I also added a CTE that just gets me the total count.

    , cteCount
    AS
    (SELECT total = COUNT(*) FROM day4)

    Next, I added a CTE that would get my unique passphrases from the first CTE.

    , cteUnique
    AS
    (
    SELECT ctePass.passphrase
    , cnt = COUNT(cnt) 
      FROM ctePass
      GROUP BY ctePass.passphrase
      )

    Finally, I subtract the invalids from the total to get the vlaids. This gave me the answer that solved the puzzle.

    SELECT total - COUNT(*) FROM cteUnique, cteCount
    GROUP BY cteCount.total

    Now let’s load the actual data. I cut and pasted data into a file. Let’s load that.

    TRUNCATE TABLE day4
    GO
    BULK INSERT dbo.Day4 FROM 'e:\Documents\GitHub\AdventofCode\2017\Day4\Day4_Input.txt'
    GO
    SELECT 
      *
      FROM dbo.Day4 AS d

    I can see I have 512 rows in my table, so my result must be between 1 and 512.  When I ran this, I got the correct result.

    A slight admission here. I misread at first, so I didn’t have the total and just got the 57. When that didn’t work, I re-read things and realized this. I tried to check for the valids, but realized that it was simpler if I just subtracted.

    Part II

    Part II is tricky, and I made a mistake the first time I tried to solve this. It took a bit, but I eventually figured out the issue.

    In this part, we have to avoid anagrams. That means if the phrase as ‘car’ and ‘rac’, it’s invalid. Getting anagrams is tricky, and I looked around to find a post on SO that covers this. I adapted that code from Pawel Dyl to search for anagrams.

    My first task was to put this into a function:

    CREATE OR ALTER FUNCTION dbo.CheckAnagram
         (@w1 VARCHAR(50)
         , @w2 VARCHAR(50)
    )
    RETURNS int
    AS
    begin
    
    declare @r INT;
    
    WITH Src
    AS
    (
    SELECT T.W1 ,
            T.W2
    FROM
    (
         VALUES
             (@w1, @w2)
    ) AS T (W1, W2)
    ) ,
          Numbered
    AS
    (
    SELECT Src.W1 ,
            Src.W2 ,
            Num = ROW_NUMBER() OVER (ORDER BY (SELECT 1))
    FROM Src
    ) ,
          Splitted
    AS
    (
    SELECT Num ,
            Word1 = W1 ,
            Word2 = W2 ,
            L1 = LEFT(W1, 1) ,
            L2 = LEFT(W2, 1) ,
            W1 = SUBSTRING(W1, 2, LEN(W1)) ,
            W2 = SUBSTRING(W2, 2, LEN(W2))
    FROM Numbered
    UNION ALL
    SELECT Num ,
            Word1 ,
            Word2 ,
            L1 = LEFT(W1, 1) ,
            L2 = LEFT(W2, 1) ,
            W1 = SUBSTRING(W1, 2, LEN(W1)) ,
            W2 = SUBSTRING(W2, 2, LEN(W2))
    FROM Splitted
    WHERE LEN(W1) > 0
           AND LEN(W2) > 0
    ) ,
          SplitOrdered
    AS
    (
    SELECT Splitted.Num ,
            Splitted.Word1 ,
            Splitted.Word2 ,
            Splitted.L1 ,
            Splitted.L2 ,
            Splitted.W1 ,
            Splitted.W2 ,
            LNum1 = ROW_NUMBER() OVER (PARTITION BY Num ORDER BY L1) ,
            LNum2 = ROW_NUMBER() OVER (PARTITION BY Num ORDER BY L2)
    FROM Splitted
    )
    SELECT @r =  CASE
                       WHEN COUNT(*) = LEN(S1.Word1)
                            AND COUNT(*) = LEN(S1.Word2) THEN
                           1
                       ELSE
                           0
                   END
    FROM SplitOrdered AS S1
         JOIN
         SplitOrdered AS S2
             ON S1.L1 = S2.L2
                AND S1.Num = S2.Num
                AND S1.LNum1 = S2.LNum2
    GROUP BY S1.Num ,
              S1.Word1 ,
              S1.Word2
    
    IF @r IS NULL 
      SET @r = 0
    RETURN @r
    end
    GO
    

    Next, I wanted to get a list of items to check. Again, STRING_SPLIT is what I used, but a Jeff’s code works as well. I decided to use another function, which will split the string and then do a cross join to run both combinations of works against each other to check for anagrams.

    CREATE OR ALTER FUNCTION dbo.SplitandCheck
    (@string AS VARCHAR(100))
    RETURNS int
    AS
    BEGIN
    DECLARE @r INT = 0;
    
    WITH mycte
    AS
    (
    SELECT checks = CASE WHEN a.value = b.value then 0
    ELSE dbo.CheckAnagram(a.value, b.value)
    end
    FROM STRING_SPLIT(@string, ' ') AS a CROSS JOIN STRING_SPLIT(@string, ' ') AS b
    )
    SELECT @r = SUM(a.checks)
    FROM mycte a
    

    This seemed to work, but when I got the final result with this code, it was wrong. Too high.

    WITH mycte
    AS
    (
    SELECT d.passphrase, IsAnagram = dbo.SplitandCheck(d.passphrase)
    FROM dbo.Day4 AS d
    )
    SELECT mycte.IsAnagram, COUNT(*)
    FROM mycte
    --  WHERE mycte.IsAnagram = 0
    GROUP BY mycte.IsAnagram
    

    I had checked for 0s, but also included other results to try and debug my code. As I went through here, I realized that some fo the input data had a string like “oicgs rrol zvnbna rrol”. Clearly “rrol is an anagram of “rrol”. Initially I was knocking those out as a cross join includes those anyway.

    As a result, I added this code to my function.

    IF @r = 0
    AND EXISTS(
    SELECT COUNT(*)
    FROM STRING_SPLIT(@string, ' ') AS ss
    GROUP BY ss.value
    HAVING COUNT(*) > 1
    )
    SET @r = 1
    

    This will check for duplicate values. If there are dups, then clearly we have an issue already.

    This give me a set of items with 1 dup as well as those with anagrams. The 0 result is the count of valid passphrases.

  • SSMS Line Numbers

    I typically don’t turn on line numbers if SSMS, but while working with someone on a bit of code, they were referencing line numbers in a large script. By default SSMS shows you the line, column, and character at the bottom (as well as insert/overwrite status) in the status bar. See the image below, where my cursor is on line 597.

    2018-01-16 13_30_26-Semicolons.sql - [ReadOnly](local)_SQL2016.RLS (PLATO_Steve (64))_ - Microsoft

    However, it’s visually harder to see lines here if those are the way you’re getting oriented with a script. It’s actually easier to have the line numbers on the left side.

    This is easy to turn no in SSMS. First, click the Tools menu and choose options (as shown here).

    2018-01-16 12_28_26-

    Next, go down to the Text Editor section on the left, expand that and then expand the Transact-SQL area. Click General, and you’ll see a checkbox for Line numbers on the right. Click that.

    2018-01-16 12_28_38-Options

    And line numbers appear.

    2018-01-16 12_28_45-Semicolons.sql - [ReadOnly](local)_SQL2016.RLS (PLATO_Steve (64))_ - Microsoft

    I do find these distracting most of the time, but there are situations where the line numbers are handy, especially when collaborating with others.

  • Finding the Titles in R

    PASS has released the videos to members from this past Summit. I say TJay Belt today ask about relating a video name to a session. I have the USB drive, so I looked on there. Here are the videos:

    2018-01-10 13_25_47-Video

    Not terribly helpful. If you run the HTML file from the stick, you see this:

    2018-01-10 13_26_23-PASS Summit 2017

    If I hover over a title, I see the link as a specific video file. For example, the first one is 65545.mp4. With that, I looked around and found a javascript file with information in it.

    The structure was like this:

    //SID
    Col0[0] = "65073";
    Col0[1] = "65091";
    
    …
    
    //Speaker Name
    Col2[0] = "Steve Stedman";
    Col2[1] = "Kellyn Pot'Vin-Gorman";
    
    …
    
    //Session Name
    Col4[0] = "Your Backup and Recovery Strategy";
    Col4[1] = "DevOps Tool Combinations for Winning Agility";

    All the data is in one file, but the index in each array matches. So Col0[0] is the SID for video 65073, which has Col2[0] as the speaker and col4[0] as the title.

    Now I want to get these in some sort of order. First, let me copy this data into separate files. That will make importing easier. I’ll copy the SID array into one file, the speaker array into a second file and the title array into a third.

    This gives me data like the list above, but I need to clean that. This is easiest in Sublime, with a few replacements. I did

    • “COL[“ –> “”
    • “] = “ –> “,”
    • “;” –> “”

    This gives me a clean file that looks like this:

    2018-01-10 13_29_18-e__Documents_R_titles.txt - Sublime Text

    Working in R

    I almost started to move this into T-SQL and a table, but since I’ve been playing with R, I decided to see what I could do there. First, I know I need to load data, so I the first file into a data frame.

    session.index = read.csv("e:\\Documents\\R\\videosid.txt", sep=",")

    The column names aren’t great, so we’ll fix those:

     colnames(session.index) <- c("Index", "SessionSID")

    Now
    let’s get the other data.

    session.speaker = read.csv("e:\\Documents\\R\\passspeaker.txt", sep=",")
    > session.title = read.csv("e:\\Documents\\R\\titles.txt", sep=",") 
    > colnames(session.speaker) <- c("Index", "Speaker")
    > colnames(session.title) <- c("Index", "Title")
    

    I have three data frames. I want to combine them. Let’s do that. I’ll use the merge() function to do this. Since I’ve got common column names, I’ll use those.

    > pass.videos <- merge(session.index, session.title, by="Index")
    
    > pass.videos <- merge(pass.videos, session.speaker, by="Index")

    This gives me a data frame with the index, title, and speaker. Now I’ve got the data merged, let’s produce a file..

     write.table(pass.videos, file="e:\\Documents\\R\\passvideos.txt",sep=",")

    With that done, I can see I have a list of video numbers, titles, and speakers.

    "Index","SessionSID","Session","Speaker"
     "1",1,65091,"DevOps Tool Combinations for Winning Agility","Kellyn Pot'Vin-Gorman"
     "2",2,65092,"Oracle vs. SQL Server - The War of the Indices","Kellyn Pot'Vin-Gorman"
     "3",3,65112,"Make Power BI Your Own with the Power BI APIs","Steve Wake"

    I did something in R. Smile

  • Reading GDPR

    In case you’re interested, the GDPR law is actually not bad to read. You might be affected by this, so go through the regulations. I’m doing that this week.

    You can also see a nice article from David Poole at SQLServerCentral.

    I do think GDPR will affect many of us, but to what extent, I’m not sure. Comments on what you think welcome.