Category: Blog

  • Parsing SQL Saturday Data – Looping Through And Loading All XML Files

    After my last post on parsing the XML, I decided to continue forward and get ready to put the data in a database. For that, I’m really looking for this data:

    • event ID
    • session title

    With this, I can easily insert data into a table. I’ll have separate tables for the events themselves and the speakers, but for now, I can easily showcase the titles of the sessions.

    With that in mind, I decided to start expanding my efforts and building a series of loops that get all the data from the XML documents.

    Looping through all files

    The first thing I needed to do was loop through all the files I’d downloaded and get the documents loaded. I decided to use a DO loop for this, since I should be doing this at least once each time. Eventually I’ll add logic to avoid downloading files I’ve downloaded already.

    Here’s the basic code:

    $loop = 1

    $loopend = 450
    $doc = New-Object System.Xml.XmlDocument

    do {
    #start large loop

      # get the filename
      $sourceURL = $baseURL + $i + ".xml"

      # do other stuff

      $i++

    #end outer loop
    } while ($i -lt $loopend)

    This is the basis for looping through all the file names, based on my downloads. A quick test shows this is building all the filenames I need.

    Loading files

    The next step is to actually load each XML file in and start querying it. I changed from the parsing code to use a loop since I’ll need to insert each item separately and I don’t think the code I had from the previous article will work. At least, I haven’t found a way.

    If you know of one, let me know.

    I used the Test-Path method to be sure that the XML exists, as there was at least one lost event in my initial download. I think that’s fixed now, but in any case, I added this code:

    #test the path first. If it exists, load the XML
    if (Test-Path $sourceURL) {
      $doc.Load($sourceURL

    #trap the event number. This will be the ID I use in the database table.
        $event = "SQL Saturday #" + $i

    That seems to work fine, and with with $event variable, I know which event the sessions are associated with.

    Next Steps

    That’s all I wanted to put here, giving me a nice, simple way of going through a series of files in a pattern. From here I’ll add more detail to the inner loop that gets the session titles out of the XML document and displays it.

  • The Vacation Conundrum

    Would I want unlimited time off? It’s an interesting question. I think I might like something more like a minimum time off, or perhaps an allowance, with the tracking and offer to work with employees that need more. There are problems with unlimited vacation, and it’s often because of poor management and social pressure of colleagues.

    Or, of course, workaholic syndrome, which is common in the technology industry.

    Really, I’ve never had an issue with technology, personally. I work mostly the stated hours, but because I’m productive, I never have had problems with managers if I come late or leave early. To be fair, that’s my experience, and if everyone acted as I did, perhaps that would be problematic.

    At Red Gate, I’ve been able to take days off for something, usually skiing, as I want. Actually, I usually only take a partial day off, and have to work part of the day to be sure the SQLServerCentral newsletter is prepped and going out. Or answering email, dealing with site issues, etc. I’ve even taken meetings on the road, or halfway to the mountains, pulling over to chat.

    The last few years I’ve not wanted to deal with that and typically book a day off in our HR system and plan to be away. If things break, I ignore them. Or someone else needs to be ready to handle them since I’ve taken vacation and I’m not going to be pushed into working when I have booked time off.

    The downside of this, which is more specific to my job, is that I have daily things that need handling, like the SSC site. As a result, if I take a day off, that usually means I’m putting in more time before the holiday to prep things. I’ve never done that as a DBA or developer. I had deadlines, but I didn’t work extra because of vacation, unless I had put myself behind.

    If someone else sends an unrealistic schedule, I’m not killing myself, or skipping vacation.

    It’s a tough question, and I need to think more about it. The pieces linked above are interesting and worth reading.

  • Desktop Repairs – Power Supply Issues

    My desktop didn’t restart after vacation a few weeks ago. That wasn’t what I wanted to have happen with a lot of work to get done before some travel. Needless to say I wasn’t thrilled when it happened.

    Photo Jan 30, 8 35 47 AM

    I was worried that something major had died. However I’d replaced the main boot drive recently and I had the old one. If I could get power.

    I emailed Glenn Berry, who’s my go to hardware consultant. He suggested I check the power supply since I couldn’t get anything to happen. On his advice, I went to the local Microcenter and bought a new 650W power supply. Here’s the old one, with a stock CPU cooler. I bought a new one of those as well, since I find the heat alarm going off during some video editing.

    Photo Jan 30, 10 11 59 AM

    I didn’t remove the old power supply at first. I actually disconnected it and connected the new one to the motherboard from outside the case and powered things on. Sure enough, everything came on. Lights, drives, etc.

    I then powered things down again and set about replacing the power supply. It’s actually easy, and I’m surprised I’ve never had to do this before in my life. Probably because I’ve tended to replace machines so often.

    In any case, it was a matter of removing 4 or 5 screws from the back of the case, carefully slipping out the old PS and then putting the new one in there.

    The CPU cooler was trickier. The tabs push through the motherboard and hook, but they are somewhat fragile. I was worried, and sure enough, the first time they weren’t tight. The cooler needs to be held tight to the CPU to pull off heat. It was loose and I got heat alarms once I started using the machine.

    However I kept messing with it, slowly trying to get the tabs in there and eventually they are holding fairly tight. I still get heat alarms during heavy duty video processing, so I think this cheap $25 cooler isn’t good enough. Or I don’t have enough thermal grease. Either way, I’ll probably replace this soon.

    Here’s everything together.

    Photo Feb 13, 10 50 03 AM

    And a happy Steve.

    Photo Feb 13, 10 50 45 AM

  • tSQLt – SQLCop – Checking Naming Conventions

    I’ve been using tSQLt a bit to do some testing and one of the things I’ve tested is standards for code. I’ve been using a framework on top of tSQLt called SQLCop. These are a series of tests written to look for specific things. One of the items I do check is for sp_ named procedures. I’ve mostly gotten out of the habit of doing this, preferring spProcName, but at times I make a mistake in typing. This catches those simple errors.

    Using SQL Cop

    You can Download the SQLCop tests and install them in your database after you’ve setup tSQLt. If you are using SQL Test, then you also get the SQLCop tests installed when you add the framework to a database. For me, I see the tests in the SSMS plugin.

    tsqlt7

    There are a lot of tests, but in this piece, I’ll look at the Stored Procedures Named sp_ test.

    If I edit the test, I see it’s fairly simple code. I’ve included it here.

    USE [EncryptionPrimer]
    GO
    SET ANSI_NULLS ON
    GO
    SET QUOTED_IDENTIFIER ON
    GO
    ALTER PROCEDURE [SQLCop].[test Procedures Named SP_]
    AS
    BEGIN
    -- Written by George Mastros
    -- February 25, 2012
    -- http://sqlcop.lessthandot.com
    -- http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/don-t-start-your-procedures-with-sp_

    SET NOCOUNT ON

    Declare @Output VarChar(max)
    Set @Output = ''

    SELECT @Output = @Output + SPECIFIC_SCHEMA + '.' + SPECIFIC_NAME + Char(13) + Char(10)
    From INFORMATION_SCHEMA.ROUTINES
    Where SPECIFIC_NAME COLLATE SQL_LATIN1_GENERAL_CP1_CI_AI LIKE 'sp[_]%'
    And SPECIFIC_NAME COLLATE SQL_LATIN1_GENERAL_CP1_CI_AI NOT LIKE '%diagram%'
    AND ROUTINE_SCHEMA <> 'tSQLt'
    Order By SPECIFIC_SCHEMA,SPECIFIC_NAME

    If @Output > ''
    Begin
    Set @Output = Char(13) + Char(10)
    + 'For more information: '
    + 'http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/don-t-start-your-procedures-with-sp_'
    + Char(13) + Char(10)
    + Char(13) + Char(10)
    + @Output
    EXEC tSQLt.Fail @Output
    End
    END;

    This code looks at the meta data in the database for an routines, stored procedures, that start with sp_ as part of their name. If any results are returned from the query, the IF statement will be true and the @output will be returned as part of the tSQLt.Fail call.

    Using the Test

    Let’s write a stored procedure. If I do this:


    CREATE PROCEDURE spLetsTestThis
    AS
    BEGIN

    SELECT TOP 10
    e.EmployeeID
    , e.EmpTaxID
    , e.FirstName
    , e.lastname
    , e.lastfour
    , e.EmpIDSymKey
    , e.EmpIDASymKey
    , e.hashpartition
    FROM
    dbo.Employees AS e;

    RETURN 0;
    END;

    GO

    This is a simple procedure. I wrote it, execute it a few times and be sure it’s what I want. I’ve done basic testing, not let’s check it before I commit it to VCS.

    The easy way to execute all the SQLCop tests is to right click them in SQL Test and execute them. I can also use T-SQL to run tests. However since I just want to show this one, I’ll right click it and select "Run Test".

    tsqlt8

    This runs the test selected. I can also run an entire class, or all tests, but clicking in the right spot. In this case, the test passes and I see a green mark.

    tsqlt9

    Now let’s write a new procedure:

    CREATE PROCEDURE sp_GetArticles
    AS
    SELECT *
    FROM dbo.Articles

    GO

    This is a bad procedure for a variety of reasons, but let’s execute my test. I see it fail, and a red mark appears next to my test.

    tsqlt10

    In this case I also get a window from SQL Test popping up with more details. This contains the output from the test, which is also inserted into a table by the tSQLt framework.

    tsqlt11

    Note that there is a URL with more information on this particular test. That is a part of the SQL Cop test code above. I could easily replace this with something particular to my environment if I chose.

    At this point, I can rename the object, drop and recreate it, etc. to correct the issue. However running this test helps me to be sure I’ve gotten good code into the VCS. If I have this also run as a part of a CI process, it then prevents bad code from other developers appearing.

    Meeting Standards

    There are all sorts of SQLCop tests, and I’ll write about more, but this is an easy one to implement to prevent a bad practice in your coding by a team of developers. Allowing each developer to test themselves, as well as an overall check by some CI process means that our code quality improves.

    If I have other standards, I can even write my own tests to enforce them, which I’ll do in another piece.

    Downloads