Author: way0utwest

  • Are You Always Up?

    I’ve had the opportunity to work for and with quite a few companies in my career. In some sense working as a database developer or database administratrator is very similar at many places. The “job’s the job,” as I’ve said in more than a few interviews, as I often ask to meet with others and better understand the environment. I’m rarely sold on the technology in use, usually more concerned with management and co-workers, though you might view employment differently.

    At the same time, it can be amazing how different the environment may be and how varied the requirements are for my application and systems. One employer was a small, but busy and growing import/export company. We stretched the limits of technology at the time, and it was a challenge to meet the desires of management. At the same time, we were primarily a weekday, roughly 9-5 shop and I had the opportunity to bring down systems at night or on weekends for maintenance.

    This was in contrast to my prior position at a nuclear power plant. There systems were expected to be up 24×7, with (grudgingly) scheduled maintenance once a quarter. That was the environment with the most stringent requirements, though I also learned over time that I preferred to have a bit more latitude in how I could architect systems or manage databases. Though that was when I had children, and I might be willing to work in a similar environment now.

    Many of you probably have varying requirements by application, but I suspect more of you are expected to keep systems up more often than in the past. It seems the world has moved to a greater dependency on database platforms, with the expectation that the data is always available.

    When you plan upgrades or maintenance, can you work with a flexible schedule? Or are changes always carefully managed? I’m sure some of you prefer one type of environment over the other, so chime in today with your preferences and restrictions. I’m sure some of you work in less busy organizations, and I’m curious if that’s the case.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.7MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Advent of Code 2017 Day 5–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

    This is day 5 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1. This is going to be a crazy looping item, since it will move through the list, relative to the current spot, and incrementing items, I know this won’t be good in SQL.

    Still. Worth solving.

    Let’s load the data. I’ll use a table, but first, I’ll also add an identity. This will help me number instructions and figure out what the next one is.

    CREATE TABLE Day5
    ( InstructionKey INT IDENTITY(1,1)
    , Instruction INT)
    GO

    There are issues with identities, but this is a great trick:

    CREATE VIEW Day5V
    AS
    SELECT d.Instruction FROM dbo.Day5 AS d
    GO
    -- reusable code
    BULK INSERT Day5V FROM 'e:\Documents\GitHub\AdventofCode\2017\Day5\Input.txt' WITH (ROWTERMINATOR='\n')
    GO

    Now I can get to work. Here’s the logic I used.

    I wanted to first set some starting points. I have a counter (0 based, increment first). This determines how many times I jump around. I also need to track the current instruction key and the next key. And, of course, I need the instruction value.

    The identity is the array index, or the instruction key (which place am I in). In this case, I’ll try to follow this logic.

    Get the end (out of bounds, which is the max + 1). I loop until I get an jump outside of the end range. The loop does these items:

    • Get the current instruction jump
    • Set the next location to be the current key + the current jump
    • Update the current jump to increment by 1
    • Set the current instruction key to the next key
    • loop

    This seems to be what I need. On the test set, this worked fine. When I first set this up, I used this code:

    DECLARE @end INT ,
             @CurrentInstructionKey INT = 1 ,
             @Instruction INT ,
             @NextInstructionKey INT ,
             @counter INT = 0;
    SELECT @end = MAX(InstructionKey) + 1
    FROM dbo.Day5 AS d;
    
    -- SELECT [end] = @end;
    
    WHILE @CurrentInstructionKey < @end
    BEGIN
         SET @counter = @counter + 1;
         SELECT @Instruction = Instruction
         FROM Day5
         WHERE InstructionKey = @CurrentInstructionKey;
         SELECT @NextInstructionKey = @CurrentInstructionKey + @Instruction;
         UPDATE dbo.Day5
         SET Instruction = Instruction + 1
         WHERE InstructionKey = @CurrentInstructionKey;
         SET @CurrentInstructionKey = @NextInstructionKey;
    --PRINT @CurrentInstruction
    END;
    
    SELECT Counter = @counter ,
            [current] = @CurrentInstruction;

    When I ran this, it chugged for some time. I bet in Python or C#, which would solve quickly with arrays. With updates, it’s slow. Like minutes slow for 1074 rows.

    However, it worked.

    Part II

    In this part, this instructions are almost the same, but based on the current instruction value, we either increase or decrease the value. Not a big change. Our new update looks like:

    UPDATE dbo.Day5
    SET Instruction = Instruction + CASE
                                         WHEN @Instruction >= 3 THEN
                                             -1
                                         ELSE
                                             1
                                     END
    WHERE InstructionKey = @CurrentInstructionKey;

    This also works, albeit slowly. I left this around 5:30 and went to the gym.

    One of the easier puzzles.

  • Are You Patched?

    It’s been a few weeks since the Spectre/Meltdown bugs were announced for most CPUs. Microsoft has been working hard to build patches, and they’ve provided fixes for Windows and SQL Server. Other manufacturers have released fixes for other platforms, though I wouldn’t be surprised if more patches are coming. We put together a page at SQLServerCentral with information and links, and if you haven’t checked it out, you should.

    If you haven’t patched systems, patch them ASAP.

    This is a bad bug, affecting many CPUs, across multiple architectures, and includes potential issues with virtual machines. The guidance and conversations I’ve heard from various vendors is that many of them aren’t completely sure of all the potential risks or attack vectors, but they are worried that customers will leave this vulnerabilities open in the future. Since this affects hardware, it’s entirely possible that an exploit could read memory from other applications and processes.

    Again, if you haven’t patched systems, patch them.

    There are reports of potential issues, so everyone certainly needs to test systems. Perform a P->V (Physical to virtual conversion) and patch a VM. Make sure the server still runs. If you’re on VMs, snap a copy and patch it as a test. Older processors might see a performance penalty with the patch, but worse performance is better than having a security hole in your CPU available to operating systems.

    This is the type of fundamental architectural bug that’s is very worrisome. The race to be efficient, to copy what works from others, this leads to less innovation not more. I hope that this is a bit of a lesson that we do need separate architectures and approaches to computing problems, both in hardware and software. I love relational databases, but I’m glad that there are other types of systems being used for data storage. I think Windows works really well, but I like competition and think it’s good that we have MacOS, Linux, and more.

    It’s good to have standards and interoperability, but I do think that a heterogeneous environment is good for security, and I hope the world continues to try new architectures as we advance computing ever further.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • tSQLt tests for Day 5 Advent of Code 2017

    This is day 5 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1.

    As I worked through the puzzles, I decided that I should be testing using their test sets and solving the issues that way. This lets me use the sample data, but also add in my own sets to cover strange situations.

    Here are the tests that I used for each part.

    Part I

    For part 1, only a short test is needed. Since we’re looking for a scalar value, I could easily just added an INT variable for the actual and expected values. I set this to the value given in the problem.

    Then I fake the table and insert the test set. From here, I can call my proc that implements the algorithm and get the result value.

    EXEC tsqlt.NewTestClass @ClassName = N'tDay5';
     GO
     CREATE OR ALTER PROCEDURE tDay5.[test day5 a initial set]
     AS
     BEGIN
    
    -- Assemble
     DECLARE @actual INT = 0, @expected INT = 5;
    
    EXEC tsqlt.FakeTable @TableName = N'Day5' , @Identity = 1
     INSERT Day5
     VALUES
     (0 ), (3), (0), (1), (-3);
    
    -- Act
     EXEC @actual = SolveDay5a;
    
    -- Assert
     EXEC tsqlt.AssertEquals @Expected = @expected, @Actual = @actual, @Message = N'Failed to account'
    
    END

    Part II

    The test is the same, just calling a different procedure that implements the part II algorithm.

    CREATE OR ALTER PROCEDURE tDay5.[test day5 initial set]
     AS
     BEGIN
    
    -- Assemble
     DECLARE @actual INT = 0, @expected INT = 10;
    
    EXEC tsqlt.FakeTable @TableName = N'Day5' , @Identity = 1
     INSERT Day5
     VALUES
     (0 ), (3), (0), (1), (-3);
    
    -- Act
     EXEC @actual = SolveDay5b;
    
    -- Assert
     EXEC tsqlt.AssertEquals @Expected = @expected, @Actual = @actual, @Message = N'Failed to account'
    
    END