Category: Blog

  • tSQLt – SQLCop – Checking Naming Conventions

    I’ve been using tSQLt a bit to do some testing and one of the things I’ve tested is standards for code. I’ve been using a framework on top of tSQLt called SQLCop. These are a series of tests written to look for specific things. One of the items I do check is for sp_ named procedures. I’ve mostly gotten out of the habit of doing this, preferring spProcName, but at times I make a mistake in typing. This catches those simple errors.

    Using SQL Cop

    You can Download the SQLCop tests and install them in your database after you’ve setup tSQLt. If you are using SQL Test, then you also get the SQLCop tests installed when you add the framework to a database. For me, I see the tests in the SSMS plugin.

    tsqlt7

    There are a lot of tests, but in this piece, I’ll look at the Stored Procedures Named sp_ test.

    If I edit the test, I see it’s fairly simple code. I’ve included it here.

    USE [EncryptionPrimer]
    GO
    SET ANSI_NULLS ON
    GO
    SET QUOTED_IDENTIFIER ON
    GO
    ALTER PROCEDURE [SQLCop].[test Procedures Named SP_]
    AS
    BEGIN
    -- Written by George Mastros
    -- February 25, 2012
    -- http://sqlcop.lessthandot.com
    -- http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/don-t-start-your-procedures-with-sp_

    SET NOCOUNT ON

    Declare @Output VarChar(max)
    Set @Output = ''

    SELECT @Output = @Output + SPECIFIC_SCHEMA + '.' + SPECIFIC_NAME + Char(13) + Char(10)
    From INFORMATION_SCHEMA.ROUTINES
    Where SPECIFIC_NAME COLLATE SQL_LATIN1_GENERAL_CP1_CI_AI LIKE 'sp[_]%'
    And SPECIFIC_NAME COLLATE SQL_LATIN1_GENERAL_CP1_CI_AI NOT LIKE '%diagram%'
    AND ROUTINE_SCHEMA <> 'tSQLt'
    Order By SPECIFIC_SCHEMA,SPECIFIC_NAME

    If @Output > ''
    Begin
    Set @Output = Char(13) + Char(10)
    + 'For more information: '
    + 'http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/don-t-start-your-procedures-with-sp_'
    + Char(13) + Char(10)
    + Char(13) + Char(10)
    + @Output
    EXEC tSQLt.Fail @Output
    End
    END;

    This code looks at the meta data in the database for an routines, stored procedures, that start with sp_ as part of their name. If any results are returned from the query, the IF statement will be true and the @output will be returned as part of the tSQLt.Fail call.

    Using the Test

    Let’s write a stored procedure. If I do this:


    CREATE PROCEDURE spLetsTestThis
    AS
    BEGIN

    SELECT TOP 10
    e.EmployeeID
    , e.EmpTaxID
    , e.FirstName
    , e.lastname
    , e.lastfour
    , e.EmpIDSymKey
    , e.EmpIDASymKey
    , e.hashpartition
    FROM
    dbo.Employees AS e;

    RETURN 0;
    END;

    GO

    This is a simple procedure. I wrote it, execute it a few times and be sure it’s what I want. I’ve done basic testing, not let’s check it before I commit it to VCS.

    The easy way to execute all the SQLCop tests is to right click them in SQL Test and execute them. I can also use T-SQL to run tests. However since I just want to show this one, I’ll right click it and select "Run Test".

    tsqlt8

    This runs the test selected. I can also run an entire class, or all tests, but clicking in the right spot. In this case, the test passes and I see a green mark.

    tsqlt9

    Now let’s write a new procedure:

    CREATE PROCEDURE sp_GetArticles
    AS
    SELECT *
    FROM dbo.Articles

    GO

    This is a bad procedure for a variety of reasons, but let’s execute my test. I see it fail, and a red mark appears next to my test.

    tsqlt10

    In this case I also get a window from SQL Test popping up with more details. This contains the output from the test, which is also inserted into a table by the tSQLt framework.

    tsqlt11

    Note that there is a URL with more information on this particular test. That is a part of the SQL Cop test code above. I could easily replace this with something particular to my environment if I chose.

    At this point, I can rename the object, drop and recreate it, etc. to correct the issue. However running this test helps me to be sure I’ve gotten good code into the VCS. If I have this also run as a part of a CI process, it then prevents bad code from other developers appearing.

    Meeting Standards

    There are all sorts of SQLCop tests, and I’ll write about more, but this is an easy one to implement to prevent a bad practice in your coding by a team of developers. Allowing each developer to test themselves, as well as an overall check by some CI process means that our code quality improves.

    If I have other standards, I can even write my own tests to enforce them, which I’ll do in another piece.

    Downloads

  • Detecting Encryption

    I ran across an article recently from MSSQLTips by my friend, Brian Kelley. It talks about the ways you can detect encryption in use in your database. Brian’s approach, which is one I agree with, is that you can look for symmetric keys, asymmetric keys, and certificates in the system tables. The tables you query are:

    • sys.symmetric_keys
    • sys.asymmetric_keys
    • sys.certificates

    That’s a good way to detect SQL Server encryption in use, but not encryption in general. One of the things I’ve advocated for applications that contain sensitive data and need to be protected from the DBA is to have the application create temporary keys or use .NET libraries to encrypt data. In that case, SQL Server just sees data, and doesn’t detect encryption.

    Brian offers a solution that is to examine any columns containing these data types

    • binary
    • varbinary
    • image

    That’s a good start, but how do you detect that this string is encrypted?

    504b 0304 1400 0000 0800 1a86 4640 0d41 …

    That’s actually not encrypted; it’s the start of a zip file. However it could be a jpg, a tiff or some other binary format. The only way I thought of was mirrored in this Stack Overflow note: you’d have to compare known file types and look for a pattern in a header of some sort that doesn’t match. It wouldn’t be sure you didn’t have encryption, but you might make some educated guess if no file type that might fit the data matches.

    There was also a link in the comments to a Stack Exchange discussion on the same topic. It’s similar, though I saw the use of the KEY_NAME() function in there. I hadn’t used it, perhaps because of the poor documentation of encryption in SQL Server. I also found a KEY_ID() function that works similarly, returning the ID for the name of a key.

  • T-SQL Tuesday #63 – Security

    tsqltuesdayIt’s T-SQL Tuesday time again and this month we look at security. Kenneth Fisher has chosen this as his topic for February and you can read his invite here. There are lots of choices on what you write about, and I’m looking forward to reading what people choose.

    You can join in, by writing a post today and publishing it with a link in Kenneth’s invite. Or you can write later and just put your own thoughts down on the subject.

    T-SQL Tuesday is the idea of Adam Machanic (b/t) , and it’s a monthly party where everyone writes on a specific topic. The first Tuesday of the month usually has a new invitation issues, and you have to watch for it. I’d recommend putting a reminder in your calendar. The second Tuesday of the month is when we publish posts.

    If you’d like to host, contact Adam.

    Security Across Environments

    At one point in my career, I worked with a startup company. We had a number of experienced people working in development, and we wanted to set up a series of environments early on to perform agile development. When I arrived, the application we built had been running for about 4 months, and we were looking to improve our data handling and development processes.

    At the time, we had a production server and a development server. There were accounts for the web application and the initial security had been to grant security on tables as appropriate for the web application. Any tables that existed for administrative use were limited to sysadmin access.

    This wasn’t a bad plan, but as we implemented a test environment, I knew this would be an issue. We didn’t want to give testers (or developers), access to the production AD account that was used by the web application. We also didn’t want any problems during deployment.

    Moving to Roles

    It can be hard to change security around on an existing application. Fortunately we had limited numbers of objects and applications accessing our SQL Server database, so I could easily determine if refactoring was going to break anything.

    My first refactor was to create two groups in each environment. I used code similar to this in Development, QA, and Production

    CREATE ROLE WebAppUser;

    CREATE ROLE WebAppAdmin;

    By creating these roles in each environment, we had a consistent place to set security for objects. We proceeded to assign generic read/write/execute security to objects to these roles as needed. The WebAppAdmin role accesses all objects (essentially as datareader/datawriter). to grant rights to the WebAppUser role for specific objects, we scripted out the rights assigned to the current WebUser user and then granted those rights to the role.

    The last step was adding the WebUser to WebAppUser. Once this was done, we essentially had duplicated permissions for the user, WebUser, through the user account and the role.

    Our test procedure for the change was to begin removing the rights granted to the user in the QA environment. Once we verified the web application still functioned, we made one final change.

    In the Development environment, we created a new user, WebAppUserDev and put this user in the WebAppUser role. We then changed IIS to use this user account. From this point forward, development was separated from production. The Windows admin changed the WebUser password and development was locked out of production. We did the same thing in QA and created a new account there.

    Once everything was done, we removed all direct object rights from the WebUser account in production. This was a scary day, as we were counting on our role having all the correct rights. Fortunately our process had worked, and the application continued to function.

    That was a lot of work, but a refactoring that doesn’t break anything can take some time. There will be multiple steps and it can take days or weeks to implement.

    Moving Forward

    From this point forward, development proceeded without problems. All of our object code now included one, or both, of these lines at the bottom of the script.

    GRANT EXECUTE on MySP to WebAppUser;

    GRANT EXECUTE on MySP to WebAppAdmin;

    We migrated object code between environments, but not security. Security for each environment was handled separately, with separate accounts added to these roles. Our deployments became much easier.

    When we needed a new role (for client auditors that could access specific tables), we added the role as a part of our deployment and assigned security to the role. The role deployment was handled separately from application deployment with a new role being deployed in a script, but a different user added in production by the sysadmin to AD and the role. Later deployments had permissions grants for the role that were the same in development, QA, and production.

    When we added a Staging environment, it was as simple as restoring the production database, deleting the orphaned WebAppUser user from SQL Server and creating a new user for that environment.

    Consistency

    One of the big issues with deployments not proceeding smoothly comes when one environment is not set up the same as others, and the scripts run in one environment do not execute on another. When users are included in all environments, you have a security hole, but when different users get different security, you create scripts that must be edited, and potentially mid-edited.

    The more you can abstract away portions of your application, whether this is through roles, synonyms, linked servers, or other items that can be named consistently, but configured differently in separate environments, the smoother your deployments will be.

    There are certainly challenges with deploying new items across environments, but that’s a discussion for another day. For security purposes, I think that roles are an important way to ensure that security is maintained, but deployments are not impacted.

  • The Dreamliner

    I flew on the Boeing 787, the Dreamliner last week on my travels to London. United flies a few of these new planes, and the IAH-LHR is one of the routes. Since I go this way often, I decided to try and see how the plane compared.

    My wife was wondering what was special about the plane, and if it was huge, like the Airbus A380. It isn’t, and is actually designed to be a mid range plane. Boeing didn’t try to outdo the 747 with this plane. It was built to connect more airports around the world on a point to point basis, rather than the hub and spoke method many airlines use. Hub and spoke works well, but there are times that it isn’t as efficient as schedules get delayed trying to get people to a hub.

    I didn’t figure this out. I heard about it at a tour of the Boeing Factory in Washington. That’s where I saw some of the innovative, additive manufacturing they used for the plane. I also saw one of the huge DreamLifter planes they use to get parts from around the world to the assembly plant.

    Photo Nov 02, 12 11 15 PM

    Not to diverge too much, but those place are huge. There are only 4, and they are needed to bring some of the huge pieces of the Dreamliner to Seattle in one piece, inside this plane. That was a piece of engineering all on its own, converting and enlarging a 747.

    Photo Nov 02, 12 11 04 PM

    My flight on the Dreamliner started when I got to Houston last week. I had booked a window seat, which I rarely do, but I wanted to see what it was like on this plane.

    Normally when I fly overseas, I like a side seat, since the plane is rarely full and I have a good chance of getting the two side seats to myself. However when I got into the Dreamliner, they had a 3-3-3 configuration in economy. Not what I wanted, and the downside was that I had the window, but someone else had the aisle. I had a little space, but not a lot. However, it did seem to be a bit more front-to-back room than I normally see on United’s 757 and 767 planes.

    Photo Feb 02, 7 34 30 PM

    One thing I did notice is that it’s a tall plane. On most planes, even if my 6ft head isn’t close to the ceiling, I can often reach up and touch it. I try to stretch a bit, and I’ve noticed this. One the 787, I wasn’t close. The ceiling is high. That added to an open feeling.

    The windows are also larger. They don’t have shades, using electronic shading to dim, but not close.

    Photo Feb 02, 8 26 34 PM

    It was dark, and I was tired as we left Houston for the overnight flight to London. However I did notice that the wings were unusual. They are curved, bending in a shape that’s unique. They flex more than I like, but they are also new. Shinier than any plane I’ve been on. I could see the lights of Houston reflected as we flew.

    Photo Feb 02, 8 36 14 PM

    It wasn’t amazing, but it was a better trip for me. Especially when I turned around in Heathrow and flew on an old, Lufthansa flight to Frankfurt a couple hours after landing.

    The way home was more interesting. I got on the plane at 11:10 or so, ready for an 11:40 departure.

    Photo Feb 07, 3 57 46 AM

    I was thinking that this would be an easy flight, but that wasn’t the case. I had a book on my iPad I was reading with headphones on. After a bit I noticed that we hadn’t taken off. It was 12:00 and we were at the gate. We got a few announcements that there was an actuator on one of the wheels that controls the brakes. The pilot didn’t want us to worry, and let us know multiple times that there were a number of actuators (I think he said 12) on each wheel, and we could fly fine without a couple of them working. However we had one that was reporting seized.

    I’ve had numerous plane issues in the past, on multiple airlines, but mainly United. I chose the 787, thinking it was new and unlikely to have issues. I had a somewhat tight connection in Houston that I thought I’d make.

    Time went on. The pilot reported that they thought it was electronic, since most of the plane is electronic. They couldn’t clear the indicator, and decided they’d need to check the actuator. We heard, and I’m not kidding, that they needed to jack up the plane to relieve the weight on the actuator and release it. They didn’t need it to work for the flight (or landing), but they did need it released. They did that, and found the actuator was fine.

    It was getting later, and they couldn’t clear the issue, so they rebooted the plane. I’m not kidding. They told us they needed to power down and power up, which meant that they turned everything off. No air, no lights, nothing. It was strange to see the plane power down and then reboot, which took more than 10 minutes, and may have been as much as 20. I was reading and semi-watching the time.

    Eventually they cleared the light with the reboot and we took off. Two hours late. We made up time, coming in only about an hour later, but I missed my plane. Fortunately I’d tweeted United during the delay and they had rebooked me while I was in the air.

    It was an interesting flight to be sure. All day, we flew with the sun over us and to my left. However the windows were dimmed the entire time, making it seem like twilight. I could see the sun and clouds, but the flight was pleasant.

    It was a long journey back, and the equipment issues were annoying, but I enjoyed the plane more than others. I hope I don’t get delays again, and I worry about the software and electronics a bit in modern planes as we might be a bit overconfident in how we build them, but certainly the plane was a better built one for me as a passenger than previous ones.

    Now if I can only figure out how to get a first class ticket for one of these flights…