Author: way0utwest

  • Job Interviews: What is Normalization?

    This is part of a series that looks at interview questions that you might be asked.

    One of the cornerstones of a relational database is normalization. This is a database design technique that is used to decide how we might distribute data among tables and columns. If someone were to ask you “what is normalization” in your next job interview, what would you say? How do you answer this question?

    Depending on the position, and your experience, perhaps you’d talk about this in practical terms, with an example. Maybe you have a scenario you use to explain the concepts. I tend to lean towards an e-commerce type database, with orders, customers, and products. That’s an easy concept for people to grasp. I can then give examples of how normalization might change the way we store data.

    Do you there are various forms of normalization? We can talk about first, second, third, fourth, or other normal forms of the database. Perhaps you can explain these from memory? Maybe you know the academic definitions. Maybe you have your own description. I certainly think if the position may involve designing tables that you should be able to talk about the differences between these forms and determine what form a particular database is in.

    My answer would include quick explanations of how I would avoid repeating groups, columns not dependent on the PK, and how sometimes the third normal form starts to impact performance because of the number of joins required for simple queries. I would also be ready to expand on these topics, explaining what a PK is and how to choose one.

    This is a complex topic,and I doubt an interviewer would expect anyone to provide a thirty minute lecture, which wouldn’t completely cover the topic. Instead, I would think that if an interviewer is interested in this topic, they will ask probing questions to determine if you understand the concepts. This means you can’t just memorize the definition. You should ensure that if I have you a table, you could break it down into the entities for 1st, 2nd, and 3rd normal form. Perhaps you should read a few basic articles to get some understanding. Maybe you need to delve a little deeper into design and the anomalies that normalization attempts to mitigate.

    Prepping for an interview shouldn’t be a cram session to learn more, but really a review of concepts you understand. Make sure you have some examples to explain your thoughts and practice a few of these questions with a friend by giving them an answer and listening to their feedback.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Advent of Code 2017 Day 2–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

    This is day 2 of the Advent of Code 2017. If you want to read about the puzzles, start with Day 1. Here is a look at Day 2, which deals with checksums of rows. However, it’s not just rows, but rows and columns.

    Part I

    In part 1, we are given a series of numbers in rows and columns, essentially a spreadsheet. We need to look at each row and compare all the values in columns.

    This is slightly tricky for T-SQL. The input is a series of text values, so I cheated slightly and inserted those as rows into a table (I like tables). If I were taking this as a programming item, I’d have an import process, so I’ll do that here inline.

    I have some idempotent logic here as I was testing and didn’t want to recreate tables and inserts, nor did I want to comment/uncomment things in/out.

    IF NOT EXISTS( SELECT name FROM sys.sysobjects AS s WHERE name = ‘Day2’)
    CREATE TABLE Day2
    ( DataRow VARCHAR(1000)
    )
    GO
    IF (SELECT COUNT(*) FROM dbo.Day2 AS d) = 0
    INSERT Day2 VALUES
    ( ‘179    2358    5197    867    163    4418    3135    5049    187    166    4682    5080    5541    172    4294    1397’),
    ( ‘2637    136    3222    591    2593    1982    4506    195    4396    3741    2373    157    4533    3864    4159    142’),
    ( ‘1049    1163    1128    193    1008    142    169    168    165    310    1054    104    1100    761    406    173’),
    ( ‘200    53    222    227    218    51    188    45    98    194    189    42    50    105    46    176’),
    ( ‘299    2521    216    2080    2068    2681    2376    220    1339    244    605    1598    2161    822    387    268’),
    ( ‘1043    1409    637    1560    970    69    832    87    78    1391    1558    75    1643    655    1398    1193’),
    ( ’90    649    858    2496    1555    2618    2302    119    2675    131    1816    2356    2480    603    65    128′),
    ( ‘2461    5099    168    4468    5371    2076    223    1178    194    5639    890    5575    1258    5591    6125    226’),
    ( ‘204    205    2797    2452    2568    2777    1542    1586    241    836    3202    2495    197    2960    240    2880’),
    ( ‘560    96    336    627    546    241    191    94    368    528    298    78    76    123    240    563’),
    ( ‘818    973    1422    244    1263    200    1220    208    1143    627    609    274    130    961    685    1318’)

    The input might be different for you, so you’d have to do your own work.

    Splitting a row of data into more rows seems like something easier done with the STRING_SPLIT function, so I’ll use that. This will give me a series of values from the table. I decided to use an inline TVF for this, since it’s a bit easier to read.

    The trick here is that the data is tab separated, and \t doesn’t work in SQL Server. However, CHAR(9) works, so let’s use that. Here’s the function:

    CREATE FUNCTION AdventChecksum ( @input NVARCHAR(200))
    RETURNS TABLE
    AS RETURN
    SELECT diff = MAX(CAST(myint AS INT)) – MIN(CAST(myint AS INT))
      FROM (SELECT myint = CAST(value AS INT) FROM STRING_SPLIT(@input, CHAR(9))) AS ss
    GO

    I need to get the difference between the largest and smallest values. STRING_SPLIT will return strings, so I cast this to an INT, then use MAX() and MIN() for the row, subtract one from the other, and return that.

    Now I have a way to get a row checksum, so let’s get the entire table. We’ll do that with a CROSS APPLY, and then sum up all the values returned:

    SELECT SUM(b.diff)
      FROM day2 a
      CROSS APPLY dbo.AdventChecksum (a.DataRow) b

    I guess it works, because I got the right value from a test set, and the puzzle shows as solved. Smile

    Part II

    Part II adds a twist. Now I don’t need to just compare two values, I actually need to see if any of the values are equally divisible by the other. This means I need to compare all values against each other.

    The STRING_SPLIT() function still works well here, however, I need all combinations of the values divided by each other. An evenly divisible set of numbers would have this pseudocode:

    INT(a) / INT(b) = FLOAT(a) / FLOAT(b)

    Or, there’s another way to look at this. The remainder is 0, so the remainder of the values, using the modulo (%) function, is 0.

    One last trick, each number is divisible by itself, so let’s avoid those. The instructions don’t mention the possibility that the only values evenly divisible are matches, so we’ll assume that’s not the case. Checking inputs, this appears to be OK.

    I’ll create a new function, where I CROSS JOIN the STRING_SPLIT to get all combinations.

    CREATE FUNCTION AdventChecksum2 ( @input NVARCHAR(200))
    RETURNS TABLE
    AS RETURN
    SELECT divmatch = CASE WHEN (CAST(a.value AS INT) % CAST(b.value AS INT)) = 0 AND a.value <> b.value
            THEN (CAST(a.value AS INT) / CAST(b.value AS INT))
          ELSE 0
          end
           FROM STRING_SPLIT(@input, CHAR(9)) a
       CROSS JOIN STRING_SPLIT(@input, CHAR(9)) b
    GO

    Now I cross apply as in Part I and sum the values, which works. The puzzle is solved.

  • Fixing logins

    I ran into an issue recently where I couldn’t get a Windows login to work with some application software. This software had been configured with a user account, which appeared to be in SQL Server, but it couldn’t connect.

    It turned out that the local name of the workstation had changed. In this case, I’d renamed a Windows host in a VM, and my login was automatically renamed in Windows, but not in SQL Server.

    A quick fix. In this case, I needed to just alter the computername in the login with this code:

    ALTER LOGIN [WIN2016\fileshareuser]
    WITH NAME=[SJ-DEMOSITC1\fileshareuser]
    

    Once I did this, everything started working again.

    It’s rare that we rename items, but it does happen, especially in lab environments. It’s good to know how to remap any of those principals that might be affected if it happens to you.

  • T-SQL Tuesday #098–Technical Challenges

    tsqltuesdayIt’s the first T-SQL Tuesday of 2018, being brought to you by Arun Sirpal. His invitation this month asks for you to talk about some technical challenge that you conquered in your career. Some type of issue that you had to troubleshoot and discover what the issue was, as well as the item you corrected.

    I’ve written about lots of items in my career, so I decided to reach back to pick one that wasn’t too hard to diagnose, but was hard to solve.

    Inconsistent Errors

    A long, long time ago, in a company far, far, away, I was a DBA. Actually, not that far away. Just down the road in Englewood, CO, and it wasn’t that long ago, but it was prior to the year 2000, which was it’s own adventure (and non-adventure).

    In any case, I was a DBA for a company that had a fairly large and active application in use by hundreds of clients. There were a number of issues I had to solve here, but one of the most memorable came when a customer called and said they were getting an error in our application. We tended to hide most errors and return generic ones to the user, so I had a developer get some debug logs and we discovered a severe error from SQL Server that would drop the connection.

    This was annoying since the VB6 app wouldn’t reconnect by itself and the user had to close it and restart. What was interesting was that the user didn’t get this for all activity, just a few items.

    I tested a few queries on dev systems and they seemed to work from isql/w. We tried the application and that worked. That led me down the path of a data issue. I suspected some data might have had strange characters that the app couldn’t handle.

    As we dug into this on production, it seemed our test accounts and various others we tried would work, even for this customer data. Along the way we were running SQL Trace (this was v6.5) and app logging to debug. A few hours in, I stumbled on a few pieces of data for this customer that caused a broken connection.

    The error led me to believe there was corruption in the database and I immediately opened a call to Microsoft while alerting our management. Since we were a company in the financial area, this was a big deal. Fortunately it was near the end of the day and we could take some emergency downtime on the system. Since we traded mutual funds at the time, we weren’t involved in real time activity outside of NY business hours.

    At this point I’d been working on this a few hours, and in talking with Microsoft, we started some diagnostics, including CHECKDB work. We ran this periodically, but if I recall, this was weekly.

    Eventually we discovered that there was corruption in part of one table. With out indexes and a spread out client load, many queries read around the corrupt section, which explained the behavior we saw. Unfortunately, backups wouldn’t help here, nor could we select out the table to get data. As I was handed off around the world to different customer service centers, my apprehension grew.

    I was told that we’d need to move data out of the table and rebuild it, no easy (or quick) task. Since we didn’t know exactly where the corruption was, I was given quite a few queries to slowly work through sections of the clustered index and find what was readable and what wasn’t. Eventually we boxed in the bad sections and moved good data from other areas into a new table.

    When we thought we had it all, we dropped the table and renamed a new one. However, I wasn’t done. While this was going on, I was also restoring a few other backups to try and find out when the corruption started and hopefully recover other data.

    I worked all night, and into the next day. A couple cat naps while some things ran, but I’d been up close to 40 hours by the time I could leave. I’d recovered most data in the corrupt areas from backups, leaving notes for our service people to try and recreate the rest. We could enable our FKs again, though a few dummy records were needed in places where we weren’t sure what the data should read.

    Most customers never knew about the issue and only a few were upset. We replaced a lot of disks and rechecked server hardware, planning on moving to new metal as soon as we could, though the nature of v6.5 made this a challenge in getting disk systems setup correctly.

    That was one of my more memorable days at the office, though not the only overnight session. I kept a pillow and blanket in my desk, sleeping on the floor 4 or 5 times that year. Eventually I moved on, and was glad to do so.