Voice of the DBA

Tag: Common SQL Server Mistakes

  • Common SQL Server Mistakes – Functions in the WHERE Clause

    This continues my series on Common SQL Server mistakes, looking at more T-SQL mistakes.

    What’s Wrong?

    If you saw a query like this, would you see a problem?

    select
      o.OrderID
      , o.CustomerID
      , o.Qty
    from Orders o
    where datepart( yyyy, o.OrderDate) = '2010'

    If there are 1,000 orders in this table, there probably isn’t an issue. But if there are 1,000,000, then this is an issue.

    Why? Let’s examine the execution plan:

    This table has 1000 rows in it, but it doesn’t use indexing to find those orders that were placed in 2010. Instead it scans all rows. The reason is that the function being used in the WHERE clause means that the index cannot be used.

    Instead, what you would want to do is write the query like this:

    select
      o.OrderID
      , o.*
      , o.Qty
    from [OrderItems] o
    where o.OrderDate >= '20100101'

    In this way, we eliminate the function from the WHERE clause and allow the query optimizer to take advantage of the indexes on the column OrderDate.

    You see similar issues with queries like:

    select
    lastname
    from Person.Contact
    where left(Lastname, 1) = 'S'

    This can be fixed as:

    select
    lastname
    from Person.Contact
    where Lastname like 'S%'

    Basically you want to move the function away from the column and put it on the other side of the comparison so that indexes can be used.

    Too often we have developers writing queries like this, assuming that the functions are efficient. They are, but when they are executed against every row in a table, an index can’t be used for seek operations, which are always quicker than scans for any significant data set.

    When you are writing queries, do your best to avoid functions against columns in your tables. Instead try to rework the query to move the function. An alternative that I’ll blog about another time is computed columns.

  • Common SQL Server – Not Indexing FKs

    This series looks at Common SQL Server mistakes that I see many people making in SQL Server.

    Foreign Keys

    It’s way too often that I see people building databases without including declared referential integrity (DRI) in their databases. Even when I see people setting a primary key on tables, it seems that often they ignore foreign keys and creating linkages between tables that link them together.

    However, even when people have declared a FK, they often don’t create an index on that column. Perhaps they assume that SQL Server will create the index like it does for PKs, but it does not.

    If I create these two tables and join them with a FK:

    CREATE TABLE [dbo].[Products](
        [ProductID] [int] NOT NULL,
        [ProductName] [varchar](50) NULL,
    CONSTRAINT [PK_Products] PRIMARY KEY CLUSTERED
    (
        [ProductID] ASC
    )
    ) ON [PRIMARY]

    GO
    CREATE TABLE [dbo].[ProductDetails](
        [ProductDetailID] [int] NOT NULL,
        [ProductID] [int] NULL,
        [SKU] [varchar](50) NULL,
        [Price] [numeric](18, 2) NULL,
    CONSTRAINT [PK_ProductDetails] PRIMARY KEY CLUSTERED
    (
        [ProductDetailID] ASC
    )
    ) ON [PRIMARY]
    GO
    ALTER TABLE [dbo].[ProductDetails]  WITH CHECK ADD  CONSTRAINT [FK_ProductDetails_Products] FOREIGN KEY([ProductID])
    REFERENCES [dbo].[Products] ([ProductID])
    GO

    ALTER TABLE [dbo].[ProductDetails] CHECK CONSTRAINT [FK_ProductDetails_Products]
    GO

    If I go and check indexes on ProductDetails, I find that there is only one index, the index for the PK.

    FKIndex_a

    Why is this a problem? It’s because of performance. We should realize that indexes speed up performance by reducing the amount of work that SQL Server has to do.

    With FK columns, what I’ve often found with child tables is that I know the value of the FK column I am searching for and don’t need to join with the parent table. However without an index on the FK column, this query requires a table scan.

    select
    sku
    , price
    from ProductDetails pd
    where pd.ProductID = 3

    If you are creating FKs in your database, don’t forget to index them where appropriate.

    Auto Creation

    I’ve seen some people ask why SQL Server doesn’t automatically create indexes on those FK columns. I am torn on this, but I like the 80/20 rle. If 80% of the tables would benefit from it, I think it should be done. I am leaning towards some intelligent mechanism to do this.

    The main issue is that you might not want just an index on the FK column. You might want some sort of covering index that includes columns in addition to the FK column to prevent key/bookmark lookups to the clustered index. If you can avoid those, you can drastically increase performance.

    There is also the chance that with your query load, you never use these indexes. That can be horrible for performance as well since there is overhead to maintain these indexes on all insert/update/delete operations.

    The Advice

    Look at the queries that are coming into your database. Check the missing index DMVs and if you find that the FK columns are being used, index them.

    If you’re not sure, or don’t know how to look for missing indexes, here’s a reference.

  • Common SQL Server Mistakes – Equals NULL

    One thing that I don’t see a lot, but it still happens with people new to SQL Server is the comparisons they’ll make with NULL values. Often those people new to T-SQL will write this:

    select CustomerID, CustomerName
    from Customers
    where SalesRepID = NULL

    The thought here is they are looking for those customers that don’t have a salesrep assigned. Or they might enclose the NULL in quotes, but this won’t work.

    The correct way to do this is:

    select CustomerID, CustomerName
    from Customers
    where SalesRepID Is NULL

    Note the “Is NULL” that will correctly return those customers who have a NULL value stored in that column.

    Why?

    NULL is an unknown value. We just don’t know what value it is, so it’s not a variable in algebra like “x”. In algebra, x=x, but NULL != NULL. Since we don’t know what the value is, and since each row could potentially have a different value (remember every NULL’s value is unknown) we can’t expect any NULL to equal any other NULL.

    NULL isn’t a placeholder like a blank or space, or even zero. It’s an unknown value, so equals (and not equals) does not apply. Instead you need to use “Is NULL” or “Is Not NULL” for your comparisons.

  • Common SQL Server Mistakes – Indexing Every Column

    If one index helps speed up queries, than more indexes should help more, right? They do, but they also come at a price. Both in performance during data modifications (insert/update/delete), and in terms of space since each index must be stored somewhere.

    I have never bothered to index every column of a table. Actually I’m not sure if I’ve indexed every column of any table. Perhaps that’s because I originally came from a dBase/Clipper/Foxpro environment and I had to manage every index manually. Adding too many indexes resulted in a lot of coding.

    In SQL Server, each index is useful in two ways. When someone puts a filter in the WHERE clause, like this:

    select
      CustomerID, TerritoryID, AccountNumber
    from Sales.Customer
    where CustomerID = 10

    then an index on CustomerID will speed up this query. Instead of having to scan all rows of the table, the index on CustomerID can be searched, the correct row found. If this is a clustered index, then the data can be read. If it’s a non-clustered index, the server can retrieve the rowID and then go get the data from the clustered index without reading all the rows in the table.

    In more recent versions of SQL Server, multiple indexes can be used. For example,

    select
      CustomerID, TerritoryID, AccountNumber
    from Sales.Customer
    where CustomerID = 10
      and TerritoryID = 23

    In AdventureWorks, there are indexes on both CustomerID and TerritoryID for this table. It is possible (with lots of data), that the optimizer might choose to scan the CustomerID index for all matching rows and then the TerritoryID index for matching rows, join those results together to get a set of rows for the overall query and read the clustered index for those specific rows. However that’s not something you can count on in SQL Server. Typically one index is used in many queries.

    So why not index every column?

    First, if the columns are large, like varchar(max), text, or varbinary(max) columns, then it doesn’t make sense to build large indexes unless you often query these fields. Even then, a full-text index is likely a better choice.

    Second, each time you change data (insert/update/delete), then all indexes must be updated at the same time. This means that your write performance suffers, and that impacts the server read performance as well since resources are being used to perform those updates. The more indexes, the more work that has to be done in support of any DML statement.

    Lastly, you typically find that most of the time you query a table based on 3-5 fields, and those are the best candidates for indexes. For transactional tables, this is typically the number of indexes that you want to put on each table. Reporting tables, or OLAP type tables, might have more indexes, but these are tables that typically receive mostly read activity, and rare write activity.

    Which columns do you index? Pick those columns that often appear in your queries, and that are fairly selective. You can always query the missing index DMVs for help in choosing which indexes the optimizer things it might use.