Tag: sql server

  • Containers make easy upgrades

    I’ve been working with containers and writing a bit about them for awhile. I find them to be fascinating and useful as a technology, and I’ve come to really believe these will change how we work with the SQL Server data platform.

    In any case, I wanted to get the latest SQL Server 2019 version recently, so I decided to download the container image. I’ve had a few on my machine, and so upgrading to the latest was a question of running this:

    docker image pull mcr.microsoft.com/mssql/server:2019-CTP3.1-ubuntu

    The main SQL Server 2019 image exists in a few flavors on my machine, so as you can see, I don’t even download the entire image.

    2019-07-03 19_15_29-cmd

    There were only 4 layers I needed to pull since I already had the other 5 layers. This took about 10 minutes on my nice, but not great, 15MBps download speed. That’s much quicker than I could download an iso to install, plus I’d need to uninstall the previous CTP.

    With containers, I can just run any CTP by picking the image from my list.

    2019-07-03 19_18_01-cmd

    If you haven’t tried working with containers, I’d urge you to start learning a bit. This is amazing technology that your developers already use, or want to use, and I expect data professionals to follow suit in the next few years.

  • Killing FTS

    In almost every application that I’ve helped build, we had a need to perform textual searches of fields. Sometimes this was in short fields, like names, where we usually needed some sort of wildcard search to let users find information. When we got to larger volumes of text, such as note or description fields, it was essential that the application allow a way to find information that doesn’t exactly match some term.

    SQL Server has had the Full-Text Search (FTS) subsystem for a long time. Built as an improvement over the LIKE keyword, FTS implements its own index system that better understands language and takes some burden off the developer when trying to find keywords and terms inside of a volume of text. However, this system hasn’t advanced much since 2005, with limited improvements in each version, and more stable performance, but not substantially improved. New languages get added, but not development language improvement.

    These days, many of the individuals that need to search text fields used ElasticSearch instead. In previous years, Lucerne was a popular choice, along with other software add-ons. In fact, most people want to use anything other than SQL Server’s FTS. Even at SQLServerCentral, our experimentation with FTS led us to abandon this early on as the work required to structure a UI around the CONTAINS code, along with poor search performance, made the decision easy. Google custom search or another package are far superior.

    I wonder if it’s time for SQL Server to abandon FTS. Maybe their limited development resources would be better spent integrating other third party engines into SQL Server. Or maybe some of their profits are better spent licensing or purchasing a different technology for searching. Certainly they should deprecate the current FTS implementation and allow applications using it to continue to do so, but cease additional work beyond limited maintenance.

    At the very least, they should move in some direction. I’m sure Microsoft could come up with ways to improve FTS, but I don’t know they can do it quickly enough, or that it’s a good use of developer resources. There is other technology that does this very well, so take advantage of it and build hooks that make integration simple. That might be the big search win for SQL Server databases.

    Steve Jones

    Listen to the Podcast at Libsyn.

  • Technology Flows Downstream

    A few years ago, I was listening a session on Azure SQL Data Warehouse at an event. At the time, one of the features I found fascinating was the separation of the storage from the compute portions of the service. We could load a lot of data in the system and pay for the storage, but we could spin up and down compute nodes to essentially scale out query processing to match our workloads. Azure Data Lake has something similar, and I was excited.

    I wasn’t excited about ASDW or ASDL because I don’t really use those platforms in my work, and I don’t have customers to talk to about those. I think they are useful, just not for me. I was, however, impressed with the technology, hoping that it would be coming in the next few years to SQL Server in the box and Azure SQL Database.

    It’s coming in SQL Server 2019, or at least, the start of it. We will get compute nodes, managed by Kubernetes, that will query other storage nodes that scale out I/O access. We also get this with Azure SQL Database in the Hyperscale edition, where there will be query servers and cache servers, designed to scale to infinite, or at least multi-petabyte, ranges

    I wasn’t always sure that the investments in Azure would pay off for the database side of things. It felt like Azure Stack was a natural place to help us manage and deploy SQL databases, and that the contained database work started in 2012 would get pushed by the needs of the Azure cloud, evolving and growing in future versions. We haven’t seen those pieces of technology grow, but I do think technology in other parts of the data platform has flowed to the “regular” OLTP database space.

    The separation of compute and storage really begins with the SQL Server 2019 era release, and I suspect that’s just the first step of how our SQL Server database systems will evolve in the future.

    Steve Jones

    Listen to the podcast at Libsyn.

  • Simulating Load

    I needed to generate some load for a demo and wasn’t looking forward to doing it. I was in a hurry, and didn’t want to deal with a lot of setup. Fortunately, I’d seen a technique for doing this in one of Brent Ozar’s classes. I’d taken a couple, and they’re good. One of the items I took away was the need to simulate a load that might help me analyze my system.

    Brent’s written about his technique, and I decided to adapt this to my own demo database. I took his code, and then added a few items.

    Examining Parameters

    I was happy with a load of ten items, so that makes things simple. I could reuse most of Brent’s code, but unlike his system, I have different parameters in use for different types of calls. In examining the types of procedures that might generate load, I found I had these types of parameters

    • Integers
    • Dates
    • Strings

    The basic code uses RAND() to generate a number, but what about the other items? I didn’t want scalars, so I needed random other items.

    For dates, I still used a similar random integer, but then used that with DATEADD() to alter an existing date. My code would generate a separate random number and then use that to go backward in time a certain number of days.

    SELECT @off = RAND() * 1000;
    SELECT @dt = DATEADD(DAY, 0 - @off, SYSDATETIME());

    For strings, it was different. In this case, it was search items and while there can be a (seemingly) infinite number of possibilities, I can simulate this. I can select from the table and use a random ordering to get different terms. If this isn’t efficient, that’s fine. I want a load on the system.

    DECLARE @term VARCHAR(100);
    SELECT @term = SearchTerm
    FROM dbo.SearchTerms
    ORDER BY NEWID();
    SELECT @term;

    Once I had these techniques, I built a stored procedure just like Brent’s, and then replaced his procs with my own. I built parameters in front of the CASE statement, and then use the appropriate random parameter for the procs as needed.

    Running the Load

    I used the same SQLQueryStress tool that Brent did, setting the connection credentials, threads, and iterations as needed to generate load. This gives me control over the load I need to generate.

    I had 10 possible procedures, each of which runs with different parameters each time the proc is called. For 100 iterations of 4 threads, I generate 400 calls to my load procedure, which generate 400 random calls to 10 other procs with different parameters

    A quick and dirty load on my server. Not representative of any production load, but good enough to stress the system and let me look for places to tune code. And, a way that I might get some of the weird random things users do.