Voice of the DBA

Author: way0utwest

  • Testing is Your Best Investment

    I signed up for the FlowCon 2014 conference in San Francisco this September. It’s a two day event about software development and how we can do better. My job is starting to encompass more work in this area, and I’m excited to go see some of the ThoughtWorks developers talk about what they do well. Part of the reason I wanted to go came after watching a few videos from last year, including this one from Randy Shoup.

    There are some interesting things in the talk, but one thing really caught my eye at the 9:52 mark. Mr. Shoup made a statement that “tests help you go faster” and “the best investment you can make in your own code …[are] tests for your code.” Those statements are a short part of the talk, but they make a lot of sense to me.

    It’s easy to ignore testing, after all, it’s hard to test for everything that can possibly happen, and writing tests can be extremely tedious. However building tests JIT, when you find bugs or problems, can give you good code coverage, especially in the areas where you find developers are likely to make mistakes.

    And like with all other software development skills, the more you work on writing tests, the easier, and faster you become at building them.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 1.6MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • The Flash Database

    I caught this piece on the impact of flash storage on database engine design this week. It’s interesting as there’s been debate for years as to whether SQL Server should alter its behavior if it detects SSDs being used for storage instead of physical disks. It doesn’t, and perhaps that’s fine, though the article makes me think there are performance gains to be had if behavior changes.

    The article really looks at a few of the NoSQL products, though the design changes aren’t necessarily specifically limited to those products. I particularly thought that two ideas in the piece were interesting: indexes in memory with data on disk and the realization that threading can be he bottleneck with SSDs. I’m not sure if Windows and/or SQL Server could use these ideas, but they are interesting.

    I do wonder sometimes if a little more control of indexes would be helpful in SQL Server. Imagine if I could limit a large slice of memory to strictly non-clustered indexes and then have other data on SSDs. Would there be a way to tune SQL Server to run better for some workloads? Perhaps the algorithsm that choose query plans would change if they knew a scan of an NCI could be completed in a fraction of the time that a seek took place on a CI? Maybe we’d be willing to perform more seeks on in-memory indexes before performing lookups on disk.

    The idea of more concurrent operations, requiring more threads, also seems to be an area where I could suspect that both Windows Server and SQL Server could benefit from SSDs. If the systems changed their read and write algorithms and used many more threads with SSDs, could we get more throughput? Should our systems be more aware of how many controllers and paths might be on a system? I wonder, especially as some of this hardware becomes cheaper and cheaper. I could certainly see more organizations looking at using lots of smaller SSDs for a few servers that require high performance than a SAN.

    However it’s not as though SQL Server isn’t trying to take advantage of technology changes. The In-Memory OLTP system  and Buffer Pool Extensions in SQL Server 2014 are designed to take advantage of more memory and SSDs to dramatically improve performance. I don’t know what else might be coming in the next version of SQL Server, but I do hope that as new ideas emerge, SQL Server considers taking advantage of them.

  • The ALS Ice Bucket Challenge

    A few friends challenged me yesterday, including Grant Fritchey and Aaron Bertrand. Last night, when I got to the Denver SQL Server User Group meeting, Todd Kleinhans was ready to take the challenge and asked if I’d participate.

    I was speaking, but happy to do it afterwards. Here we are:

    I tagged Allen White, Erin Stellato, and Jes Borland.

  • Production Subsets

    Continuous delivery recommends developers never use production data. It’s too big, too cumbersome, and slows the process too much. Developers should have enough data to determine if their solutions work as they build them. Testing should have enough to do some tuning, but unless you plan on full performance/load tests (which you should), then you don’t need the full set of production data.

    It’s an interesting idea, and overall I agree. A subset of data, hundreds of rows, can usually tell you if you’re writing code that works if you profile the code and look for inefficiencies. Note that profiling code doesn’t mean use Profiler. It means examining the resource used by your code in terms of CPU, I/O, memory, etc. There are tools to help you, and at some point in your development process, you should be using them.

    However it can be time consuming and cumbersome to build small development data sets. There are lots of choices in how you might do this, and I thought this would make an interesting poll. For those of you that deal with development, whether that’s T-SQL, .NET, or something else, what do you think?

    Should we have a subset of production data, a custom data set, or perhaps deal with complete production data?

    Some of this depends on the size of your production data, and I hope, it’s contents. I would not want any PII, PCI, medical, etc. in any development area. However if that’s not the case, then what do you prefer?

    Whether you have  custom data set or a subset of production, it can be cumbersome to keep this up to date. Your data may evolve over time and there’s overhead in maintaining some scripts that would produce the data you need. Perhaps that’s the cost of writing good software, but I’m curious how many of you feel.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.