Voice of the DBA

Tag: software testing

  • Demo Data for Everyone

    As someone learning about DevOps, I follow a number of people, one of whom is Gene Kim. When I see him get excited about a post, I usually read it. That’s how I found this post on Demo Data as Code. It’s a short, but interesting read. I think this is actually something more people ought to implement in their environments and not just for demos.

    DevOps is about reliability and repeatability, among other things, but those two are tackled with automation for a known process. We don’t want simple, silly mistakes, or even complex errors that might undermine our ability to move forward and create value. We don’t want simple errors eating up resources and time from expensive talent with unnecessary work. Part of ensuring both repeatability and reliability involves using data in our databases to evaluate our application. This isn’t necessarily for demos, though it could be used for demos.

    Once of the areas that is often left out of the process is the data that we use in our building our systems. We need some data for developers, for QA, and often for demos. In all of those cases, when humans need to repeatedly look at how well the software performs, and want to re-test things, they need some consistent data. I’d also argue that the need for agility means that we need a manageable data set. I think SQL Provision from Redgate is amazing, but I still don’t want to always develop with 2TB of masked data. I certainly don’t want to demo this for customers from a laptop, and might not want to share this in the cloud.

    At Redgate, we sell masking with SQL Provision, and it supports most of the process that’s outlined in the Demo Data as Code article. What it needs, however, is a small set of data that can be masked in a deterministic fashion. What I recommend to most clients is that they build a known set of test data, which could be used for demos. This can include all your edge cases and show off new features. It’s helpful for developers, testers, and salespeople, who will always have a known, useful set of data.

    This can’t be a build it and forget it, much like what is emphasized in the article. This will need to be altered over time. There ought to be a process to build this dataset, likely from production data that gets sanitized. This can then be distributed through SQL Provision (or similar technology), with backups, or even as a set of scripts in your VCS. Ensure an environment can be hydrated instantly on any platform, from a developer workstation to a sales laptop to a QA server. Once you have this, everyone can work on evaluating your software from a known baseline.

    And if you find the need for more data, then just add it. You have a process, so add an additional step that will cover the holes you inevitably find.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Republish: A T-SQL Code Testing Guide

    The day after  Thanksgiving, and a holiday. So you get to read A T-SQL Code Testing Guide.

    Please do, I’m a fan of testing.

  • Practical Refactoring

    Today I hosted a webinar with Gene Kim (@RealGeneKim) and we had a fantastic discussion. I was slightly star struck since I’ve been reading his work and quoting him for years in talks about Database DevOps. It feels like I got to work with someone really famous, and I’m hoping I didn’t appear too nervous on the webinar.

    In any case, we had a great discussion, and I think you can still register to watch the recording. If not, we should have this on our Redgate YouTube page soon. We discussed the State of DevOps report, and specifically how the findings relate to databases. It was a good discussion, but when we talked testing, we both had some links to ways that we could build better software.

    In my case, I referenced this talk, Practical Refactoring, which I think is great. A bit is the technical approach, but mostly I find the philosophy and freedom that comes with having tests in place to be invaluable.

    This is based on a real project that these consultants worked on. The code is mocked, so don’t get caught up in the actual methods and structure, but think about how you could apply the ideas to your own work. How can you make the code better in a few minutes.

  • Building Test Data

    One of the debates I’ve seen over the last few months is about test data in development environments. As I’ve been preparing for and learning more about the GDPR, it seems that many companies are concerned about holding sensitive data in their development systems. I think it’s a valid concern, and I’ve often had to deal with this issue in the past, before any regulation impacted my work.

    In one of my early jobs, we stored emails from customers in a table. We also had an email feature for our application. Needless to say, we needed to test that, though we didn’t want to obviously send emails to real customers when we were testing some sort of feature. I’ve done that and usually it results in a complaint and some scolding of the development staff. As a result, I learned to ensure that anytime we restored production to our QA system, we ran a script that would either change all emails to invalid values, or reset them to something we could use in a test system. In some cases, we’d reset them all to a specific address that we could check to see if the emails actually were sent.

    In talking to many people, they often don’t build test data for development systems because the data isn’t valid. What a developer thinks about, or what might be randomly generated by some utility often isn’t seen as valuable. Most developers want to see real data, perhaps because they can then better relate their work or a specific feature to the actual live system. Maybe it’s easier to see actual customers, products, accounts, etc. when working with clients or testers, but I do think that certainly in light of the GDPR and other regulation, there are risks here.

    While many people want to just restore production to refresh environments, I do think it’s a poor idea to use actual sensitive data. Even if you trust your developers, there have been no shortage of attacks against development systems, loss of laptops or other files with production data that were intended for developers. We just don’t secure test and development environments like production, and that means we are making a fundamental error in how work habits.

    I’ve gone through different views on this topic across the last couple decades, and now I want some real data, some not real data. What I’d really want is a bunch of random data that is close to production, mimicking the shape and skew of production, but without any sensitive data. Then what I’d like is a set of known cases of data that are the types of data that we need to ensure works in our system. Various cases of transactions and values designed to cover the functional edges that we support.

    Of course building these sets isn’t’ always easy, and it’s never going to be done. As long as we write software, we will need to maintain tests and data alongside the code. I do think this is a worthwhile investment in regulated industries, and likely worth doing in all industries. The thing is, it’s not interesting or fun, and likely not to be ever be done in most organizations. I’d like to change that, and I hope you do as well.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.3MB) podcast or subscribe to the feed at iTunes and Libsyn.