Voice of the DBA

Category: Editorial

  • Patching Challenges

    In my career, I’ve had to manage many production database instances. In fact, there was a time when two of us were patching hundreds of instances (600?) when MS released patches. That wasn’t too often back then, unlike the every other month(ish) schedule that we have with 2017/2019 and the Cumulative Updates.

    My process was often to test patches on QA/Test servers and then start to roll out the patches to production. We didn’t quite follow what Brent Ozar recommends, in that we often patched development servers later. We didn’t control those, and unless production was patched, we couldn’t get developers to patch their systems.

    We didn’t patch DR servers first, but that’s a good idea. It’s one I think makes a lot of sense. We did patch secondaries first, ensuring that if there were issues here, we wouldn’t impact production, and these days with Availability Groups, hitting the secondary replicas, especially read-only ones, is a good idea.

    The big thing for me wasn’t so much the type of servers, but in having a series of rings to roll out the patches in groups. We used automated processes (no one wants to click next-next-next) and while we might patch a lot of servers, we never wanted to patch them all. We typically had 3 rings. Ring 1 was test servers, and ring 2 was most of production. Ring 3 was any production servers that got an exemption from initial patching. There were times when some process was important and couldn’t be interrupted or a client needed a few more weeks to test. We’d let them delay a month, but not longer.

    I think it’s important to have a strategy, and as Brent notes, also a protocol for how you handle things. I’ve often depended on our normal backup processes, especially in large environments, and the patches tended to stop the SQL services, so I don’t know how important it is to stop client apps, but think about it.

    One note about backing out changes is that containers make this a lot easier. If you move to production containers (linux, HA challenges, features missing, etc.) you can swap out an updated container with a new (or old) patch level as needed. There are caveats here, and certainly I’d start implementing this in a dev area first to understand the implications, but I expect over time containers will make patch deployment and rollback much easier.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Where Does NoSQL work well?

    It seems not too long ago that NoSQL was the hottest thing on the market. MongoDB was leading the charge, with tremendous growth as a company and many developers looking to adopt their platform. I remember years ago going to visit a number of customers in New York City, where Mongo had just grown their offices. Every company was considering migrating their RDBMS to MongoDB, trying to calculate the cost and return.

    Now, I certainly hear about companies adopting database that are in the NoSQL family, but not always (or even often) to replace a RDBMS. Instead, it’s for a new application or alongside an existing RDBMS. While there are applications that are well suited for a non-RDBMS data store, there are plenty that perform worse on those platforms. This might be because the platform doesn’t handle their workload, or because the developers aren’t writing good code, but I know I’m not worried that RDBMS usage is going away, or even down.

    I ran across an article earlier this year that wonders if NoSQL is a good choice for your application. The article covers a bit of the types of these stores and how they work. It’s a basic look, but it makes sense and it addresses the benefits of NoSQL, like scaling. It also notes there are issues, which is something to think about. Not that I see a lot of developers thinking about the issues. Often they think the can code around them.

    The piece notes that for some problems, these databases work great. However, for a side project or a simple site, a relational store is likely better. NoSQL classes of databases are suited to scale well, but with quite a few tradeoffs.

    Are there places you’d use NoSQL? I’m curious. If your organization has adopted these stores, or rejected them, it would be good to share some information.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Deceptive Visualizations

    I’m sure there are plenty of examples, but I’ve seen deceptive data visualizations far too often in mass media. I don’t often watch the news, but at times I’ve been at the gym, saw a graph on a television and been a little surprised. The issue is similar to what’s discussed in this SAS article on deceptive graphs.

    In this case, the problem is likely what the author describes. An axis was moved, which what I often see in the news. Rather than starting at 0, often the image might start at 20 or 30 on a scale of 1 to 100. As a result, the differences between two bars, lines, or whatever is on the graph is distorted.

    This might be unintentional, as more and more graphing tools try to “size to fit” in a space, and can alter graphs, but in many cases, I think the author of the report is deliberately emphasizing something to evoke a reaction in an audience.

    I suspect this also happens inside companies. Someone might change scales or axis starting points to emphasize or de-emphasize some part of the data. I could see sysadmins wanting to de-emphasize downtime, but they might want to emphasize a cost savings to get management to make the decision they would prefer.

    The article linked above makes a good point. If you do this, your credibility is undermined if someone notices. Maybe the media doesn’t care, but I know that trying to deceive the person that signs your checks, or has some impact on your future employment, is a bad idea. These days, it is hard to find good help, but that doesn’t seem to stop a lot of companies from getting rid of bad help a times.

    Present data the way you’d want to see if your positions were reversed. Or if you wanted a decision to be the opposite of that you are hoping for in this instance. You might get a short term win, but in the long term, I would argue your credibility and reputation are worth more than a short term win.

    Steve Jones

  • Evaluting a DBA

    Interviewing is difficult for many organizations. Often they don’t have a consistent process, and very often the people that do initial screening don’t know much about what a DBA does. Many of the somewhat trivial questions I see on Internet lists (what’s a clustered index?) are suited for someone getting started in the business, not for hiring an experienced DBA.

    Sure, an experienced person ought to be able to answer those questions, but the real world often requires more nuanced questions that relate to the situations we find ourselves in. I found a list from Joey D’antoni recently, which has lots of open ended questions, designed to allow someone to talk about what they know and think, giving the interviewer the chance to gauge their level of expertise.

    I especially like the questions around tuning and configuration. Tell me what you’ve done in the past and why. That way someone can tell you a story about their experience and then relate this to practice solutions. There is no good way to talk about these situations if you haven’t lived them. You might read someone’s blog, but a few follow-up questions on details will show whether you really understand the solutions or not.

    Of course, you need a good interviewer that knows something about these topics to understand whether the answers make sense or not. An HR person or manager isn’t likely going to be able to judge how a candidate performs. While a developer or sysadmin might be able to gauge whether the person has spent time with SQL Server, if you need a really experienced person, you need someone equally experienced to interview them. That can be a challenge for some companies.

    Hiring is difficult, but with more junior or intermediate positions, it might not matter as much. Evaluate whether the person has some knowledge and fits with your team. Can they work with others, which isn’t the same as just agreeing with them. Maybe more importantly, can they learn. Do they learn from your questioning, or have they shown they are learning.

    For more senior people, references and networking matter a lot. If others recommend them (or don’t), that says a lot. It’s good to initially decide if this person is a good fit, and then engage someone like Joey or Glenn Berry (or Tim Mitchell in the BI space) that might provide a better evaluation of your short listed candidates.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.