Voice of the DBA

Category: Editorial

  • A Work Conference

    I like the idea of a work conference. We’re actually having one at Redgate this week, though it’s all virtual. I’d like an in person one, and it sounds like that will happen at Atlassian four times a year when they require employees to be in the office. That’s it.

    Four times a year.

    They are going all in on work from home, allowing employees to be in any place that Atlassian has a legal presence for work. As someone that’s dealt with this, it is a hassle for a company doesn’t have any setup for an employee in a new locale, especially in the US. So I understand them limiting employees to places where the company has a corporate entity. Still, that’s a lot of places for Atlassian.

    Many Atlassian employees do expect to go to offices about half the time, but they won’t be required to do so. I think there are lots of their staff that want to be able to go to an office, at least some time.  I do wonder how they will handle meetings and collaboration with people being able to move their schedule around and be in or out of the office. At Redgate we’ve said remote first, but sitting at my desk for a meeting with one person that’s at their desk in the same room and one person that is remote feels strange.

    Getting everyone in a region together a few times a year is a really interesting idea. It will feel like a SQL Saturday or other event when you know everyone will show up. I can see this generating some excitement. Especially if those days are different from normal workdays.

    It’s something I think I’d be interested in, and perhaps excited by. I think I’ll suggest each Redgate office have a few days a year, and then see if I can get invited to each office on those days. Fingers crossed that it happens.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Gift of Asking Stupid Questions

    I am a fan of teaching someone to fish, rather than giving them fish. But when teaching someone, I want to allow them to make mistakes and not just dictate how they work. I want them to learn how to solve problems themselves, gain knowledge, and improve skills.

    I was watching some of the Pluralsight Tech Skills day recently and in the Scott Hanselman interview, I really appreciated his answer on how to approach a problem. He talks about asking bisection questions at scale, asking lots of yes/no or querying questions that lead you towards the solution.

    From here, he notes that experienced or senior developers often know the questions to ask because of their past. For junior developers, they need to be encouraged to ask questions and solve problems. However, they need to be able to do this safely, meaning that we encourage and help lead them to answers without berating them for asking questions.

    This is one of the things we’ve tried to do at SQLServerCentral. We have a few guidance articles (etiquette and examples) on how to post a question, but often we ask people what they’ve tried, what works, what results they get, and try to help them find answers. Not all of us, but many of us do this. I do too often see someone posting code as a solution without explaining or helping someone learn, but it’s less often than I see a response trying to help someone learn and think.

    I’ve always disliked the RTFM answers I see on the Internet, but I really hate seeing this inside an organization. I get that some people will take advantage of others and avoid work, but often someone wants to learn. I do think that it’s good to allow someone to ask about a topic once or twice, or even about how to learn. Give them ideas, and remind them to put forth some effort, but don’t automatically discourage questions.

    Like Scott Hanselman, I had the gift of being able to ask stupid and silly questions. While I was sometimes abused for these, most of the time someone helped me increase my knowledge. I haven’t always been a good listener for others’ questions, but I have been working on being more patient and understanding with others. I’ve also started to appreciate how hard it can be to search out some answers, so I remember that when I see the same question again and again from different people. They’re all trying to learn and I want to give them the gift of being able to ask questions, no matter how simple they might seem to me.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Incident Response Data

    I was watching a PoSh session at the PowerShell + DevOps Global Summit recently where a sysadmin had a series of scripts to run when there was a problem. One of these was Rapid Response, which gathers information from a machine(s) and stores it in a series of files. It’s a grab bag of various items, but the data can be used to help determine what’s wrong.

    Some of us have monitoring tools for our databases, and some don’t. I’m wondering, in each case, is there a set of data you want or need when an incident occurs? Do you have separate types of incidents that require disparate data? Perhaps you respond differently to performance issues than security incidents than hardware problems and want different types of data gathered.

    I know that in the past, I’ve often had scripts I ran to respond to some issues, but not others. I’ve also depending on monitoring systems (bought or built), but usually they don’t have all the information I need when something goes wrong. Capturing all the data I need in an incident is often too much to store for any length of time, but it is data that I need for specific issues. Having a series of automated processes that might start collecting data when an incident occurs, perhaps filtered based on an instance, database, user, or some other value, would be helpful. However, I think I’d need a lot of incidents to build the list of scripts myself for different issues.

    A crowd sourced series of scripts, developed by people responding to different problems, would likely be the best way to capture this information. I do see some good resources (PDF, GH) for certain types of problems, but in order for these to be useful to you, some knowledge and familiarity is needed. You need to know what scripts are useful in which situations.

    This is really the best reason for blameless RCA (root cause analysis) work after problems occur. If you have runaway blocking, constant security probes from unknown clients, or any other issue, it becomes important to analyze what happened and how people responded. Build up a protocol for how to respond and ensure that the knowledge is distributed to others that might need it. Practice running scripts and looking at information, perhaps even in a controlled replay of the problem.

    When an incident takes place, you’ll be glad you are prepared. Whether it’s small or large, a little practice will help you get through things more efficiently, and likely with less stress.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Checking Up on Vendors

    Many years ago I was in the Denver airport, waiting to board a plane. At the time I was consulting with a small startup company and they called me in a panic, having had an issue with their database deployment and needing to restore the previous version. However, they were getting the “The media set has 2 media families but only 1 are provided. All members must be provided.” error. Fortunately, I knew what was wrong and verbally walked them through the process. When I landed, I logged in and verified things were working and that they’d followed my instructions.

    A few years later I was shopping for furniture with my wife one Saturday night and I got a call from a fellow Operations person at JD Edwards. They had some security issue with a SQL Server, and since I was the SME (subject matter expert), they called me for help. Sitting in a comfy lounge chair (which I never purchased), I helped them solve the issue. Later, I verified things were working.

    In these cases, I was using remote hands from trusted individuals at an organization. However, these days, many people call cloud vendors (or other hosting companies) for the same type of remote hands support, but without necessarily having the accountability from that staff. Sure you can check if they solved an issue, but are you sure they didn’t read some data or change anything else?

    It’s a concern, but it’s one that I think can be solved with help from vendors. Microsoft recently announced there’s an audit capability for Azure SQL Servers, allowing customer to determine what actions Microsoft support engineers might take on your system. Since we may not necessarily see what is happening when someone else is working on our PaaS or IaaS systems, it’s good to have something tracking their moves. I don’t know this gathers all queries run, but it certainly should.

    I’ve seen some presentations on how customers have to authorize engineers in VMs with Customer Lockbox, and the technical details are interesting. These are time limited and audited access controls to ensure that customers are aware when a Microsoft employee touches their systems. I don’t know if this is comprehensive enough, but it is a good start.

    Over time, more and more of us have to delegate some work to others, either inside or outside of our organization. We need strong controls and good tools to ensure that we can determine what happened, especially in the event of some incident taking place. Building these controls is important, and really, they should be easily enabled in all our systems. While I don’t know we have a lot of rogue employees accessing things, we do have some, and we have more that make mistakes. Auditing their actions should be something we can easily do.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.