Voice of the DBA

Category: Editorial

  • Expect the Unexpected with DiRT

    Disaster recovery is one of the core tasks that many DBAs think about on a regular basis. Ensuring that we can get our data back online, available, accessible, and intact is important. More than a few DBAs that haven’t been able to recover systems, find themselves seeking new employment.

    That’s not to say that most DBAs perform perfectly under pressure. Plenty make mistakes, and there may be times when they can’t recover all data. There does seem to be a correlation between how often DBAs practice recovery skills and how well they perform in an actual emergency. I know that at a few companies, we scheduled regular disaster tests, though often with simulated recovery of a systems that didn’t expect to actually take over a workload. Arguably not a good test, but better than nothing.

    Google takes things a step further. They have annual, company wide, multi-day DiRT (Disaster Recovery Testing) events. These are across many departments and can be substantial in terms of the disruption that the these events cause to their infrastructure. This is a way for the various individuals responsible for infrastructure to actually evaluate if they are prepared for potential issues.

    If you read the article, you find that Google started small with these and progressed them to larger, more inclusive tests, like taking down a data center. They also whitelist some servers, knowing they cannot pass a test, so there is no reason to actually take them down. After all, business still needs to work.

    It’s good to have tests and walk through actual events, like call lists and bridges to be sure that communication and documentation work. This might be especially important when teams often expect that all their written procedures are available. I went through an audit with one company, where we failed immediately when all our DR plans were on a network share. In this simulation, we had experienced a network failure and servers had crashed. We were supposed to bring up the systems on spare hardware, but some critical documentation wasn’t available without a network. We started printing things out right away so that we could continue on with the simulation (as well as have this in a binder in our office).

    Not everyone can schedule large scale tests, and certainly many managers don’t see the point. They’ll often want to gamble that staff will “figure things out” if there is an incident. That doesn’t mean that DBAs and sysadmins can afford to wait for a disaster to practice some skills. Be sure that everyone on your team can recover databases, they know where backups are (or how to determine this), and multiple people have access to resources. The last thing you want is a disaster to occur during your vacation and have managers calling you to cut short your holiday because you’re the only one that knows where something is or has the authority to access a resource.

    Think about this ahead of time and prepare.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • HSA for Training

    Most of us don’t get any sort of training budget from our employer. In fact, some of our employers expect us to learn more and get certified on our own time and our own dime. Relatively few of us get to attend a conference that our employer pays for, which is a shame. I think conferences can be exciting and encouraging in helping you drive your career and knowledge forward.

    This is one reason that Andy Warren and I started SQL Saturday. We wanted to bring the conference experience to many people whose employers might not pay for a trip. That has grown to over 100 events a year, all around the world, primarily thanks to tons of local efforts and the support of PASS.

    For most of us, however, training involves some cost, often financial. Some people pay for a class or conference themselves, many pay for books, and some get their employer to provide funds. Asking an employer, however, is a chore, and this can be intimidating for many people.

    Managers often are whimsical about approving training budgets, even for things like books. Either the employee really has to justify the cost (conference) or they may feel silly about asking for a small amount (a book). Managers also need to track and somehow justify their spending on training and balance this among employees. This creates friction and hassles for everyone.

    Perhaps there’s a better way. For those in the US, you might be familiar with the Health Savings Accounts (HSA) for medical expenses. In these, an employee can put away their own money for use on whatever medical costs they may incur. There are tax advantages here, but perhaps that would be something that would work for training.

    Imagine an HSA for training, where the employer would designate an amount that each employee could use for training. This would be per year, with carry over amounts. The total budget per year might also be based on job and seniority. There also wouldn’t be a need to seek approval, but rather, notify someone that you were using funds for some educational experience.

    In any case, the employee would have a fund that they could use at their own discretion, and be both responsible and accountable for training. Maybe there are caps on accrual, or a limited list of places to spend the money, but this would remove some of the friction of keeping employees trained. We treat them like adults and expect them to make good decisions for their career. If there are issues, we handle them on a case by case basis.

    I’m wondering if any of you have a training budget that is better structured. Perhaps some of you always get your requests approved, and perhaps some of you never ask. Let us know today what you think of this idea, and would this encourage more learning at your organization?

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Ransomware Hero

    One of the common attacks against many organizations today is ransomware. I’m constantly surprised when I see friends and they tell me they’ve been attacked, but the incident never made the news. I think many people would be upset with businesses if they knew the extent of attacks, but I also think that this would force more organizations to implement better security if shareholders and customers were aware of the incidents. This might even get employees and others to be more careful about the things they click and do, though that might be wishful thinking.

    In Illinois, in the US, there is a guy that is fighting ransomware and helping out both organizations and individuals. For free, which is much better than paying the fee, something plenty of organizations have done. Michael Gillespie runs ID Ransomware, a site that helps you determine what type of attack you’ve had. It can also possibly give you or generate a key that will allow you to decrypt your files.

    This man does this in his spare time, while working as a computer support person. It’s an amazing story, and one that is a little sad. Here is this man that doesn’t want to charge, and wants to help, but has struggled financially while doing so. Things have gotten better, and more people are supporting his efforts, including Emsisoft, who has hired him part time.

    If you get struck, it might be worth a visit to ID Ransomware and see if you can get some help. If you do, convince your boss to donate some payment to support Michael’s efforts. It’s a good cause, and certainly cheaper than paying a fee or losing lots of work while you try to sort out the problems.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The 2019 Techie Gifts

    It’s Black Friday in the US, and I assume, most of the world. With the advent of so many online options, I’m not sure how big a day this is for many people, but there certainly are many deals from local retailers. While I do like to buy local, I don’t plan on doing any shopping today. It’s just too crowded and busy for me. Likely a day of chores around the ranch for me.

    There have been a number of Black Friday deals already going on for days. Brent Ozar has had a sale on training all month. Pluralsight had one last year, and I assume another coming. I expect there are other deals as well, but this week I wanted to talk about a few items that I use already or need to purchase. It’s the season for giving, so perhaps this will give you some ideas for the tech friends in your life.

    I have a couple wireless charging pads I use regularly: one on my desk and one by the bed. It’s fantastic to be able to pick up or put down my phone without wires. I also have quite a few long Anker cords for my phone (here’s a USB-C and a lightning one). The more I use a 6ft cable in different places, the more I appreciate the length. Especially when charging in a public place. To help, I also have a travel adapter with USB plugs. I’ve used this often in the US when I need to share an outlet with someone.

    I love music, and while I get by with cheap headphones often, I am tempted by a better set of noise cancelling ones. Maybe my wife will read this and get me something. She does carry a JBL speaker with us on trips, which we often use in our hotel room for some nice ambiance. I also carry around a wireless HDMI adapter, which comes in handy on the road, for work and pleasure.

    I’m about ready to get a new laptop, and I’ll likely go with an HP Spectre again, though the Lenovo X1 and the Dell XPS both look really nice. I’m still debating, and if there’s a good sale this weekend, I might grab one. These days I think most laptops will do the job for me, but I like the small, lightweight ones for traveling. I’ve had no shortage of adapters, but I am thinking to get a small USB-C hub that might work well for a variety of tasks.

    I’ve tended to stay away from any of the voice powered devices, mostly for privacy reasons. I have an older tablet, but most of the time my phone or laptop are what I use, so I don’t have a need there, but the new iPads look really nice, as do the Galaxy’s. I have a smart watch, and I love it. I certainly see the appeal of the Apple Watch or the Galaxy Watch. It’s on my list to pull down my data and start playing, something every data pro might enjoy.

    There’s a lot more to the holiday season than shopping, but since it’s a big part of today, I wanted to list a few items that I’ve enjoyed, and encourage you to add more in the discussion. Perhaps you’ll give someone a good idea for a gift, or an item to put on their list.

    Disclosure: These links are affiliate links for me, and I get a small commission from any purchases.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.