Voice of the DBA

Category: Editorial

  • Hard Drive Reliability

    Do any of you still use hard drives? I’m sure some of you do, and I’m also sure plenty of you aren’t sure if you’re using HDDs or SSDs. For some of us, the hardware is just a set of resources with a certain quantity, capacity and performance. While we used to spec, purchase, and assemble database servers in the past, these days may of us just specify cores, RAM, and IOPs or some storage capacity, not worrying about how our resources are actually implemented. I know that I assume SQLServerCentral is using SSDs in our hosted systems, but I honestly don’t know, much less care about the type of drives in use.

    I saw Backblaze release their 2018 hard drive stats recently, looking at the reliability of their storage. Since BackBlaze is in the business of storing data, they need to ensure that they have quality storage. I’m a customer, and I appreciate that they’re not just buying and using devices, but also tracking how well the different brands and models work. Since they have over 100,000 drives, this means they can provide some interesting information on reliability.

    As our storage needs grow, many of us won’t use SSDs for all our data. Even all the data in our database servers might not be SSD as the cost is high. I’m sure HDDs are well suited for backups, and potentially even still useful in many servers for data storage when workloads might not be as high. Even at home, I like to use SSDs as much as possible, but for some types of storage, especially pictures, audio, and video, HDDs are much more cost effective with their higher density.

    The Backblaze stats are interesting in a few ways. Certainly it’s amazing to see how larger and smaller drives perform. It seems that most drives have a very low failure rate, though there is an order of magnitude difference between different models of similar sizes. It doesn’t seem that the 10-14TB drives are worse than the 4TB drives, which makes me think that vendors are doing a great job with quality control in manufacturing. I used to worry about larger sizes being released and having lower reliability, but this doesn’t seem to be the case.

    Some drives have very low failures, but these are averages, so I’m not sure they’re necessarily better, but worth keeping an eye on. I also think that since many of these drives fail at < 1% annualized rate, they’re good choices for your data center. Certainly the HGST ones seem to have very low failure rates. I know I may pick up a couple of these for more photo storage as I’m running out. It seems that I never have enough space for all the pictures I take.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • DR Prep Can Miss the Little Things

    The other day we had a blizzard in Colorado and we weren’t quite prepared. My wife and I were away on a trip and watching the weather. We were concerned about flights and about the conditions at the ranch with someone else in charge of horses. We were lucky in that the weather didn’t come as quickly as predicted and we made it home before the snow started.

    Depending on moisture and temperature, we sometimes put blankets on horses, which is a chore that I’m not very good at completing. Fortunately the kids were around and at 8pm, we all dressed in warm clothes and headed out to the barn. The horses depend on us to provide for some of their needs, so we each had jobs to do. One kid secured a temporary holding area to keep horses nearby. One gathered 14 or 15 blankets from storage and laid them out. My wife made grain to facilitate attracting the horses and giving us a way to hold them while blankets went on.

    Me? I had the envious task of double checking electricity and tank water headers. Our employee had thought that one was shorting out and at night, with a flashlight and plug tester, i got to debug 4 water heaters, including one that’s semi-remote from the barn. Testing for shorts involves touching water to see if there is any flowing power (this is very low amperage), which isn’t terribly dangerous, but is a little daunting. We managed to get everything done, and felt like we were prepped for a 4-10″ snowfall, 50mph winds, and a 0-5F temperatures.

    Later I lay in bed, trying to relax with a little TV before calling it a night. While the wind was howling, I heard a pop and the power went out. We have a generator, and I wasn’t too worried, but I did want to ensure it came on. I walked downstairs and across the house to listen for the engine noise. I heard cranking, but the engine didn’t catch. It was then I remembered the tank was low the last time I checked, and with a weekly diagnostic auto run for 5 minutes, I was likely out of propane. Since we needed power to heat water for horses, I had to get up around 11pm and take care of things.

    Fortunately I keep a spare bottle of propane and I went outside in the blowing snow to change it. A true work-at-home-techie, I did this without pants and got things running. Back to bed, though worried a bit about how to get more propane in the morning, just in case of an extended outage. The power came back on in the morning, but I still needed to get more propane just in case.

    Extended disasters sometimes cause problems with our plans because many of us focus on the immediate reactions. Longer term, things like additional fuel, food for humans, even shelter and child care are issues that I’ve had to deal with in teams that no one had planned for. We’ve seen this over and over again in the world, especially when supply chains break down quickly in disaster situations.

    I could have prevented some of the issues and stress if I’d prepped things better. I should have filled propane tanks earlier, knowing that bad weather can come at any time, and likely will at some time. Double checking heaters and power earlier might have made the blanketing process go quicker. There are always things we miss in prep, often ones we don’t consider to be important, but may be difficult to deal with in the moment. Take a few minutes and think about the little things you might dismiss as not important. Imagine how hard some of them might be to deal with under the pressure and stress of a DR event. If you can do a little more prep, some maintenance, or get some work done early, now might be the time to do it.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Learn More than Technology

    I’ve been lucky and successful in life in many ways. Certainly I’ve worked hard, may long days and nights, but plenty of others have as well. In fact, there are plenty of people I know that have worked more hours, but been less successful. Conversely, I know some that have worked much less and been more successful. It’s hard to know what the best way is for each of us to examine how to move forward in life, but it’s also not really helpful to compare yourself with others as a measurement. What I’d note you should do is decide if you think you’re becoming more or less successful and find your own path.

    This could mean very different things to different people, and that’s fine. I don’t want to live other people’s lives. I want to live my own, learning from positive and negative things that happen to others, but following the path that best suits my wife and me. I include my wife because we’re a partnership. We decided to spend our lives together, and that means we work with each other, sharing success and sacrifice, making joint decisions. It’s worked out well for us, and I am looking forward to the next few decades of continuing on that path.

    I got inspired to write this by Troy Hunt’s 10 Personal Finance Lessons for Technology Professionals, which is better written than what I’d do if i wrote it. My wife and I haven’t been as focused as Troy has, but we’ve gone through a similar set of thoughts and execution in our careers. We were less money focused, and we don’t live on the Gold Coast, but we’re pretty happy with where we are. We’ve made a few real estate investments, along with some traditional ones, and I think we’ve been fairly successful.

    What inspired me, however, isn’t the advice about how to make money as much as the need to understand what money means to you and the implications of your choices. I do think money is necessary, at least some to live in the US. I also think money can make life better, but you have to learn to enjoy life first. Money should help you with things you enjoy, not make you buy things you think you (or others) will enjoy. However, to ensure you can make the choices you want to, learning a bit about money is useful.

    Most of us don’t get good financial literacy, and more importantly, we don’t think about what money really means. It’s a representation of time, and your time is the most valuable asset you have. I don’t advocate that everyone try to be rich, or retire at 40, or save every penny/pence/shilling/lira/yen and live like a pauper.

    Enjoy your life as you live it and learn to appreciate the small things that touch your days. Along the way, learn a bit about money so you can make choices. Develop a hobby outside of work, and enjoy the moments with friends and family. Life is short. Try to actively live to enjoy it.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.5MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Forgiveness

    We are frail, fragile creatures. I think this is one of the main reasons I push for and try to work in an agile, DevOps style of development. I know that I make mistakes, as do others. My goal is often to ensure our system, our process, our way of working can transcend those mistakes and we can still be effective. We can use automation and regular examination of what works and what doesn’t to ensure our system works well and isn’t overly dependent on one person.

    Part of dealing with mistakes and failures is learning from them. Not only do we need to learn ourselves, and avoid repeating mistakes, but we also need to learn to forgive others. I thought about this as I was reading Derik Hammer’s post on Active Forgiveness. In his post, he covers the situations where we acknowledge mistakes without a punishment. Instead, we forgive the act and go to work on finding ways to prevent the same situation from happening in the future.

    I think this is one of those concepts and practices that are very difficult for most leaders, and maybe more followers. We don’t want to excuse the behavior, and we certainly don’t want to tolerate this in the future. What we want is for this person, and others, to learn from the mistake and work to prevent it in the future. What often happens is that we blame the person, we’re glad we’re not the person, and we may not learn a lesson because we think that we wouldn’t make the same mistake.

    Throughout my career I’ve seen the same mistakes occur over and over. Sometimes the same person makes a mistake again, but often different people make the same mistake because there wasn’t a product retrospective where everyone learned for the future. Having a blameless meeting where discussions take place, mistakes aren’t minimized, and everyone learns is hard, but it is important if we are going to improve our teams performance over time.

    I’d encourage you to try to find ways to learn from failures and grow. Forgive others actively, but don’t minimize the mistake. Hopefully you can also help your management learn to do this.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.1MB) podcast or subscribe to the feed at iTunes and Libsyn.