Voice of the DBA

Category: Editorial

  • Downtime

    Most of us that work with technology hate downtime. We don’t want a system that we’re using to go down. We don’t want any software that we depend on to fail when we need it. Most of all, we don’t want our phones ringing because some system we’re responsible for has gone down. We do everything we can to keep our applications online. We avoid patches. We try to test as much as possible before deploying changes. We also may apply generous amounts of hope and prayer.

    However that’s not how all companies run their internal systems. Netflix has taken the opposite approach, actually creating downtime for some of their systems using what they call a “chaos monkey,” and they think it could help you. To be fair, Netflix doesn’t take their entire application offline, but they do cause failures in the hardware and software, specifically to see if their redundant and scaled-out architectures can limit the impact on users.

    It’s an interesting idea, though one that I’ve not seen many companies be willing to implement. Netflix thinks you could benefit from it, but they also run a series of services that are scaled our across many machines. Many companies I’ve worked with have services on one machine handling an application, and they accept the risk that a system might fail and users will experience problems. Given the quality of modern hardware, that might be a good bet to place these days.

    However more and more of us are running redundant systems for some applications. If you think the Chaos Monkey could help you, let us know.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • Geeky Entertainment

    The 2014 Oscars are this weekend, and while I only have a passing interest in who wins the various awards, I did manage to see a number of the nominees and enjoyed most of them. I missed a few as well, and am hoping to find time to watch some of these movies later this spring.

    I didn’t see many technological, science fiction, or “geeky” movies this past year, but my daughter and I did enjoy the latest Marvel installment of Thor and are looking forward to Captain America later this year. I’m not sure I want to see Guardians of the Galaxy, but my “Marvel fan girl” may talk me into it. I did enjoy Ender’s Game last year, and thought it was an interesting take on a book I enjoyed many years ago.

    With a few new movies in the works (Star Wars and Star Trek), I wanted to ask you what you might hope for.

    What “geeky” movie would you like to see made?

    It could be a comic book based movie, another installment of an existing movie franchise, or perhaps a book that you wish would appear on the large screen.

    I’ve read a lot of books over the years, comic and longer format, and of all of them, I think I’d like to see the Old Man’s War series produced as movies. Or perhaps even as a TV show. I also enjoyed Red Thunder more than once, and think that might make a thrilling movie. Hyperion is an amazing story as well, though I’m not sure it would fit into a movie. Perhaps we’ll see that produced at some point as a multi-year adventure show.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • Caching

    How many of you have worked with a cache of data for your application? I would bet that the vast majority of you have never built a caching system. There’s not shame in this, as most of us don’t work with very high levels of concurrency in our applications. Even if we hit them, it’s rare enough that we wouldn’t bother with the extra coding when we designed the application because, well, it’s extra coding. It takes time, requires more testing, and our project managers might not be willing to invest in this “just in case” we hit some concurrency issue. Actually, most of us probably would view this as a “good problem”, one that would then be worth re-architecting the application.

    The problem is that often no one wants to re-architect a working application and potentially delay other, more exciting enhancements. Many highly visible web sites have gone through this, experienced growing pains, and only those that have a tremendous amount of resources to devote to the problem usually solve it.

    However it doesn’t have to be hard. Brent Ozar published a piece on caching results that uses a creative solution, scaling out to a new database and table that exists only to return results. In some sense, this is a great application of the KISS principle that doesn’t use replication or any other complex technology. It even employs a technique those NoSQL platforms often hawk: eventual consistency.

    That’s the idea behind caching. The data isn’t necessarily the most up to date, and it’s not dependent on being synced with your other data. Ideally you’d have some system that updates your cache when the base data changes, but even then there’s an “eventual consistency” delay at work. It might be tens, or hundreds, of ms, which is often what we find with distributed systems, but that’s usually fine. It’s very rare that a problem requires absolute synchronization of all data in the application.

    I’d urge you to consider thinking about  building some scaffolding into your application that can potentially be built out later to incorporate caching, or some other scaling technology, in the event that you find your system is more popular than anyone might expect. It shouldn’t be too hard, especially if you think about it early on.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.6MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • Lawsuits and Data Breaches

    After the breach of the Target payment systems, a class action lawsuit was filed against the company. The action alleges that Target failed to implement strong enough security. Regardless of the merits of this action, it does bring to light a few things that we, as data professionals, should be aware of.

    The first is that if our companies store any PII, financial, medical, or other sensitive data, we need to ensure that our management is aware of potential security pitfalls we see, as well as the possibility for legal action if the data is somehow disclosed. The risk and mitigation actions taken need to be weighed by management, and we should approach this as we might any other upgrade or enhancement to a system. With logic, and rational discussion about the issues, providing guidance and potential solutions.

    However we also should be aware that no matter what security efforts we undertake, criminals are going to be finding ways around our defenses. As this piece notes, Target likely had security in place, but it’s never going to be enough because the attack vectors and techniques are out-pacing the ability of security techniques to provide protection. The solution, or at least a potential mitigating action, is one that data professionals can help with.

    We, and the businesses that employ us, should be incorporating analytics into our defenses to detect abnormal actions, by both external and internal, users. We should be looking for potential ways that data is disclosed, and perhaps even scanning the Internet for potential leaks of data. We won’t prevent all problems, but if we can detect them early, we can limit the damage.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.