Voice of the DBA

Category: Editorial

  • An Out of Date CAP

    I was reading this piece from Martin Kleppmann asking us to stop using the CAP theorem to talk about distributed systems. It’s of interest to me because we often see this applied to databases, and it seems I’ve seen database platforms described as fitting along one of the sides of the CAP triangle.

    However the complaint is good, although tough, reading. As I read through it, a number of the concepts and tradeoffs, and concerns are similar to what I see discussed in Azure systems, or any “cloud-like” distributed systems. We expect consistency and availability, as well as scaling through partitions. However, it seems that we are always trading consistency (what Keppmann notes is really linearizability) and availability in the data world. We simply can’t guarantee both of these things in a data world.

    Or can we? At some point, perfection isn’t required, and we can tolerate some level of inconsistency in the results clients might receive. I’m sure many of you have dealt with this in the past, and perhaps even gone so far as to add the report execution time to outputs to reduce client complaints. If two sets of data are compiled at different times, most clients tend to understand some discrepancies.

    Certainly many of us are starting to consider using database platforms that might not work like SQL Server. In those cases, availability and scale is often touted as the reason to abandon a RDBMS. On one hand, the lack of linearizability across nodes is often tolerable, and many of our businesses aren’t affected dramatically by them. On the other hand, if you can tolerate some delays in nodes, than perhaps SQL Server can work for you with multiple nodes in a replicated or AlwaysOn scenario.

    Distributed systems are hard, and certainly your decision shouldn’t be a simple one when it comes to choosing an architecture. However it also helps to think deeply about the problem in terms of not only the CAP theorem, but also the in terms of practical measures such as latency and practical failures.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.6MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Serious Hacking

    There’s a piece that calls the US Office of Personal Management (OPM) data breach the biggest government hack ever. It might be, but give us a few months and I’m sure something will be bigger. After all, we constantly acquire more data, so the next breech is likely to contain more information. I’m also not sure most of us are actually getting much better at security.

    There were a few notes about this that would apply to every company I’ve worked in. Such as the OPM not having a comprehensive list of devices and databases. I’m not sure any company does, and having worked with people that run SCOM-type systems, it’s a hard problem to solve. This doesn’t even cover the problems of Excel. Access, and soon, PowerBI data being scattered across systems.

    However there was one problem I think we could fundamentally improve in most companies. The article noted that OPM didn’t have control over how it’s systems were configured, meaning an attacker could reconfigure things. Far, far too many companies allow a (too) large group of people to deploy changes to servers. Even when larger companies limit rights for developers, I’ve too often seen operations staff log in and allow developers to change systems to get them working.

    As an industry, we really need to solidify and build better systems for ensuring the security of our hardware and software and preventing, or detecting, unauthorized changes. Certainly there will always be social engineering and other techniques that bypass security, but we should be able to prevent malicious changes to systems with solid architectures from our vendors/FOSS developers. We should also decide upon, and be sure, that our staff learn, understand, and follow, best practices.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Bad Management

    Many people know that a good manager can make a huge difference in the productivity of a staff. A bad manager can also lower productivity, but only to an extent. Fear of losing a job, or avoiding punishment can motivate plenty of staff to get work done, even when you have a poor, or even absent, manager.

    Some companies don’t value strong management skills much, some do. I think most of us would prefer to work for a company that does, but we don’t always get the choice. That’s one reason why I do promote the idea of branding yourself and ensuring you have other options for employment. The more options you have, and the more demand there is for your staff’s services, the more likely that your company will make an effort to work with, rather than lord over, their employees.

    For those of you that are managers, or those that might want to help their manager build some skills, I ran across an interesting story from David Haney on his experiences as a manager at StackOverflow. It’s a good set of principles that I tried to keep in mind as I managed people in the past. For those managers that might not know why their staff leaves, there’s a good warning that good IT staff are in short supply, and companies should make an effort to retain them in various ways.

    For many of us in the IT sector, we can provide a strategic advantage for our businesses as we enhance and extend software. Many of us are important insurance that keep systems running, hopefully at a peak level. Bad management can impact the effectiveness of our technology organizations, and it’s silly to accept bad managers. This is something that can be fixed with some training and accountability for the manager, with a little effort. Make a few suggestions, or try to show how a better manager (or style) might help you get more done.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.6MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • The Poor Soul

    This editorial was originally published on May 11, 2011. It is being re-run as Steve is away at SQL in the City.

    I recently had someone post this after I made a comment about someone taking responsibility for their database server.

    “So Steve, what would you recommend then for those poor souls?  As being one of them it drives me nuts to constantly be told I can’t get more training as it’s not my true job position yet I’m responsible for making the databases work. “

    I’ve been in this spot a few times, in and out of IT. I’ve been tossed into cooking or bartending jobs without training and had to learn quickly how to do the job, and I’ve had the same thing happen in IT. In a few cases the companies knew it was a bad situation and they eventually got me training, in others they didn’t. Here’s the advice I have for you.

    Ultimately you are responsible for the job. Tough love, but you’re being paid for that job, so you are responsible.  That means you have to learn how the technology in your environment works and how to solve the problems you have.

    First, get your resume up to date. Make sure it is ready for submission, and you are prepared to get fired every week.  Keep an eye on the job market and save some extra money, because to me, the financial security for my family comes first.

    Second, learn to restore data and then make sure you have backups in place. This is secondary because if you have a failure quick, you want to be ready to get a new job. But ultimately no matter what breaks or doesn’t work, getting data back first is crucial.

    As you go through all of this, you might be fighting fires. So while you practice restores or document the environment, you might be trying to fix things and asking questions of others, but invest the time to get yourself into a solid position.

    I’d also talk to my boss regularly. Every time I found a place I didn’t know something, I’d make a note and let my boss know this is a hole. Maybe I can learn it, maybe I can’t, but I could use help. That might be the best ROI for conferences. Go, make friends, get contacts that can fill your knowledge holes. Or find consultants you can call.

    Steve Jones