Voice of the DBA

Category: Editorial

  • Big Data Problems

    Nate Silver is a “data guy.” He has made a career of analyzing data, and doing it well. His site, FiveThirtyEight, has become very popular in the last few years after some of their political predictions were surprisingly accurate. Now the site publishes analyses of sports, politics, economics, and more.

    Mr. Silver works with a lot of data, as does the rest of his company, and arguably, he’s one of the people that understands just what “big data” is and isn’t. We could argue about what the term means, but no matter what “big” is to you, Mr. Silver has some thoughts about the challenges we all face with larger data sets.

    In an interview, he noted that storage and access are always issues. Many of are us know this because we are charged with managing data. However he also talks about bias, false positives, and complexity being problems with an analysis that isn’t performed well. It’s a short piece, and I wish there were more depth since many of these topics could be the subject of a book all by themselves.

    While many of us aren’t in charge of performing the analysis, we do often work with those that do. We assist others in writing the queries or assembling the data. Our skill in helping perform analysis, understanding the meaning of data stored in our systems, and learning how to separate the signal from the noise will be talents that your employer will appreciate.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Working with People

    Many DBAs have gotten the reputation of being difficult to work with. I think some of this is based on the impedance mismatch between developers and DBAs that seems to cause issues in many organizations. Many developers want their changes to be deployed quickly, while DBAs want extensive review and testing to be sure that no problems will occur. This prioritization of stability over enhancements by DBAs does make us seem difficult to managers, PMs, and no shortage of clients.

    As the job of data professional has morphed and matured, many of us that might have been strictly DBAs or developers in the past now often need to work with many other people. We find all types and ranges of personalities that we must deal with, and I would guess many of us find other people difficult to deal with.

    Learning to work with difficult people is a skill itself, and I ran across a piece that talks about a few ways that each of us might work with those we find difficult. Maybe more importantly, if we are perceived as difficult, perhaps we can learn a few things about ourselves and how we might adjust our own personality to work with others, or even perhaps we can give others ideas on how to best interact with us. The piece looks at emotional intelligence (EQ), which isn’t necessarily correlated to any other sort of intelligence, so don’t think a low EQ implies anyone is lacking technical talent.

    Our interpersonal interactions are important. As important as our technical skills, if not more so. Learning more about ourselves and learning how to better work with others are important skills for us that can help ensure we have an enjoyable, as well as successful, career.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • The Scientific Method

    What’s the greatest invention in the history of humankind? We probably each have some ideas, and we could certainly debate the answer, but Dave Farley says it’s science, and the scientific method. Many of us probably learned about the scientific method. Look at a problem, develop a hypothesis, test it, alter our hypothesis based on results. Repeat. Eventually come up with an explanation that’s proved by the evidence.

    This sounds like a good way to actually build a piece of software. However, is this what most of us do? Or do we continue to pile additional caveats and restrictions to our hypothesis and try to force the system to work a certain way.

    Is the scientific method something we need more of in software? I was watching the talk from Dave Farley, and he mentioned this. He talked about the need to examine a problem, guess how to solve it, try implementing your guess, and then examining the result. If it’s not correct, then you change something and try again. It’s an agile process that tends to work well when there are fast feedback cycles.

    However the one thing in the talk that was the most telling to me is that something isn’t finished until a customer can see it and give feedback. Until then, it’s really in limbo. I would guess that if more development efforts focused on making this happen quicker, we’d end up with better software, written more quickly.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • NoSQL Inside SQL Server

    There has been quite a debate the last few years between using NoSQL and SQL Server as data stores for different applications. We’ve seen a great deal of interest from developers in document stores as the data structures are closer to what a C# or Java developer sees in their code. Graph databases are becoming popular for certain types of data as well, since their query performance can be extremely powerful in some cases. There are also the scalability aspects of NoSQL systems touted as a better way to build Internet based applications.

    However NoSQL databases aren’t the ultimate data store. There are pros and cons to using them, and while it might be easier to being development on one for an application developer, I would argue it’s just as easy to get caught mis-using a graph database or a document database as it is to poorly program a relational database.
    This week, in advance of the SQL Server 2016 release, I caught a post from the SQL Server storage engine team showing how JSON storage inside of a table can provide some NoSQL-like benefits to applications. It’s a good idea, but like many NoSQL solutions, you need to spend a bit of time examining your particular requirements. If you need to update your JSON often, especially in large batches, this might not prove to be a good idea, especially if you need consistency.
    I think ultimately that everything does relate to the CAP theorem, precisely because data has to exist somewhere, and anytime you need to duplicate it, move it, aggregate it, or anything to relieve some other issue, then there isn’t any magic taking place. You’re going to have to do work to move the data. Some platforms may make that easier, but I bet there are other drawbacks that you will have to consider.
    Ultimately I think you need to pick the platform that best suits your environments. That’s not only the application requirements, but also the developers’ skills, the operational staff’s ability to manage the platform, and the maturity of the platform. Those are tough questions, and while I’m glad people experiment with new systems, I do hope that they think carefully about the positives and negatives, and more importantly, share their experiences with the rest of our industry.

    Steve Jones