Voice of the DBA

Category: Editorial

  • Super Nerds

    I enjoy the game of baseball, playing as a kid and then for a decade as a 40 year old adult. I gave it up a few years ago, worried about the wear and tear of sporadic play. I’m happy now coaching kids and participating in individual sports for exercise, but I still love the game.

    A few years back I read the book, Moneyball. The book is about the use of metrics and analysis in addition to the human scouting evaluation in the Oakland As organization. There was a movie made as well, and both are worth consuming. Since that time,  other baseball teams have adopted some of the ideas, and I’ve been hearing that both basketball, NFL football, and other sports in the US are using more metrics. There’s even a Revisionist History episode that talks about football/soccer and the way that makes the most sense to improve your team.

    Recently I ran across an article with a great headline: Jayson Werth rails against ‘super nerds’ that are ‘killing the game’. In the article, a player rants about the way in which data and statistics are being used to make decisions. Instead of allowing players to just play, analytics become a part of the decision to play a certain way. For example, bunting and stealing have dramatically declined, mainly do to the analytics that show these are lower percentage actions compared to other choices.

    It’s interesting to hear players rant against the user of statistics. I completely get the annoyance at losing control of your choices, and certainly appreciate that the game becomes less exciting at times. However, I also know that players, and even coaches, may make emotional decisions, or base decisions on poor information. Most of us humans can’t remember all the tendencies and likelihoods. In a modern world where skill levels have dramatically increased in many ways, there are often better ways to build a strategy.

    Data is valuable, and certainly can help in sports. It isn’t the end-all be-all, and it can be misleading when applied to individual humans. At the professional level, where more data is available, I think it makes sense to use data more as a significant part of your decisions, though not the only factors. For me, as a coach of younger kids where I have relatively little data, I still use the eye-test for most things, relying on data to double check my thoughts. The world moves fast, and it can be easy to forget how individual players have performed across an event, especially when I’m trying to manage the game as well. Data helps me remember how the day is going.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Destroying Data

    For many of the data professionals out there, our job is to ensure that data is protected. We work to protect the information in our databases in a variety of ways that allow our organization to continue to work in adverse situations. We often schedule backup schemes to ensure we can recover data from losses. We perform maintenance to ensure efficient access to information, and we move data from one system to another so that a wide variety of applications and clients can use it. We spend most of our workday trying to ensure that data is where it needs to be.

    It’s worth considering that there may be times when we are asked to destroy data as well. If hardware is being upgraded, then it might be a part of our job description to ensure that no traces of data exist in old systems. Even if it’s not our job, we might want to ensure that hardware is destroyed or permanently wiped. It’s unlikely to be an issue, but there are numerous stories of someone buying used equipment with old data on it. This is such a low bar of something to check that there isn’t an excuse to avoid removing old data.

    If you aren’t sure what to do, don’t have time, or want independent proof, maybe you want to use a service. There are numerous ones, which makes sense with the growth in legislation requiring stronger data protections as well as the embarrassment from management that wants to outsource their risk to another entity. This might be especially handy for larger organizations that could have multiple types of devices, some of which aren’t as accessible as a hard drive in a server chassis. There are services to ensure wiping of phones, tablets, and more.

    It’s a little funny to me to think of destroying data when most of my career has revolved around protecting data. What I mostly think about here is that too often many organizations have thought about data as an asset of the company to be used as needed, but not necessarily protected throughout its life cycle, and certainly not classified according to it’s sensitivity.

    I hope that with the growing value of data, and regular misuse of information, and the security issues that appear in the news that most companies would start to actively manage their data in the future, from capture to destruction.

    Steve Jones

  • Data Has a Dollar Value

    It seems that every year we see new ways of analyzing information that companies are adopting. In this era of Big Data, with the challenges of real time BI analysis of (often) streaming sets of data, companies search for ways to handle the load. We had map-reduce methods to process bits a few years back and lately there has been a growing popularity of machine learning (and deep learning) used to gain insights from the massive data sets we have.

    The problem is that in trying to analyze data, we find that we often don’t have enough data in many cases. While some parts of our organizations face a surplus of data, others trying to provide an analysis might face a shortage, at least for some types of data. This might be especially true when business people want to engage in a new type of business or a new way of working with customers.

    The last couple years have given rise to a number of companies that actually are gathering and selling labeled data, or even generating synthetic data that can be used to build and train models for analysis. As we look to let machines learn to solve some problems on their own, we need to provide them lots of data, which has become big business. I have heard of companies paying six or seven figures a year to get data sets for their data scientists.

    In some sense, as noted in this keynote, data is the more valuable part of these systems. Staff matters, and certainly the software and models are important, but the data is key. Good data, with lots of features, can produce a better trained system than poor data. Many of us that work in traditional software know this as well. If we use poor data sets in development, with limited values, and not in the skew and selectivity that we’ll see in the live system, we often build lower quality software with more bugs.

    In some sense, I think that our data is more valuable than we realize, and far too many developers don’t take advantage of using the data our organizaition does have to build features and properly test them. Actually, too few of us actually test things well, but certainly we often can’t without a good set of data. I’ve been disappointed with random generators, though they are useful in that they can find unexpected issues from the random values, including NULLs, that will creep into systems. I really wish we had better subsetting tools that would help us use a portion of our production data. Redgate is working on tooling, but I’d think this was a problem we’d have gotten better at solving, between software people and database staff.

    I’ve had a nice career working with data, and I’m glad that the recognition of the value of data has continued to grow through the years. Now I’d like to see us actually start to emphasize the importance of producing and using more useful data sets when we build software, whether in traditional means or using machine learning techniques. My guess is we’ll get more useful and better quality software if we do.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Best Days

    I recently asked about the worst days in your career, so today let’s turn it around. What are the best days of your career? If you can give us a list of 2 or 3 days, projects, accomplishments, etc., it might be fun to read.

    Throughout my career, I have lots of little successes where I accomplish something that a client needs. Those add up, and they’re good, but by nature there are so many and the numbers tend to blur the lines in my mind. Unfortunately, the mistakes I’ve made tend to be more memorable and stick in my mind. I really have to think about what I would consider the best days.

    There are a couple software deployments that I consider to be successes, though none overly memorable. Getting my code into production is great, but rarely is a change a huge win. Instead, it takes time for clients to use the code, to understand the impact, and for me to be sure that the code works as expected.

    I remember getting our first automated deployment done, almost twenty years ago, when we had moved from a manual process of collecting scripts and trying to ensure they were correctly ordered and contained all the changes we needed. After weeks of working on an automated process to collect, order, and execute scripts, we tested things in QA. Then a day later in production. Overall this was a non-event, with the deployment taking minutes. The change from constantly checking scripts, results, and manually executing code was amazing and the reduction of time spent getting ready was notable.

    I also remember a great day as a DBA, when we experienced a server crash and had to rebuild new hardware. I was away with family, but I knew my backup and restore procedures were ready. I walked a manager through restoring a database, using a script to automatically build the log restore scripts, and then executing those scripts. While driving with my family, completely hands off, I helped recover a database.

    I hope you have some good memories of your career, and can share one or two today.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and Libsyn.