Voice of the DBA

Category: Editorial

  • Destroying Data

    For many of the data professionals out there, our job is to ensure that data is protected. We work to protect the information in our databases in a variety of ways that allow our organization to continue to work in adverse situations. We often schedule backup schemes to ensure we can recover data from losses. We perform maintenance to ensure efficient access to information, and we move data from one system to another so that a wide variety of applications and clients can use it. We spend most of our workday trying to ensure that data is where it needs to be.

    It’s worth considering that there may be times when we are asked to destroy data as well. If hardware is being upgraded, then it might be a part of our job description to ensure that no traces of data exist in old systems. Even if it’s not our job, we might want to ensure that hardware is destroyed or permanently wiped. It’s unlikely to be an issue, but there are numerous stories of someone buying used equipment with old data on it. This is such a low bar of something to check that there isn’t an excuse to avoid removing old data.

    If you aren’t sure what to do, don’t have time, or want independent proof, maybe you want to use a service. There are numerous ones, which makes sense with the growth in legislation requiring stronger data protections as well as the embarrassment from management that wants to outsource their risk to another entity. This might be especially handy for larger organizations that could have multiple types of devices, some of which aren’t as accessible as a hard drive in a server chassis. There are services to ensure wiping of phones, tablets, and more.

    It’s a little funny to me to think of destroying data when most of my career has revolved around protecting data. What I mostly think about here is that too often many organizations have thought about data as an asset of the company to be used as needed, but not necessarily protected throughout its life cycle, and certainly not classified according to it’s sensitivity.

    I hope that with the growing value of data, and regular misuse of information, and the security issues that appear in the news that most companies would start to actively manage their data in the future, from capture to destruction.

    Steve Jones

  • Data Has a Dollar Value

    It seems that every year we see new ways of analyzing information that companies are adopting. In this era of Big Data, with the challenges of real time BI analysis of (often) streaming sets of data, companies search for ways to handle the load. We had map-reduce methods to process bits a few years back and lately there has been a growing popularity of machine learning (and deep learning) used to gain insights from the massive data sets we have.

    The problem is that in trying to analyze data, we find that we often don’t have enough data in many cases. While some parts of our organizations face a surplus of data, others trying to provide an analysis might face a shortage, at least for some types of data. This might be especially true when business people want to engage in a new type of business or a new way of working with customers.

    The last couple years have given rise to a number of companies that actually are gathering and selling labeled data, or even generating synthetic data that can be used to build and train models for analysis. As we look to let machines learn to solve some problems on their own, we need to provide them lots of data, which has become big business. I have heard of companies paying six or seven figures a year to get data sets for their data scientists.

    In some sense, as noted in this keynote, data is the more valuable part of these systems. Staff matters, and certainly the software and models are important, but the data is key. Good data, with lots of features, can produce a better trained system than poor data. Many of us that work in traditional software know this as well. If we use poor data sets in development, with limited values, and not in the skew and selectivity that we’ll see in the live system, we often build lower quality software with more bugs.

    In some sense, I think that our data is more valuable than we realize, and far too many developers don’t take advantage of using the data our organizaition does have to build features and properly test them. Actually, too few of us actually test things well, but certainly we often can’t without a good set of data. I’ve been disappointed with random generators, though they are useful in that they can find unexpected issues from the random values, including NULLs, that will creep into systems. I really wish we had better subsetting tools that would help us use a portion of our production data. Redgate is working on tooling, but I’d think this was a problem we’d have gotten better at solving, between software people and database staff.

    I’ve had a nice career working with data, and I’m glad that the recognition of the value of data has continued to grow through the years. Now I’d like to see us actually start to emphasize the importance of producing and using more useful data sets when we build software, whether in traditional means or using machine learning techniques. My guess is we’ll get more useful and better quality software if we do.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Best Days

    I recently asked about the worst days in your career, so today let’s turn it around. What are the best days of your career? If you can give us a list of 2 or 3 days, projects, accomplishments, etc., it might be fun to read.

    Throughout my career, I have lots of little successes where I accomplish something that a client needs. Those add up, and they’re good, but by nature there are so many and the numbers tend to blur the lines in my mind. Unfortunately, the mistakes I’ve made tend to be more memorable and stick in my mind. I really have to think about what I would consider the best days.

    There are a couple software deployments that I consider to be successes, though none overly memorable. Getting my code into production is great, but rarely is a change a huge win. Instead, it takes time for clients to use the code, to understand the impact, and for me to be sure that the code works as expected.

    I remember getting our first automated deployment done, almost twenty years ago, when we had moved from a manual process of collecting scripts and trying to ensure they were correctly ordered and contained all the changes we needed. After weeks of working on an automated process to collect, order, and execute scripts, we tested things in QA. Then a day later in production. Overall this was a non-event, with the deployment taking minutes. The change from constantly checking scripts, results, and manually executing code was amazing and the reduction of time spent getting ready was notable.

    I also remember a great day as a DBA, when we experienced a server crash and had to rebuild new hardware. I was away with family, but I knew my backup and restore procedures were ready. I walked a manager through restoring a database, using a script to automatically build the log restore scripts, and then executing those scripts. While driving with my family, completely hands off, I helped recover a database.

    I hope you have some good memories of your career, and can share one or two today.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

     

  • What Data is Really Needed?

    The GDPR took effect in May of this year, at least with regards to enforcement. A few days after the May 25 date, a German court ruled against ICANN, the company that registers domain names on the Internet and manages the global WHOIS database. The case revolves around the information collected when you register a domain. ICANN wants multiple contacts, which they’ve required for decades. However, a company in Germany that is a partner, argued that the additional technical and administrative contacts were not required for fulfilling the business that both ICANN and EPAG (the German registrar) are engaged in. ICANN Is appealing the ruling, citing the need for clarification of what this means with regard to the law.

    This is interesting to me, because a) it concerns data, and b) there is an interesting argument here to be made about what data is needed for a business purpose. I could see this being argued successfully either way, and not just in court. As a domain holder, does the registrar really need multiple different sets of personal information from me? Arguably, this is a convenience for them, one that is based on tradition. However, one could argue the other way.

    It is a little scary that a court, with no expertise in some industry (Internet domain registration, in this case), will decide if there is an actual business need. After all, can a lawyer or judge really understand what data a business needs in their daily activities?

    Maybe, maybe not, but I do think this forces businesses to actually stop and think about what data they collect, have a justification, and document that. That’s a good thing, because often I find business people just asking to collect data without any idea what they’ll do with the information. I also find technical people collecting data, not maliciously, but often to anticipate what might be asked of a system, or because they want to avoid rework and just decide to collect everything they can.

    Data is precious, and while I don’t want to put many limits on what data businesses can collect, I also don’t want to them be able to collect anything, not disclose what they’ve collected, and not secure it properly. Having some limits, or at least forcing them to consider the risk of holding old, useless data, is likely a good thing for all of us.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.7MB) podcast or subscribe to the feed at iTunes and Libsyn.