Voice of the DBA

Tag: data

  • Data Clarity

    There was an Associated Press (AP) report recently that noted Google applications track your location, even if you’re turned off Location History on your Android phone. The article has details about what the AP noted, as well as the report from some researchers that were testing the functionality. You can read the details, but the issue doesn’t seem to be as simple as the headline of the report.

    Google has responded to the claims, saying that they document and explain the various settings that need to be changed in the applications themselves to prevent any tracking. That might be the case in the eyes of the engineers that built the functionality, but I would tend to argue that the expectations, the descriptions, and explanations we use as technology professionals might not be clear enough for most users. We ought to be documenting, explaining, and even coding systems for users that aren’t as familiar as we are with the technology.

    This is an interesting issue. Not the location tracking, since I assume Apple, Google, government, and more can track my phone if they really want. To me, the issue is that we have data practices that are not clear to the end user. What Google documents, what they do with new services and features, and what the clients expect are not necessarily the same. That’s an issue, and I suspect it’s a similar issue for many companies.

    Most of us collect some level of detail from our software on how the user interacts with it. This might be a local log, or it might be some sort of telemetry, similar to what Microsoft collects from SQL Server. In either case, I think it’s important to spell out what data is being collected and to what extent this data is related to a specific individual or company. The changes to data handling as a result of the GDPR and other legislation might require that we do a better job of disclosing any data we collect, and in which specific circumstances.

    I know that data matters, but I also think that lots of the information that is collected doesn’t need to be related to a specific individual. Aggregates or tokenized data is often enough, though if you need to track a particular individual over time, such as the features they use in their install, be sure that you are very careful with any sensitive data, such as names, locations, etc. Most of us don’t have Google’s resources to combat legal action if customers find we are infringing on their privacy.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Destroying Data

    For many of the data professionals out there, our job is to ensure that data is protected. We work to protect the information in our databases in a variety of ways that allow our organization to continue to work in adverse situations. We often schedule backup schemes to ensure we can recover data from losses. We perform maintenance to ensure efficient access to information, and we move data from one system to another so that a wide variety of applications and clients can use it. We spend most of our workday trying to ensure that data is where it needs to be.

    It’s worth considering that there may be times when we are asked to destroy data as well. If hardware is being upgraded, then it might be a part of our job description to ensure that no traces of data exist in old systems. Even if it’s not our job, we might want to ensure that hardware is destroyed or permanently wiped. It’s unlikely to be an issue, but there are numerous stories of someone buying used equipment with old data on it. This is such a low bar of something to check that there isn’t an excuse to avoid removing old data.

    If you aren’t sure what to do, don’t have time, or want independent proof, maybe you want to use a service. There are numerous ones, which makes sense with the growth in legislation requiring stronger data protections as well as the embarrassment from management that wants to outsource their risk to another entity. This might be especially handy for larger organizations that could have multiple types of devices, some of which aren’t as accessible as a hard drive in a server chassis. There are services to ensure wiping of phones, tablets, and more.

    It’s a little funny to me to think of destroying data when most of my career has revolved around protecting data. What I mostly think about here is that too often many organizations have thought about data as an asset of the company to be used as needed, but not necessarily protected throughout its life cycle, and certainly not classified according to it’s sensitivity.

    I hope that with the growing value of data, and regular misuse of information, and the security issues that appear in the news that most companies would start to actively manage their data in the future, from capture to destruction.

    Steve Jones

  • Data Has a Dollar Value

    It seems that every year we see new ways of analyzing information that companies are adopting. In this era of Big Data, with the challenges of real time BI analysis of (often) streaming sets of data, companies search for ways to handle the load. We had map-reduce methods to process bits a few years back and lately there has been a growing popularity of machine learning (and deep learning) used to gain insights from the massive data sets we have.

    The problem is that in trying to analyze data, we find that we often don’t have enough data in many cases. While some parts of our organizations face a surplus of data, others trying to provide an analysis might face a shortage, at least for some types of data. This might be especially true when business people want to engage in a new type of business or a new way of working with customers.

    The last couple years have given rise to a number of companies that actually are gathering and selling labeled data, or even generating synthetic data that can be used to build and train models for analysis. As we look to let machines learn to solve some problems on their own, we need to provide them lots of data, which has become big business. I have heard of companies paying six or seven figures a year to get data sets for their data scientists.

    In some sense, as noted in this keynote, data is the more valuable part of these systems. Staff matters, and certainly the software and models are important, but the data is key. Good data, with lots of features, can produce a better trained system than poor data. Many of us that work in traditional software know this as well. If we use poor data sets in development, with limited values, and not in the skew and selectivity that we’ll see in the live system, we often build lower quality software with more bugs.

    In some sense, I think that our data is more valuable than we realize, and far too many developers don’t take advantage of using the data our organizaition does have to build features and properly test them. Actually, too few of us actually test things well, but certainly we often can’t without a good set of data. I’ve been disappointed with random generators, though they are useful in that they can find unexpected issues from the random values, including NULLs, that will creep into systems. I really wish we had better subsetting tools that would help us use a portion of our production data. Redgate is working on tooling, but I’d think this was a problem we’d have gotten better at solving, between software people and database staff.

    I’ve had a nice career working with data, and I’m glad that the recognition of the value of data has continued to grow through the years. Now I’d like to see us actually start to emphasize the importance of producing and using more useful data sets when we build software, whether in traditional means or using machine learning techniques. My guess is we’ll get more useful and better quality software if we do.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Editorial Republish: The Value of Data

    I’m busy today, in the Redgate office, rehearsing and practicing for the SQL Privacy Summit.

    Today I’m republishing The Value of Data.