Voice of the DBA

Tag: GDPR

  • Losing Track of Data

    I saw this article a few months ago, which talks about engineers at Facebook not knowing where their customers’ personal data is stored. The engineers were being questioned in a legal matter, where they were asked to definitively state where all personal PII data for any human was stored by Facebook. Their answer was that they didn’t think anyone in the company would be able to answer that question.

    Facebook has been controversial over the years, and plenty of people dislike the way the company conducts business. I noticed no shortage of data people (and many others) commenting on this situation, saying that Facebook should be shut down because they don’t know where data is being stored.

    However, I don’t agree. In working with lots of customers, on all aspects of how they handle, process, and manage data, I expect this to be a problem in many organizations. Whether large or small, whether they have few or many software engineers, it is highly possible that there isn’t a good list of where personal data is being stored. As we work with customers to classify data with SQL Data Catalog, that process takes a long time, and very often the system administrators or developers who undertake take the task are unaware of all the places where data is stored.

    That’s just in relational databases, ignoring all the Excel spreadsheets, text exports, mail merge operations, and uploads to services for mailing, analysis, or something else. Very often the control of personal data is fragmented among groups, with there being few efforts made to coherently manage a customer’s data.

    The world has adopted computing at an incredibly fast pace, often by people with little knowledge or forethought of the implications of gathering and processing data. In many cases, probably most cases, there is no overriding strategy. Just like with applications slapped together quickly, we find data being gathered and stored based on the requirements and demands of business people, with no planning for management or archival, and often not even with any security requirements.

    I liked the GDPR as a step forward, asking companies to not only handle data appropriately, but remove it when not needed, not use it without consent, and to be able to keep track and delete it if not necessary. I don’t know that this has been successful, but it has changed handling practices in some organizations. At least in responsible organizations, and many of them have had to track down personal data to delete it. I’m not sure they know where it all is, but I at least assume they know where all of the data about a person is in their various relational stores.

    As a technical person, do you know where all data is stored about a customer? Are you sure you know where marketing has been keeping information and what other mailing, analysis, reporting, CRM, etc. systems they’ve put data? Any idea how many copies the operations group keeps? Test systems, QA, UAT, and others? What about test data sets, are they sanitized? Perhaps legal or finance has gotten extracts of data to reconcile their systems.

    Tracking down all data can be hard, and I’m not surprised Facebook struggles. I would guess engineers in many organizations would have similar answers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A Data Controversy

    Quite a bit has changed since this article about airlines and the US government.  Since very few people are flying, or even can fly, perhaps this disagreement is moot, but I bet it comes up again. Now, separate from the idea of the actual disagreement here, there is an interesting discussion about the data involved here. In short, the US government wants airlines to collect data about passengers to help track the COVID-19 virus. Airline executives say they can’t easily get this data, other than on paper, without spending a few months on development.

    Certainly having a way to gather additional information in a digital form can require some development work. There are all sorts of software decisions to be made about when, where, and how users might input information. We have mobile devices, kiosks, laptops, and more, all of which might require separate interfaces for software changes. There is also the testing, validation, and verification we want to ensure the software works well and doesn’t introduce instability.

    In today’s world, with growing legislation, there is also a question of privacy. These requests may or may not conflict with other laws that airlines are bound by. There is likely to be more conflict here as the world changes and laws are slow to change and converge in some type of consistency. Rapidly changing requirements, as have been pushed during the COVID-19 pandemic, can potentially put us technical people in a difficult position. We have to balance the urgency of meeting requirements with the potential liability of violating privacy. I’d hope we could find some balance there, especially in a crisis.

    We do need to be flexible and ready to adapt to changing requirements. If regulations change, our organizations ought to be able to prioritize these changes and rapidly deploy them. In today’s world, where many high performing DevOps companies can get new software out in hours or days, governments may expect large companies to be prepared to follow suit. In that case, especially where new data is needed, having a software development process that includes the database is critical.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Don’t Get On This Page

    The GDPR has been in effect for over a year. While the press has died down a little, outside of a few record fines, there is still plenty of activity. At Redgate, we have customers that still worry about compliance and are doing their best to ensure they properly handle and secure data. It’s a tough job, but one that many organizations need to continue to focus on.

    And there’s a good reason. Who among you would want their boss to come talk to them about mishandling data? Who wants their boss to come after reading this page, with your organization and fine listed? I’m guessing most of us would prefer to not be on that page, or at least not want our boss to know.

    There have been a lot of fines handed out, though most are relatively small. Still, every amount spent towards a fine is money that could be used for shareholders, investment, or even better security and systems to prevent future issues.

    What’s interesting is that many of these fines aren’t for data breaches, but rather for other issues. There are some security issues (unauthorized access), and some inappropriate storage. There are also quite a few consent fines, where data is used without the appropriate permissions from the data subjects.

    I find the list interesting, and I hope this is the type of thing that does drive change in organizations. Many of these fines might easily be eliminated with a few process and procedure changes and adding some security to prevent unauthorized access. While some companies might be willing to pay fines, I expect that subsequent amounts will rise, and it behooves organizations to change both their behavioral and technical practices.

    This might be building archival processes, which would be one of the few ways that might reduce the amount of data that we need to manage and query. Hopefully, something that might improve performance for your clients.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Death of the GDPR

    When I first heard of the Sarbanes-Oxley Act, I was disheartened. It sounds like another over reaction by politicians to a crisis that would create lots of work for companies. At the time I was working for a software vendor and while we weren’t bound by many regulations, we did value our ISO certification. As we went through the SOX audit, it felt a lot like our ISO audit the previous year. In fact, we re-used most of our work, and re-did some, making it compliant with both items.

    A few years ago, we started GDPR preparations at Redgate, as well as talking and working with customers. We built some products, like SQL Provision and our Data Catalog, with the hope that they would be useful to organizations looking to ensure compliance. We have seen many companies trying to comply, and I’m glad that we can help. I think the GDPR is a step in the right direction for the future of how we manage and deal with data.

    I’m also glad that companies are taking the idea of data rights and data privacy more seriously. I think those of us in technology have been lax, and management of many companies even worse. I think I may some some evidence in this piece about possible lack of enforcement of the GDPR by Ireland. Not to discuss politics, but I completely understand when a very large company brings a lot of revenue to a country, there will be hesitation in disturbing that relationship.

    Regardless of the politics, I did find this quote interesting. A report in 2011 on Facebook’s data handling practices had this quote: “We do not consider that reliance on developer adherence to best practice or stated policy in certain cases is sufficient to ensure security of user data…” This didn’t prompt Facebook to make changes to process, and Ireland has not pressured them to do so, even with the GDPR taking effect last year. I’m hoping this isn’t the death of the GDPR as a set of regulations.

    I completely agree with the sentiment of that quote. We can’t trust that developers will adhere to best practices, or that companies will protect data. Sometimes this is ignorance, sometimes human error, and other times willful disregard for rules and regulations. We do need some sort of enforcement of the data handling practices that we as a society want to see in place. We certainly have to decide what practices to codify in our framework and we need some enforcement.

    I don’t know that very large tech companies will get fined or forced to better handle data, but for many of us that work in smaller organizations, we don’t have the clout or impact of a multi-billion dollar revenue organization. We might see fines that can very much hurt our businesses. We ought to be ensuring we not only follow best practices, but we build data security in by design and default. It’s cheaper and easier to do so, and it ensures we don’t fail to go back and fix things later.

    It’s also something each of us, as data professionals, ought to take pride in doing.

    Steve Jones

    Listen to the podcast at Libsyn.