Voice of the DBA

Category: Editorial

  • Protecting Data Between Services

    I saw an announcement this week that Microsoft is shutting down their HealthVault service, where someone might have chosen to store their medical records in a way that it could be shared with health professionals, but an individual could maintain control of the records. They could share them out to friends, download them, etc. It was a good idea, and I was interested in it for the future. Since I have been fortunate to mostly avoid doctors and hospitals, I never signed up, but I still could as of this week.

    Google used to have a similar product, but shut it down a few years ago. I suspect that this commercial space just isn’t as lucrative and valuable as either company thought, as I don’t really see any competitors out there that might allow a user to transfer their records. While I don’t know that there needs to be a service for this, I did think of the hassles and potential issues that might exist if there were. Imagine your mother or grandmother keeping their health records here for the last decade and now needing to download them and manage them before they could be used again. What if someone downloaded these to a PC and had a hard drive failure?

    Backups are needed, we know that, and I’d hope most consumers know that, but in a rush or in the worry that these need to be copied, but securely, would you want to keep this data on a tablet or PC? Or would you want an encrypted drive. If that’s the case, would you want to ensure you have 2 (or more) of them? Easy to plan this out, hard to think about if you get a notice about the service shutting down. Imagine that your spam filter knocks this down or you’re inundated and miss it and get a final notice on Sept 1 or Oct 1? A real hassle.

    The higher level view of this in my eyes is that we need better data formats for capturing and keeping lots of our data in systems. Those of us that are impacted by the GDPR (or similar laws) might be thinking about this already, as we have a need to provide data in response to requests. Providing a report, and packaging this up, is no small task. I assume I’d use a .zip file, but maybe a .nuget with a manifest is actually a better idea for customers.

    As the use of services grows, I expect that we will want to get more portability for data over time. Certainly vendors that provide services have an incentive for tooling. WordPress has import/export for other blog services, Microsoft will help you move data from Oracle (as will Oracle in the other direction), and there are specialized vendors in niche applications doing the same, but really I’d like to see us have more open, and extensible, data formats that relate to the types of data in our lives. I know we’ll have more disparate types of data, in various formats, so why not an easy way for each of us to store text, images, and more as a service that contains some metadata, some indexing. A personal data lake of sorts.

    We could have other services, like image services, visualization services for numbers, and more be authorized (or de-authorized) for our data. Imagine a way for us to allow a company to hold our data, but we disperse that to other vendors as needed for services. I could easily imagine various “storage” vendors competing and allowing us to “port” our data to a new service as easily as we port phone numbers for mobile phones. Perhaps a whole new era of data storage and management is coming.

    Or maybe we’ll just stick with the current separated, proprietary, limited view of data that we manage on hard drives and USB sticks. It’s more likely, but much less preferable in my mind.

    Steve Jones

  • Protecting Data Between Services

    I saw an announcement this week that Microsoft is shutting down their HealthVault service, where someone might have chosen to store their medical records in a way that it could be shared with health professionals, but an individual could maintain control of the records. They could share them out to friends, download them, etc. It was a good idea, and I was interested in it for the future. Since I have been fortunate to mostly avoid doctors and hospitals, I never signed up, but I still could sign up as of this week.

    Google used to have a similar product, but shut it down a few years ago. I suspect that this commercial space just isn’t as lucrative and valuable as either company thought, as I don’t really see any competitors out there that might allow a user to transfer their records. While I don’t know that there needs to be a service for this, I did think of the hassles and potential issues that might exist if there were. Imagine your mother or grandmother keeping their health records here for the last decade and now needing to download them and manage them before they could be used again. What if someone downloaded these to a PC and had a hard drive failure?

    Backups are needed, we know that, and I’d hope most consumers know that, but in a rush or in the worry that these need to be copied, but securely, would you want to keep this data on a tablet or PC? Or would you want an encrypted drive. If that’s the case, would you want to ensure you have 2 (or more) of them? Easy to plan this out, hard to think about if you get a notice about the service shutting down. Imagine that your spam filter knocks this down or you’re inundated and miss it and get a final notice on Sept 1 or Oct 1? A real hassle.

    The higher level view of this in my eyes is that we need better data formats for capturing and keeping lots of our data in systems. Those of us that are impacted by the GDPR (or similar laws) might be thinking about this already, as we have a need to provide data in response to requests. Providing a report, and packaging this up, is no small task. I assume I’d use a .zip file, but maybe a .nuget with a manifest is actually a better idea for customers.

    As the use of services grows, I expect that we will want to get more portability for data over time. Certainly vendors that provide services have an incentive for tooling. WordPress has import/export for other blog services, Microsoft will help you move data from Oracle (as will Oracle in the other direction), and there are specialized vendors in niche applications doing the same, but really I’d like to see us have more open, and extensible, data formats that relate to the types of data in our lives. I know we’ll have more disparate types of data, in various formats, so why not an easy way for each of us to store text, images, and more as a service that contains some metadata, some indexing. Something like a personal data lake of sorts.

    We could have other services, like image services, visualization services for numbers, and more be authorized (or de-authorized) for our data. Imagine a way for us to allow a company to hold our data, but we disperse that to other vendors as needed for services. I could easily imagine various “storage” vendors competing and allowing us to “port” our data to a new service as easily as we port phone numbers for mobile phones. Perhaps a whole new era of data storage and management is coming.

    Or maybe we’ll just stick with the current separated, proprietary, limited view of data that we manage on hard drives and USB sticks. It’s more likely, but much less preferable in my mind.

    Steve Jones

    Listen to the podcast.

  • Scary Data Collection

    Most of us would feel fairly creeped out by finding out an AirBnb or hotel had security cameras watching us. I’m not a woman, and I’m sure ladies are especially bothered by this, but there was an AirBnB rental where a guest found a camera using a little technology scanning. While cameras are allowed, they have to be disclosed.
    While many of us would prefer not to be surveilled, we are on a regular basis. Governments are watching our vehicles, all commercial activity is tracked in multiple ways, our locations are captured and sold to anyone. And it’s not even the carriers, it could be software that we think is innocuous and helpful. I would think most people reading this know that everything you do online is tracked, and often tracked from site to site with Facebook, Google, and other APIs, even if you don’t use those companies’ services. What’s disconcerting to me is how extensive data gathering and tracking has become and most people aren’t aware how comprehensive it has become.
    And in a wonderful set of timing. As I was writing this, I got a great article about how Google apparently isn’t perceived as invading privacy to the extent that they are. We likely trust them more than we should.
    The capture and misuse of data continues to grow. Whether this is by criminals, governments, or commercial businesses, it’s something we have to deal with. This isn’t necessarily any particular organization or situation that stands out, though the larger organizations likely have an out-sized impact and benefit from this. This is one of the reasons why the GDPR and similar legislation was passed. It’s a first attempt, and arguably weak attempt, to limit the use of data by organizations in ways that might be contrary to the wishes of the human that generated the data.
    Personally I like the GDPR, and while it might need alteration over time, it does start to to examine the idea that humans ought to be in control of data about them, just as we are often in (some) control of many of the physical items in the world we own. There are rules and regulations, restrictions, and even legal processes that provide recourse over our possessions. Those ought to be extended, and certainly adapted, to digital data, with the corresponding rights that we currently have and perhaps even new ones.
    I think this is going to impact our jobs as data professionals in the future. While we will have more requirements, more hassles from security, and more restrictions, this is also going to ensure that organizations need data professionals for a long time.
    Steve Jones

  • Securing Code Early

    Last year I started to get alerts from Microsoft Repos that someone had put a piece of security information in their code that pertained to one of my Azure services. At first I was worried, but then I realized this was the public version of AdventureWorks we maintain in Azure. We’ve published the login so people can test code against this if they want, and I started ignoring the warnings. Well, not ignoring. I still glance over them to verify the issue, but I’m less concerned.
    That doesn’t mean that you shouldn’t be concerned about sensitive information in repos. I saw this quote: ” Bots are crawling all over GitHub seeking secret keys, a developer served with a $2,375 Bitcoin mining bill found.” This follows a sentence that says “It once caused Uber to leak the contact details of 75m users“. These are from an interesting look at a way to secure code that might leak API keys. The idea is that you secure code on local commits and prevent secrets from being stored in your VCS.
    That’s a great idea. Can we prevent passwords in SQL code or ASP.NET config files? Can we actually start to teach developers to use secrets and other run-time mechanisms and prevent them from hard coding anything into a VCS? Perhaps, but we have a lot way to go and certainly more tool chains need to be updated to prevent what is a simple, but common, mistake.
    We need to get better at security, and we are. I see more Static Code Analysis tools being used in all sorts of companies, and I’m glad when they start to impact developers. While I get that changing your favorite method of writing a query or procedure is a pain, often we can reduce potential problems by enforcing some standards and avoiding poor coding practices. We do need to have exceptions since a rule for code might really be a general guideline 99% of the time with a few edge cases.
    Catching issues early in development is one of the goals. Having things like inline SCA in SQL Prompt (live demo), or continuous testing in Visual Studio, scanning with Sonarqube, and other tools are improving our code, while allowing us to build applications faster. There is plenty of other work to be done, especially for database code, but we are improving as an industry.
    If you haven’t used any of these tools and you write code, start learning a bit about them. You’ll likely appreciate their benefits once you get over the learning curve. You’ll also start writing better code.
    Steve Jones