Voice of the DBA

Author: way0utwest

Protecting Data Between Services

I saw an announcement this week that Microsoft is shutting down their HealthVault service, where someone might have chosen to store their medical records in a way that it could be shared with health professionals, but an individual could maintain control of the records. They could share them out to friends, download them, etc. It was a good idea, and I was interested in it for the future. Since I have been fortunate to mostly avoid doctors and hospitals, I never signed up, but I still could as of this week.

Google used to have a similar product, but shut it down a few years ago. I suspect that this commercial space just isn’t as lucrative and valuable as either company thought, as I don’t really see any competitors out there that might allow a user to transfer their records. While I don’t know that there needs to be a service for this, I did think of the hassles and potential issues that might exist if there were. Imagine your mother or grandmother keeping their health records here for the last decade and now needing to download them and manage them before they could be used again. What if someone downloaded these to a PC and had a hard drive failure?

Backups are needed, we know that, and I’d hope most consumers know that, but in a rush or in the worry that these need to be copied, but securely, would you want to keep this data on a tablet or PC? Or would you want an encrypted drive. If that’s the case, would you want to ensure you have 2 (or more) of them? Easy to plan this out, hard to think about if you get a notice about the service shutting down. Imagine that your spam filter knocks this down or you’re inundated and miss it and get a final notice on Sept 1 or Oct 1? A real hassle.

The higher level view of this in my eyes is that we need better data formats for capturing and keeping lots of our data in systems. Those of us that are impacted by the GDPR (or similar laws) might be thinking about this already, as we have a need to provide data in response to requests. Providing a report, and packaging this up, is no small task. I assume I’d use a .zip file, but maybe a .nuget with a manifest is actually a better idea for customers.

As the use of services grows, I expect that we will want to get more portability for data over time. Certainly vendors that provide services have an incentive for tooling. WordPress has import/export for other blog services, Microsoft will help you move data from Oracle (as will Oracle in the other direction), and there are specialized vendors in niche applications doing the same, but really I’d like to see us have more open, and extensible, data formats that relate to the types of data in our lives. I know we’ll have more disparate types of data, in various formats, so why not an easy way for each of us to store text, images, and more as a service that contains some metadata, some indexing. A personal data lake of sorts.

We could have other services, like image services, visualization services for numbers, and more be authorized (or de-authorized) for our data. Imagine a way for us to allow a company to hold our data, but we disperse that to other vendors as needed for services. I could easily imagine various “storage” vendors competing and allowing us to “port” our data to a new service as easily as we port phone numbers for mobile phones. Perhaps a whole new era of data storage and management is coming.

Or maybe we’ll just stick with the current separated, proprietary, limited view of data that we manage on hard drives and USB sticks. It’s more likely, but much less preferable in my mind.

Steve Jones

April 22, 2019
Protecting Data Between Services

I saw an announcement this week that Microsoft is shutting down their HealthVault service, where someone might have chosen to store their medical records in a way that it could be shared with health professionals, but an individual could maintain control of the records. They could share them out to friends, download them, etc. It was a good idea, and I was interested in it for the future. Since I have been fortunate to mostly avoid doctors and hospitals, I never signed up, but I still could sign up as of this week.

Google used to have a similar product, but shut it down a few years ago. I suspect that this commercial space just isn’t as lucrative and valuable as either company thought, as I don’t really see any competitors out there that might allow a user to transfer their records. While I don’t know that there needs to be a service for this, I did think of the hassles and potential issues that might exist if there were. Imagine your mother or grandmother keeping their health records here for the last decade and now needing to download them and manage them before they could be used again. What if someone downloaded these to a PC and had a hard drive failure?

Backups are needed, we know that, and I’d hope most consumers know that, but in a rush or in the worry that these need to be copied, but securely, would you want to keep this data on a tablet or PC? Or would you want an encrypted drive. If that’s the case, would you want to ensure you have 2 (or more) of them? Easy to plan this out, hard to think about if you get a notice about the service shutting down. Imagine that your spam filter knocks this down or you’re inundated and miss it and get a final notice on Sept 1 or Oct 1? A real hassle.

The higher level view of this in my eyes is that we need better data formats for capturing and keeping lots of our data in systems. Those of us that are impacted by the GDPR (or similar laws) might be thinking about this already, as we have a need to provide data in response to requests. Providing a report, and packaging this up, is no small task. I assume I’d use a .zip file, but maybe a .nuget with a manifest is actually a better idea for customers.

As the use of services grows, I expect that we will want to get more portability for data over time. Certainly vendors that provide services have an incentive for tooling. WordPress has import/export for other blog services, Microsoft will help you move data from Oracle (as will Oracle in the other direction), and there are specialized vendors in niche applications doing the same, but really I’d like to see us have more open, and extensible, data formats that relate to the types of data in our lives. I know we’ll have more disparate types of data, in various formats, so why not an easy way for each of us to store text, images, and more as a service that contains some metadata, some indexing. Something like a personal data lake of sorts.

We could have other services, like image services, visualization services for numbers, and more be authorized (or de-authorized) for our data. Imagine a way for us to allow a company to hold our data, but we disperse that to other vendors as needed for services. I could easily imagine various “storage” vendors competing and allowing us to “port” our data to a new service as easily as we port phone numbers for mobile phones. Perhaps a whole new era of data storage and management is coming.

Or maybe we’ll just stick with the current separated, proprietary, limited view of data that we manage on hard drives and USB sticks. It’s more likely, but much less preferable in my mind.

Steve Jones

Listen to the podcast.

April 22, 2019
Enable a Named Instance to Connect on 1433

My default setup for a few years is to only use named instances, usually with a standard SQL2014 or SQL 2016 as the name of the instance. Recently, I set up a new instance and needed to connect from another machine. I’ve done this lots of times, but in this case, I wanted to both enable TCP/IP as well as move the port.

This post explains how to get your named instance to listen on 1433.

First, run the Configuration Manager and check your protocols for the server. You likely see something like this, where TCP/IP is disabled.

Right click TCP/IP and enabled it from the menu.

You’ll have to restart the instance, but wait. Cancel the message that appears. Now open the properties for the TCP/IP service.

At the bottom, there’s an IPAll item. This is likely set for Dynamic ports for your named instance. Remove the 0 and enter 1433 for the TCP Port line. You should get something like this:

When you click OK or Apply, you’ll see this message.

Now restart your database engine service and you should be able to connect over 1433.

April 22, 2019
SQLServerCentral Design–QotD
This is part of a series of posts that look at the changes to the SQLServerCentral design, bringing us to v3 of the site. You can see the other posts with the SQLServerCentral tag.

We’re a few weeks in and the site is stabilizing a bit, but a lot of work still to go. This week, I wanted to write about the changes to our Question of the Day section, which are likely more backend than front end, but still an important part of the site.

When we started the question of the day, it was a whim from Brian Knight, one that annoyed me because I ended up being responsible for most questions. Over time we allowed users to submit questions and enhanced the way the testing worked, but it took a bit of a step backward in v2 of the site. We had more restrictive forms for loading the questions, not supporting images in the explanation or even any HTML at first. We enhanced this over time, but the marking of multiple choice questions, scoring, awarding back points, and more were somewhat broken for a long time.

I got quite familiar with running direct “fix-em-up” queries against the database to correct issues.

When we talked with the developers about the new system, I wanted a more flexible way to get questions written, scheduled, scored, and even fix when we corrected issues. There are a number of plugins for WordPress that allow quizzes, but none that really fit our paradigm.

A custom plugin was written which allows a more flexible editing system for the question and explanation, similar to what most users are familiar with for writing posts in WordPress. This is more visually appealing, and easier to author a question.

On the front end, things are very similar, though we have reformatted a few things in the last couple weeks to show more information:
- We let you know if you were right or wrong
- We give you the correct answer, or if you were wrong, we tell you what you picked
- We give you the percentage and count of people answering each choice.
Many of these changes were simple, and some added back functionality that was lost when we launched the new site, but the changes were quicker and easier than in the past.

That’s a good sign because in the process of building this out, there are a few things missing. We still need to award back points for poor questions. Right now I’d be reduced to making a database query, but I need a “award everyone points regardless of answer” button built.

We also want to start collecting items into a test that might quiz a user in some particular area. In other words, rather than a question a day, we want to put questions more into a group like a certification test. Since the items are all linked in a more normalized fashion, and this would in some sense mirror what we’ve done with the Stairway series, I’m hopeful we can add this.
April 19, 2019