Voice of the DBA

Tag: ETL

  • ETL Security Holes

    The addition of DTS to SQL Server meant that more and more of us could grab data from a variety of sources, easily, and move it to SQL Server systems. SSIS enhanced our capabilities and made it fairly simple for non-programmers to grab web service data, and other formats, and quickly decompose them into relational formats. This made it more likely that we would build data warehouses or reporting systems, and easily keep them up to date with data from our OLTP systems.

    However the ease of using ETL (extraction, transformation and loading) processes to move data did not include the moving the same security controls and restrictions to these new systems. That can create a big security holes, especially when you have secondary systems used for decision support that might not be contained in the production network and not audited to the same level as other production systems.

    Whenever you move data around, there ought to be some set of guidelines for the security protocols required. That can be hard to track when you are pulling data from a secure system and your connections look like every other connection. One of the dangers that I suspect we will run into with Powerpivot is that secure data will be downloaded to Excel worksheets on insecure systems, and all of the controls that we have in place for protecting sensitive information are ignored when we pull data for analysis elsewhere.

    Our data security has gotten better, but we have a long way to go. If you are working with ETL processes, and especially with end-users, make sure that some sort of security policy is in place to help protect this data from being accidently disclosed. It’s not a great suggestion, but until we have better security tools to allow policies to follow the data, that is all we can do.

    Steve Jones

    (originally published at http://www.sqlservercentral.com/articles/Editorial/72157/)

    Podcasts

  • Data Portability

    It seems that there is a never ending list of formats that data professionals need to deal with. As I read through posts and articles talking about Integration Services, I’m struck by how many unique solutions people come up with to move data in and out of Excel. They know how to handle multiple worksheets, headers, practically anything that people put into Excel. If there’s a pattern in there, chances are some SSIS worker has a solution.

    This week I saw that Facebook had announced a data portability initiative, giving users the ability to download all their Facebook data into one file. The file can include images, movies, posts, events, etc. and gives you a way to not only back up your data, but perhaps use it for other applications. I’m sure it’s only a matter of time before we have an SSIS package that can load up your information into a SQL Server database. I suspect someone will do it just for the challenge, and not because they have a practical use for it.

    Facebook seems to have been under attack constantly from groups that are concerned about the privacy policies and features of a company that has so much data about individuals. However this new feature seems to show that the company isn’t looking to lock you and your data into their own system. You can now backup your data and shut down your account if you with to move on to another service.

    Recognizing that data is important to users and allowing some level of portability is a feature that I wish more companies would embrace. I’d love to be able to easily move my medical records, my financial investment history, or even my credit data to my own systems as a backup. Or even as a way to document the current state and perhaps ensure that my data is correct.

    Steve Jones