Voice of the DBA

Category: Editorial

  • Automatic Refaction of PII

    Data privacy and protection has become a hot topic. Across the last few years, as I’ve worked with customers that deal with the GDPR or other laws, they’ve become more concerned and careful with their use of data. I find less and less resistance from developers about using sensitive production data in development environments, but still too much.

    Using data that’s stored in databases or other text files is one thing. What about data in less structured forms? I’ve dealt with a few customers over the years that recorded customer interactions for various purposes. Call center or financial organizations commonly do this, and sometimes deal with sensitive information. I know I’ve given my date of birth, credit card number, bank account number, and other data to representatives at various times.

    Those calls are often recorded, and the IT staffs have often had to ensure extra security is applied to these files. Not everyone has access to listen for various reasons, but certainly when there is sensitive information inside the audio (or video), this data needs the same protection we’d apply to data in other forms. Providing protection, or redacting the information, isn’t an easy task.

    I saw recently that the Amazon Transcribe service will now redact some PII information automatically. This service can be configured to automatically remove the information in text, which is fantastic. This is a great way to start to use technology in a safe way to ensure that we have less data  leakage when we re-use data. Certainly people might look at this data to better train reps, but it’s also likely someone wold look through transcripts to determine why customers are calling in and use that information to better design applications. In either case, there isn’t any need to expose PII data to them.

    This doesn’t protect against the data inside the audio, but perhaps companies can delete and remove those recordings sooner with transcripts available and more quickly reduce their potential attack surfaces. We’ll always have some liability, but reducing that and not unnecessarily creating issues is part of what we want to do when protecting data.

  • Reviewing Your Past Efforts

    It’s about the start of a new year for me now. I know, it’s late February, and we’re almost a sixth of the way through the year, but I’ve been on sabbatical. In fact, with planning for the break, I’ve really missed almost two months of work at this point. I deliberately did not make any resolutions or planning at the end of 2019, since 2020 wouldn’t really start for my career until about now.

    Now is the time to plan, and I will do some writing on my blog for the things I want to tackle, but I also want to think about my plan. The main reason for this is the SQLskills newsletter from Jan 7 (#214), in which Paul Randal wrote an interesting editorial on the start of the New Year. There are a few things in the newsletter, but one of the items was about resolutions and planning for the new year, but by starting the process while looking at the previous year.

    I wrote about learning goals last year, and then blogged a bit about them throughout the first third of the year. I then pivoted a bit, with an outside commitment that ate up a bunch of my personal time. Despite having a reminder in my calendar, I never got back to the books I’d planned on reading.

    2019 Rating : Failure

    I’ll do more of a review on my blog, and then planning, but the important things I got from Paul’s note is that I didn’t go a good job of not only reviewing last year, but also I didn’t end up reviewing it periodically after a few months. Not only did my goal fail, but my review of my goal failed.

    There’s a lot to be gained from some self-analysis, and maybe even a review of your analysis by a spouse, partner, mentor, or close friend. In my case, I ended up doing more speaking and traveling than I’d expected, didn’t even complete my commitment, focused my reading time on fiction and escaping from life, and really ignored a bit of my career goals.

    I don’t know if I’ll do better this year, but just writing this and thinking about the process will help me do something different. Hopefully you are keeping up with your goals, but if not, maybe you need a mid year review, as I’m doing now.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Data Orchestration

    We continue to deal with larger and larger data sets all the time. In fact, it seems that most people find themselves outgrowing the capabilities of some of their OLTP databases, often RDBMS stores, and need to upgrade hardware or re-architect software. It doesn’t matter if you have a 100GB database on a few cores or 10TB on dozens of cores, there is often a need to upgrade to meet our workload demands.

    In addition to the transactional needs, there is a growing demand for reporting and analysis workloads. Some people use a separate warehouse, and some want to just query data where it is. Certainly ETL processes and platforms have grown tremendously over the last few decades for those that want to implement the former process, but there is plenty of demand for the latter. In fact, I’m amazed how many customers have inquired if Redgate’s SQL Clone product will enable them to do this and spread their workload to other systems (it’s not designed for this).

    I’ve been thinking that with SQL Server 2019 we will start to access data where it lives, not move it to another place we want it. To me, this is more of what future data orchestration might involve. I ran across an article that takes a slightly different approach, thinking AI and other products will help better move data around, and perhaps that’s true, but I do think more and more we want to query data where it lives, and use larger, distributed compute platforms to do this.

    The scale out capabilities of SQL Server 2019, with the separation of compute and storage in Big Data Clusters, is a huge change that I think will be the future for many of us that look to meet reporting needs. The ability to grow hardware to match the workload needs is huge. This alone is a good reason to think about doing this in a hybrid or public cloud scenario.

    Of course this doesn’t come cheap, easy, or quick. There is work to be done to evolve systems, but it is an area I think is worth experimenting in during the coming year. I bet many companies would be interested in some PoC work here to determine how to better meet the reporting requirements of larger data sets. Perhaps this is something you suggest to someone in your organization.

    Steve Jones

  • Back from My Sabbatical

    It’s been six weeks away from work, but I’m back now. It was a great break, I’m refreshed, and I am ready to get back to writing, speaking, and working with the data platform again. I haven’t done much SQL Server work, but I have been busy. With this being my first day back, I did spend a few minutes looking back at what it was like to have a voluntary, extended break from work.

    The time away was very refreshing. I was able to spend days on my own personal growth in a few areas as well as tackle personal projects that my wife and I have wanted to do around the house. I avoided the computer for the most part, only using it for personal tasks, like watching guitar videos on YouTube. I used my hands often, rarely sitting at a desk.

    At the same time, this was an enlightening break. I realized I enjoy my job, and I enjoy technology. During a two day stretch near the end of my time away, I attended a conference and spoke. I engaged with other technologists and realized that I really enjoy working with and talking about databases and technology.

    I am ready to go back to work and looking forward to what 2020 brings to my career. I’ve missed Redgate, and I’ve missed interacting with many of you on a daily basis.

    Perhaps the best part of the sabbatical was that it have me a glimpse of retirement. I puttered around the house, did things I wanted to do, at my pace, without pressure. I handled it well, though I also think I do need a little more purpose or focus when I do retire.

    It was a good break for me, and I’m looking forward to the next sabbatical in a few years. In the meantime, I’m ready to get back to writing and speaking and SQLServerCentral.

    Thanks to Kathi, Kendra, Grant, and the others at Redgate for covering my work while I was away, and I hope everyone out there enjoyed their stewardship if the community.

    Steve Jones