Author: way0utwest

  • TDE and DDM

    Someone asked a question about TDE (Transparent Data Encryption) and DDM (Dynamic Data Masking), which are two different technologies that are in the security area. As I’ve mentioned in the Stairway to Dynamic Data Masking, DDM is not a security technology. It makes programming some obfuscation easy, but it’s not really security. It can be easily bypassed.

    In any case, how do these work together? Or do they work together? Good questions, and I’ll answer them.

    First, TDE is a technology that encrypts your data at rest, meaning when on storage devices. This handles the encryption and decryption as data is read or written from storage, without coding for it or the user having to do anything.

    DDM is a technology that takes results from a query and replaces some of the data with masked values. This works in memory, after a query runs. Query processing, with WHERE, GROUP BY, etc. clauses is not affected by DDM.

    These work together since data is encrypted on disk. It is read into RAM and decrypted. The query processor then assembles some of this data into a result set. Once this is done, before the results are sent to the client, DDM will make data.

    Demo

    I’ll setup TDE and then add DDM and run a query. Here’s a quick table and enabling TDE:

    CREATE DATABASE TDE_Primer;
    GO
    -- create and populate a table
    USE TDE_Primer
    go
    CREATE TABLE MyTable
    ( myid INT
    , myname VARCHAR(20)
    , mychar VARCHAR(200) 
    ) ;
    go
    DECLARE @i INT = 65;
    WHILE @i < 92
      begin
       INSERT mytable SELECT @i, 'Steve Jones', REPLICATE(CHAR(@i), 200);
       SELECT @i = @i + 1;
      END;
    GO

    From here, let’s enable TDE. If you have a master key in the master database, you’ll get an error, but it won’t really affect things.

    CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'AlwaysU$eaStr0ngP@ssword4This';
    go

    -- create certificate to secure TDE
    CREATE CERTIFICATE TDEPRimer_CertSecurity WITH SUBJECT = 'TDE_Primer DEK Certificate';
    go

    USE TDE_Primer;
    GO
    -- Create DEK
    CREATE DATABASE ENCRYPTION KEY
    WITH ALGORITHM = AES_128
    ENCRYPTION BY SERVER CERTIFICATE TDEPRimer_CertSecurity;
    GO

    ALTER DATABASE TDE_Primer
       SET ENCRYPTION ON;
    GO

    Once this is done, let’s alter our table. We also need a user that isn’t privileged.

    ALTER TABLE dbo.MyTable ALTER COLUMN myname ADD MASKED WITH (FUNCTION = 'partial(1,"XXX",0)')
    GO

    CREATE USER JoeDBA FROM LOGIN JoeDBA

    GRANT SELECT ON dbo.mytable TO JoeDBA

    Now we can check the execution of a query.

    2018-11-14 10_37_40-Microsoft Edge

    Masked data with TDE.

  • Moving to Query Store

    In SQL Server 2016, Microsoft introduced the Query Data Store (QDS) as a tool that would capture data about the execution of queries inside of your database. This was a project that had been in the works for a number of years, and one that many of us that were bound by NDA agreements had been following. We were excited by the chance to actually gather some information on the.

    Are you using Query Store? You should be, as this tool will become more valuable over time. I know that there are potential overhead issues (3-5% for most people, but possibly larger). I would argue that the potential for better performance and understanding of our systems outweighs the overhead. After all, if we’re unwilling to devote some resources to measuring our systems, how do we really know what to improve?

    We upgraded the SQLServerCentral servers to SQL Server 2017 this year (2018), and I’ve been wanting to enable the Query Store. I’ve been slightly hesitant with over 75,000 air miles and 5,000 driving miles on the road since the upgrade. Being distracted and out of my routine isn’t the best way to document and carefully observe the effects of a change. Not to mention concerns over data leakage for a company bound by the GDPR. After a little discussion and debate, and my schedule slowing, I’m looking to change that soon.

    I don’t expect that a lot of improvement at SQLServerCentral from changing this, as our third party forums and much of the internal code is batch SQL, and quite a bit generated on the fly. However, there are some stored procedures, and I might be very wrong. While we’re over provisioned with resources to avoid any performance problems, I do expect that we’ll learn a few things. I hope we find places to better tune code, and with some documentation of the process, hopefully some of you out there might spot things our team doesn’t.

    If you’ve got stories of the QDS working well or not well, let us know. Certainly let Microsoft know as well. The QDS is a major part of the SQL Server platform improving in the future and there are enhancements in SQL Server 2019. While I don’t know that the QDS and some of the automatic tuning features remove the need for a data professional to watch a system, I’d like to think they do provide opportunities and insight for how we might better structure and develop applications, as well as help us find better patterns that are useful in our initial database coding.

    If you’ve got stories, Erin Stellato wants to know (and she has a few in the post). If you’re concerned about overhead, read her other post. If you’re confused, we’re working on some articles to help you learn more. Give the QDS a try, especially if you’ve got some less critical systems. Part of our job is learning how to use new tools, and this is one that ought to be on most DBAs ToDo list.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Choosing a Career

    This was a short week in the US with the Thanksgiving holiday. This is a time when many people in the US have a long holiday weekend and often get together with family for some time away from work. It’s also a tradition for many people to “give thanks” to the blessings and people in their life. It’s a good time to reflect and appreciate the things that go well for you.

    That doesn’t mean that many, or even most, of us aren’t looking to grow our careers. For some of us, that may mean moving into a new field. Is that a good decision? Will we like the day to day work? Those are tough questions, and while you might be enamored with some technology as a hobby, when you do the work every day, things aren’t always the same. I’ve certainly seen this with friends and colleagues that have dramatically changed the lives into a completely new field. What is fun a few hours a week may not be as interesting when it’s a 40-50 hour a week grind.

    At SQLServerCentral, we often see posts where a community member asks if they should learn x or y. Should they move into a new career, or what’s it like working as a person that does z? Those are good questions, and I ran across a nice blog post from Dev Nambi on what it’s like being a data scientist. If you’re interested in data science, this is a nice list of notes about his job, but even if you were thinking about becoming an HA specialist, or Power BI report writer, or something else, the post talks about things that make impressions with his job. I’d like to hear more about how others would answer some of these questions about their own work. I need to do some of this myself.

    There are also a few very good items that I wanted to highlight. First, Dev notes that you don’t learn everything in school and there is more to work than is learned in school. That being said, there are core concepts to understand. I think this is true in most fields. We need some good base knowledge, but there is more than that knowledge. We need to learn how to put concepts and requirements together, as well as continue to ask good probing questions about the task.

    The other item that is pointed out a few ways is teamwork. Communication skills, including body language and humor, are important. We are social creatures and while I think having a variety of skills, views, and ideas is important, we need to get along with each other. We need teamwork within an organization, but for our careers, we also need a team of people to help us. That’s our network. Those skills are important for all of us, no matter what the job.

    If you want to move into a new role and grow your career, you have options. You can use schooling, you can try to grow in your role, or you can learn things on your own and apply for new positions. They all work, but they all have some drawbacks as well. They also all require effort and motivation, so if you want to make a change, set some learning goals, and start working. Making strides in a direction will help you grow and get you closer to your goals.

    Steve Jones

  • Republish: A T-SQL Code Testing Guide

    The day after  Thanksgiving, and a holiday. So you get to read A T-SQL Code Testing Guide.

    Please do, I’m a fan of testing.