Voice of the DBA

Tag: SQLNewBlogger

A Basic Encryption Primer for SQL Server
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

Encryption is a function call in SQL Server, where we pass in the text to encrypt and a key. That’s really what we are doing with encryption. I pass a key and data into a function and get either encrypted or decrypted data. Here’s a short example:

Basic Encryption Demo

Let’s say I have a simple message, like “Let’s meet at Kunjani Coffee at 8am.”. I want to protect this message.
```
DECLARE
     @data VARCHAR(500) = 'Let''s meet at Kunjani Coffee at 8am.'
   , @encrypteddata VARBINARY(500);

SELECT 'Plain Data', @data

SELECT
     'Encrypted Data'
   , ENCRYPTBYPASSPHRASE ('mysecretk#y', @data) AS EncryptedData

SELECT
     'Decrypted Data'
   , CAST(DECRYPTBYPASSPHRASE ('mysecretk#y', ENCRYPTBYPASSPHRASE ('mysecretk#y', @data)) AS VARCHAR(100)) AS DecryptedData

SELECT
     'Almost Decrypted Data'
   , DECRYPTBYPASSPHRASE ('mysecretk#y', ENCRYPTBYPASSPHRASE ('mysecretk#y', @data)) AS AlmostDecryptedData;
```
In this code, I’m using EncryptByPassPhrase and DecryptByPassPhrase to encrypt data. I pass in a key, which is my passphrase. That is “mysecretk#y” i this case. I then pass in the data and get the result returned. You can see the four results below:

This is a quick example of the functions working together. I use one to hide the data, and another to reveal it.

The Encryption Hierarchy

Encryption is expensive in terms of resources (time, CPU, etc.), which means we want to minimize it. Part of what we do is try to minimize the work done while maximizing protection.

We can double encrypt things, meaning I could take the result of the second query, and use that “encrypted data” string and use that in another encryption function and provide even more protection. The encryption of each string, however, is time and CPU intensive.

Rather than encrypt the data again, we often just encrypt the keys. One key encrypts another and then the resource cost of decrypting a key is much lower than decrypting the data over and over. If you look at the encryption hierarchy on MS Docs, you’ll see that keys are stacked on each other, each one protecting the layer below. Typically we are only encrypting the actual data with a password or a symmetric key, which are essentially the same thing and the quickest way to perform the encryption and decryption.

Let’s lightly look at the parts of this hierarchy.

Asymmetric Keys

The stronger keys are the asymmetric ones. They are called this because the key used for encryption is different than the key used for decryption. Typically these are paired together, and you can give someone 1 key, so they can only perform one operation.

Certificates are asymmetric keys, with other metadata, and have the two keys as the public and private keys. This is how we do a lot of encryption across distances where we need to exchange data on insecure channels, like the Internet.

These are computationally intensive, meaning lots of CPU and time, so we don’t usually want to use these to encrypt lots of data. Instead, we use these to encrypt or symmetric keys.

Note: The strangeness in the SQL hierarchy is that the SMK and DMK are symmetric keys.

Symmetric Keys

The Symmetric keys are used for both encryption and decryption of the data. This doesn’t mean any key works, but the key used to specifically encrypt a set of data is the key used to decrypt it. This means we need to send the key to all parties that do encryption and decryption.

These do require lots of CPU, but much less than asymmetric keys. Typically we use these to actually encrypt the data.

Hash Functions

This isn’t really encryption, but some hash functions are used for things like passwords, where we can have one-way encryption. These are often used to transform the data into a hash, or representation, and then we can compare the hash together.

Using Encryption in Practice

Each of these ideas is just a function call, and that is how we’ve implemented encryption in SQL Server. Whether you use TDE, Always Encrypted (AE), column level encryption, or anything else, you are making function calls in some way.

For TDE and AE, SQL Server handles much of this work for you. It gets keys, does the function call. For the code above, or any of the ENCRYPT/DECRYPT functions, you are writing code and using those in your work.

However, keep in mind that key management is the key to protecting things. This means how you protect them, where they are, how you copy them to other places, and change them over time.

Summary

Encryption is just a series of function calls. We have different types of functions and different parameters, but that’s really the core of what is happening.

We can use these functions with the outputs of one as the inputs of another, or more often, the keys used as inputs instead of text, allowing us to encrypt and protect the keys themselves.

There is a lot more to learn, but this gives a basic look at encryption in SQL Server.
May 26, 2021
Computed Columns for Grouping–#SQLNewBlogger
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

I ran into someone trying to do some grouping for an accounting report. In this case, they had a number of criteria where certain accounts were used to produce groups of values. For example, I might have these criteria:
- Accounts from 400000 to 490000 are Operating Expenses
- Accounts from 500000 to 502999 are Personal Expenses
- Accounts from 503000 to 503999 are Materials and Services
There would be other items, but if I have accounts and values, how do I sum and group in these ways?

There are a few possibilities, but since I might move accounts around, I thought that computed columns might help with grouping here. This post looks are a way you can do this.

Imagine I have some values like this:
```
CREATE TABLE BudgettoActual

(accountid INT

, budget NUMERIC(10,2)

, actual NUMERIC(10,2)

)

GO

INSERT dbo.BudgettoActual
     (accountid, budget, actual)

VALUES
     (400010, 300, 299),
     (501010, 100, 102),
     (502010, 200, 150),
     (503010, 400, 150),
     (507010, 800, 150)

GO
```
Now I can use a SUM with a CASE, but that makes a complex query. One way to simplify this for others is to use a computed column in the table that might include my criteria. I can use a CASE statement to create my groupings.
```
ALTER TABLE dbo.BudgettoActual 

ADD AccountGroup AS CASE 
    WHEN accountid>= 400000 AND accountid < 489999 THEN 1
    WHEN accountid>= 501000 AND accountid < 503000 THEN 2
    WHEN accountid>= 503000 AND accountid < 504000 THEN 3
    WHEN accountid>= 507000 AND accountid < 508000 THEN 4
  ELSE 5
  END
```
With this, I see this in my table:

Now I have these different groups that I can use in a query and group by them. For example, I can get a quick look at my different categories with this code. I’ve put the categories in a CASE in the column list, but it could come from another table.
```
SELECT
          CASE
              WHEN ba.AccountGroup = 1 THEN
                  'Operating Revenues'
              WHEN ba.AccountGroup = 2 THEN
                  'Personal Expenses'
              WHEN ba.AccountGroup = 3 THEN
                  'Materials and Services'
              WHEN ba.AccountGroup = 4 THEN
                  'Reserves'
          END 'Object'
        , SUM (ba.budget) AS Budget
        , SUM (ba.actual) AS actual

FROM     dbo.BudgettoActual AS ba

GROUP BY ba.AccountGroup;
```
This gives me a look at my financial numbers quickly.

Rather than numbers, I could have used the title in the computed column, but that causes issues with ordering. With these numbers, I can choose numbers that are the order I need them in for a report (which can matter for financial reporting).

I’d prefer to use a separate table mapping the AccountGroup to a title and then joining that in my report.

This isn’t the only way to do this, but it is one way to handle complex grouping in a way that can make it easier for clients that might need to query this data.

SQLNewBlogger

This post took me about 10 minutes to write, but about 15 minutes to setup, which might be most of a writing session for a blog. However, it’s a good showcase of a creative way to solve an issue.

Any of you could put together a similar post on a query issue you’ve run into and want to share.
May 19, 2021
Basic XML Queries–#SQLNewBlogger
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

I ran across a question recently on querying an XML document. While I think XML is a pain and it’s not the future, there is a lot of it out there that you might need to deal with in a database. Legacy stuff will be there for awhile.

In any case, someone was struggling with this code.
```
DECLARE @x XML = 

'<?xml version="1.0" encoding="UTF-8"?>
    <PartyID>
     <PartyID>147</PartyID>
     <CampaignID>
       <CampaignID>1</CampaignID>
       <Arc>A</Arc>
       <TicPosition>2</TicPosition>
     </CampaignID>
     <CampaignID>
       <CampaignID>1</CampaignID>
       <Arc>A</Arc>
       <TicPosition>13</TicPosition>
     </CampaignID>
   </PartyID>'
SELECT

Data.Col.value('(./PartyID)[1]', 'int') As Party_ID,

Data.Col.value('(./CampaignID)[1]' , 'int') As Campaign_ID,

Data.Col.value('(./Arc)[1]', 'varchar(1)') As Arc,

Data.Col.value('(./TicPosition)[1]', 'varchar(10)') As TicPosition

FROM @x.nodes('/PartyID/CampaignID') As Data(Col)
```
The person got results where the Party_ID was NULL. Some of you might get what’s wrong, but it’s a question of understanding your context.

In this case, the FROM clause helps us understand this. When we specify the node() method, we choose a path in the document. The path we pick is PartyID/CampaignID. This puts us here in the document:
```
    <CampaignID>1</CampaignID>
       <Arc>A</Arc>
       <TicPosition>2</TicPosition>
     </CampaignID>
     <CampaignID>
       <CampaignID>1</CampaignID>
       <Arc>A</Arc>
       <TicPosition>13</TicPosition>
     </CampaignID>
```
If we are trying to specify paths on the current position with the period (.), we can only see these values. There is no PartyID here.

However, similar to a folder navigation from the command line, if I use two periods (..), I move up one level. From here, I can get the PartyID. Therefore, my code is:

SQLNewBlogger

As soon as I saw this question, I knew the issue. It was a good reminder to me to watch the path, which is why I thought this was a good thing to post about. It cements this in my memory.

In 10 minutes, I did this, just as you could.
May 10, 2021
Basic OFFSET–#SQLNewBlogger
Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

The other day I saw an article on the OFFSET clause in a SELECT. I had seen this come out and looked at it briefly in SQL Server 2012, but hadn’t done much with it.

NOTE: if you use this, be sure you read about potential performance problems and solutions.

The basic structure of this clause is that it is a part of the ORDER BY section of a query. After the column ordering, I can enter OFFSET and a value, which will skip those rows. I can optionally enter a number of rows to fetch.

The structure is:
```
<query>
ORDER BY col1, col2
OFFSET n ROWS FETCH NEXT 10 ROWS ONLY
```
This code:
```
WITH myTally(n)

AS

(SELECT n = ROW_NUMBER() OVER (ORDER BY (SELECT null))
  FROM (VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) a(n)
   CROSS JOIN (VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) b(n)

)

SELECT *

FROM myTally

ORDER BY n
```
Will get me numbers from 1 to 100, each in a separate row. A tally table, with partial results shown in this image.

If I change this, and add an OFFSET, I can skip some rows. For example, I can skip 7 rows by adding that clause, as shown below.

If I only want a certain number, say 6 rows, I add the FETCH clause.

This is useful for pagination, saving some network bandwidth, and less buffer space on the client. Not necessarily helping the query processor, but it does make it easy for developers and with small result sets (and source table sizes), this is nice.

It’s a fairly easy clause to use, but it can still require the full work on the server for looking through data, so be sure you read the link in the note above.

SQLNewBlogger

I was testing some code I’d seen from someone and it occurred to me to document the process a bit. I used a tally table, and wrote this around a couple of my experiments.

You can do this as well, show some learning, testing, understanding of code in ten minutes.
April 21, 2021