Tag: encryption

  • Mongodb Encryption

    The last decade has seen a number of new database platforms get implemented in a variety of organizations. Most of the newer popular platforms have been NoSQL (Not Only SQL) products, and one of the first that became quite popular was MongoDB. I first ran into a few customers using this platform around 2012/2013. At the time some customers inquired whether we had any products to help with development. We did, somewhat, as we invested in 3T.

    MongoDB has enjoyed a lot of popularity and set some standards for how to store and retrieve data. CosmosDB (and other products) has API compatability with MongoDB, precisely because of its popularity. While I’m not sure MongoDB is the replacement for your RDBMS, I’m sure it has places and problem domains where it shines. I’ve also watched the product improve and grow to meet the requirements and desires of more customers. One recent addition was field level encryption.

    MongoDB has had storage level encryption, similar to TDE in SQL Server. It also has had transport encryption (think SSL network protection). This new feature is more like Always Encrypted (AE) in SQL Server. Clients can perform the encryption and decryption, assuming they have the keys. This isn’t quite like AE as it appears to need code changes, but it does protect the data from system administrators, which is a concern for some applications and industries.

    How well does this work? There certainly is a challenge with managing keys, distributing them to clients, revoking them, and more. All of these are the same challenges more encryption technologies have to deal with, including AE. This is a good addition to all data platforms, which ought to enable encryption, but having, or even using, the encryption, isn’t a panacea for the problem of protecting data. We need lots of other things to be designed, implemented, and operated securely.

    I’m not a big fan of MongoDB, but it’s a fine platform, and it likely works well for some organizations. I’m glad that it continues to enhance its security capabilities, and I hope that anyone implementing it enables all that they can. Too many installations in the past haven’t had good security, and that’s a problem. If you manage data, secure your system, and be sure you continue to monitor the platform for security issues and keep learning more about how to best protect your data.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Encryption Libraries

    One of the ways we work to implement better security is through the use of encryption. I remember the early days of PGP, VPNs adding end to end encryption, IPSec becoming available, and more. I was excited to see column level encryption come in SQL Server 2005, but it was a bit complex to implement and had issues. The evolution over time with TDE and Always Encrypted were welcome additions, but we can do better.

    Handling encryption in the database can provide some protection, but really, we’d like to see an end to end solution, that ensures data is protected on the wire as well as in the db. Microsoft does a lot of research, and released a Simple Encrypted Arithmetic Library (SEAL) as open source code. This is a C++ library that developers can use to protect their communications. Last year they also released a .NET wrapper to make it easier for a wide variety of developers to use.

    There is a lot more to deal with when handling encrypted at scale. Indexing and search operations are difficult and complex, and can require lots of resources. I think that the Always Encrypted in SQL Server 2019 with Secure Enclaves is a good evolution, but it’s complex to set up, tooling is poor, and maybe more disconcerting, there are potential vulnerabilities with secure enclaves on Intel chips.

    Certainly a combination of developers being able to use something like SEAL and a linkage with a secure enclave on the server for set-based operations might be something we can look forward to in the future. I hope so, as security continues to be a challenge for many of us.

    I’d also like to see more software that is built to ensure users can’t access huge amounts of data, and perhaps easier ways to control the export of large amounts of data. We do need this capability, but we also want very few people, or no people and only a process, with rights to do so.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Secure Enclave Concerns

    We worry about security as data professionals. It’s a constant issue, though it seems that so many of the data breach issues come from simple mistakes. We have lots of automation, configuration as code, and more. These days we should have a protocol and automated process for setting up systems, file shares, and other ways of allowing others to access data.

    Awhile back we had attacks against hardware (Meltdown and Spectre), which required patches to systems to secure them. This was disruptive, but I haven’t seen any widespread security issues as a result of these flaws. Many systems were patched, though it’s entirely possible that there have been exploits.

    In a report this week, researchers have found a way to read data in the secure enclaves of some chipsets. That’s disconcerting because lots of secure software counts on the Intel Software Guard Extensions (SGX) protecting data. This is part of the Always Encrypted enhancements in SQL Server 2019, which I was happy to see added. These are supposed to help ensure you can protect data, even if you don’t have control of the physical machine, which is common in hosted and cloud scenarios.

    A new exploit could be a problem. Intel says it’s not, but they have an interest in saying that. The research looks complicated, and I could certainly see this is hard to do, but attacking a large environment, like a cloud subscription, might be something hackers try. Even if they aren’t successful, there may be performance issues with SGX calculations from a software patch.

    The point of this piece isn’t to alarm you. Likely this is more speculative than reality, but you should be aware there are potential issues. Encryption and security are complex topics, and I’m hoping Intel and Microsoft (and others) come up with better protections and mitigation for the issue. However, ultimately it’s up to us tech professionals to understand what possibilities there are and ensure we are taking every precaution we can.

    Steve Jones

  • Backwards Compatible Symmetric Keys in SQL Server 2017+

    I discovered recently that there was a change made in SQL Server 2017 to the way that symmetric key passphrases are hashed. There’s a KB article that notes the fix, but basically the passphrases used to be encrypted with SHA1. That’s cryptographically insecure, so the algorithm was updated to SHA2.

    This is a problem, and can cause some issues. I’ll show the issue and then how to get around it.

    No More Decryptions

    Let’s say I have a SQL Server 2016 instance and database. I run this code:

    CREATE SYMMETRIC KEY SalaryKey
    WITH ALGORITHM = AES_256
    , IDENTITY_VALUE = 'Salary Protection'
    , KEY_SOURCE = 'Protect this from hackers'
    ENCRYPTION BY PASSWORD = 'SomeReallyStr0ngP@ssword';
    GO
    OPEN SYMMETRIC KEY SalaryKey DECRYPTION BY PASSWORD  = 'SomeReallyStr0ngP@ssword'

    UPDATE dbo.Employees
      SET EncryptedSalary = ENCRYPTBYKEY(KEY_GUID('SalaryKey'), CAST(Salary AS VARCHAR(50)))
    GO

    I can easily decrypt this data:

    2019-12-05 12_12_54-SQLQuery2.sql - Plato_SQL2016.sandbox (PLATO_Steve (66))_ - Microsoft SQL Server

    Let’s now say I move this data to SQL Server 2017. It could be a restore, some ETL, replication, etc. In any case, I have the data there.

    Now, if I drop the symmetric key, or it doesn’t exist, I need to recreate it. These are supposed to be deterministic, which means I can run the code above and get the same key. I’ve done this on SQL 2014 and SQL 2016 databases, and I can decrypt data encrypted in another database if I use the same code to create the key. Let’s try this. I’ll run this code:

    CREATE SYMMETRIC KEY SalaryKey
    WITH ALGORITHM = AES_256
    , IDENTITY_VALUE = 'Salary Protection'
    , KEY_SOURCE = 'Protect this from hackers'
    ENCRYPTION BY PASSWORD = 'SomeReallyStr0ngP@ssword';
    GO
    OPEN SYMMETRIC KEY SalaryKey DECRYPTION BY PASSWORD  = 'SomeReallyStr0ngP@ssword'
    SELECT top 10
      e.EmpID
    , e.EmpSSN
    , e.Salary
    , CAST(DECRYPTBYKEY(e.EncryptedSalary) AS VARCHAR(50)) AS DecryptedSalary
    , e.EncryptedSalary
      FROM dbo.Employees AS e
    GO

    I get this:

    2019-12-05 12_15_24-SQLQuery1.sql - Plato_SQL2017.sandbox (PLATO_Steve (54))_ - Microsoft SQL Server

    Why do I get NULL? SQL Server can’t decrypt this data, so it returns a NULL This isn’t supposed to happen, but the hash change caused this.

    Let’s fix this.

    A Trace Flag

    The KB article linked above mentions that trace flag 4631 will fix this. Let’s try it. I’ll run this code:

    DROP SYMMETRIC KEY SalaryKey
    DBCC TRACEON( 4631)
    GO
    CREATE SYMMETRIC KEY SalaryKey
    WITH ALGORITHM = AES_256
    , IDENTITY_VALUE = 'Salary Protection'
    , KEY_SOURCE = 'Protect this from hackers'
    ENCRYPTION BY PASSWORD = 'SomeReallyStr0ngP@ssword';
    GO

    Now, let’s open the key and requery:

    2019-12-05 12_19_32-SQLQuery1.sql - Plato_SQL2017.sandbox (PLATO_Steve (54))_ - Microsoft SQL Server

    Hmm, this doesn’t seem right. With a little experimentation, I discovered the trace flag needs to be global, or it can be enabled instance wide. Let’s do that.

    DROP SYMMETRIC KEY SalaryKey
    DBCC TRACEOFF( 4631)
    DBCC TRACEON( 4631, -1)
    GO
    CREATE SYMMETRIC KEY SalaryKey
    WITH ALGORITHM = AES_256
    , IDENTITY_VALUE = 'Salary Protection'
    , KEY_SOURCE = 'Protect this from hackers'
    ENCRYPTION BY PASSWORD = 'SomeReallyStr0ngP@ssword';
    GO

    Now we query, and this works.

    2019-12-05 12_21_38-SQLQuery1.sql - Plato_SQL2017.sandbox (PLATO_Steve (54))_ - Microsoft SQL Server

    Most people don’t deal with column encryption, but if you do, be aware of this.