Voice of the DBA

Category: Editorial

  • Badly Encrypted Databases

    I ran across a blog about encrypted databases linked from Bruce Schneier’s blog. I follow his musings and writings on security ,and he recommended we read it with this sentence: “Even the summary is too much to summarize, so read it.” Good enough for me, so I clicked the link and read about encrypted databases.

    I like the idea of stronger encryption in databases, and I’ve given a few talks on the subject. At times there are attendees that will debate that encryption in the database doesn’t do a lot of good. Often they dismiss the idea of TDE, since administrators can still read the data and break the encryption, and normal users aren’t affected. Many also note that database encryption does nothing for data on the wire, which is true. Most people want to do the encryption and decryption on the client, which has other challenges and is fairly hard to do well.

    I think that security a series of layers, and as noted by the author of the blog, most criminals are lazy. If they can copy a backup file or data file, they’ll just do that and read the data. TDE isn’t perfect, but it does limit these simple attacks. Always Encrypted was developed to try and make it easy to include encryption from the client side, but in SQL Server 2016, it has lots of limitations. In SQL Server 2019, we get secure enclaves, which should help adoption somewhat, but we will see once developers start to experiment with the feature.

    The blog talks about problems with encryption, spending quite a bit of time on approximate database reconstruction, which is essentially guessing data values with some information and by watching queries and results. It’s somewhat fascinating, and also scary, but complex and likely requiring lots of queries. To me, this is an area we ought to focus, and really an area that all our protocol libraries and possibly database firewalls (or built in limits) ought to focus efforts. We shouldn’t be most clients to make large queries of all data in a table. Really at this point, we ought to have build in limitations of queries to ensure that users are exporting all data from a table. I’d like some throttle that might prevent the return of very large result sets to clients.

    At the same time, there ought to be some way to analyze the queries coming in and see if an attacker is “guessing” values. I won’t pretend to know how we might do this, but it would seem that in the same way we detect lots of login attempts, we could have some alert being raised when we had xxx of the same type of query in yyy time. That might alert us to potential problems.

    Or users running lots of searches.

    I don’t know the best way to protect data, but I do know that too many of us aren’t doing a good job of this. We need to get better, both in production and development environments. We need to be better at protecting databases and the data within them.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Database Brainstorming

    I read a post recently that captured my attention for more time than I expected. The piece fascinated me, and I sent it to my son and a couple friends. In fact, reading it caused me to stop working for a bit as I thought through some of the concepts and ideas. It’s about divergence and convergence, and how you can use these ideas when trying to be creative. I recommend you take a few minutes and go through it.

    Is this something we ought to do a bit in database design? Should we spend first some time on divergent, creative thinking for how to implement some new entity or design some schema? Do you ever do this in your job, especially in a team? Or do you just push forward with your first thoughts, relying on experience and instinct? That’s what I’ve often done.

    We certainly need convergence thinking. We have to get things done and deliver software. Most of us are probably good at the convergent side of thinking and writing code or queries or configuring systems. Most of us probably do some amount of divergent and convergent thinking at the same time.

    However, I’m not sure that’s the best way to tackle creative work, especially when building complex systems or working to solve new types of problems. There is some value in separating out the brainstorming portion from the actual, let’s write some code and see if it works. Getting a good list of potential ideas and spending a few minutes thinking about them before writing code helps us to avoid getting trapped pursuing into the first thing that works and possibly see other approaches.

    In today’s world of DevOps, trying to move faster and get small chunks of work to the customer quicker, we are under pressure to do more convergent work, quicker. If we want to ensure we do model our entities well and follow good database design practices, we need to spend some time with divergent thinking without doing work, perhaps speculating on potential future design challenges in advance. If we periodically spend this time, then we can rely on our previous brainstorming when it is time to converge and get things done.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.3MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Common Data Challenges

    Working with data has proven to be a challenge for most of my career. It’s been fun, and certainly fulfilling, but there are constant challenges involved. Let’s take away the hardware and admin challenges of keeping systems running, backed up, and performing well. I’m thinking today of the struggles of just data.

    There is a short article that talks about three common data issues, and these are some of the same ones I’ve struggled with for most of my career. Are these challenges any different than they were 20 or 30 years ago? I’m not sure, and I was working with databases and software nearly 30 years ago.

    ETL is a constant challenge, even today with tools like SSIS and Biml that make it much easier to build flows that migrate data from one database to another. ETL is such a challenge that many people make a very comfortable living helping organizations meet their every changing needs to move and prepare information for end users.

    The other challenges noted in the article are getting a complete data picture because of missing data and not trusting or believing in data. The latter hasn’t been as much of a problem for me.  I might describe it differently as more often we aren’t sure what weight to place on certain data. The world is messy, and often we collect data that we think might be valuable only to realize later that it doesn’t mean what we thought or our our hypothesis was incorrect in the first place.

    I think the challenges are part of what makes this work interesting. Our employers and clients might view the effort and time involved as frustrating, and I wish I had solutions to make our process quicker and smoother. Actually, I do think that advances like SSIS have made things quicker, but the world has grown more complex. We deal with more data from more systems, in the still chaotic, messy formats of the world.

    What are your challenges with data today? Are they getting better, worse, or still the same as they always were?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Data Incident

    I was reading a Google whitepaper and I think this was the first time I’d seen a potential loss of data referred to as a data incident. The paper deals with the response from the Google Cloud team if there are potential data loss issues, outlining a four part process for responding and ensuring the customer can get back to work. Azure has a similar plan, though AWS seems to expect the customer to do more of the work.

    This is the first public disclosure of a process, and it’s a good one. In fact, everyone should use this or have something similar in their organization. The way that the world seems to be moving, it’s likely that most of us will suffer a breach at some point, and we ought to know how to respond. Without some plan, it’s easy to have a situation spiral out of control.

    I’ve had formalized DR plans as well as incident responses for events such as a DDOS or virus attack, but the issues with data are somewhat different and probably deserve a dedicated plan of some sort. While I’ve often seen these handled under the security team response, that might not be enough for data services, especially when you have responsibility for data that is different than managing a software system or service.

    Data security is becoming a more important part of the data professional’s job, with legislation and best practices starting to require that we recognize a shared interest in sensitive data that extends beyond our organization. Whether this is something you like or dislike, it is becoming a part of our work.

    Personally I like having a crafted plan that is available for use in crisis situations. It can be hard to remember all the possible issues, and certainly crisis seem to occur when our best people aren’t available. Even more important, a plan means we have something we can practice to be sure that everyone understands how to react, limit the exposure of data, and even ensure that we can get services back up and running as soon as possible.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.