Voice of the DBA

Tag: security

  • Insiders Selling Data

    There is a report that the lowest-paid employees at various banks are selling customer data, which is sad and also not unexpected. Those who have access to sensitive information can be tempted to part with it for a quick profit. I’ve seen this in the past in restaurants and retail stores. Sometimes with credit card or other data, but also with knowledge (or keys) that would let someone burglarize the premises.

    There’s a great quote in the article that says: “The more employees there are inside a company with access to sensitive customer information, the higher the risk that access is going to be abused.” I think that’s very true in most companies, which is why governance and data controls are becoming more important in some organizations. I wish it were a global trend at all companies, but far too many don’t think there is a high risk, or that their employees will abuse their privileges.

    I’d like to think most IT people wouldn’t be tempted to sell or disclose info, but I’m not sure that’s true. While many IT staffers are paid well, well is relative. Some might not think they’re compensated enough or they might feel pressure from their own financial stresses to compromise that data of others.

    Privileged access is just that: privileged. While it can be a pain to have a separate account for certain access, the intention is that you are reminded that this account can access things that others should not see. In my career, I’ve tried to remember this and be careful with my access, though admittedly, I’ve done plenty of non-privileged things with a privileged account, being lucky enough that nothing bad happened.

    Security continues to be a problem in so many places, and with the vast amounts of data we collect, and the growing desire for many organizations to let AI LLMs view that data, I suspect we’ll likely have more problems in the future not less. While we might not have as many humans that decide to sell our data, I suspect we’ll have many more AIs that can be tricked into disclosing it.

    Steve Jones

  • The Cloud Security Problem

    Your management gets a great demo from a cloud vendor and decides that the organization needs to implement the new service/application/etc. quickly. Your team tries to comply, furiously learning and experimenting with integrations, software changes, infrastructure configuration, and more. Things get deployed are working. Clients and management are happy with the new capabilities and you breathe a sigh of relief.

    After a bit of time there’s a security issue and all of a sudden there’s blame pouring down on everyone. The vendor takes a hit because it’s a public security problem, but the reality might be that your organization didn’t completely understand how to configure strong security. The public doesn’t blame your organization, but internally your team don’t know how to make changes to ensure future security.

    That’s a bit of what happened with the Snowflake customer hacks, and a good description of some of the issues is in this piece from Joey D’Antoni. Snowflake didn’t necessarily have bad security, but they allowed customers to have bad security and also limited the options for customers to implement stronger security.

    I think about this all the time as I look at the challenges of security that many organizations face. In my decades of working in different situations, the one thing I know is that pressure to move fast often creates security shortcuts. Much of the early security problems with SQL Server could easily be traced to a) allowing installs with no password, b) developers not understanding the security model and granting sysadmin (or using sa) in their applications, and c) management agreeing that this was OK because a deadline needed to be met.

    In all these situations, workers had the best of intentions to go fix things later, but there was rarely the time or energy to do so. As someone who forced apps to change away from sa in a large org, and required separate accounts and passwords for servers, I can tell you no one liked me and I got a lot of pressure to leave things alone. That is until I could explain the risks to managers with security people present. Even then there was no shortage of people who wanted me to assume the risk of apps using sa and let things be.

    Cloud security is pretty good from the major vendors. Most of the smaller co-location facilities I’ve worked with in the past also had good security. It was the clients that caused problems, often because of a lack of knowledge or the desire to hurry.

    Joey says Entra is a great system. I agree, and I love the SSO capabilities I’ve seen implemented. I also know that trying to set things up and configure them has been difficult for me, and I think I can be pretty sharp about lots of technology. Joey has patiently answered many of my ignorant questions because I didn’t understand how some part of Entra (AAD) works. I think lots of tech people don’t understand this and don’ t necessarily have a Joey to call on.

    Most of us working in technology need more security education, better habits, and the patience to implement strong security. That includes management. At the same time, I wish that the solutions out there were easier to understand, or maybe better explained for those of us who need to use some portion of the authorization and authentication systems in our software.

    Unfortunately, security software is still software and those vendors push out changes and updates quickly as well, leaving education and documentation to the user. This is a bad situation that continually arises with regular issues in many organizations.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Everything is Code

    I posted a note on Twitter/X with this quote: “The content updates had not previously been treated as code because they were strictly configuration information.” This is from testimony given by Crowdstrike to a US Congressional committee in trying to explain how they grounded much of the airline industry a few months ago. That was a mess of a situation, and apparently, the vendor didn’t think their configuration was part of their code.

    That’s an amazing viewpoint to me. The fact that any developer or manager thinks that their configuration data isn’t a part of their code is worth testing. Yet, I see this attitude all the time, where developers, QA, managers, and more think that the code is the only thing that changes or doesn’t change, ignoring the fact that there are configuration items that affect the code and need to be managed appropriately. Certainly, if the config data were in enums rather than in a file or database they’d feel differently.

    I think part of the reason that people try to ignore config data is that it is hard to manage. Often config data might change between dev, test, and prod. Dealing with that, and testing appropriately is hard. I haven’t ever seen a good solution for getting data into an environment the first time. That’s the hard part. Once the data is there, you can use it as a token where it is needed, and hopefully, the value has already been tested. At the very least, you can test how that data affects that environment.

    I am glad to see Crowdstrike publicly recognize that they need to dogfood not only their code changes but also their config changes. However, for a company that hasn’t shown a rigorous engineering approach, I suspect they’ll test very simple and basic config changes and not necessarily do a good job of carefully testing a variety of potential problem vectors. That takes work, and excluding config data from testing is a sign (to me) of a technology group trying to avoid doing too much work. It’s likely more management and leadership than technology workers, but the entire organization is showing signs of shortcutting good engineering.

    My view is that developers should be free to experiment and try lots of things, and have a lot of freedom on how they build software. I think the same thing for infrastructure people as well. However, as we start to move our changes towards production, everything should be in code, version-controlled, and promoted through PRs. In other words, get everything stored as code and nothing gets changed outside of development. It only gets approved to move forward or rejected, after it’s well tested.

    That’s a tough process to implement, and one many companies don’t spend the time doing, but for those that do, they end up deploying many  fewer bugs.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • No Backdoors

    Every once in awhile I hear about someone in law enforcement sure that tech people can build in a safe, secure way for data to be unencrypted by the company or vendor. The latest appears to be from Australia, where the Security Intelligence Organization wants tech companies to build this into products.

    Backdoors never work. Anytime an encryption key is stored, it could be stolen. We see this all the time. Keys are just data, and companies lose data all the time. At scale. Governments are certainly not immune from this. One of the reasons that Azure allows a BYOK (bring your own key) for encryption mechanisms is that many organizations don’t want to trust Microsoft to store their keys. I’m guessing Microsoft doesn’t want the liability, either.

    Many of the data protections an organization might implement are outside the scope of data professionals, but we certainly have responsibilities in this area. We should certainly manage our keys appropriately, and my recommendation for many people is that they have a separate repo and pipeline for the deployment of privileged objects, such as encryption keys. Your organization also ought to have a process (and test it) for key rotation. Keys expire relatively infrequently and the person who last rotated the key might not be available when you realize this process needs to be completed.

    It’s especially important everyone knows how this works for those emergency situations where something expires and none of you realize there’s a problem until a client calls.

    There are plenty of other security mechanisms we ought to be using. Secure your backup files and limit access to shares or even processes that move these files around. In general, limit access to those who need access to everything. Using organizational groups is the best way to do this, along with regular audits to ensure that those who change jobs are removed from groups that aren’t needed. I haven’t seen any organization that has a good process for knowing what positions need what access. I often only see people given new access for a role change, giving them the new roles that match some other employee. This is usually done without any previous access removed when it is no longer needed. The most senior people often have the most access, not because they need it, but because they keep getting new roles.

    Managing security with roles too granularly can become a nightmare, though ensure you use roles everywhere you can while trying to limit the numbers of roles. For most databases, we are giving access to nothing or everything, but there are reasons to limit access for certain data. You might consider two roles by default in most places: one for privileged users and another for everyone.  Easy to ensure new objects get grants to one or both roles and let access be managed by role membership.

    Keep it simple, but keep it secure.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.