Voice of the DBA

Tag: security

  • Has Everyone Had a Data Breach?

    Almost every time that I attend an event, I’ll end up meeting someone that has had security issues at their company. I’m always surprised how many people have had ransomware or other security problem that didn’t get well publicized. It’s not like everyone has had one, but out of 100 people, it seems there is at least one issue.

    Many of us work with third-party companies for either products or services. It’s become standard to use other firms for specific things your organization needs. However, since that’s the practice, many of those firms you partner with have their own partners. After all, they’re using other companies for specialized work just like you are.

    These third- and fourth-party relationships have changed our security and risk profiles for the worse. As the numbers of data breaches and security issues grow, it’s likely that someone in your partner network has had an issue, which might mean that you have an issue. This depends on what you have contracted with partners for, but it seems more and more often this is some sort of service provided, often with your data being shared with the partner. Which could mean your data is shared with their partners.

    An article recently noted that the number of partners are going up and many organizations are not aware of the risk this creates for them and their customers. There are more and more third- and fourth-party partners who have suffered data breaches, and if they have shared our data, we may have liability. The weakest link in a supply chain is the problem, and many of us have lengthened our supply chains quite a bit without paying attention.

    I don’t know that there are good solutions here, but I am seeing more and more companies demanding that suppliers of services prove they have strong security practices and protocols in place. It’s not perfect, but it does help us remember that security is everyone’s business, or at least everyone with whom we share our data.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Stale Data Causes Security Issues

    Security has become better and better in many organizations. At the same time, hackers and malicious actors are doing a better and better job of finding new ways to attack systems. Some work to target specific individuals, often because of government or industrial espionage. Most of us aren’t likely to deal with those issues, unless we work with (or are) someone that is very important in a particular situation.

    Instead, many of us deal with wider spread attacks that look to exploit vulnerabilities in technology or humans at scale, hoping to find the weak links. Lots of people I know have dealt with viruses in the past that shut down systems, and more recently, had to rebuild systems crippled by ransomware.  Despite their best efforts, this often means lots of extra unexpected work, combined with the stress of falling behind on our commitments. We have plenty of other work to do.

    Windows has been vulnerable throughout its history, and it appears, lately through a data problem. There is a class of attacks that look to use approved, though old and unpatched, drivers as a vehicle for gaining a foothold inside a network. This was a problem (called the BYOVD issue) and Microsoft addressed this with a block list that was used to prevent the loading of vulnerable drivers. They updated this list through Windows Update.

    Except they didn’t. In database terms, we had an eventually consistent set of data, which was being updated at Microsoft, but not being sent to client workstations. There were over 3 years of the list not being updated, despite assurances from Microsoft that Windows 10 PCs were protected. There are instructions for manually updating your machine.

    I don’t envy this being a process I’d want to build. Getting data from security researchers or elsewhere, putting it in a database (I hope), then exporting this into a text format and getting that loaded into the Windows Update process, all while trying to ensure the process is secure along the way. That can’t be easy inside a large company like Microsoft. At the same time, not noticing this wasn’t working isn’t excusable. Likely there were issues, but my guess is someone didn’t want to admit a failure and get a bad annual review.

    Data sync issues are nothing new, and many of us struggle with these on a weekly basis. However, these are important issues. Replication can fill log files (and disks), broken ETL processes can cause execs to make poor decisions, and in the security space, not updating drivers and block lists leave us vulnerable.

    This situation isn’t excusable and Microsoft ought to be ashamed. Some of these execs ought to lose bonuses, at the very least. It’s also not excusable in our orgs. We ought to be sure we’re patching on a regular basis and minimizing the attack surface area we present. It’s the least we can do as IT professionals.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Code Supply Chain Security

    There have been a number of attacks in the last few years on source code. In fact, I saw a new one this week for an e-commerce WordPress plugin. This time hackers got access to the distribution server for the company, Fishpig, and altered the plug-ins that their customers download.

    A few years ago this was big news, with the SolarWinds exploit. There was also an attack on PyPy, a popular Python package that many people include in their code.  There have been no shortages of problems in npm packages as well. I’m sure this has happened in other software packages, which is scary. In the days of DevOps where we publish code from a repository, an exploit against your developers might go unnoticed. Then again, maybe not.

    Would any of us notice new code in a file share or a folder on our system? We might just compile a large project without realizing it. At least with DevOps, we have the opportunity to include security scans and code analysis checks, some of which could look for known patterns of exploits. I know some companies use these, and often compromised or vulnerable packages are stopped by the automated pipelines.

    In the US, various security agencies have released a set of recommendations, as has the Open Source Security Foundation. Both of these are designed to help developers secure their supply chain against attacks. This is likely going to be a continuous problem for software vendors in the future as it’s much easier to attack one vendor whose software many people use than each individual company. I shudder to think about what happens if someone manages to get a ransomware package into a vendor’s codebase.

    Ultimately, there will still be problems. Many new projects begin with poor practices precisely because they’re experiments and the authors don’t know if others will find the software valuable.  While we can have good templates and security controls, I’m not hopeful. To me, the best solution for stopping code is to have patterns detectable by security checks in the pipeline. Checks that can be expanded and enhanced as new issues are determined.

    Of course, that means the makers of security software need to ensure their supply chain is protected as well.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Reducing Debt and Increasing Security

    Many of us know that reducing financial debt in our lives leads to a bit more security in our minds. We are better able to cope with unexpected expenses when we have the ability to get credit or pay for something in cash. If you’ve ever had a major house repair, car calamity, or (hopefully not) medical issue, you know what I mean. At least in the US. That last one is less of an issue in many other countries.

    We know that our code has technical debt, and often we fight a battle with project managers and business people over whether we should spend time paying down that technical debt or developing new features. We certainly can do both, but often that’s not a choice we are given. If we do get the chance to reduce technical debt, it’s a tiny fraction of the time that is allocated to adding something new.

    There might be a better argument for reducing some technical debt: security. There’s an article that talks about the security risk posed by moving too fast and not ensuring there is rigorous security testing. This is a common problem in many companies that adopt some parts of DevOps, but forget that quickly deploying new code isn’t the only principle. We also need to learn from our mistakes and quickly correct them.

    I am a big fan of releasing often, especially for security purposes, but those releases need to build on each other with not only new code but refactored old code that doesn’t perform well or has a security issue. More importantly, we need to ensure when we find issues in code, that all developers are aware of the problem and don’t repeat the issue. We also need to update our testing to ensure that humans don’t continue to write code in a style that causes known issues. We want to avoid antipatterns in our code.

    This is less of a developer issue than a managerial one. The C-suite and all management that are involved in software need to prioritize security so that known issues are fixed quickly. This also should involve ensuring that technical debt is kept under control as a lot of this poorly written code increases the chances that a vulnerability is in the code. Either because the code isn’t well written or because developers avoid touching the code in the future.

    The best solutions, to me, involve DevOps. Make small changes, deploy them, instrument them, and learn if they work well. If not, fix them quickly. While this might end up delaying some features, ultimately it ensures that our software is in better shape, with a reduced risk of simple security mistakes upsetting our customers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.