Voice of the DBA

Tag: security

  • Stale Data Causes Security Issues

    Security has become better and better in many organizations. At the same time, hackers and malicious actors are doing a better and better job of finding new ways to attack systems. Some work to target specific individuals, often because of government or industrial espionage. Most of us aren’t likely to deal with those issues, unless we work with (or are) someone that is very important in a particular situation.

    Instead, many of us deal with wider spread attacks that look to exploit vulnerabilities in technology or humans at scale, hoping to find the weak links. Lots of people I know have dealt with viruses in the past that shut down systems, and more recently, had to rebuild systems crippled by ransomware.  Despite their best efforts, this often means lots of extra unexpected work, combined with the stress of falling behind on our commitments. We have plenty of other work to do.

    Windows has been vulnerable throughout its history, and it appears, lately through a data problem. There is a class of attacks that look to use approved, though old and unpatched, drivers as a vehicle for gaining a foothold inside a network. This was a problem (called the BYOVD issue) and Microsoft addressed this with a block list that was used to prevent the loading of vulnerable drivers. They updated this list through Windows Update.

    Except they didn’t. In database terms, we had an eventually consistent set of data, which was being updated at Microsoft, but not being sent to client workstations. There were over 3 years of the list not being updated, despite assurances from Microsoft that Windows 10 PCs were protected. There are instructions for manually updating your machine.

    I don’t envy this being a process I’d want to build. Getting data from security researchers or elsewhere, putting it in a database (I hope), then exporting this into a text format and getting that loaded into the Windows Update process, all while trying to ensure the process is secure along the way. That can’t be easy inside a large company like Microsoft. At the same time, not noticing this wasn’t working isn’t excusable. Likely there were issues, but my guess is someone didn’t want to admit a failure and get a bad annual review.

    Data sync issues are nothing new, and many of us struggle with these on a weekly basis. However, these are important issues. Replication can fill log files (and disks), broken ETL processes can cause execs to make poor decisions, and in the security space, not updating drivers and block lists leave us vulnerable.

    This situation isn’t excusable and Microsoft ought to be ashamed. Some of these execs ought to lose bonuses, at the very least. It’s also not excusable in our orgs. We ought to be sure we’re patching on a regular basis and minimizing the attack surface area we present. It’s the least we can do as IT professionals.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Code Supply Chain Security

    There have been a number of attacks in the last few years on source code. In fact, I saw a new one this week for an e-commerce WordPress plugin. This time hackers got access to the distribution server for the company, Fishpig, and altered the plug-ins that their customers download.

    A few years ago this was big news, with the SolarWinds exploit. There was also an attack on PyPy, a popular Python package that many people include in their code.  There have been no shortages of problems in npm packages as well. I’m sure this has happened in other software packages, which is scary. In the days of DevOps where we publish code from a repository, an exploit against your developers might go unnoticed. Then again, maybe not.

    Would any of us notice new code in a file share or a folder on our system? We might just compile a large project without realizing it. At least with DevOps, we have the opportunity to include security scans and code analysis checks, some of which could look for known patterns of exploits. I know some companies use these, and often compromised or vulnerable packages are stopped by the automated pipelines.

    In the US, various security agencies have released a set of recommendations, as has the Open Source Security Foundation. Both of these are designed to help developers secure their supply chain against attacks. This is likely going to be a continuous problem for software vendors in the future as it’s much easier to attack one vendor whose software many people use than each individual company. I shudder to think about what happens if someone manages to get a ransomware package into a vendor’s codebase.

    Ultimately, there will still be problems. Many new projects begin with poor practices precisely because they’re experiments and the authors don’t know if others will find the software valuable.  While we can have good templates and security controls, I’m not hopeful. To me, the best solution for stopping code is to have patterns detectable by security checks in the pipeline. Checks that can be expanded and enhanced as new issues are determined.

    Of course, that means the makers of security software need to ensure their supply chain is protected as well.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Reducing Debt and Increasing Security

    Many of us know that reducing financial debt in our lives leads to a bit more security in our minds. We are better able to cope with unexpected expenses when we have the ability to get credit or pay for something in cash. If you’ve ever had a major house repair, car calamity, or (hopefully not) medical issue, you know what I mean. At least in the US. That last one is less of an issue in many other countries.

    We know that our code has technical debt, and often we fight a battle with project managers and business people over whether we should spend time paying down that technical debt or developing new features. We certainly can do both, but often that’s not a choice we are given. If we do get the chance to reduce technical debt, it’s a tiny fraction of the time that is allocated to adding something new.

    There might be a better argument for reducing some technical debt: security. There’s an article that talks about the security risk posed by moving too fast and not ensuring there is rigorous security testing. This is a common problem in many companies that adopt some parts of DevOps, but forget that quickly deploying new code isn’t the only principle. We also need to learn from our mistakes and quickly correct them.

    I am a big fan of releasing often, especially for security purposes, but those releases need to build on each other with not only new code but refactored old code that doesn’t perform well or has a security issue. More importantly, we need to ensure when we find issues in code, that all developers are aware of the problem and don’t repeat the issue. We also need to update our testing to ensure that humans don’t continue to write code in a style that causes known issues. We want to avoid antipatterns in our code.

    This is less of a developer issue than a managerial one. The C-suite and all management that are involved in software need to prioritize security so that known issues are fixed quickly. This also should involve ensuring that technical debt is kept under control as a lot of this poorly written code increases the chances that a vulnerability is in the code. Either because the code isn’t well written or because developers avoid touching the code in the future.

    The best solutions, to me, involve DevOps. Make small changes, deploy them, instrument them, and learn if they work well. If not, fix them quickly. While this might end up delaying some features, ultimately it ensures that our software is in better shape, with a reduced risk of simple security mistakes upsetting our customers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Republish: It’s Not Yours

    I’m still at Techorama.

    Check out It’s Not Yours from the past. For some reason I didn’t get it on this blog, so here it is:

    It’s Not Yours

    For most of my career, I’ve felt a tie to the systems I’ve managed or the code I’ve written. I’ve often said that a part of the code was mine, because it was something I’d written. Having some ownership of your work has often meant more pride in ensuring things are well implemented. The same thing goes for administering databases. When an employee feels personally responsible and accountable for  the database and its contents, usually they do a better job.

    Most of us know that the systems we manage and the data they contain aren’t really “ours”, no matter how we might refer to them in conversation. We know that ultimately the data, applications, and hardware belong to our employers. Or, we should know that.

    Apparently not everyone does. A contractor just pleaded guilty to stealing classified information from the US National Security Agency, about 50GB worth of data. This isn’t a case of spying or malicious intent. Instead, the report is that the employee was a hoarder, just keeping copies of data for some unknown reason. Hopefully that’s true.

    Many of us will have copies of data that we use for different purposes. We might be building software, or testing queries, or some other task that requires data. We might want to conduct more in depth analysis in our spare time. Those are all positive goals, but none of them mean that we should have copies of data without authorization. We don’t own the data, and we should always be sure if we are moving data to other systems, that those systems are secure and someone else has approved the action.

    This becomes more important all the time. We have enough issues with data security and keeping our information away from hackers. We make plenty of mistakes with configuration in authorized development and test systems. There’s no reason to add to the issues by treating data as our own personal asset, no matter how good your intentions.

    Steve Jones