Voice of the DBA

Tag: security

  • Maybe It Is DNS

    There are jokes about how often DNS is a problem. There are stickers, shirts, and mugs you can buy. There’s a classic haiku, which leads into a rant about the struggles of sysadmins. I will admit that I know a little about DNS, but I find myself confused at times and usually seek assistance from others when something doesn’t work. The old days of a single MX record and a few A records to map names to IPs has grown into a much more complex system.

    DNS usually works. In fact, it’s lower on my list of things to check these days, though I do often get a quick DNS check when I ping an address. There are some failures, and these days when large companies have issues, it can affect a lot of people. Akamai had an issue and knocked off a large chunk of the Internet. When Facebook had an issue recently, it not only affected customers, but the engineers that were trying to fix things. Some couldn’t even get into buildings. I guess relying on a single domain might be a bad idea.

    Most of us rely on DNS in our organizations, which means that a rising level of DNS based attacks is disconcerting. These are still behind the issues we have with ransomware and DDoS attacks, but when DNS doesn’t work, a lot of stuff shuts down. I could see a lot of people that work inside a company’s domain for most of their work being unable to accomplish anything.

    I tend to use a number of domains for my daily work. Evernote and Dropbox are essential, but if one were down, I could still do things with local copies. Red-gate.com is certainly important, but as long as SQL Server Central is up, I can do something different. Actually, if both were down, I could still write or create questions. I’m more bothered by slowness in my connection or machine than losing any particular service.

    I haven’t every been too worried about DNS security, but I left sysadmin work when it was a simpler time. These days, I suspect many organizations might want to be sure they are paying attention. If they use a lot of services connected to one domain, I might think about having a secondary domain connected to a few. If I were in charge, a sqlservercentraladmin.com might be one I’d set up.

    Steve Jones

  • How Often Do You Patch?

    I saw two things recently. First, CU25 for SQL Server 2017 was released. I realized that I was CU8 locally, so I had work to do. Second, I noticed that SQL Server 2016 fell out of mainstream support on 13 Jul, 2021. Amazing that it’s been around for 5 years already and has moved into Extended support. We’ll still get security updates, but nothing will be fixed from that version, which it seems many people are running.

    Microsoft also announced they will put out a final Service Pack for SQL Server 2016. I am glad to see that, as I’ve often wanted to know there is a final patch that is released for each version, as I can know when I’m done patching.

    Today I’m wondering if you track this at all. Do you actively look to ensure your instances are patched? Are you on a schedule of some sort, maybe matching Microsoft’s every-other-month pace, or perhaps just a few times a year? Or is this more ad hoc?  When I go visit companies, I often find instances at all different levels, often without any reasoning as to why systems aren’t consistently patched.

    I used to try and ensure we patched every quarter, though certainly, I faced resistance from some business owners of systems over the worry that a patch might break something. Valid concerns back then, and still somewhat valid today, though if a patch gets through a few months without reports of issues, it’s probably safe for most systems. Still, make sure you test, especially for business critical workloads.

    These days, with so many hacking attempts, ransomware distribution channels, and no end to phishing, it makes sense to stay on top of patches and make sure you are up to date or at least planning to do so. Some of the high-profile hacks, like the Equifax one, have occurred on systems where patches were available but not applied. I don’t know that many of the patches are closing holes in SQL Server, but there are definitely issues with Windows where you would want to ensure your host OS was patched. Maybe that’s something you ought to check on today and ensure you have a plan to apply those updates.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Danger of Management Access

    First we had the Solarwinds hack, and now we have a Kaseya ransomware epidemic. It seems the criminals are moving up the stack. We used to see physical attacks on tapes and keyboards, then we saw OS level attacks. Now we seem to be getting to the management layer for software that is used to help us run systems at scale. Since we often require some level of privileged access for monitoring and management systems, this is scary. I certainly wish that we didn’t require admin access for monitoring, but unfortunately platforms sometimes do.

    Many of us depend on some standardization and some sort of software to ensure we can manage systems at scale. I don’t know about the OS world, but in the SQL Server world, there are relatively few vendors that provide software for managing systems. If one of these were compromised in some way, this could be very bad for many database administrators. Fortunately, many of us know how to air gap backups and ensure that we are prepared for disasters.

    Or we should. If you don’t know how to do this, you ought to be learning right away. Review backup plans, ensure you can rebuild systems, test restores, and brush up all your recovery skills. Be ready for whatever a criminal might throw at you, including having gotten ransomware into some of your backups.

    This attack seems to have taken advantage of a zero day, or very early, vulnerability that was discovered by a Dutch security research firm. The firm looks into management software, especially admin interfaces, specifically because they are worried about the lack of security in many products. In this case, Kaseya builds tools that allow admins to distribute software to other systems on the network. In this case, criminals used the management software to distribute ransomware.

    The updates from the Kaseya are less than stellar, and if I were a customer, I’d be rather upset. They seem to keep setting unrealistic plans to restore service and then constantly revise them across a few days, all the while with customers that are likely stressed and overworked. I’d also be upset in that they claim only a few of their thousands of customers are affected, but they neglect to admit that some of those customers affected as Managed Service Providers, who themselves have thousands of customers using this software.

    There are some technical  details in this piece, in case you want to check your own systems. If you think you have multiple pieces of software that might protect you, read the article. This deployment shuts off some other products, like Microsoft Defender.

    I feel bad for many people here. IT staff at affected companies that have likely been incredibly stressed and overworked recently. The consumers of some affected customers, like those that might shop in the Swedish grocer, Coop, who shut down more than 400 stores. I don’t know the state of grocery shopping in Sweden, but this might dramatically impact many people that just want to buy food for their families.

    Ransomware continues to surprise and worry me. Large profile hacks keep coming, affecting lots of people. Often these are because of previously undiscovered software vulnerabilities or simple mistakes made by privileged users. I hope that at some point insurers and governments start to put more pressure on companies that make widely used software to ensure they are adhering to best practices and have detailed security practices in place to ensure their code is constantly checked for issues, and that they have detailed plans for responding to and patching customers when there are issues. Because, they likely will have an issue at some point.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes

  • Hostage Data

    Ransomware continues to surprise me in different ways. Recently there was an incident where data was not only encrypted, but also copied back to the criminals. In this case, Apple was the target through a supplier with the ransom note saying that without a payment, the data would be auctioned off.

    That’s crazy. Not only might you have business issues where you can’t access data, but now you have the stress of the data possibly being released or sold. While not many of us work in organizations where our data would be worth $50mm, it might be worth a significant amount, especially if the data were customer data. This alone might be a good reason to ensure that you have local data already encrypted without the keys present. At least then the criminals couldn’t read your data.

    This doesn’t help with SQL Server and TDE. In that case, the certificate would be inside the local master database, and if someone could attach it and get access to the master database, they could read your databases. If you have an SMK and a DMK, perhaps this might offer some protection, but I don’t think so. This does mean that Always Encrypted might help, unless you have lots of servers or other machines on your network with the certificates, in which case someone might be able to piece together the keys and read data.

    Attacks are becoming more numerous and creative. Having backups might have protected you against some ransomware, but not if copies of your files are sent to criminals. Perhaps the access from servers to the outside world needs to be more reigned in. Not much fun for administrators, but this might be the future of protecting systems.

    The arms race between sysadmins that protect infrastructure and criminals seems to have taken a leap forward here, and I’m not looking forward to the next step.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.