Tag: security

I remember working at a large organization with a team of other IT Operations staffers. We rotated this one job every month amongst a few people, each taking turns, where we’d lose a day to update all the privileged passwords for our servers. This was before Managed Service Accounts and the cloud, when we were required to change these every 30 days and then store the new ones in an encrypted store.

What struck me when I got stuck with this wasn’t the requirement to change every 30 days; that seemed normal. The thing that bothered me was how manual this was. As a former developer, I wrote some scripts to automated this, pre-PowerShell, and make the task easier on my fellow sys admins. I had scripts to generate a password, change it in AD, then print the pwd to be copied into our secure storage (no API there). This ran in a loop so I didn’t lose a whole day to changing password.

These days, we have lots of alternatives to managing passwords, and in fact, much of modern guidance isn’t to require password changes so often. For systems, use an automated process such as an MSA or GMSA. For users, we’ve mostly given up on changes and are trying to get people to use decently long passwords and disparate ones across services.

Modern guidance from Microsoft says to avoid using common passwords (asdfasdf, password1, ec.) and don’t use the same password in multiple places. MFA is also recommended, but the anti-patterns for success are requiring long, complex passwords or frequent changes. Studies show these lead to less security because users do stupid things.

These days, I would guess many of you managing database systems use some sort of integrated security with AD, Entra, OAuth, etc. However, I know there are still places where passwords are in use. Do you require changes often? Do you change any of your passwords regularly?

Security is always hard, and it’s even harder when the recommendations and rules aren’t consistent or even enforced. I don’t know what to do, but I try to use disparate, long passwords and MFA wherever I can. So far that’s worked well.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

I read a piece recently that got me thinking that data breaches might be inevitable. Disclosure: This was written by Redgate, for whom I work, titled “Data breaches May Be Inevitable—Compliance Failures Don’t Have to Be“. It’s based on our research with the State of Database Landscape survey as well as feedback and conversations with customers.

The thing that caught my eye was the first part: data breaches may be inevitable. Do you think that’s true? Are we doomed to lose data in our organizations, not as a possibility, but something that will happen at some unknown time in the future? Those of you who have suffered breaches might agree with this, but for those of you who haven’t had to deal with that situation, are you resigned to it happening at some point?

I’ve gone through phases in my career. In a few jobs I worried about this. When we started SQL Server Central, we were very careful and cautious, trying to be sure we wouldn’t be caught losing email addresses of the community members. We never had an issue, and I’m proud of that. These days at Redgate, I worry a bit more, mostly because we’re growing and it only takes one person to click on a phishing email to cause problems. Fortunately our IT group is on the ball and even conducts plenty of “mystery shopper” email campaigns to test our awareness with follow-ups to help ensure people are aware and careful with how we handle data.

The criminals always have an advantage in that they are focused and only have to take advantage of one mistake to win. We have to constantly be vigilant, across all our staffers and systems. Simple mistakes, or worse, typos in complex configuration files could result in a data breach. One of the places this can happen is in test/dev environments where more people the make changes. If we have sensitive data in those places, we’re increasing our risk.

I don’t think breaches are inevitable, mostly because companies are getting better at simple security measures and reducing the ways in which unauthorized access takes place. There’s also the fact that the world is huge and there are more and more systems all the time, which might reduce our chances of the criminals targeting us. Don’t get complacent, as automation gives criminals a tool to attack us at scale, in the same way it lets us get more done with less.

One criminal can do a lot of damage, so remember that the next time you’re annoyed by your MFA or other security measures.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

The latest code supply chain attack isn’t a direct attack, but a failure of a system designed to be efficient. There is a Go (Go-lang) module that had a malicious module inserted into it years ago. Someone caught this and removed the module from GitHub, but Google had cached this and has kept it alive for the last three years.

This attack worked because it relied on the Go Module Proxy services, which prioritized caching. Even when the source changed, the cache wasn’t invalidated or reloaded, which seems like a major ovesight or even larger design flaw, but I know everyone trying to maintain software archives at scale tends to cache a lot. After all, so many developers might load modules in their daily work, in CI, etc. that caching matters.

However, the bigger issue is that criminals are getting more enterprising and aiming for supply chain attacks where possible. One would hope that any PRs (pull requests) are carefully examined, but the truth is that a lot of people might depend on their unit tests passing in a CI build. They are not necessarily looking to see if anyone has actually entered malicious code or even poorly written code. I can see some people worried more about code structure and naming (or tabs/spaces) than what the code does. Imagine seeing a change that looks innocuous at the top of a file, all tests pass, and you merge the change, but didn’t notice there was a few new functions added below in the file because they’re not obvious in the UI.

Someone maintaining a popular code repo as a side project might be fooled here, but if this were in a corporate repository we might be even more susceptible. We have more reasons to trust the PR isn’t a problem if the code passes tests inside of an organization. After all, who things criminals might insert code into their corporate repo? Not many people do, but we’ve seen quite a few successful supply chain attacks in the past. Who knows how many more we don’t know about.

Security is a hard business, and when it’s extended to the code we write, it might even be harder. I know there are security scanning solutions you can integrate into your codebase, but those detect what they know about and criminals keep finding new ways to attack us. Ultimately I think we depend on code maintainers carefully examining PRs from outside their circle of trusted individuals, and even then, things can slip through.

Some days I think it’s truly a mad, mad world.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

It’s no surprise that our systems are under attack by all sorts of criminals. Some organized, some opportunistic, some just aiming for vandalism. We need to protect our digital systems to prevent issues, and a part of better protection is reducing the number of places that are vulnerable. Those places include databases.

This article discusses the rising costs of data breaches and the increased frequency of attacks. It also examines the increasing number of regulations that are demanding proof of stricter security measures. It can be hard enough to defend production systems, let alone protecting dev/test environments. I see an increasing number of organizations that limit access to production systems, even to the point that this impedes some of the daily work habits of technology professionals, but that is probably a good thing. Too many of us are too lax when it comes to security.

There are lots of approaches to getting better at security, but one of the easier ones is to avoid making copies of sensitive data. About half of you (I hope it’s not more) still use production restores in dev/test environments. That’s a tripling of the places your data could be attacked if you restore a database to a development server and a test server. If could be even worse if you make more copies.

An easy solution in today’s world is to build a better test data management process for either anonymizing and obfuscating your sensitive data or generating synthetic data. Both have their challenges and I suspect that most organizations need a combination of both approaches to both protect their data as well as build better software for their customers. After all, a huge amount of bugs are data related, where developers have not tested their software against enough different data elements. Both synth data and anonymized data help here to produce enough different edge cases that your testing is thorough enough to increase quality.

Of course, you need extensive testing, which means automation. Ideally an automated DevOps flow that subjects your software to increasingly complex tests as it moves through your pipeline to ensure it’s ready for release. This also means a good set of test data, not only for QA, but for automated tests. You need a test data management strategy.

Securing digital systems is a complex task, but we ought to try and make it easier on ourselves by developing good habits in how we manage both access and by limiting the copies of sensitive data.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.