Voice of the DBA

Tag: DevOps

DevOps and OSS Can Scale

Microsoft has changed quite dramatically in the last decade, in many ways. One of the more surprising ways to me is their embracing of open source software, while still building products that customers want and use. This is quite a change from the Microsoft that I knew back in the early 90s when I started working with SQL Server. Open Source software was seen as a complete anathema to the Microsoft way.

One of the very interesting ways that Microsoft works in the open is their use of GitHub as a VCS (version control system) for many projects. From .NET to Azure Data Studio to the new virtual kubelet code, thousands of Microsoft engineers are writing and committing code in a very transparent way. Not all their products are here, but there are more than I would have ever expected.

Part of the challenge of using a (formerly) third party platform is the infrastructure and administration of users. Microsoft has a scale issue here that most us do not deal with. While we work with dozens or hundreds of developers, they deal with thousands. There’s a good blog post about how they’ve scaled up to ensure 25,000+ developers can work.

The post has lots of information, both at a high, conceptual level as well as at a detailed, technical level in places. Some of the big themes I see in the post is that automation is important, and that there are tools that facilitate the collaboration of their developers by sharing information, triggering notices and reminders, and by enabling the discovery of others. Collaboration is a huge part of DevOps, and the more software can encourage this while reducing the burdens of tedious work, the better the system works.

None of this works without a culture change. If Microsoft can do this with thousands of developers and thousands more non-technical staff, any of us can. Microsoft has grown in the recent years to be one of the more valuable companies on the planet. Many of our organizations can learn from this by adopting DevOps, but not just in name or at a technical level. The technical tools are easy, but to get your organization to perform better, you need to ensure culture grows with the Three Ways and a team environment. Let your management know that you can all do better, but only if they truly embrace all the concepts of DevOps, teamwork, feedback, and learning from our previous actions to drive continuous improvement.

Steve Jones

Listen to the podcast at Libsyn, Stitcher or iTunes.

July 17, 2019
Ransomware and DevOps

Ransomware.

A scary topic and one attack that is apparently more common than I suspected. Before you go further, if you haven’t restored a database backup in the last month, stop and go verify your DR plan works. That’s one of the overconfident issues facing lots of government and businesses. While this might not help your entire organization, at least you’ll have some confidence in your process and that you can recover a database.

This is a great article from Ars Technica and worth reading: A take of two cities: Why ransomware will just get worse. I’d recommend you read it and think about a few things. First, do you have insurance because things (or substitute your own word here) happen? Second, have you really tested a DR plan for some sort of software issue like this? You might think about a way to restore systems in an air-gapped manner that prevents them from re-triggering encryption from a remote source, or maybe even in a scenario where you reset dates/times to prevent timer triggered issues. If you don’t think you need to, read this article as well.

Perhaps the bigger issue is are you actually patching and updating systems? Too many organizations can’t, don’t, or won’t. The former means you aren’t sourcing software properly. Either you’re using vendors with poor practices or have a poor development process. Organizations that don’t or won’t bother prioritizing patching, especially security issues, are likely those that will have issues as more criminals spread and use ransomware and other attacks for profit. Software and environments continue to be more complex, which means that the less you ensure the system is patched, the more likelihood there is of a vulnerability in your environment.

DevOps and the cloud PaaS/SaaS platforms are attractive for a few reasons. One is that the platforms are constantly kept up to date, forcing you to move along with them. SaaS cloud vendors know this and are constantly patching and updating their software in order to keep it running. DevOps asks that we always have the ability to release, that we have the ability to patch on demand, not only at certain intervals. This is something I try to emphasize when talking about DevOps. It isn’t necessarily about velocity, but it is about being able to release when you need to, whether that’s today or next month. This is especially important for security issues.

I have had hope for a long time that insurance would drive software to higher quality, and I still do. With the attacks and issues of ransomware, and who knows what other techniques that will be developed, I still believe more companies will buy insurance. I then hope, because of selfish motives, the insurance companies will require frequent patching, regular vendor certification of new platform versions, and better development processes. If insurance drives DevOps, I’m all for it, but I’d prefer you decide to adopt it yourself and start making changes today.

Steve Jones

Listen to the podcast at Libsyn, Stitcher or iTunes.

July 15, 2019
Building Helpful Commit Messages

Using a VCS (Version Control System) is important in any modern software development environment. I still can’t believe that both application and database developers are using a VCS at less than the 95% level, but surveys constantly show this. I’m not sure if this is because of laziness, ignorance, stubbornness, or something else, but it’s a bad idea. Start using a VCS. Talk to your management, and let them know Git is free and can be run on premises, much like a file share for code, except it’s better. Even if you hand craft scripts (or use the SSMS GUI), just start saving those scripts in a VCS.

There are lots of tutorials and examples of how to use git (or any VCS), and this is a skill you want to build over time. To start, just learn to build a habit of committing code to the repo as you make changes. This is fairly easy, but there is one skill you need to develop over time: how to write a good commit message.

When we commit code, we usually need to enter some comment. This is a bit of an art, and should be designed to quickly convey information to the next developer (who may be your own future self). While people may have their own view on what to include, my thought if usually to include the “why” of the commit. This helps me to go back later and understand what I was thinking. I don’t need “what” was changed, because a quick diff of the file will explain that. If possible, I like to relate a commit to a specific request, like a Work Item, a help ticket, or some place that describes the requirements in more detail.

I ran across a nice post that describes the art of the commit, and I think there are some good ideas here. Certainly being brief and understanding that often your message is a headline is good advice. The most important parts of the post, for me, are the need to understand that I’m working with others, and it’s important that my message help them and vice versa.

Learning to use a VCS is a critical skill for anyone working in the technology field. We know most application developers need this, and slowly I find more database developers doing the same. However, as we move to GitOps and Configuration and Code, as compliance becomes more impactful with more auditing requirements, as we try to do more with less, the infrastructure staff and others need to learn how to capture their changes in a VCS as well.

Do yourself a favor and start learning how to use a VCS. Git makes it easy or you can play with Azure DevOps in the cloud. Just start saving some files and building skills. Your career will thank you. Learn to write good commit messages and your team will appreciate you.

Steve Jones

July 8, 2019
Getting Started with Spawn

I’m excited that the Redgate Foundry is working on some cool projects. The Future of DevOps is one and part of that work is with Spawn. I’ve played with this a little, and I wanted to write about a few adventures I’ve had.

First, you need an account as Spawn is hosted and there is a login. The idea here is that the service (hosted now, but could be on premise) handles all of the complexity of databases in containers, spinning them up and managing them, with you just connecting to an endpoint to use the database. We can spin up SQL Server, PostgreSQL, MongoDB, MySQL, and Redis in this fashion. It’s truly a Database-as-a-Service (DBaaS).

Spawn on Windows

For once, this is easier, or I think it is. I downloaded the spawnctl.exe CLI and then put it in a folder in my path. For me, I have a “utilities” folder where I drop stuff and that’s where this lives. I can check this working with a version parameter.

So far, so good. Now we need to log in. I like integrated authentication schemes, and this one works well. I start with a “spawnctl auth” call from the command line.

This pauses and opens a web browser window. It’s here I need to log in. We support Github and Microsoft, but have mostly been using Github for our demo work.

Once I click that, since I have a Github session elsewhere, I’m logged in and I get a code back.

I paste that back in the cmdline and I am authenticated. From here, I can then see what images I have. As you can see, there are both MSSQL and PostgreSQL images for me.

I also have a few containers from my demo work.

Let’s make a new container and connect to it. I’ll use Azure Data Studio, since I can connect to both SQL Server and PostgreSQL.

Logging In

To get a database, I’ll use a CREATE verb and the DATA-CONTAINER noun, along with an image. When this runs, I get a note this container is being created, then I get a response with login information.

Let’s put that in ADS. I can copy/paste my credentials into a connection dialog and then hit Connect.

As you can see, I’m connected.

I see the demo database we’ve been using and the tables in there. How this works is worthy of more posts, but for now, this is pretty cool.

I can do the same thing with PostgreSQL. Create a container.

Then connect (need to specify the port in Advanced)

And query:

Pretty cool. Simple, easy, and I think about as fast and easy as any other cloud database. Here, though, I am spinning up a container from an image template. That’s valuable for dev work, and I’ll talk about that in the future.

July 1, 2019