Voice of the DBA

Tag: DevOps

You Always Have a Software Pipeline

I was working with a customer recently and they said they didn’t want to set up a pipeline. Somehow they wanted magically to see changes that developers made to their database appear in their production database without making any effort to build or configure a pipeline in a tool like Octopus Deploy.

As I was thinking about how to find them a solution they would accept, I realized that I’ve always had a pipeline for software. Early on, this was a copy of all the files in a folder sent to the client for Clipper and dBase apps. Later, my pipeline was copying around all the VB6 files from a .zip file and executing a few SQL scripts that were sent to me in emails.

At one point, I had developers just tell me the software pipeline was them copying files to a server while I had to move all the changes from the QA server to production. I didn’t like that one for two reasons. One was that the developers would sometimes forget to move a change from dev to QA, despite claiming they’d tested the application. Two was that the developers sometimes moved too much to QA, and had changes to the schema that they hadn’t finished in their app. SQL Compare couldn’t help me in either of those cases.

Over the years, I learned that a slightly thought-out pipeline is better than nothing. I wrote my scripts to simulate what Flyway does today, executing the scripts I’d placed in a folder. I added logging that would capture output results and save them for later review. I learned how tools such as Team City or Jenkins could execute the same thing I could but in a more repeatable, reliable manner. After all, my reliability at 4:55 p.m. on Friday isn’t as good as it is at 2 pm on a Tuesday.

DevOps is about learning and taking advantage of tooling. Look at what works in your pipeline and find repeatable, reliable ways of accomplishing those tasks without humans doing more than clicking a button to approve something. It’s about using your creativity to improve the process and codifying it in a way. It’s about being better than we were last week while ensuring the process isn’t dependent on any particular person. After all, I might leave for vacation.

Or find another job.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

October 13, 2023
The Database DevOps Roadshow Continues (with fun photos)
At SQL Saturday Denver 2023, I had a few people ask me about the Redgate Database DevOps roadshow and how it was going. A few people asked about the van as well.

Here’s the van, with my ready to rock in Chicago:

Chicago is done, but we have a few more dates coming:
- Oct 11 – Atlanta
- Oct 19 – Jacksonville
- Oct 26 – San Diego
- Nov 1 – San Francisco
- Nov 14 – Seattle
All these are on the Redgate Events page.

I won’t be doing any in October, as you can see above, I’m tired. With the 3 US ones and the 3 UK ones I did across Aug/Sept, I was gone 24 days from Aug 23-Sept 21. I need a break, as shown below:

The show goes on, and I’ll see (some) of you in Nov.
October 6, 2023
The Need for DevSecOps

One of the things that happens with many companies that start adopting DevOps is that they release new features constantly. They publish their lists of changes, and they try to attract customers and grow their businesses. They may make some mistakes, but they fix those quickly and keep pushing forward. That’s the idea, and it works well.

However, many of the developers (and most managers), don’t think about the security side of their changes. This piece looks at the way hackers and criminals view DevOps, often using release notes and feature changes as a target to focus their efforts. In this way, they exploit holes and vulnerabilities in software to attack data storage. The examples include S3 buckets of storage and Elasticsearch, which is notoriously poorly secured by many people.

I’m sure there are hacks that also expose relational data stores and NoSQL stores, but those are often more secure and harder to directly attack. Certainly, hackers do get credentials and can query data, it’s more often I hear about data breaches from other sources than direct relational database access. SQL Injection is definitely still an issue, and I hope that more and more developers are learning patterns that avoid these vulnerabilities.

DevOps works. I think it’s great. However, it’s not enough to trust developers to build features without including static code analysis, pen testing, and other security evaluations before you release code. Often developers can build features just as fast with good patterns as bad. Use automation to catch bad patterns and force developers to learn new ones.

Also, avoid letting developers implement data stores out of convenience and speed. Ensure that strong security practices, long passwords, service accounts, secrets, and more are implemented from the start. It’s easy to shortcut these, but it’s also harder to explain to customers and investors why we didn’t do better.

If you’re a developer, the main thing to keep in mind is that you will often be the scapegoat for these issues. Upper management might not support you and pressure you to move faster, but when there are issues, they’ll also be quick to blame you and let you go first. Push back and ensure that you have the tools and the process to evaluate if you are building problematic code. Always use long passwords, and document what you do for others to follow.

And if your boss insists on cutting corners, get that in writing. It might not save your job, but that documentation has served me well in the past when issues come to light.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

October 2, 2023
Creating an Empty Repo in GitHub–#SQLNewBlogger

I saw someone struggling with getting started with a Visual Studio project and Azure DevOps. They got a conflict, which I’ll show and then get you started with an empty repo.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

The Problem with Initialized Repos

I wrote about this recently with Azure DevOps and empty repos.. If I initialize a repo in GitHub, or any online provider, and try to push up a local repo, it returns an error that I have conflicts.

I’ve learned that when I want to start a new project, either I create the repo locally and then make an empty one online, or I create it online, clone it, and then start working.

I try to do the second (create online first), but I don’t always remember.

Creating an Empty Repo

In GitHub, if you go to your repositories page, you can see a “New” button on the right.

Click this and enter a name, as I’ve done below. BTW, I love the suggested name.

If you look at the image above, near the bottom, you can see my mouse pointer.

DO NOT check this box if you want an empty repo. Don’t add a .gitignore either. You’ll be adding code from your own project, which likely has a gitignore file and might have a repo. If not, just create a readme.md file in your local git repo.

Don’t add a license file either.

Pushing My Project

Now when you create this, you get the details of how to push your repository. If you have never used git, just open a command prompt and paste these lines in one by one and run them.

That’s it and now you can work and commit in either repo and deal with merges.

SQLNewBlogger

This post was easy for me, but I’ve been in the situation where I’ve made this mistake. In this case, I’d written the other post, so I knew what to do here. I spent another 10 minutes getting GitHub set, taking images, and writing this.

Easy to do, 2 posts from one idea, and showcasing knowledge on another platform. I could do another one with BitBucket or somewhere else, and maybe I will. I have customers that use other platforms.

Maybe you should do this and showcase your versatility to the next person that wants to interview you. If you want to work with software and Git, show them you know the basics. Or even more.

September 20, 2023