Voice of the DBA

Category: Editorial

  • We’re Not Faster with AI

    At Redgate Software, we’ve been trialing Copilot from GitHub with our developers. I managed to get access for this experiment and have tried a few things, though I’m not sure I’ve found it very useful. I’ll continue to work with Copilot, but for now, I just don’t find Copilot AI helping me with the types of tasks I do.

    However, our chief of technology, Jeff Foster, was interviewed about how he sees AI and Copilot in Redgate. It’s a good read, and I found a few fascinating things in the piece. First, I was worried about this a bit, as we have explicit guidelines in developers that say we can’t take the output from an AI and put it in our codebase. We worry about legalities since we sell software. How does that work with Copilot?

    Well, I learned that there are a couple of things in here. First, with a business subscription, we can have the AI not show us copies of public code. If more than 150 characters of public domain code is in a suggestion, it’s hidden. This is in the Copilot docs as well. I don’t know if 150 is a good length or too long, but at least there are limits. We also found that Copilot doesn’t retain any snippets, which is important for us. We don’t want to share Redgate code with others, especially competitors unless we are explicitly putting a public repo out there.

    The second thing I found really interesting is that we have a lot of guardrails around the way we produce code. We have tests, lots of automation to evaluate things, and plenty of peer review. I see requests for PRs in Slack constantly amongst our teams. We know that the code which gets generated likely needs some editing, but also that we want to be sure that whether it’s sent as is or edited by a developer, the code is still reviewed. Having these guardrails, test, etc. help us experiment with new ideas without impacting our productivity.

    Perhaps the most interesting part for me was near the end. Jeff doesn’t think that Copilot makes us faster. He says “We probably invest ten times more time finding the right problem to solve than on the actual coding.” That makes sense, given what I’ve observed over the last 15 years. We are deliberate and careful about picking problems to solve and producing high-quality code. Sometimes it makes me a little crazy as I want things to move faster, but I’ve learned to appreciate that overall, this is an approach that doesn’t get us into places where we struggle to onboard developers to new projects or adjust our code to meet new requirements.

    I’ve got a few quiet weeks coming up, so I’m going to give in more and use Copilot a bunch to see what I can get out of it. If you’ve tried it, let me know what works or doesn’t. If you want me to try and solve some problem for you, drop a note in the discussion with the challenge you have.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Large PRs are Bad

    I heard a joke years ago that went something like this. When a developer gets a pull request for code review that’s 100 lines long, they will open the file(s), look at the code, and ensure standards are being followed. They might run some code, they’d walk through the algorithm in their mind, and spend 10, 20, or more minutes examining how the change is built. If you give them a code review of 1000 lines, they’ll just assume it’s great and approve it in seconds.

    I know that many developers don’t find that funny. Often I meet people that think they’re professional and they spend the time needed to examine the changes and ensure they work. I’m sure that many people do this often, and some people are very meticulous about their evaluation of the change. However, everyone gets busy and I know that often depending on who submitted the PR and how busy the reviewers are, the joke does reflect reality. The longer the PR, the less likely someone will either a) spend the time to carefully review it, or b) catch small mistakes.

    There have been a few large profile outages in 2023, one of which was an Azure DevOps outage in Brazil. I’m not picking on Microsoft as AWS, GitHub, GCP, and others have had issues. I know GitHub is part of Microsoft, but it’s also a separate enterprise that really runs on its own in many ways. The point is more that there will be issues, and some of these are related to the rapid changes of a DevOps or GitOps workflow where PRs aren’t always reviewed clearly and cleanly.

    In this case, there was a typo in how a process worked. A cleanup process was supposed to delete databases in Azure, but the typo had it deleting the logical servers. Those had many databases, not just old ones that needed cleanup. This PR, however, contained a lot of changes, as there was an upgrade to swap out older Azure Manager packages with Resource Manager packages. I don’t know if the cleanup job was related here or included in a large PR, but in any case, the PR was reviewed as in the joke above. It was approved and things started failing.

    This wasn’t caught in testing as there wasn’t great test coverage. You can say MS should have more tests, and they should, but there will never be enough test coverage. There also weren’t any systems in their ring 0 (first) deployment that triggered this typo, so no one realized there was an issue. Again, ring 0 systems might not be representative of larger rings. Another reality that we aren’t likely to fix in every situation.

    Microsoft recovered the data, but it took a long time. I don’t know how easy or feasible it is to create smaller PRs with something like this when you are upgrading packages in many files. I just know that the time that I make large-scale changes in code, with large PRs, often I find some problem somewhere. Especially if there are changes that aren’t all the related. If too many different things get included, the potential for mistakes and problems goes up.

    I think this is actually a good DevOps story. They ran their process and there was a mistake. They fixed it and have started to adjust their process to add more testing in this area and potentially ensure this doesn’t happen again. The logging helped them diagnose the issue quickly once it was reported. Their ability to deploy on-demand meant that once the problem was understood, a fix could be quickly deployed. That’s what DevOps is: it’s not perfect, but it does allow us to understand, learn, and adapt quickly.

    Now we just need to ensure that humans use the process in a way that other humans can more easily understand, with smaller PRs.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Republish: The Minimum Upgrade Point

    I guess I’m technically working today. I’m at That Conference in Wisconsin, speaking and networking with fellow software geeks, but I’m expecting to be mostly out of touch, I’m republishing The Minimum Upgrade Point

  • The Choice of SQL Server Version

    Every quarter Brent Ozar publishes some data from his SQL ConstantCare® service. This is a service where companies contract with Brent to install a service on their instance, collect data, and give them simple, short daily emails on things they should check. It’s a good service for companies who don’t employ skilled DBAs and may relay on a developer or sysadmin to manage a SQL Server instance. While this is a self-selecting group of organizations, across his 3,100+ monitored servers, there are likely trends that could apply to the world of SQL Servers in general. After all, for every gung-ho, let’s-upgrade DBA, there’s probably a sysadmin with a similar mindset.

    In any case, the summer 2023 report form Brent shows that SQL Server 2022 adoption has slowed. His report is down, though I doubt anyone downgraded. Perhaps someone was testing and added a 2022 server in the spring they removed. Or maybe they tried 2022 and then went to the cloud. He does show 2019 growing and 2016 shrinking, which dovetails with what I see from my memory of various questions at SQL Server Central. I see people asking about moving from older versions to 2019 much more than 2022.

    I wonder what that is? Brent thinks this is because people standardized on 2019 installs and haven’t moved to 2022. So anyone adding new instances likely uses an image/setup/process for 2019. That matches with a few of my customers, who haven’t had some of their install or security processes updated and are still adding 2019 instances. I think that’s short-sighted as 2019 is 4 years old, but I also understand that people get busy and updating anything for a new version isn’t a priority.

    There have also been some problems with updates, and Brent thinks companies are skipping 2022. I don’t know, but I do wonder what you think about your estate and how things are changing. I assume if you are still running 2014- at this point, you’ll just live with the server as long as you can. I hope you’re at least on a VM so you can restore quickly if there are issues (assuming you back up VMs).

    If you run 2016/2017, are you looking to upgrade? Considering 2022 or stick with 2019? Or kick the can and hope that SQL Server 2024 or 2025 will be better? Actually, take a guess as well on the next release date. I’ll take a page from Brent and run a contest for you to guess the next release date.

    Steve Jones