Voice of the DBA

Category: Editorial

  • The Quiet Part

    Apparently, Meta did what a lot of employees suspect their management will do: use AI to lay off people. In this case, there is a report (and lawsuit) that Meta used it’s AI-integrated HR platform to make decisions about who to let go in a layoff. The rumor is that the AI used productivity metrics to choose who was the target of the layoff. As with a lot of AI failures, this appears to be another case of poor communication guiding the AI, or the AI not actually taking individual situations into account.

    A number of the people terminated were on maternity/paternity leave, which is a protected activity. Others may have been on other medical leave, though for privacy reasons, the article doesn’t have firm data to prove this. The lawsuit will likely bring more of this out, but this appears to be a case not just of AI making decisions, but of poor behavior from management. Plaintiffs were discouraged from taking leave off, which is something sh****y humans have said to people for decades. We need you; your baby or family isn’t important, so don’t use your leave. It’s one aspect of working in the US that is way worse than overseas, where there are more employee protections.

    Meta disputes the case, saying the AI didn’t make decisions. However, that brings out an interesting point. If the AI recommends things, who’s responsible? The humans, right? They still have to sign off on the decision. If they don’t perform due diligence, they’re still responsible, correct? I think so; after all, I’m responsible for code the AI writes if I commit it. Even if an agent does the work, I have to oversee it and approve (or grant permissions for) the actions.

    I. Am. Responsible.

    Trusting an AI to do a lot of work is like trusting a lot of smart, but very inexperienced staffers to work in your environment. There are always inconsistencies and reasons why we might code something, configure something, or deploy something a certain way. What seems like a good way to tackle the situation from the outside doesn’t always make sense when you have experience. Humans often have experience that AI agents lack.

    LLMs are relatively stateless, and while we can provide context, give them guidance, and provide comprehensive codebases, they still sometimes do silly things. This can be problematic, especially with database changes, which are stateful and disruptive to rollback.

    AI agents can make mistakes much like humans, only faster. Much, much faster.

    Labor is one of the most expensive parts of many organizations’ budgets. Plenty of management would like to replace relatively expensive humans with cheaper tokens. That isn’t working out as well in practice, despite lots of experiments. Hopefully, other organizations realize that AI is a tool, not a replacement for humans, and we can’t trust it or even believe it’s outputs without some human judgment.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Are You Working More Hours?

    Recently, I noticed my son was coming home later and going to work earlier. He typically works a 9/80 schedule as a software engineer, but at times he might work extra hours on a deadline. His company tries hard to keep employees working a set schedule without overtime. Unless there is a need, after all, this is software. However, when they are on a deadline, management warns people, and they try to manage overtime to reasonable levels to avoid burning out employees.

    I asked him if he was extra busy or if AI was encouraging extra hours. He told me this was a short-term project with a few more hours, but mostly the team was decompressing a bit after work. I completely understood that as I’ve spent my share of time with co-workers at theend ofday, sometimes sharing a drink or meal nearby, sometimes just chatting in the office or the parking lot.

    There have been a number of studies and reports that AI is making people work more, not less. The time spent on a variety of tasks, especially those where we are reading or interacting with text, has grown. I suspect there is additional pressure from management as well to get more done since everyone has an AI assistant around. There is also the excitement of tackling new projects, which can lead people to spend more time at work. While I appreciate that AI can be exciting, and you might love it, don’t lose yourself in it at work. It’s still just a tool.

    And, of course, there’s the token spend. Some managers are measuring people on the amount of work their AI assistants do, rather than by what gets done. I’ve certainly seen plenty of people using AI to perform busy work just to be seen as using AI tools.

    At times, it seems people are using AI to process more information, which they then use AI to summarize. They post these summaries, which grow in number because the AI can produce more of them. Others use their AIs to read these summaries and produce their own summary of what they should be reading. It certainly seems like AI is producing more copies of information, summarized over and over, giving all of us more things to keep an eye on.

    That seems silly. While I like AI, I fall into that group that uses it lightly for targeted tasks. I certainly try not to create more reports, summaries, or even emails to send to others, especially large groups of people. We have enough to review without AI creating even more tasks to deal with every week.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Finding Bad Queries

    T-SQL Tuesday #200 was in July, hosted by Brent Ozar, and it was a great topic: How do you recognize a bad query? In the age of AI, when lots of people will get queries written by others (people or AIs), how can you easily and quickly review code? Review is already a challenge in the software world, and I am sure it’s going to be even more challenging as people let machines author more database code.

    Lots of you might hope that an AI agent will write better code than your average developer, but I don’t know if I’d count on that. There is a ton of poor query examples on the Internet and that’s where AI models are trained. You need some sort of feedback loop, good testing, and strong guidance if you want better query code. I think it’s as likely as not that AIs will produce poor queries just like humans.

    Just faster.

    How can you recognize poor queries? How do you judge if there are issues? Brent’s roundup shows what a number of people think, and there are some good “quick checks” to use to evaluate your code. The things that people wrote in their blog posts are the types of things that I’d want added to Flyway’s Code Analysis rules to catch problems before they get deployed to production. After all, anything that gets to production usually lives for 10 years (my rule of thumb).

    Getting feedback early in the development process is critical. We know from over 50 years of software development that finding problems early is less expansive than finding them later. Whether we use humans or AI agents, we don’t want unnecessary time spent on simple bad code things that we know cause issues. Shift-left is still a challenge, even in the automated era of DevOps and guided AI agent skills. We need reliable, repeatable ways to analyze code and limit the downstream effects of poor quality.

    Especially these days, where the volume of changes is overwhelming CI/CD systems. This was a problem already in organizations with too many teams in one codebase. It’s a growing problem in more organizations as users use AI and submit more changes.

    Plus wasting time gets expensive. Human labor is slow and spending time on more tokens for rework gets expensive.

    Don’t let bad queries flow through to CI/CD systems if you can avoid it. Learn to provide quick feedback early to whoever, or whatever, writes your SQL.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Fixing P1 Queries

    When we look at the performance of software, we use Pnn notation to indicate the latency of an issue. A P95 problem is one that exceeds the time that 95% of the other queries take. In other words, this is the 5% slowest things happening, which can include database slowdowns that impact your application.

    For many years in software development, we have tended to work on the P95 or P90 issues, the slowest items. This is primarily because those can make a big difference to the system’s performance. If I fix the slowest things, then the system feels faster. Certainly, I know lots of DBAs and developers will apply this logic to database queries. They focus on the slowest queries and tune them to improve the system. If most things are quicker, especially the things most users notice, then the system feels faster.

    We’ve often limited our work on latent items because the time involved in working on other issues is hard. If you choose to work on items in the P70 range (30% slowest) or the P50 items, does that make sense? There are slower items, so doesn’t it make sense to focus on those P95 or P99 items when trying to increase the speed of the user experience? In software, I think this does make sense.

    However, in the database world, we aren’t often thinking of one user’s experience. We’re thinking about the volume of requests from all clients. We might find there are queries that seem fast, say 400ms queries, but if these are running thousands of times a minute, we might see more benefit from clients if we reduce this to 200ms than if we speed up a 30s query to 2s. For the database world, we often think about the rate of items, not just their latency (or speed).

    For database queries, how do we choose what to focus on if it’s not the slowest queries? we might not also want to tackle the most frequent queries either. My question might be: do we need to choose?

    In the age of AI, especially if tokens aren’t too expensive, do we need to spend engineering time deciding? Would it make sense to let our AI coding agent evaluate and propose changes to all the queries? With a team of agents, we might have time to investigate the entire workload and try to speed everything up. This does mean we need a good system for guiding agents, separating out work, and ensuring that each agent isn’t trying to add an index for every query. That’s something humans have done, and agents likely will try if they don’t have to coordinate their work.

    The effort of this work isn’t zero, and I’d bet there will be a significant amount of time to get this set up in some organizations, though I’d hope as people discover ways to organize agents for database work, with controls to scope their efforts and validation techniques, this becomes easier and easier. I’d like to think that at some point, DBAs will work in the same way software developers are hoping they work: thinking about problems, evaluating potential solutions, and managing AI helpers who type the actual code.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.