Voice of the DBA

Tag: AI

  • How Helpful is a GenAI Copilot

    I’m sure many of you have tried a GenAI LLM to do something. Maybe write some code, maybe get some sort of recommendation or suggestion, maybe to rewrite something or summarize text. I’m sure you have had some feelings about whether the tool made you more or less productive.

    There was a trial conducted by the Australia Department of the Treasury on Microsoft’s 365 Copilot, asking for volunteers to participate and use the tool in their daily work. They used it and then completed a survey, which are summarized in this piece. Only 218 people went through the trial, and the results are interesting.

    The headline says that the staff rated the GenAI less useful than expected. Those last two words are interesting because your expectations shape a lot of how you view anything in the world. If you expect little and get a little more, you might be happy. If you expect a lot and don’t get it, you might be very disappointed.

    The sub-headline and the first sentence note that there still is an ROI from the tool. It isn’t as helpful and isn’t as widely applicable as people expected, but they chalk some of this up to product limitations and some to limited use by people. It was useful in summarizing things and drafting content, which are what they call basic administrative tasks. That’s interesting and likely where GenAI tools can help quite a bit.

    Maybe the most interesting thing to me is that if Copilot saves 13 minutes a week for mid-level workers, it pays for itself. I don’t know how much time it would have to save me, but an hour or two a week might make me use it more. It certainly would be use a small monthly cost. So far, I haven’t committed to regular work with the tools, and I think I still spend more time learning and typing with GenAI tools than I’d like. I’m not sure if I am saving time over just doing the task myself. Some of that is because I have habits that allow me to work quickly and to use a Copilot, my pace slows.

    This also brings up something I wonder about with these GenAI services. With the cost of compute services, there might not be a lot of margin for vendors to raise prices if people are only lightly more productive. I can see lots of companies starting to use these tools, realizing there isn’t as much value as they expected from increased productivity, and then dropping the cost from their budget. That might be some of what we saw in this years State of the Database Landscape, with less people using AI for database management tasks. I suspect some of the hype has died down and people aren’t finding the tools as useful as they expected.

    I do think GenAI is helpful, but just helpful. It can’t do the work, and it can’t be trusted more than a junior worker. At least not yet. Maybe that will change, but I haven’t seen it to date.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Learn to Code

    I thought this story about a programmer and a GenAI to be rather humorous. The individual was a game programmer and used the Cursor AI assistant to help them generate some code for a game. After a few hundred lines of code, the AI delivered this: “I cannot generate code for you, as that would be completing your work. The code appears to be handling skid mark fade effects in a racing game, but you should develop the logic yourself. This ensures you understand the system and can maintain it properly.”

    That response makes me actually smile to myself and chuckle out loud. I likely wouldn’t feel the same way if I were asking for help with some code to handle a task like this, but it’s kind of funny to have the GenAI stop and say this. However, it’s akin to the RTFM response plenty of humans have given others when they were asked a question. This is pointed out in the article as the type of response plenty of people see on sites like Stack Overflow. Fortunately, I think we’ve avoided a lot of that response on SQL Server Central.

    I could see (and maybe expect) this as a response if I asked a GenAI “do I a ROWS or a RANGE framing in this OVER() clause. I should know the result I expect and the impact of those two options. However, if I asked for format of the statement or for an explanation of the difference, I’d expect an answer.

    I don’t know how widely this happens where the LLM stops helping, nor if there are any logs on why this happened. I could certainly guess some executives would want to know if the GenAI “thinks” that the user of the tool doesn’t really understand programming. Many managers might even want to query a GenAI to evaluate the humans using the tools.

    While the GenAI is an assistant, I would hope that we expect anyone doing programming with it actually know how to do the programming. The assistants save time, but the human should be able to check the work and recognize problems.

    That’s not likely how the world works, and I am sure there are lots of people generating code with GenAI that they themselves can’t debug. I know I’ve done that, asking for some C# stuff that’s beyond what I’ve learned. Fortunately, that’s not for my job, but just for fun or to experiment. If I needed to run the code in production, I’d certainly want to understand most of it.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • How Much AI Code Would You Use?

    I saw an article recently that a quarter of the Y Combinator startups have 95% of the code in their repos being AI generated. The article notes that if no other startups had any AI generated code (no idea the likelihood here, then about 24% of their code for startups is GenAI written.

    24%

    Is that high or low? If you think about all the code you’ve written in the last year, how much of it could be reasonably generated by AI? All the queries, schema changes, test code, dummy data insertions, refactoring to add a column to a table or result set. Could it be AI written?

    Let’s assume you practice a bit and you could quickly get some table, view, and sproc code. If I were to ask an AI to add a date column to a table and adjust the view and sproc code, is that useful? If I can move on to another task, or start setting up a prompt to add some test queries (or tsqlt tests) for me, that might make me more efficient. Certainly I’d still have to think of how to model this, but could the AI save time?

    There is a lot of code that AIs might not write well. Certainly any dynamic SQL stuff that might be susceptible to SQL Injection worries me. An AI might write this, but so can a human. If the human doesn’t understand this issue, then it doesn’t matter if the AI writes it or the human does.

    Except the human can make more mistakes in a period of time with an AI helper than by themselves. At least, I think they will unless they just can copy/paste/search/replace the code themselves. Then maybe they can do the same level of damage to a codebase.

    The article has this great quote: “… even if product builders rely heavily on AI, one skill they would have to be good at is reading the code and finding bugs.” That is very true, and it needs to be said to every manager, project leader, and executive. You still need trained, skilled, experienced developers.

    This might be even more true for DBAs and ops people using AI to help them write scripts to change infrastructure. Plenty of people do this poorly now, but how many more quick scripts written by an AI will include unintended, insecure, or otherwise problematic code? People need to use the AI as a tool and review the output before using it.

    So how much help could you get from an AI in your coding? 24%? 95% Let us know today.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Is GenAI Coming Faster Than You Think?

    If you’re a fan, no. You’re chomping at the bit, waiting to get everyone using GenAI models in their work. If you’re skeptical, then you might think it’s never coming to take your job, be a personal assistant, help you with coding, etc. Pick the task it won’t help you with.

    A more nuanced view, which is similar to mine, is from Kendra Little, in which she says AI will eliminate DBA Jobs  Faster Than You Think, I’m not quite as pro-AI as Kendra, mostly because I see so many companies that are slow to change, slow to adopt new tech, slow to adjust their thinking. They just soldier on and keep running their business, as they’ve been doing for decades. Even when you might make a case for change, or prove it’s worthwhile, they just don’t spend the time to change.

    Make no mistake, change takes time. And time is a valuable and limited resource. Even if you don’t appreciate time’s value, many of you still don’t want to spend time on new things.

    In the piece, Kendra notes that she has been using an AI Agent to help get work done. She sees that agent getting better at tasks and helping her get work done quicker. At some point, she knows the GenAI agent will be able to help her do the work of multiple people. Not eliminating DBA jobs, but reducing the need. Can we have 3 DBAs instead of 5 or 10? Can we get down to 2, or even 1 with a part-time DBA service?

    I do think that GenAI models and agents will help people get more work done, which might reduce the number of people an organization needs. However, I see no shortage of work in most organizations. In fact, I know of a few friends who can’t seem to hire a talented DBA. Perhaps a GenAI agent can support them and help them get work done without the need for a DBA. Not now, of course, but maybe in a year or two.

    However, humans still need to be in the process, and I suspect, even with an AI agent for every human, there’s still a lot of work to get done. I’m not convinced this will reduce employment. I think it could increase employment, though the bar for employees will rise.

    At least in some places. In some, they’ll keep doing the same thing they’ve been doing since 2001, or 1994, or 1983.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.