Voice of the DBA

Tag: AI

  • Checking Myself with GenAI

    I had a suggestion from somone on a place where AI helps them and I decided to try it. The person had an AI summarize their work and if the result wasn’t the intention of the author, then they know know their writing wasn’t clear.

    This post looks at how that worked for me.

    This is part of a series of experiments with AI systems.

    Checking an Editorial

    I wrote an editorial on database Devops that was published recently. I decided to have a few AIs summarize this and see if the result was what I intended. First up, Claude, with a simple prompt: summarize this text (insert here). See the top of the prompt here:

    2025-05_line0025

    The result is shown below and seems to be an accurate summary of the text. This is basically what I was trying to say in the piece. Of course, this isn’t much shorter than the text, but this gives me confidence here in the ability to recognize what I’ve written.

    2025-05_line0027

    In Perplexity, I got this result. This result is similar, but doesn’t mention the author. Instead, this is a summary of the text, not trying to give a voice to the author, which is interesting. Very close to the text above, but this seems slightly drier, taking the text as fact rather than opinion.

    2025-05_line0026

    Perplexity also had some related items at the bottom, which injected a prompt back into the LLM for more info.

    2025-05_line0029

    Last, Copilot. I have a dedicated key on my laptop for this and I pressed it and entered my prompt in the app. This result is shorter and to the point. There are some additional links to click that the bottom.

    2025-05_line0028

    Clicking on one of the items at the bottom injects the text and gets a new result.

    2025-05_line0030

    I tried this with a couple other piece of work, some of which aren’t published. In each case (3 attempts), the summary made sense. I don’t know if that means I am writing anything clearly, but it does help me get a sense of what I’ve written.

  • The AI View From Above

    It likely isn’t a surprise to many of you that executives like AI. A survey shows that 74% of executives surveyed have greater confidence in AI-generated insights than advice from colleagues or friends. At the board level, even more (85%) favor AI-driven advice.

    That’s amazing to me, and while I might think this is a bit too much trust being placed in these GenAI LLMs, perhaps it’s also partially because they work with too many people who aren’t great at their jobs. Plenty of people skim through data or focus on certain things and might miss the details. While an AI can read and summarize a lot, it might not have the context we expect. I tend to be a bit skeptical of AI summaries, often because they don’t necessarily weigh the different parts of an article the same way that I do. However, they can be helpful.

    Even more interesting, 44% of executives say they would trust a GenAI to override their decisions based on insights, and 38% would trust AI to make decisions on their behalf. Business decisions based on data, or conclusions from a lot of inputs are different than producing working code, so I don’t know how accurate these models might be in this context. I do know that I want experienced people reviewing and judging GenAI outputs, and I would not allow an AI to override me without my input.

    However, I wouldn’t just discount a GenAI recommendation. I tend to have strong opinions, but loosely held. I’ll change if there is evidence or a good argument to do so. It’s possible a GenAI might see things I miss and produce an insight that gets me to change a decision.

    What’s a bit scary about the stats from this survey is that many executives see a skills gap in their staff, and their trust in GenAI might lead them to replace or augment existing staff with more GenAI tools. They might expect hiring can be delayed or slowed (or eliminated) with GenAI filling gaps. This might be especially true as many tech companies talk about how GenAI tools are making them more efficient.

    That means that tech professionals should consider a few things. First, learn to work with GenAI tools and use them to prove your value to an organization. This includes learning when not to use them. Second, continue to improve your skills to ensure you can judge Gen AI results and emphasize that you are still the expert. Lastly, as the technology improves, consider adding some skills in how to train an AI to be a better assistant for you. The more efficient you are, especially with a GenAI helper, the more likely you are to impress executives and managers that are choosing which staff to keep.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • How Helpful is a GenAI Copilot

    I’m sure many of you have tried a GenAI LLM to do something. Maybe write some code, maybe get some sort of recommendation or suggestion, maybe to rewrite something or summarize text. I’m sure you have had some feelings about whether the tool made you more or less productive.

    There was a trial conducted by the Australia Department of the Treasury on Microsoft’s 365 Copilot, asking for volunteers to participate and use the tool in their daily work. They used it and then completed a survey, which are summarized in this piece. Only 218 people went through the trial, and the results are interesting.

    The headline says that the staff rated the GenAI less useful than expected. Those last two words are interesting because your expectations shape a lot of how you view anything in the world. If you expect little and get a little more, you might be happy. If you expect a lot and don’t get it, you might be very disappointed.

    The sub-headline and the first sentence note that there still is an ROI from the tool. It isn’t as helpful and isn’t as widely applicable as people expected, but they chalk some of this up to product limitations and some to limited use by people. It was useful in summarizing things and drafting content, which are what they call basic administrative tasks. That’s interesting and likely where GenAI tools can help quite a bit.

    Maybe the most interesting thing to me is that if Copilot saves 13 minutes a week for mid-level workers, it pays for itself. I don’t know how much time it would have to save me, but an hour or two a week might make me use it more. It certainly would be use a small monthly cost. So far, I haven’t committed to regular work with the tools, and I think I still spend more time learning and typing with GenAI tools than I’d like. I’m not sure if I am saving time over just doing the task myself. Some of that is because I have habits that allow me to work quickly and to use a Copilot, my pace slows.

    This also brings up something I wonder about with these GenAI services. With the cost of compute services, there might not be a lot of margin for vendors to raise prices if people are only lightly more productive. I can see lots of companies starting to use these tools, realizing there isn’t as much value as they expected from increased productivity, and then dropping the cost from their budget. That might be some of what we saw in this years State of the Database Landscape, with less people using AI for database management tasks. I suspect some of the hype has died down and people aren’t finding the tools as useful as they expected.

    I do think GenAI is helpful, but just helpful. It can’t do the work, and it can’t be trusted more than a junior worker. At least not yet. Maybe that will change, but I haven’t seen it to date.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Learn to Code

    I thought this story about a programmer and a GenAI to be rather humorous. The individual was a game programmer and used the Cursor AI assistant to help them generate some code for a game. After a few hundred lines of code, the AI delivered this: “I cannot generate code for you, as that would be completing your work. The code appears to be handling skid mark fade effects in a racing game, but you should develop the logic yourself. This ensures you understand the system and can maintain it properly.”

    That response makes me actually smile to myself and chuckle out loud. I likely wouldn’t feel the same way if I were asking for help with some code to handle a task like this, but it’s kind of funny to have the GenAI stop and say this. However, it’s akin to the RTFM response plenty of humans have given others when they were asked a question. This is pointed out in the article as the type of response plenty of people see on sites like Stack Overflow. Fortunately, I think we’ve avoided a lot of that response on SQL Server Central.

    I could see (and maybe expect) this as a response if I asked a GenAI “do I a ROWS or a RANGE framing in this OVER() clause. I should know the result I expect and the impact of those two options. However, if I asked for format of the statement or for an explanation of the difference, I’d expect an answer.

    I don’t know how widely this happens where the LLM stops helping, nor if there are any logs on why this happened. I could certainly guess some executives would want to know if the GenAI “thinks” that the user of the tool doesn’t really understand programming. Many managers might even want to query a GenAI to evaluate the humans using the tools.

    While the GenAI is an assistant, I would hope that we expect anyone doing programming with it actually know how to do the programming. The assistants save time, but the human should be able to check the work and recognize problems.

    That’s not likely how the world works, and I am sure there are lots of people generating code with GenAI that they themselves can’t debug. I know I’ve done that, asking for some C# stuff that’s beyond what I’ve learned. Fortunately, that’s not for my job, but just for fun or to experiment. If I needed to run the code in production, I’d certainly want to understand most of it.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.