Tag: AI

  • Using AI to Test a SQL Query

    Someone sent me this code.

    WITH p AS
     ( SELECT ID, ROW_NUMBER() OVER (ORDER BY ID ASC) AS RN
     FROM wp_posts
     WHERE post_parent = 94341; )
     UPDATE p SET menu_order = RN

    That’s not great code. In fact, it won’t run. I know that, but does an AI? Let’s see how Copilot does.

    This is part of a series of experiments with AI systems.

    The First Test

    I entered this prompt, but accidentally hit Enter. When I then pasted in the code and hit enter, I got this:

    2025-05_line0012

    That is just about what I would have sent the author to let them know this code doesn’t work. It’s good to see a GenAI LLM can recognize this.

    I then pasted in the entire prompt which was: “give me a table with the schema to match this code: WITH p AS ( SELECT ID, ROW_NUMBER() OVER (ORDER BY ID ASC) AS RN FROM wp_posts WHERE post_parent = 94341; ) UPDATE p SET menu_order = RN”

    The Gen AI response was polite, with this answer:

    Certainly! Based on your SQL code, it looks like you need a table schema that matches the structure of wp_posts, particularly including ID, post_parent, and menu_order. Here’s a possible schema:

    I then had this code, with a copy button that I could copy/paste into SSMS.

    CREATE TABLE wp_posts (
         ID INT PRIMARY KEY,      -- Unique identifier for each post
         post_parent INT,         -- Parent post ID (for hierarchical structures)
         menu_order INT           -- Order value for arranging posts
    );

    That’s what I would have done to mock this up for testing. You might guess the next step: data.

    Here’s my prompt: create some test data for the table with Ids that increment but are not necessarily sequential and post_parent values that match the ID values

    That got my this response.

    2025-05_line0016

    That’s not bad, but it’s a limited set of data. Let’s ask for more.

    2025-05_line0031

    Below this the GenAI LLM gave me a summary that explains what it did.

    2025-05_line0032

    Next, I wanted a test. I used this prompt: write a tsqlt test to verify that the update statement works to reset the menu_order to sequential values for a particular post_parent

    I got a nice response and the code I’m showing below the image.

    2025-05_line0033

    Here is the code:

    -- Step 1: Create a test class
    EXEC tSQLt.NewTestClass 'TestMenuOrderUpdate';

    GO

     

    — Step 2: Define the test
    CREATE PROCEDURE TestMenuOrderUpdate.[test_UpdateMenuOrder_Sequential]
    AS
    BEGIN
    — Arrange: Fake the wp_posts table
    EXEC tSQLt.FakeTable ‘dbo.wp_posts’;

     

    INSERT INTO wp_posts (ID, post_parent, menu_order)
    VALUES
    (101, 94341, 5), (102, 94341, 3), (103, 94341, 8),
    (104, 94341, 1), (105, 94341, 7);

     

    — Act: Run the update logic
    WITH p AS (
    SELECT ID, ROW_NUMBER() OVER (ORDER BY ID ASC) AS RN
    FROM wp_posts
    WHERE post_parent = 94341
    )
    UPDATE wp_posts
    SET menu_order = p.RN
    FROM p
    WHERE wp_posts.ID = p.ID;

     

    — Assert: Check if menu_order is sequential
    EXEC tSQLt.AssertEqualsTable ‘ExpectedResults’, ‘wp_posts’;

     

    END;
    GO

     

    — Step 3: Create expected results table for assertion
    CREATE TABLE ExpectedResults (
    ID INT PRIMARY KEY,
    post_parent INT,
    menu_order INT
    );

     

    INSERT INTO ExpectedResults (ID, post_parent, menu_order)
    VALUES
    (101, 94341, 1), (102, 94341, 2), (103, 94341, 3),
    (104, 94341, 4), (105, 94341, 5);

     

    — Run the test
    EXEC tSQLt.Run ‘TestMenuOrderUpdate’;

    
    

    This is where I stopped. This code runs, though it’s brittle in that the Expected Results table is a real table and not a temp inside the test. I can easily edit that and likely quicker than the GenAI can. It’s copy/pasting some code for me from one spot to another. I also need to add a CREATE OR ALTER to the proc.

    Summary

    A quick use of an AI that easily saved me 5-10 minutes to mock of a scenario and show someone what they were doing wrong and also testing their code. I mock up things regularly to test them, and often do something like this to answer questions. This is a good place where AI can save some time, and might be worth using. I’m also glad to see it recognize and write some decent SQL code for me, especially the insert statement for test data.

  • Checking Myself with GenAI

    I had a suggestion from somone on a place where AI helps them and I decided to try it. The person had an AI summarize their work and if the result wasn’t the intention of the author, then they know know their writing wasn’t clear.

    This post looks at how that worked for me.

    This is part of a series of experiments with AI systems.

    Checking an Editorial

    I wrote an editorial on database Devops that was published recently. I decided to have a few AIs summarize this and see if the result was what I intended. First up, Claude, with a simple prompt: summarize this text (insert here). See the top of the prompt here:

    2025-05_line0025

    The result is shown below and seems to be an accurate summary of the text. This is basically what I was trying to say in the piece. Of course, this isn’t much shorter than the text, but this gives me confidence here in the ability to recognize what I’ve written.

    2025-05_line0027

    In Perplexity, I got this result. This result is similar, but doesn’t mention the author. Instead, this is a summary of the text, not trying to give a voice to the author, which is interesting. Very close to the text above, but this seems slightly drier, taking the text as fact rather than opinion.

    2025-05_line0026

    Perplexity also had some related items at the bottom, which injected a prompt back into the LLM for more info.

    2025-05_line0029

    Last, Copilot. I have a dedicated key on my laptop for this and I pressed it and entered my prompt in the app. This result is shorter and to the point. There are some additional links to click that the bottom.

    2025-05_line0028

    Clicking on one of the items at the bottom injects the text and gets a new result.

    2025-05_line0030

    I tried this with a couple other piece of work, some of which aren’t published. In each case (3 attempts), the summary made sense. I don’t know if that means I am writing anything clearly, but it does help me get a sense of what I’ve written.

  • The AI View From Above

    It likely isn’t a surprise to many of you that executives like AI. A survey shows that 74% of executives surveyed have greater confidence in AI-generated insights than advice from colleagues or friends. At the board level, even more (85%) favor AI-driven advice.

    That’s amazing to me, and while I might think this is a bit too much trust being placed in these GenAI LLMs, perhaps it’s also partially because they work with too many people who aren’t great at their jobs. Plenty of people skim through data or focus on certain things and might miss the details. While an AI can read and summarize a lot, it might not have the context we expect. I tend to be a bit skeptical of AI summaries, often because they don’t necessarily weigh the different parts of an article the same way that I do. However, they can be helpful.

    Even more interesting, 44% of executives say they would trust a GenAI to override their decisions based on insights, and 38% would trust AI to make decisions on their behalf. Business decisions based on data, or conclusions from a lot of inputs are different than producing working code, so I don’t know how accurate these models might be in this context. I do know that I want experienced people reviewing and judging GenAI outputs, and I would not allow an AI to override me without my input.

    However, I wouldn’t just discount a GenAI recommendation. I tend to have strong opinions, but loosely held. I’ll change if there is evidence or a good argument to do so. It’s possible a GenAI might see things I miss and produce an insight that gets me to change a decision.

    What’s a bit scary about the stats from this survey is that many executives see a skills gap in their staff, and their trust in GenAI might lead them to replace or augment existing staff with more GenAI tools. They might expect hiring can be delayed or slowed (or eliminated) with GenAI filling gaps. This might be especially true as many tech companies talk about how GenAI tools are making them more efficient.

    That means that tech professionals should consider a few things. First, learn to work with GenAI tools and use them to prove your value to an organization. This includes learning when not to use them. Second, continue to improve your skills to ensure you can judge Gen AI results and emphasize that you are still the expert. Lastly, as the technology improves, consider adding some skills in how to train an AI to be a better assistant for you. The more efficient you are, especially with a GenAI helper, the more likely you are to impress executives and managers that are choosing which staff to keep.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • How Helpful is a GenAI Copilot

    I’m sure many of you have tried a GenAI LLM to do something. Maybe write some code, maybe get some sort of recommendation or suggestion, maybe to rewrite something or summarize text. I’m sure you have had some feelings about whether the tool made you more or less productive.

    There was a trial conducted by the Australia Department of the Treasury on Microsoft’s 365 Copilot, asking for volunteers to participate and use the tool in their daily work. They used it and then completed a survey, which are summarized in this piece. Only 218 people went through the trial, and the results are interesting.

    The headline says that the staff rated the GenAI less useful than expected. Those last two words are interesting because your expectations shape a lot of how you view anything in the world. If you expect little and get a little more, you might be happy. If you expect a lot and don’t get it, you might be very disappointed.

    The sub-headline and the first sentence note that there still is an ROI from the tool. It isn’t as helpful and isn’t as widely applicable as people expected, but they chalk some of this up to product limitations and some to limited use by people. It was useful in summarizing things and drafting content, which are what they call basic administrative tasks. That’s interesting and likely where GenAI tools can help quite a bit.

    Maybe the most interesting thing to me is that if Copilot saves 13 minutes a week for mid-level workers, it pays for itself. I don’t know how much time it would have to save me, but an hour or two a week might make me use it more. It certainly would be use a small monthly cost. So far, I haven’t committed to regular work with the tools, and I think I still spend more time learning and typing with GenAI tools than I’d like. I’m not sure if I am saving time over just doing the task myself. Some of that is because I have habits that allow me to work quickly and to use a Copilot, my pace slows.

    This also brings up something I wonder about with these GenAI services. With the cost of compute services, there might not be a lot of margin for vendors to raise prices if people are only lightly more productive. I can see lots of companies starting to use these tools, realizing there isn’t as much value as they expected from increased productivity, and then dropping the cost from their budget. That might be some of what we saw in this years State of the Database Landscape, with less people using AI for database management tasks. I suspect some of the hype has died down and people aren’t finding the tools as useful as they expected.

    I do think GenAI is helpful, but just helpful. It can’t do the work, and it can’t be trusted more than a junior worker. At least not yet. Maybe that will change, but I haven’t seen it to date.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.