Voice of the DBA

Tag: AI

  • Who is Irresponsible?

    There was a post on X recently from a founder in the EU about an engineer using Claude and ChatGPT to build a feature. I am not sure how true these posts are or if they are designed to just create engagement, but it’s still an interesting topic. The part that makes me think is that (supposedly) the engineer was fired because their “data” (code) was sent to American servers. The code was then deleted and the feature will be built without AI.

    First, read some of the responses before you form an opinion. There are some funny ones in there. There are a few I think are overblown and silly, and I skim past them. Someone is always more upset than I am, and more than I think they rationally should be, so I tend to let their outrage flow by me.

    There are two interesting things here. First, the debate about sending data to America. There certainly is some cause for concern here if data is being sent to a place outside of the EU where GDPR rules might apply. There possibly could be some legal issue here, though I doubt some of the responses about all code being compromised are an issue here. I don’t think code is PII, though if it were re-used or appears in AI output, perhaps investors could sue this company.

    The second thing here is whether someone should be fired for doing this. There might be a policy and some training about not doing this, and in that case, perhaps the person should be fired. However, I find this kind of thing happening too often, and it’s the type of thing that has happened before AI where people used outside sources (SQL Server Central, Stack Overflow, etc.) to post code in a question and get an answer. And then often use that code without changing or testing it.

    Is this rational? Some people might say yes, some no, many unsure. In the past, before AI, what would you think? To me, sometimes there have been solutions engineers have found but couldn’t use code written by someone else. There are real IP/copyright concerns here. You could rebuild the solution, rewriting the code, which in some sense is what Google did with Java APIs and successfully defended that effort. If another human or an AI gives you code, can you rewrite that code, keeping the same idea for the solution?

    I think that in most cases this is acceptable. I use AI for a lot of things and I throw away a lot of AI output, but it often gets me started down a path, whether in writing, coding, or something else.

    Who was more irresponsible here, the founder or the engineer? I think the former.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Barely Reviewed Code

    Years ago I was giving a talk on software development and asked the audience how long it takes to review a PR that has 10 lines changed. Answers were in the minutes to tens of minutes range. I then asked how long it takes to review a PR that has 1,000 lines changed. Some people said hours, but a few people said seconds.

    I’ve often taken the latter, pessimistic view. Not because I don’t think engineers want to do a good job, but because I know human behavior. Most humans will get bored, lose focus, and end up skimming through a large amount of code. Many (most?) people don’t want to spend all that time, after all they have they their own code to write. They’ll just approve the PR and assume testing will catch any major issues.

    Even if a reviewer wants to do a great job, they likely will still miss things. It’s very hard to focus across that much code.

    This is a funny visual (from X) about code reviews. It’s titled “me reviewing code written by Claude before pushing it to production.” Plenty of people are probably laughing, or thinking this is a good reason to not use an AI to write code.

    However, I don’t think the problem is an AI writing code. If you trust the AI without reviewing things, that’s on you. You deserve blame if things fall apart.

    The bigger problem is that an AI can write code so quickly and can make so many changes that PRs will tend to be large. These changes will tend to not get human-reviewed with any level of focus or quality control. The problem is volume, not who wrote the code (or the quality). Certainly quality matters, but it’s easy to catch changes if you have a small volume of code. Harder if you have a lot.

    The more I use AI for spot work, to handle tedious things, to do something like subtly adjust spacing in a UI or focus on adjusting a few things in a data model, the easier it is to judge the focus and quality of the code. Is the change doing the job I need done, and is it doing it well?

    Code quality is a problem we’ve had ever since we started writing code. AI can make the problem worse, not because of poor coding, but because it will write so much code in a PR that you can’t review it appropriately.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Prompt Requests

    One of the challenges of AI-assisted coding agents is that they tend to produce A LOT of code. Even in refactoring or migration changes, the AIs can work quickly and generate such a volume of code that the process starts to become overwhelming. For pull requests, for CI/CD build systems, and certainly for human reviewers, they can be overwhelmed. This can become a real problem with OSS projects, where submissions can grow exponentially to the point that maintainers stop looking at pull requests. I suspect the same thing might happen in corporate repositories when lots of developers can refactor or submit huge amounts of code produced by AI agents in a fraction of the time it took a year ago.

    I was listening to an interview with an experienced software developer and OSS project maintainer who said that he preferred getting a “prompt request” that contained a description of a problem and the specification for a solution that he could submit to his own LLM to get the code. Rather than use an AI to review a code in a PR written by a human or AI agent, a great prompt that can communicates the problem and solution is preferred.

    That’s a fascinating idea to me. Specify what you want and let the code owners send it to an AI and get a code response whose quality and focus they can decide to implement, based on their own context provided to an LLM (standards, style, patterns, etc.)

    Of course, for open source projects, perhaps the maintainer doesn’t want to spend time managing AI agents or working through quality, but this does allow them to focus on the idea being suggested rather than attempting to review code, test it, judge the quality, and perhaps request changes from the submitter. They can take your idea and implement it. If it’s a simple fix, even better, as the maintainer might get quick help from an AI, using the style of code they are used to (their own).

    Software engineering is changing a lot in the age of AI, and this seems to be one of the more interesting things I’ve seen suggested. Not YOLO or vibe-coding, but rather a prompt that suggests the idea and turns contributors into a specification written for the robot coder.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Everything is the right question away

    When I was young and needed to learn about something, I had to go to a library or a bookstore to get information. I often started by looking through an encyclopedia. I had to wander between entries to learn more about the topic I was researching. A few lucky friends had their own copy of an encyclopedia, which was nice since we could research at home. At some point these collections of information were moved to CD/DVDs, which made them more portable and more accessible to a larger group of people than in the past.

    In the 90s we had the innovation of search engines, which allows us to more quickly move through information. There was more information available on the Internet than was ever published in encyclopedias. Over the years, these search engines improved their ability to rank and recommend information that is relevant to your query. However, you still need some idea of what you are trying to learn about. You have to direct the searches, although the Google auto-complete felt very predictive at times.

    However, I heard a quote recently that is the title of this piece: everything is the right question away.

    That might seem like something a search engine or even an encyclopedia would help with, but consider the fact that with an AI LLM you don’t have to specify much at all to get started. You can even ask it the question of how do I do/learn/find something and get a result that seems better than any computer system in the past. It might be better than what you get from most humans as well.

    Of course, you might not get the answer you want or need, though you can continue to ask the LLM and refine what you need. What’s even better is that once you get a good answer, you can shortcut the route to that knowledge by asking the LLM to provide you with a better prompt to get you to the place you end up in faster.

    Asking the right question to get an answer is an age-old human problem. Philosophers and religious figures have debated and hinted at this for centuries. You still need to build strong communication skills to ask a clear question and some expertise to judge the results. AI LLMs, however, make this a much easier and quicker process than at any time previously in human history.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.