Voice of the DBA

Tag: software development

  • Barely Reviewed Code

    Years ago I was giving a talk on software development and asked the audience how long it takes to review a PR that has 10 lines changed. Answers were in the minutes to tens of minutes range. I then asked how long it takes to review a PR that has 1,000 lines changed. Some people said hours, but a few people said seconds.

    I’ve often taken the latter, pessimistic view. Not because I don’t think engineers want to do a good job, but because I know human behavior. Most humans will get bored, lose focus, and end up skimming through a large amount of code. Many (most?) people don’t want to spend all that time, after all they have they their own code to write. They’ll just approve the PR and assume testing will catch any major issues.

    Even if a reviewer wants to do a great job, they likely will still miss things. It’s very hard to focus across that much code.

    This is a funny visual (from X) about code reviews. It’s titled “me reviewing code written by Claude before pushing it to production.” Plenty of people are probably laughing, or thinking this is a good reason to not use an AI to write code.

    However, I don’t think the problem is an AI writing code. If you trust the AI without reviewing things, that’s on you. You deserve blame if things fall apart.

    The bigger problem is that an AI can write code so quickly and can make so many changes that PRs will tend to be large. These changes will tend to not get human-reviewed with any level of focus or quality control. The problem is volume, not who wrote the code (or the quality). Certainly quality matters, but it’s easy to catch changes if you have a small volume of code. Harder if you have a lot.

    The more I use AI for spot work, to handle tedious things, to do something like subtly adjust spacing in a UI or focus on adjusting a few things in a data model, the easier it is to judge the focus and quality of the code. Is the change doing the job I need done, and is it doing it well?

    Code quality is a problem we’ve had ever since we started writing code. AI can make the problem worse, not because of poor coding, but because it will write so much code in a PR that you can’t review it appropriately.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Prompt Requests

    One of the challenges of AI-assisted coding agents is that they tend to produce A LOT of code. Even in refactoring or migration changes, the AIs can work quickly and generate such a volume of code that the process starts to become overwhelming. For pull requests, for CI/CD build systems, and certainly for human reviewers, they can be overwhelmed. This can become a real problem with OSS projects, where submissions can grow exponentially to the point that maintainers stop looking at pull requests. I suspect the same thing might happen in corporate repositories when lots of developers can refactor or submit huge amounts of code produced by AI agents in a fraction of the time it took a year ago.

    I was listening to an interview with an experienced software developer and OSS project maintainer who said that he preferred getting a “prompt request” that contained a description of a problem and the specification for a solution that he could submit to his own LLM to get the code. Rather than use an AI to review a code in a PR written by a human or AI agent, a great prompt that can communicates the problem and solution is preferred.

    That’s a fascinating idea to me. Specify what you want and let the code owners send it to an AI and get a code response whose quality and focus they can decide to implement, based on their own context provided to an LLM (standards, style, patterns, etc.)

    Of course, for open source projects, perhaps the maintainer doesn’t want to spend time managing AI agents or working through quality, but this does allow them to focus on the idea being suggested rather than attempting to review code, test it, judge the quality, and perhaps request changes from the submitter. They can take your idea and implement it. If it’s a simple fix, even better, as the maintainer might get quick help from an AI, using the style of code they are used to (their own).

    Software engineering is changing a lot in the age of AI, and this seems to be one of the more interesting things I’ve seen suggested. Not YOLO or vibe-coding, but rather a prompt that suggests the idea and turns contributors into a specification written for the robot coder.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Testing is Becoming More Important

    Many of us know that testing our code is important. The adoption of unit testing by many software application developers as a normal course of business has dramatically improved the quality of applications. Mobile software, especially, has benefited from the requirement for most software to include, and constantly run, a suite of unit tests.

    For database software, I find relatively few organizations formally test their database code. A few people have adopted tSQLt or the Microsoft Unit Testing Framework, but most don’t bother. In fact, many queries that are embedded in application code, or built by ORMs, aren’t tested beyond a developer looking at the results from their own (limited set of) test data. That often doesn’t catch errors until someone in production runs their application against a larger set of data.

    What might be worse is that refactoring those queries might produce different results that aren’t tested against regressions.

    In this new age of AI-assisted coding, testing is becoming more important. Grant wrote an interesting post on LinkedIn that discusses your job changing in the age of AI. You need to have more testing that ensures you validate code that the AI produces, which is going to be more important as the amount of code grows. AI will produce lots more code, and potentially, lots more poor code. We will need to ensure that the generated code  has some validation that the results are what we expect.

    Unit tests help here, and while I know these can be tedious to write and maintain, this is a great use for AI assistance. Generating unit tests, with default data based on data in current tables, is something AI agents can do well. They can also use these to verify functionality as code is generated and refactored. Of course, humans still need to be in the loop as there are plenty of reports where AI Agents write tests that return success without actually testing code. This is something humans have done as well.

    You need to validate the tests, and ensure your AI uses those tests to validate its work. Those tests can also be used by humans if they write code.

    AI is an amazing tool, but like an intelligent, over-eager, junior developer, it needs clear communication and strong guidance.

    And a little review of its work.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Deep Learning and Craftsmanship Matter

    There’s concern about the future of AI and how it may affect jobs and employment for the masses. I see plenty of people on both sides of the issue. Some are sure AI technologies won’t replace people; some are concerned their jobs will be eliminated, and some are hoping that we will eliminate some jobs and create many more.

    Sometimes that’s the same person.

    A GenAI can replicate a human, or maybe more accurately, mimic one. That might work well enough for some people to trust the technology more than humans. Or maybe it works well enough, enough of the time, and lots of us are OK with well-enough. After all, I think a lot of us already work with “well-enough” or “good-enough” code on a daily basis.

    However, the GenAI is based on what other humans have already done. It’s “trained” on lots of existing ideas, decisions, codebases, etc. It can recall and use those quicker, and often as well as many humans. It might be a light craftsman, but it can’t be a great one.

    Humans will be able to deeply understand problems and create better craftsmanship for many systems. Across time, an AI can learn from these craftsman and repeat their work in other systems, but an AI will often struggle to understand the entire context of whether we would apply that solution or a slightly different one this time.

    That’s the human advantage. Deep learning and craftsmanship will differentiate us from the AIs because we can contextualize things better than an AI. Or really, we can internalize the context better than another human can express it to the AI. That will be the difficult part of working with AI LLMs, agents, and whatever comes next: explaining what is really needed in a new situation.

    Communication is hard. Because many humans aren’t good at communicating, they won’t be able to use an AI to replace other humans. They’ll struggle with the results, and they will need to hire a craftsman to help. However, that also implies that more of us need to become craftsmen, not only for the advantage it gives us over AI, but because those skills will help us better judge AI output, as well as express what we want to see the AI do.

    There will be lots of work in the future, even with AI, but I also believe that the jobs that are desired, that will pay better, will go to those who learn to use AI tech and who can judge when the quality of the work is appropriate for the situation.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.