Voice of the DBA

Category: Editorial

  • There Are a Lot of Databases

    I was reading Andy Pavlo’s end-of-year review of the database world. He’s done this for a number of years, and there are links to previous recaps in the piece. He is an associate computer science professor at Carnegie Mellon University, working on quite a few database-related projects. In the review, he tends to track the database world from the perspective of business success and money. There are certainly parts of it that discuss technical changes, but my overall impression is more about the business and usage success than it is about the way database systems work.

    The main thing that struck me after reading the review was how many database systems there are in the world. I hadn’t heard of any of these: RaptorDB, TigerData, Tembo, StormDB, Translattice, FerretDB, DocDB, SpiralDB, Tantivy, SkySQL, HeavyDB, and more. I’m sure I missed listing some I didn’t recognize, and quite a few of these are PostgreSQL-based systems, but still, that’s a lot of database systems that exist and are having success.

    Last year, I ran into someone who worked at a company that had implemented ArangoDB for the software their company sold. This system had something to do with tracking parts and managing schematics for machines, which is a great place to use a graph database. I asked them why they didn’t pick a more well-known and used graph database like Neo4j. He answered that cost was a big reason, but if Arango failed to wrok, they felt could port their data over to another platform. He did mention that training new people was a challenge, which I believe is a good reason to stick with more mainstream systems. However, I understand that people placing bets on less well-known technologies is how the popularity of those platforms grows.

    As a side note, I keep confusing ArangoDB with AvacadoDB. Maybe because I like guacamole.

    If I look at DB-Engines, I see lots of platforms I recognize and a few I don’t, but overall this is a long list. Some you could argue aren’t really database platforms, but these are platforms people report they are using. There are 429 ranked, which is quite a few. I’m not sure there are that many different models of cars being produced in the US each year.

    Many of these are specialized platforms and might be suitable or even preferred in certain situations. I wonder if any of you reading this are running Hazelcast or Presto. Or anything else unusual. If you are, why? What’s better about one of these systems than the top 5-10 in any category?

    As I look around I realize there are so many databases available to choose from. Perhaps it’s just me, but I prefer choosing from a small list rather than a huge one. Do you feel the same way?

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • More Documentation is Needed

    AI is everywhere, and if you spend any amount of time looking for answers on the Internet to your coding challenges, you’ve likely encountered a lot of poor, average, good, bad, amazing, and just-helpful-enough AI content. For awhile, I was avoiding the AI summary from Google as the quality seemed slightly off, but lately it’s gotten good enough that I tend use it to decide which links to click on in the results. The summary helps me better understand the context Google sees in my search query.

    I ran across a post on coding documentation and how helpful these docs are in onboarding, code reviews, and more. The teams that worked smoothly together often had good docs that helped them function as a cohesive group. At least to some extent. Over time, teams start to depend on tools and lose some of that cohesiveness since they rely more on tools than docs. I agree with the piece that this is a part of the reason many teams don’t really function as teams over time.

    In the age of AI, this becomes more important. These AI agents are smart, but gullible and prone to making inconsistent decisions if you let them. In the piece, there’s a great quote: “When your codebase follows consistent patterns, AI assistants become force multipliers. When it doesn’t, they become chaos amplifiers.” Or as we data people know it, garbage in, garbage out.

    The lack of documentation means a lack of guidance for both humans and AI agents. It’s easy to say AI makes crazy decisions when we feed it our code, but humans do the same thing. I can’t even begin to count the number of weird decisions over structure and naming I’ve seen from other humans when I didn’t provide them guidance. It happens even when I give them standards, but at least then we can have a conversation about attention to detail if there are docs.

    I saw Brent’s predictions for AI database development in 2026, and part of the challenge in getting AI to be helpful is the lack of docs many of us have on schemas. I can’t tell you how often I’ve been asked if Redgate has tools that can doc a schema and decipher what data is being stored. Microsoft spent a billion+ on Purview, and its results in classification are a mixed bag. It’s a hard problem, and a lot of the problem is us. We don’t make good decisions about what to name columns or tables, we’re inconsistent, and we reuse columns as our requirements change, subtly altering the data being stored. Usually, this is an overloading of two types of similar, but different, data into one column. Sometimes it’s just storing whatever we want in a column (or allowing a user to do so).

    I’d like to think that the growth of AI will result in a little more attention being paid to documenting our data stores. I’d hope this results in at least using the extended properties or COMMENT capabilities of the different platforms. I think having better ER diagrams might be a second step, though certainly with some AI assistance to help keep things in sync as we evolve our schemas.

    Documentation is tedious work, and it’s not something humans are good at, or want to, update over time. However, if an AI agent were around to do the work and then let a human check the results, I suspect we might do a better job of keeping things up to date. To me, that’s another place where the AI revolution might benefit us all.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Deep Learning and Craftsmanship Matter

    There’s concern about the future of AI and how it may affect jobs and employment for the masses. I see plenty of people on both sides of the issue. Some are sure AI technologies won’t replace people; some are concerned their jobs will be eliminated, and some are hoping that we will eliminate some jobs and create many more.

    Sometimes that’s the same person.

    A GenAI can replicate a human, or maybe more accurately, mimic one. That might work well enough for some people to trust the technology more than humans. Or maybe it works well enough, enough of the time, and lots of us are OK with well-enough. After all, I think a lot of us already work with “well-enough” or “good-enough” code on a daily basis.

    However, the GenAI is based on what other humans have already done. It’s “trained” on lots of existing ideas, decisions, codebases, etc. It can recall and use those quicker, and often as well as many humans. It might be a light craftsman, but it can’t be a great one.

    Humans will be able to deeply understand problems and create better craftsmanship for many systems. Across time, an AI can learn from these craftsman and repeat their work in other systems, but an AI will often struggle to understand the entire context of whether we would apply that solution or a slightly different one this time.

    That’s the human advantage. Deep learning and craftsmanship will differentiate us from the AIs because we can contextualize things better than an AI. Or really, we can internalize the context better than another human can express it to the AI. That will be the difficult part of working with AI LLMs, agents, and whatever comes next: explaining what is really needed in a new situation.

    Communication is hard. Because many humans aren’t good at communicating, they won’t be able to use an AI to replace other humans. They’ll struggle with the results, and they will need to hire a craftsman to help. However, that also implies that more of us need to become craftsmen, not only for the advantage it gives us over AI, but because those skills will help us better judge AI output, as well as express what we want to see the AI do.

    There will be lots of work in the future, even with AI, but I also believe that the jobs that are desired, that will pay better, will go to those who learn to use AI tech and who can judge when the quality of the work is appropriate for the situation.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Republish: Chopping Off Data

    It’s our first big volleyball tournament of the season. Today, Saturday, and Sunday I’ll be downtown in Denver coaching a team of 13 year olds.

    My break is your break (from me). Re-read Chopping Off Data today.