Voice of the DBA

Category: Editorial

  • Using Feature Flags

    The use of feature flags in software development has become more and more prevalent over time, especially as teams move to DevOps-style development with frequent releases. I’ve often thought that using feature flags allows technical people to separate out the deployment of some feature or change from the release of that to users. There are a number of articles on this style of work (feature flag driven development, Why Use Feature Flags?) as well as a discussion at Reddit.

    I am a big believer in feature flags helping with improving your software in many ways. These articles (and others) highlight the advantages that a software organization gains by using feature flags. Failed releases become less of an issue, as the specific change that doesn’t work can be turned off. This can even work with databases. I can deploy a database change and at a later time have the code (or new table/column) start being used when a feature flag is set. If there is an issue, I can turn off the feature flag and stop using the code (or populating the schema). I can then clean things up, even saving data before I make a change.

    I don’t love the idea of using feature flags to handle security access to features, which is pointed out in a few places. If this is for testing or evaluation by customers, perhaps. If this is to get access to data from a security standpoint, this is a bad idea. I hope most of you are savvy enough to realize this.

    Feature flags are not a panacea for preventing issues. They do clutter up code and make it harder to read. Once a feature is done and permanently enabled, the code for switching flags should be removed. It is also hard to stack versions of features up behind one flag, which can increase coding mistakes. Adding flags in stored procedures or functions also can wreak havoc on query optimizers, so I’d recommend you don’t do that. Instead handle feature enablement in the application code and use multiple procs/functions for the different functionality you might need.

    To use feature flags appropriately with database changes, you also need to be able to dark deploy those changes, with your application code able to handle additive changes to the database. A new column, a new table, or a new parameter should be easy to add without breaking the app code. This requires the use of defaults as well as good coding practices (no select *, inserts with column lists), but it can be done. Once you are in this place, life becomes a lot less stressful and feature flags work amazingly well.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • A Domain for Data

    A domain is a set of possible values (among other definitions). I use this word a lot in my work, often with a problem domain (the thing you’re trying to solve) or the domain of possible values (like the US States and Territories list). That last one is interesting, as this is often the set of data we stick in some reference or lookup table to use in a form on a screen.

    There is a domain as part of the SQL specification, which I never knew about. I was reading an article from Joe Celko on the CREATE DOMAIN statement. This doesn’t exist in SQL Server and T-SQL, but it has been a part of PostgreSQL for quite a few versions. The article talks about the definition of a domain from a few very experienced database design people.

    What’s interesting is that this could be a list of values, but it could be an expression against which values are checked. For example, we might choose the domain of positive integers, which might require that the value passes some x>=0 where x is the value. That seems OK, though this looks like a CHECK constraint to me, which we already have.

    The article is a little light and doesn’t give many examples of how or why a domain might be needed or why it might be better than a constraint. Perhaps there are good reasons, but I struggle to think of any. Certainly I am cautious of tightly binding which data is allowed, especially when I find many businesses have rules that often have exceptions.

    Would you use a domain for any data in your system? Do you think you have business rules that are firm and set enough to add domains? Perhaps you do, and if you do, that’s great. I suspect many of you are like me in that you are careful of where and how many check constraints you use. Those can be very hard to change and remove when an exception occurs, so the fewer the better.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Database DevOps Metrics

    The DORA organization is dedicated to helping others build software better and faster, at a higher quality, and in a way that is more efficient. They continue to compile and publish the Accelerate State of DevOps report every year, which is a fascinating read.

    As a part of the report, they have identified four key metrics that identify high performing organizations in terms of software. These are divided into two areas: throughput and stability. Throughput measures are change lead time and deployment frequency. Stability measures are the change fail percentage and failed deployment recovery time.

    For a long time, as I chatted with various people doing database work, it seemed that most people deployed relatively infrequently. They might deploy a couple times a week for software changes, but database changes were often less than once a week. There have always been people moving faster or slower, but that felt like the pace for a majority of people. These days, in the 2024-2025 timeframe, many people seem to be able to deploy database changes every week, often multiple times a week.

    Lots of people have moved to more throughput, with more frequent deployments and less change lead time. Most of us can’t get more work out of people, so if we deploy more often, their completed work gets released quicker. Those two metrics make some sense, and I think those are good measures, but not goals. What I find is that people often need to make changes quicker either to respond to changing needs of their organization or to fix bugs they’ve introduced. I wonder what the ratio is of the former to the latter? I suspect it might be less than one, if most of your deployments are fixing bugs. I don’t mind deploying software quicker, but the design, modeling, and testing can’t be shortened.

    The stability metrics are often high for most people I speak with about deployments. I don’t see a lot of failures at deployment time as code usually compiles and deploys. It’s often a day (or week) later that someone notices the code doesn’t do what they expect. Is that a deployment failure? I think not. What’s the MTTR if it’s fixed an hour after being reported a day after the deployment? Is the MTTR an hour or a day plus an hour? I don’t know how these metrics apply to databases, especially if data gets mangled and has to be corrected manually over hours/days/weeks. Is that in the MTTR? Can you even track it?

    Metrics are good ways to measure you progress or health, as long as the metric doesn’t become the goal. I’ve run into a lot of customers using these metrics to measure their development, and it does help most for a period of time. Whether this continues to help them improve often depends on whether they keep focusing on their goals of delivering better quality software faster, or they focus on the metrics.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Is GenAI Coming Faster Than You Think?

    If you’re a fan, no. You’re chomping at the bit, waiting to get everyone using GenAI models in their work. If you’re skeptical, then you might think it’s never coming to take your job, be a personal assistant, help you with coding, etc. Pick the task it won’t help you with.

    A more nuanced view, which is similar to mine, is from Kendra Little, in which she says AI will eliminate DBA Jobs  Faster Than You Think, I’m not quite as pro-AI as Kendra, mostly because I see so many companies that are slow to change, slow to adopt new tech, slow to adjust their thinking. They just soldier on and keep running their business, as they’ve been doing for decades. Even when you might make a case for change, or prove it’s worthwhile, they just don’t spend the time to change.

    Make no mistake, change takes time. And time is a valuable and limited resource. Even if you don’t appreciate time’s value, many of you still don’t want to spend time on new things.

    In the piece, Kendra notes that she has been using an AI Agent to help get work done. She sees that agent getting better at tasks and helping her get work done quicker. At some point, she knows the GenAI agent will be able to help her do the work of multiple people. Not eliminating DBA jobs, but reducing the need. Can we have 3 DBAs instead of 5 or 10? Can we get down to 2, or even 1 with a part-time DBA service?

    I do think that GenAI models and agents will help people get more work done, which might reduce the number of people an organization needs. However, I see no shortage of work in most organizations. In fact, I know of a few friends who can’t seem to hire a talented DBA. Perhaps a GenAI agent can support them and help them get work done without the need for a DBA. Not now, of course, but maybe in a year or two.

    However, humans still need to be in the process, and I suspect, even with an AI agent for every human, there’s still a lot of work to get done. I’m not convinced this will reduce employment. I think it could increase employment, though the bar for employees will rise.

    At least in some places. In some, they’ll keep doing the same thing they’ve been doing since 2001, or 1994, or 1983.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.