Voice of the DBA

Category: Editorial

  • The AI Budget Assistant

    I saw this article about AI helping IT leaders plan their budgets, and I had visions of certain departments getting cut, especially data-related ones. My concern is that a lot of what a DBA does can be acting as insurance or supporting others and not necessarily doing things which are easily visible or that might show up in reports, praise in Slack/Teams, or in other places an AI might be prompted to look. If an IT leader has to decide what’s important for the AI to look at, is data included?

    Of course, it’s entirely possible the AI would look at all the performance problems and want to spend more budget on the data side. Fingers crossed for that interpretation.

    The idea of an AI looking at a large amount of current data in various spreadsheets, reports, goals, requirements, usage, and more to synthesize a budget makes sense. This is the type of thing many AI-type technologies do well, finding patterns and summarizing the data in a way that assists a human in producing a final report of where to spend money.

    Just like any report produced by a human, I would hope that someone reviews the results and verifies the analysis makes sense. However, in reality, I suspect a lot of human reports are passed along as-is, with only a cursory glance. The world of AI might make this worse as we can produce larger and more detailed reports.

    The funny thing might be that whoever has to approve the budget might let their GenAI model summarize the budget for accuracy. In that case, do we need humans involved? I can see some managers trying to reduce the staff that might have internal knowledge and replacing them with GenAIs. Will we just have AIs producing things other AIs read and summarize?

    Most of us know that garbage in/garbage out is a constant problem in the world of data and reports. Judging data quality and ensuring the accurate (and complete) data is fed into any system is something that humans need to be a part of. An AI model might assist, but humans have context and knowledge that can be hard to craft into a prompt. After all, this might be the training data for your model, and I’m not sure general models will learn well without handholding from humans, especially in specialized areas like budgets.

    I’ve had to build budgets, and while I don’t think it’s hard, it’s time-consuming. There is a lot of drudgery, and I think an AI can help here. However, I’d want to audit and check the data and ensure that my helper is summarizing the data in a way that makes sense to me. Even then, what I’d most like to do is let the AI give me a few suggestions of future costs and let me fine-tune those before I present them to anyone else. After all, if we make large errors, I suspect I’ll feel the pain of failure more than my AI assistant.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • The Types of Changes

    I would guess that most of you reading this piece are database professionals in some sense. Either you work on databases as a developer of some sort (app and/or database) or manage them as a DBA or sysadmin. In working with databases, I’m sure that you need to alter object structures. Maybe you perform a lot of changes, or maybe you just alter the schema occasionally, In either case, you need to make decisions on which changes to make.

    I would guess that the most common changes are altering stored procs/views/functions. Often these are to handle new or updated business rules. In those cases, how do you ensure you haven’t added regressions? I would hope you had a set of test scripts to verify that your results are correct. If I were very optimistic, I’d hope that you had a set of automated tests, such as those you could write with tsqlt or the Microsoft Unit Test Framework.

    I’m sure many of you add new columns to tables. What are these types of changes? Capturing new data? Splitting existing columns or de-normalizing data? In these cases, are you re-examining the data model to ensure the changes make sense?

    Maybe even more importantly, if you are capturing new data, do you consider vertical partitions for the table? This can be a good way to separate out data in columns that isn’t queried as often. Despite most recommendations, lots of developers will still SELECT * from tables, which can flood the buffer cache with data that isn’t often needed by an application. Vertical partitions can reduce the impact of these types of queries.

    For those of you that add new data, do you consider using different data types for your new data, or do you have a standard string/numeric/date type that you use? SQL Server (and other platforms) often have a variety of different data types that can save space, while still accommodating the requirements for a change. As an example, many people use datetime as a standard type when they just need a date. In this case, we could save 5 bytes per row by choosing a more appropriate type.

    If you add status values or flags, have you considered bit storage for your values? Whether you use the bit datatype or bits in an int, you can dramatically save space in each row. Newer versions of SQL Server include T-SQL functions to help you work with bits, though I would hope most developers can handle simple AND/OR logic against an integer to test if a particular bit is set.

    Most software under active development has a need to continually alter and modify the database to meet new requirements. While lots of us have experience in making these changes, and do so often, once we make a change it often becomes enshrined in our production system for years. I would hope that we feel confident in refactoring our code as needed, but the reality of what I often see is that people are loathe to change the database for fear of breaking our software. Even when the refactoring actually improves the performance of the application.

    I’m curious what types of changes you often make, and how you go about deciding on the specifics of the change. Let us know about your approach in the discussion. I’m especially interested in how you might handle the various options for changes. Do you assume defaults or are you explicit in how you structure your ALTER code?

    You might have a novel way of examining requests and implementing changes that can help others in learning to ensure they make better changes in their own system.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • The Managed Cloud Database Options

    There are many, many choices for cloud database services these days. I would hope everyone is aware of the various IaaS options in public clouds with EC2, Azure VMs, GCP Compute Engine, and others. These are often the easiest way to move your workload, but you’ve really just moved a VM from one place to another (likely more expensive) place.

    For managed databases, there are lots of choices, but you might not be aware of your options. I ran across an article that discusses the various flavors of managed databases in the big three public clouds for SQL Server. In the piece, there is a section that talks about when a managed database makes sense. I like that it discloses the development on a managed service is expensive.

    The problem I have is that I know lots of companies that struggle when they don’t have a development environment that matches product. Invariably developers will use something in their local SQL Server/MySQL/Oracle/PostgreSQL database that doesn’t work in a managed service and causes no shortage of pain. Containers can help, but they’re not always available.

    I won’t delve into details, but the article lightly looks at RDS (Amazon), Azure SQL DB and Managed Instance (Microsoft), and Cloud SQL (Google). There are pros and cons for all of them, but to me, the Google option is interesting. It gives you an instance (with SSIS/SSRS) and a proxy, but it does have limit you to instance restores. Same for RDS, which is a limitation I’d be worried about. Often I have an issue with a database, really often with just a table. Restoring a whole instance instead of a db might be a delay I wouldn’t want to deal with. I hope this changes over time, as not recovering a database from an instance is a big hole to me.

    The comparison of hardware resources and performance seems to show Azure has the highest capacities, but as the article notes, if you approach the maximums, the cost is very high. I tend to agree there, and I think this is one of the things that gets lost when companies consider the cloud for workloads. They look at the maximums with an eye towards growth if they need it. Rarely, however, do they compute the increased cost if they’ve under-provisioned resources. There’s also the concern that trying to match CPU/RAM/disk from on-premises to the cloud isn’t easy. This often isn’t the same type of comparison that most of are used to with local machines.

    The main thing I find too many people forgetting is that moving to the cloud is a new architecture. You likely need to rework/refactor/rewrite your application to work better. Part of that work is reducing the amount and frequency of data queries. Without changing those things, you likely will end up spending a lot more money than you expect. Possibly without getting the workload response you expect.

    The cloud can work well for your application and database, but it’s not easy and it’s not quick.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Simple(r) Coffee

    Starbucks has too many order combinations. This article about the challenges for their new CEO notes there are over 170,000 combinations of things customers can order. As someone who has visited a store often and stood behind someone placing an order, that sounds like an accurate number. Sometimes I hear people order things to drink that I had no idea were even possible.

    When there are too many choices, sometimes there can be a delay as the person tries to process the information and make a decision. Some people can’t make a decision. Others enjoy the variety and keep making different decisions each time. That’s fine if you’re visiting a restaurant, but it’s less useful if you’re building software.

    I have worked with people who struggle to decide on an approach to solving a problem. In the C++ world, I’d see people struggling to organize classes and members, sometimes spending too much time researching and comparing different methods. They debate when is private or public appropriate, when do we need a superclass or subclass? We would see some paralysis by analysis from certain developers who are endlessly searching for the optimal approach.

    Other developers might change their approach constantly, resulting in a lack of consistency across the codebase. In the modern world, I sometimes see users experimenting with different libraries or packages (or data stores) with each new project. They might be consistent within a project for a period of time, but since many teams often shift to different projects over time (or maintain them), how helpful is this? If each new project looks different, we spend a lot of time on developers getting familiar with the technology.

    I have tended to adopt the philosophy of strong opinions, loosely held. In other words, I’ll pick an approach/platform/technology/etc. and stick with it. Moreover, I’ll ask everyone on the team to stick to a similar method of coding. If we need to change, then we can evaluate the proposed change, performance test it, and if we decide it’s better, we all adopt this and slowly work to refactor old code as we have time. We never have time, so we only refactor when we have to touch old code for some fix or enhancement. However, when we do touch old code, let’s improve it.

    My preference is to allow a limited set of technologies in our company at any point in time. We need a process to add or remove something, but this ought to be a decision made to limit our range of technologies and required expertise. I’m not against Python or PowerShell or MongoDB or any other technology, but I don’t want things adopted because an individual prefers that technology or thinks it works better. They need to prove that there is a good reason to others, and as a group, we agree that the value of adding a new technology is worth the burden.

    Otherwise, we end up in the situation I see often in job descriptions: with a requirement that new hires know dozens of technologies at an expert level because we don’t have that person on staff.

    And we’ll never find them.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.