Voice of the DBA

Category: Editorial

  • Don’t Let Corner Cases Drive Your Design

    If you graph computer/query cost against the size of data, you can get four quadrants:

    1. small data, small compute (most CRUD app queries)
    2. small data, big compute (complex BI queries for this quarter, most reporting)
    3. big data, small compute (logs, audit data)
    4. big data, big compute (complex BI queries across all our data)

    If you examine the costs here, 1 is the cheapest, with 2 and 3 having a similar cost. Number 4 is expensive, and it’s why we often have big boxes running our database server software. However, where is most of our work? The majority is in quadrant 1, with 2 getting the second most action. 3 might rarely exist, as does 4, but we often design for 4. We have to as we don’t want phone calls, ever. What we want is to provision a system large enough that we don’t hear many complaints about performance. On premises, many of us have over-provisioned systems to handle the peak load to avoid phone calls.

    Can we handle the peaks or the really important things that someone thinks are important? Everyone thinks their workload is important, and it is. To them. However, there are plenty of cases where someone could think about designing for specific types of workloads, rather than just aiming for quadrant 4. I’ve got an image of different types of workloads that I grabbed from the Small Data 2025 conference. For example, if I am working with things like Time Series data or streaming analytics, I might not need huge compute. I might be storing a lot of data, and I need space, but the compute is low. The analysis of that data, however, might be compute intensive.

    This is a reason why we might separate analytic systems out as they often are in quadrants 2 and 4, and we might want serverless or scale up/down systems to handle the rare cases, and get a real cost for them. I found it particularly interesting that the Bronze tier might be where we have big data and big compute, but once we’ve moved to Silver or Gold, we might have lower compute and data requirements. This makes sense as Bronze is more staging, but it is a good reason why we might aim for a Gold layer in our organization and only keep that data for the long term; it’s more cost-effective.

    Often, for simplicity, we build a bigger system for all types of queries. In other words, we are letting corner cases drive our design. That might be required, but it might not be. In this area of cost concerns, especially in the cloud, designing systems with appropriate resource usage is something that might override the analyst’s desire for queries across all data running as quickly as order lookups in an OLTP system. This might be even more true if we can predict some patterns in our workloads during system design. We can’t scale up or down instantly, but in a lot of places, I wish I had been able to scale financial or reporting systems up for a few days as we close out the period and scale them down for the rest of the month.

    When building a system, think about the practical nature of your requirements and assign a cost to them. Let users know what workload you’re building a system to handle and set expectations on performance and cost. If you do that, you can let others decide when we handle corner cases and when we don’t. That’s often a much easier conversation when we have cost numbers to help customers understand the implications of their request.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • What’s Your Theme Music?

    A few weeks ago, I was at the Small Data SF 2025 Conference in San Francisco. I attended the inaugural event last year and decided to go back again. It’s a great chance to hear people thinking about data and its impact on the world in a different way, recognizing that building lager and larger systems isn’t always possible. Or a good idea. We might find that smaller systems fit well, especially smaller datasets, which can both serve our purposes and create agility. The manifesto of the conference says that “We champion the power of Small Data and smart AI, believing that less is truly more.” There’s a bit more, but that’s the idea.

    The format for the conference is a little different, with 3-5 talks in a row, all on one stage, each about 25 minutes long. These are talks with or without slides, but no live demos, just speaking and expressing a point of view. What I found fun was that each person picked their own music to play as they walked onto stage (or ran/danced in the case of Glauber from Turso). It was a bit of fun, with the DJ letting the music play as the person made their way to the front and were welcomed by the audience. I heard rock, metal, hip hop, and more.

    It’s Friday during the week of the PASS Data Community Summit, and I had the chance to deliver part of the keynote on Wednesday. I’ve done this before, and no one has every asked me if I wanted a pick a piece of music, but it got me thinking. What would I pick?

    For a fun Friday during the holiday season, think about if you were going to give a presentation. Maybe to your team, maybe other groups in the company, or (for some of you) on a conference stage. What music would you choose to accompany your walk into the bright lights? Imagine you get between 10 and 30 seconds.

    Have some fun, and remember this is a professional setting. My first thought was something from the Notorious BIG, but I realized I’d have to walk fast as most of his lyrics wouldn’t be appropriate. I’m not sure, but I lean towards one of these: one, two, three, or four.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Done is Better than Good

    Mary Spender is a musician in the UK who I follow and hope to see live one day. She works hard producing content about music, that business, and, of course, songs. Recently she had a little essay on Instagram where talked about creative time and focus. In it she referenced Elizabeth Gilbert saying “done is better than good.”

    My initial reaction was “that’s right.”

    Then I thought about software, and poor queries impacting database performance, and thought, “No, that’s not right.” My next reaction was to think maybe it’s “done is better than great”. I do see plenty of engineers trying to build great software. Code that would impress their peers or their former professors. Or maybe their future self.

    Then I thought, no, Elizabeth is right. If things don’t get done, then what’s the point?

    At the same time, I think that “done” and “good” (or great) aren’t mutually exclusive. We can get things done and make them great, which is something to strive for. We can also get good things done.

    Sometimes.

    Sometimes we don’t have that luxury of time, for various reasons. If I had to make a trade, I push for good (or quality) as much as possible if the delay isn’t substantial. If it is, then done is likely the choice I’d make. I do try to return and refactor, improve, etc. to raise the quality over time, but I recognize that sometimes getting something done is important. Certainly, on the ranch, I need to fix things to get by, with the aim of doing a better job later. I have mixed success at returning to improve a patch with a better fix later, but so far, that’s worked well. I’d say the same thing has happened while building software or managing systems, with enough success to be comfortable with my choices.

    What about you? What choices have you had to make about being done over delivering something that’s good? Or maybe great.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Is Data Modeling Common?

    Recently, I had a few questions on database modeling. One was posted in the SQL Server Central forums, and a customer asked about ERD tooling on the same day. This came shortly after Redgate acquired Vertabelo (now Redgate Data Modeler). This stood out to me as very rarely in the last few years have I found people consulting and updating a diagram while performing database development.

    When I started as a developer and needed to update a database, I had to first update a diagram that was stored in ErWin. We had a dedicated computer (back when we went to an office every day) where the software was run and any developer could us this to update the diagram with proposed changes. Back then, we had to get another peer to sign off on changes before making them, and the peer was supposed to go check the diagram for the change before approving it. That’s only if they thought your change made sense and conformed to our standards (naming, design, etc.).

    Over the last decade or so, it seems there haven’t been a lot of common tools for building diagrams in the Microsoft space. The SSMS Database Diagrams haven’t been the easiest to use and I rarely see people consult it. I loved ER/Studio, but it and ErWin were very expensive and outside the budget for most groups. There have been a lot of smaller online tools, but I don’t see a lot of consensus for which ones people like. There are DbSchema and SqlDBM, among others. I see various blogs, like this one, that recommend different tools, but the uptake seems to be low in many organizations. I hope Redgate can make data modeling more common with Redgate Data Modeler (coming soon to the Toolbelt).

    My question for you is do you still maintain and use database diagrams for development? Or is this a somewhat lost art? Is it not necessary or useful for you? Do you wish your development teams did this to ensure a more cohesive and consistent model?

    I used to like having a diagram because it a) forced me to slow down and rethink things a bit as I made updates and b) let me see where there were dependencies. One of the reasons SQL Dependency Tracker exists is that I asked Redgate to extract the dependency tree from SQL Compare to allow me to see it separately. This would allow me to know what other objects might need to be changed if I refactored ObjectA. I found that to be a great “picklist” of things to check during database development.

    I don’t know how many people these days perform data modeling, but I do know a lot who don’t spend much time thinking about the changes they make. I fear data modeling has become a bit of a lost art, which saddens me.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.