Voice of the DBA

Category: Editorial

  • JSON Has a Cost

    JSON seems to be everywhere these days. Many application developers like it across all sorts of languages, C#, JAVA, Python, and more. They use it for transferring information between systems, and are comfortable serializing hierarchical object data into JSON from text and de-serializing it back into its various elements.

    For those of us working in relational databases, JSON seems like a blob of information that isn’t easily queried, indexed, or stored. We prefer working with a relational set of data, which brings us into conflict with software developers. We’d like them to convert their objects to a relational structure, and they’d like us to just work with JSON.

    SQL Server has added new JSON functions in SQL Server 2025, expanding the JSON capabilities from previous versions. PostgreSQL has JSON types for a few versions, as has Oracle. Lots of applications are storing JSON data in databases. Unlike XML, however, JSON seems to be working well (overall) as a data storage mechanism.

    Or is it?

    At a recent conference, one of the speakers noted that we do a lot of data movement these days, and there can be a high cost to this as we pay for both compute and network. This data movement often incurs a query cost to get information from the source system. JSON can exacerbate this issue as we may send more data than necessary, and we might find our compute engines don’t support predicate pushdown. Even if we index the JSON, we might still end up querying more data from disks to get what we need. Especially as we move a lot of data to warehouses and lakehouses.

    Using JSON can slow things down. It’s nice for storing a bunch of information quickly and easily, but as we need to work with specific parts of a JSON document, we would likely be better off de-serializing the important pieces into more structured formats that prevent duplication, are easily indexed, and can achieve quick query performance. Everything becomes a little smoother.

    JSON is great, and I do like it, but it’s not a substitute for relational systems and relational models.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • An SSIS Upgrade

    I came across a post recently on the Microsoft Fabric blog about the evolution of SSIS 2025..I hadn’t heard much about SSIS in SQL Server 2025, so I thought this might provide some info on the investments that Microsoft is still making in Integration Services. I’ve run into a few people in the past year who are still heavily invested in SSIS and run packages daily. SSIS seems to be a technology that isn’t even close to dying for many organizations.

    The blog starts well, delving into the security investments with the change to the SqlClient and TLS 1.3, as well as supporting Strict Encryption. I don’t know many people using this level of security, but it’s good to have SSIS support stronger security. There is also an upgrade for SSIS packages targeting Fabric Data Warehouses if they modify their approach.

    There is also a mention of lift and shift into Fabric Data Factory as an early access program. I wonder how many organizations are looking to still run SSIS packages, but upgrade to Fabric. I’m sure there must be some, and I’m interested if any of you are in this situation. Many of us buy SQL Server licenses, so I would hope Microsoft could invest in a few new SSIS tasks, like SFTP at the least.

    The rest of the post covers Fabric and Azure Data Factory, not SSIS. I understand that Microsoft would prefer everyone abandon SSIS and move to ADF/Fabric, but that’s not what a lot of customers want to do. Many of us have no real need for complexity beyond what SSIS does. What I’d really like to see is a local version of ADF. Many of us still run our own systems and plan to do so for years. If there isn’t going to be an investment in SSIS, which I do understand, then invest in a local version of ADF.

    I doubt Microsoft has much interest in doing work here, especially as the trend towards lakehouses and Parquet files seems to drift further from the idea of SSIS moving data between systems. However, there is still a need for many organizations that might want to build packages to export their data to a data lake on-premises. They might prefer to host their own storage, use Polybase to query those files, and maybe grow into Fabric and Power BI over time.

    Giving options is something Microsoft has often done in the past. Their successes often open the door wider for others to build on their platforms. Their failures are often in places where they try to force everyone to work in a particular way with a particular technology. Expecting everyone to move to ADF/Fabric feels like one of the latter decisions.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Where Your Value Separates You from Others

    I ran across a post that discusses what makes you a senior engineer (via Brent Ozar). The main point of the post is that there is a core skill that separates senior engineers from others, which is reducing ambiguity. When a senior engineer gets an ill-defined (or ill-communicated) request, they can deliver a solid, or even great, result.

    When someone says “performance is poor,” what do you do with that? Can you build a plan to identify the issues and solve them? Or do you expect the customer to explain what is slow and why it’s slow? Do you ask what metrics they have showing things are slow? A senior engineer can ask questions to find the problem and then determine how to move forward.

    The post also discusses the way many companies hire senior people, often basing decisions on years of experience and answering specific questions in an interview or on a test. It’s hard to interview and test a person who is given a vague requirement and develops a solution. Most interviewers don’t want to have to wprl that hard and compare what might be very disparate answers to ambiguous questions. How can you judge two people who give very disparate answers to questions with no clear answer?

    It’s hard, and I know this because when I’ve run interviews that lightly describe a situation and let the candidate lead me to the next question, so that I can see how they work and think. It’s very hard to judge the end result and rate the candidate as effective. Often, I find myself deciding if I like the person more and if they fit in our company. That’s if I think they can probe to find information and make decisions when that information is incomplete. If they can’t probe and find a way to solve the problem without direction, that’s an issue.

    And that really is the senior skill. In all fields, not just engineers or DBAs. Managers, customer service, analysts, and more. Can someone handle an unstable atmosphere and clarify, identify, simplify, and present a way forward? If they can, then they are likely someone to consider for a senior role. They still need expertise in their area, but can they get things done when they aren’t being directed?

    Are they self-starters?

    That might be the main thing I consider for senior people. Can they drive themselves and get things done, things we need done, but without a lot of direction and hand-holding? If a senior person struggles to move forward without more direction or supervision, then maybe they aren’t really a senior-level employee.

    Steve Jones

     

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Your AI Successes

    Recently, I was discussing AI with a friend, and they asked me to name a great success of using AI to build software. I’ve tried a few things, and I’ve worked with customers who are using AI tech. However, most of the things I’ve seen built with AI are small tasks; they’re utilities or quick wins that change a minor part of the software. The items tend to be tactical and focused in a narrow band of fixes, and they might save a programmer time, but I’m not seeing large-scale team improvements in productivity.

    Yet.

    I think there is a chance for AI to dramatically change our industry, and perhaps help us tackle a lot of small things we haven’t wanted to, or been able to, find the time to build/fix/change/etc. Certainly, in the area of migrating from one version of a platform or framework to another, GenAI can be very useful. This can be a very tedious task, and one where humans can make lots of small mistakes. An AI agent likely can do this quicker, cheaper, and more accurately than humans. The question might be whether this is a huge success, as any single organization might do this rarely.

    The biggest success in many organizations might be the ability of developers, or even business people, who can quickly build out an MVP of an idea to see if there is a project worth pursuing. These often won’t have the robust coding or security practices embedded, but they can perhaps shortcut putting more research into an idea until some value is proven. At least internally, if it doesn’t have good scalability or security, it shouldn’t be exposed publicly until it adds those capabilities.

    When or where has an AI worked best for you? What impressed you about the AI technology interaction that changed how you might work in the future? Or perhaps if you’ve had a big failure and want to share, where did AI not work well?

    I think most of my experience is that AI is still a bit of a toy and useful in small ways, but it hasn’t proven to me that it’s worth the hype we see in so much of the media. Maybe it will at some point, but so far this seems more of a dream than a reality for software professionals.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.