Voice of the DBA

Category: Editorial

  • The Network Bottleneck

    Most of us know that sending extra data back in a result set is a waste of resources. This can certainly cause delays and poor response in our apps, and across slow networks, this can even cause timeouts. Good developers try to limit the data transfer to just the data necessary, without pulling back extra fields. This is one reason a “SELECT *” is a bad idea.

    What about when we are sending just the data we need, but that data is so voluminous that we can’t easily transmit it across a network. That’s one of the ideas that is noted in this piece, which asks for innovation in the way that we move data around, especially IoT data. This type of data is growing, and potentially growing across networks that are bandwidth constrained, such as satellite and cellular links.

    Do many of you find that your network links are constrained? I certainly do at times, though I live in a place where the broadband and cellular networks are not very robust. Even so, most of the time I never notice issues with moving data around. Even when using a real time application, such as streaming data.

    Perhaps there are tricks that buffer data and hide network issues, but even if that is the case, is there a problem? After all, that’s part of what our applications should do: isolate and protect us from the inherent unreliability of hardware.

    Relational databases continue to store and process larger amounts of data all the time. Size of data operations, which grow increasingly cumbersome as data volumes grow, are a place where many vendors are trying to improve their offerings. Microsoft is one of these, and some of the improvements in SQL Server 2019, with Big Data Clusters, Accelerated Database Recovery, and scale out architectures are designed to precisely help us cope with more data.

    I have confidence that we will continue to solve these problems, and that networks will keep growing to support the demands of our ever increasing database sizes. We just need to be sure we continue to evolve and grow our applications to take advantage of these improvements.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • How Did I Do?

    I made a number of predictions for the opening keynote at the PASS Summit. I wrote the predictions a week before the event, making my own guesses about what might be revealed. I was busy in pre-conference sessions on Monday and Tuesday, so I had to write something well in advance of the event. I was flying home yesterday, so I had no chance to watch the keynote until later, so this is a look back after catching up Wednesday evening at home.

    How did I do?

    With the Ignite conference taking place this week, I had guessed that SQL Server 2019 would release there and then be a part of the Summit keynote. I was right. On Monday, as I was taking a break during the SQL in the City Summit, I saw that SQL Server 2019 had been released. A milestone, and perhaps one that I think changes the data platform. It’s an inflection point, and I agree with this tweet from Distinguished Engineer, Slava Oks. Rohan Kumar did announce it at PASS as well, so maybe half credit here?

    I didn’t mention Hyperscale yesterday, as I wasn’t sure what I knew as NDA and what was possible. The demonstration of Azure Arc, and Hyperscale for PostgreSQL, is pretty cool. While I used to think Hyperscale was only for the largest of the large databases, I now think that many smaller, 2-4TB systems (I can’t believe I think of that as small), might consider moving to Hyperscale and preparing for future growth into the low single digits of TBs. BTW, Azure Arc is amazing. I knew about it previously, but didn’t expect it this year. I did mention Azure Stack deploying on premise, and I think the Azure Arc covers this, so 1 1/2 right.

    Azure Synapse was something I didn’t expect. I have been looking at Big Data Clusters as a game changer with the relational platform. Synapse combines the Azure SQL Data Warehouse and Data Lakes into a platform. Since these were both the places where we first got compute separated from storage, this makes sense. Both of these platforms seemed similar to me, so combining them and letting customers drop structured and unstructured data.

    The Edge is a bigger deal than I thought. This was mentioned as a private preview during Build, and I should have guessed this would move to public preview. This as one of those areas that I haven’t been as focused on and wasn’t sure about information I’ve gotten under NDA, so I didn’t include this. Certainly a miss here, but I do think the demo in the keynote with Zeiss and making decisions from sensor data is something more organizations will do. Sensors generate too much data and trying to move all that across a network to the cloud doesn’t make sense. Train models and let some low power SQL Server Edge engine make a few decisions without moving data.

    There was more in the keynote, and it’s worth watching. If nothing else, I think both the opening Ignite keynote and the opening Summit keynote are inspiring for us as data professionals. You can dream about possibilities and truly see that data is an important asset that Microsoft is empowering us to make better decisions about based on analytics.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Republish: Logging Tables

    My second pre-con at the PASS Summit, so I’m re-running Logging Tables. Tomorrow, I’ll be back.

  • Republish: Custom Schemas

    Pre-con one at the PASS Summit. A busy day for me, so I’m re-running Custom Schemas