Voice of the DBA

Tag: Microsoft

  • November Data Platform Releases

    This past week was the PASS Summit in Seattle. It’s an annual conference that attracts thousands of SQL Server and data platform professionals. I am lucky in that I usually get to attend, and this year was no different, although it was a short summit for me. There are some live blogs of the keynotes from Kendra for Day 1, 2, and 3 if you want to catch up.

    Microsoft was there with a large presence, and as always, they delivered a keynote and had plenty of demos that showcase new changes and enhancements for the data platform. In this case, center stage is really SQL Server 2019, though there were plenty of other items shown as well. I was surprised to see the Azure Managed Instance get so many mentions. I suspect this is an easy for for many companies to transition away from an expensive local data center, or to find ways to use less staff and continue to run a SQL Server outside of their existing infrastructure. I don’t know if this is a good fit for most customers, but Microsoft certainly wants you to try it. The Business Critical edition (with business critical pricing) will release as generally available on December 1.

    There were a few releases as well, which you might want to play around with in your lab. First, Azure Data Studio (ADS) got it’s November release. I’m still not certain I love the tool, but Microsoft is working hard to improve it and add features. There aren’t a lot of changes this month, but there are a few more extensions and a number of bug fixes. The paradigm for ADS is just a little off for me, and I’m not quite sure why. I find VS Code to work well for C#, Python, and PowerShell, so why is ADS off for me? Not sure, but let me know in the discussion if you like the tool.

    We also have our second release of SQL Server 2019 with CTP 2.1 being announced. It’s supposed to be available Friday for download, at least as a container, but we’ll see. There aren’t a lot of changes, but there are some. What’s more impressive is Microsoft being able to release a second version a month after the first one. They hope to get to a monthly cadence, which I think is amazing for a large product like SQL Server, especially as Windows struggles with their cadence.

    There is one amazing new feature, which I think will really improve SQL Server performance for many systems: Scalar UDF Inlining. It’s not a panacea, but it should dramatically improve the use of functions in many workloads. There are restrictions, and it’s only SQL 2019, but I look forward to testing a few demos to see how well things perform with this enhancement to the query processor. You should give it a try as well, testing workloads before and enabling compatibility level 150. If you see improvement, maybe there’s a good case to upgrade your instances that might be using lots of functions.

    There are more announcements, especially in the BI area. We get some cool SSRS enhancements, and you might want to watch my friend, Patrick LeBlanc, demo the changes in Power BI. I love Power BI and I think this is going to be the de facto reporting tool for most organizations moving forward. Maybe it will even displace Excel for visuals.

    There are a lot of moving parts in the Microsoft Data Platform right now, which may feel overwhelming to many of us. That’s fine. We don’t have to learn everything, but we can pick something that looks interesting and spend a few hours playing. You never know what you might get inspired to learn more about.

    Steve Jones

  • Big Data Analytics

    How large is your analytics system? Do you have more than one machine for analytics? Do you have a cluster of machines that run Hadoop in a YARN cluster to analyze your data? Are there hundreds, or even thousands, of nodes that are being used regularly? Some of you might have what you consider to be a large system, but I bet it isn’t as large as Microsoft’s cluster.

    They think they have the biggest YARN cluster, with over 50,000 nodes in a single cluster. This is used to process multiple exabytes of data from their various properties and systems. I certainly haven’t heard of a system this large, and I really wonder what this costs to run. After all, I’d think a 50,000 node cluster has to be a significant cost, though perhaps in the grand scheme of Microsoft’s $100 billion in revenue and $38 billion in expenses, even 100,000 machines can’t really impact their numbers.

    The cluster has essentially been running a private version of Azure Data Lake for years that their internal developers and analysts use to access a common pool of data. In fact, because of their scale needs and the desire to limit the copying of data between clusters, they have contributed back to the Apache Yarn project a number of fixes to help ensure the software can scale to tens of thousands of nodes. There is some discussion of how they’ve allowed YARN to grow to larger scales, and it’s an interesting solution that essentially allows some overbooking of resources, knowing there are always some spare cycles available for processing data. It’s a great test site for Azure Data Lake, and something that more of us might use in the future.

    I doubt may of us would need to work on data sets that large, and I know I certainly wouldn’t want to be responsible for that much of a data lake, I do think these are interesting problem domains that someone should look at. Certainly there are always large organizations and governments that have ever growing pools of data that will likely end up in a data lake of some sort. And who knows, perhaps, the definition of large will continue to grow to the point where 1,000 nodes in a cluster is considered “small”, and it’s what many of our businesses might implement in the future.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • A $3 Billion Donation

    Earlier this week, Microsoft agreed to grant royalty free licenses for 60,000 of its patents for Open Invention Network (OIN) members as it joins the group. The patents include a number that affect Android, Linux, OpenStack, and more. As recently as a few years ago, Microsoft was making over $3 billion from this set of patents, including a billion dollars from Samsung.

    To be fair, licensing revenue has been declining, so this isn’t likely a $3b gift, but it’s still substantial and more important, it’s a positive step forward.

    Actually, it’s a surprising move, and one that seems to have stunned many journalists and open source advocates. There are numerous stories that seem to describe the move with disbelief, which is a position I would have held until recently. If I had seen this post from Erich Anderson, Microsoft’s Deputy General Counsel, I would have parsed the language, looking for the careful crafting of phrases that indicated some future duplicitous action.

    Instead, as I’ve worked with more and more people from Microsoft and talked with them about upcoming changes, I’ve found that Microsoft is truly changing. They’re becoming more open, and trying to prove their services and software are worth using, not trying to lock customers into an environment they can’t easily leave.

    Nowhere is this more evident than Azure, where there are quite a few Linux VMs. In fact, I thought I heard that there were more Linux than Windows VMs earlier this year, but I can’t find confirmation. In any case, Microsoft supports free and open source software (FOSS), including MySQL as a Service. Incidentally, that powers tsqltuesday.com. I moved the site there earlier this year, using Microsoft’s MySQL services.

    Microsoft is a for profit company, and I expect that they want you to use their software in addition to services. That’s no surprise, but they are becoming less cutthroat and much more welcoming. Scott Guthrie, who leads the cloud and enterprise group, noted that “It’s not just code, it’s community.”

    That’s been my philosophy here are SQLServerCentral. We’re a community, and the vision from Brian, Andy, and I was to share and help others as much as we could. We continued that with SQL Saturday, and we continue that today in our own ways. Andy and I try to support individuals in various ways. Brian has really put his money forward in this area with the Pragmatic Works Foundation.

    We’re a community. All of us. We compete, we argue, we laugh, we cry, but we’re a community and we can all try to grow our business, improve our skills, and advance our industry as a community. I’m proud to be a member of the SQL Community and the #sqlfamily.

    Steve Jones

  • The Ever Expanding Data Platform

    This past week was the 2018 Ignite conference from Microsoft, where we had a number of announcements about the data platform. You can rewatch some of the sessions from the event, and I might recommend the keynotes to see some of the demos and positioning of the data platform. That’s the direction that Microsoft is moving their database products, as a complete platform that not only includes SQL Server, but CosmosDB, Managed Instances, Data Lakes, and  more.

    If you’re a SQL Server DBA, it’s time to stop thinking yourself as a SQL Server DBA or developer. Instead, you need to be a data professional, especially on the Microsoft stack. While you might concentrate on SQL Server and live in SSMS, you ought to be aware of the growing options for working on the Microsoft stack. Azure Data Studio, which Grant wrote about this week, ought to be a tool you investigate. You also ought to be looking at the latest version of SSMS, which had it’s v18 move into a public preview this week. With these tools being free, companies ought to be moving away from the older versions that shipped with SQL Server 2014 and earlier. Instead you should at least be on a v17 version of SSMS. Talk to your IT group and give it a try today. It works fine with all your SQL Server versions, from 2005 through 2017.

    Microsoft is certainly hoping you’ll run more workloads in Azure, and that’s where the data platform is growing. CosmosDB, which I think has a lot of promise for various types problem domains, or even as a companion to SQL Server for certain types of data. There is an increase in their SLA to 5 9s, which is both impressive and ambitious. I know very few on-premises instances that get by with 5 9s across multiple years, leaving aside the ability to get 10ms write performance in the SLA. The is also multi master replication and support for the Cassandra API. While I haven’t done much with CosmosDB, it is on my radar to experiment with as a data store option.

    Managed Instances will be generally available on Oct 1, just a couple days away. While I wasn’t sure that this product would catch on, I’m not surprised that some companies would like to get away from managing most of the stuff around the database and stick with the data. To me, this, more than anything else, can mean that DBAs at larger companies need to be managing data, security, and more, without worrying too much about the basics of backups and HA. Even threat detection, something few of us are good at, is handled by Azure. The restore demo in the keynote is truly impressive. I’m not sure many of us would want to, or be able to, architect those speeds. At least not as easy as provisioning an Azure Managed Instance.

    There are lots of other announcements, which you can read. The one really interesting thing for me was the Data Box announcement. I’ve had more than a few people be concerned about the initial loads of data into an Azure database or data lake. I’ve had that concern, and actually been part of a company that FedEx shipped a rack of disks as part of a SAN to a DR site because of bandwidth constraints. The Data Box is a device that you can order and fill, shipping this back to Azure for loading. It comes in 40TB, 100TB, abd 1PB sizes. That is truly stunning to me. Drop ship 1PB if you have the need. You can even see a picture of it from Argenis Fernandez for some idea of size.

    It’s an exciting time to be a data professional, and Microsoft’s data platform continues to grow. I don’t know that any of us will know more than a tiny bit about most of the platform, but I certainly plan on increasing my knowledge in a few areas to become better aware of how they work and what they are capable of. I might not be able to use them well, but I can at least have enough knowledge to have a conversation about the technology and have an idea of whether it might solve a problem that I run into at work.

    Steve Jones