Voice of the DBA

Tag: Database Weekly

  • Microsoft, Think DevOps First

    This post from Melissa Coates is a good example of not thinking through your product architecture early on. The short version of the post is that Power Bi Desktop is a better place, perhaps really the only place, to author your Power BI Reports. Melissa covers a few potential issues while using the Power BI online editor, but the big item is that you cannot easily prevent conflicts and track versions there.

    Power BI started as the online service, with the Power BI Desktop tool seeming like a bit of an afterthought. At least from the marketing perspective as all of the early demos and media from Microsoft was about the online report service. Since then it’s evolved to become the primary way we will build reports in SQL Server moving forward, which I think is a good move.

    While I can understand the developers at Microsoft not really thinking that the service would on-premises and maybe not considering the need to provide some text format for the reports, I can’t understand why they didn’t learn from the Integration Services team and realize that binary versions of programmable items don’t make sense. We need a format that can be easily versioned, and maybe more importantly, stored in a VCS and diff’d in a way humans can understand.

    When building a format for storing code, please consider the need to work in a team and version the changes made. This means any format should not be a) binary and b) difficult to decode. Separate visual elements from logical elements and ensure a text version of this can be examined by developers. You can use XML, JSON, YAML, or any other text based format, but choose something that makes sense. Even if you add your own extension, ensure that standard tools that work with code can use this.

    I do know the PBIX format is a ZIP file, but zip files don’t easily integrate with a VCS. We could use hooks to extract/rebuild out files on commit/checkout, but that’s cumbersome and silly. I’d rather that the PBIX were a folder with the files inside. Users, including my Mom, can zip a folder and email that if needed. To me, that would have been a better structure from the beginning.

    Microsoft is supposed to be a company providing platforms that we build upon and use in our work. The decisions for the Power BI service seem to be poorly thought through with that in mind. I’d urge them to create a baseline set of rules for future products that consider DevOps, teams, and the need to track code.

    Steve Jones

  • Building or Buying Analytics

    One of the decisions that I’ve been involved with at the beginning of every software project is whether to buy software to solve the problem or build our own. This might be a quick “is there software anyone knows about to do this?” query, or an in-depth review of the marketplace or something in between. Often it’s a limited discussion on whether we think the problem is close enough to one other organizations have. If so, let’s buy something. If not, we might need to build an application to get what we want. Key card software for our doors? Buy it. Managing our core business flow, I might buy this.

    Analytics, or what has often fallen under the umbrella of business intelligence (BI) is an area where technology groups often wonder what’s best. I have seen no shortage of decisions in the past for how best to implement this feature for our organization. Ultimately, I think this piece summarizes this well: time will make the decision easier.

    If you need this done now, buy it. If you don’t have development resources, buy something. If you need similar analytics to what many companies need, buy this. Do you trust vendors to spend enough time on security, buy their service.

    Those are good reasons to adapt your process, your view, your requirements. If you can be flexible, it makes sense to buy some software that provides analytics for your data. Many companies have had success with purchased products, like Tableau, Qlikview, Cognos, Power BI, or some other pre-built package that allows users to analyze their own data. You might even just give them access to all views (or even tables) and ignore the entire part of your business.

    If you have time, however, I do find that developers working with analysts can truly add value and insight to the information a business holds. They can be responsive, while also providing knowledge about ways to structure and summarize data, and perhaps even identify those places where the way in which we gather and use data ought to change, perhaps even add new data elements.

    I do like the self-service, pre-built packages for multiple reasons, not the least of which is that users can do some Proof of Concept work themselves. I think Power BI Desktop is one of the best ways to do this today, using existing data sources that users can access. Let them go crazy and try whatever wild idea they have with data. When they find something useful, something that others need to also access, and something that becomes important to the organization, then it’s likely time to involve IT to provide stabliity. If there are also ways to better structure the information and perform a more detailed analysis, this might also be the time to customize the way the analytics package works, and again Power BI provides the ability to do this.

    Making this decision is hard, and it might not ever be done, but I do think buying is a good place to start, especially with the wide variety of packages out there, but you might find yourself changing your mind as your analytic needs mature.

  • Getting Close

    Last week we saw the release of SQL Server 2019 CTP 2.5, with enhancements for Big data clusterrs and a Java language SDK for SQL Server. This is the 6th public release of the next version, and one that’s coming at an increasing rate. If you look at the cadence, there was a release in September, then November, then December, about once a month. A long delay at the beginning of the year, but now 3 releases in a month and a half, which lead me to think that they’re getting close to being done with features. I have no idea when the product will release, and I do think that //build/ is a little too close, but I do think that it will release in the next few months.
    For fun, I’ll send someone this mug (or a fun one of your choice) if you can guess which day the new version will release. Drop a note in the discussion with your guess.
    For SQL Server 2017, we had seven CTPs and 2 RCs. For SQL Server 2016, we had 11 CTPs and 4 RCs. SQL Server 2014, I only had 2 CTPs tracked, though I think this was barely a major release. You can flip back through our build lists and check, but there has been both an increase in the pace of CTP/RC release as well as an increasing volume through the versions. I think this is a direct result of an better engineering process that has adopted DevOps to ensure the release of software is easier than ever.
    I have to admit I’m torn on this. There is some effort to download and update software, and at times these versions have to be uninstalled to install a new version. That makes sense, and I certainly don’t want them spending time ensuring they can upgrade between CTPs if they can fix some bugs instead. Even if i have time to install the software and check a few things, I often don’t want to then recheck a number of things that used to work, and like many people, I probably just check one or two new things. Or if I haven’t finished checking something from a previous CTP, I might stick with that version.
    This is one place that I think containers can really change the world. I can get the updated software without effort. Other than the download, starting a CTP 2.5 image is no different than starting a CTP 2.4 image. There is still a challenge with data files, but in a very rapdily changing world, we want to be able to upgrade quickly. With the changes made with CUs to allow roll forward and roll back, containers make even more sense. Stop one container, start a new one, same IP, same name, same data, new version of binaries. Upgrade or downgrade.
    I’m both excited and intimidated by the new version. There are lots of enhancements that are going to make this a new generation of the platform, but as developers and customers request the use of new features, I’m going to be scrambling to better learn and understand some of the tricks I need to get the system to run well. Of course, with Intelligent Query Processing, maybe I’ll have a bit more free time away from some of the poor code that I’m stuck running.
    Steve Jones

  • The End of XP

    This week we have the end of Windows XP. At least, we have the end of support, which means the lifespan of the OS was over 17 years. That’s a long time. It’s longer than I’ve run any system without an upgrade. I have some Windows 7 systems I use at times, but that’s a mere 10 years old. I know there are still a few SQL 2000 systems out there, which is older than XP, and likely a few of them are running on Windows 2000 in a VM somewhere.
    I have to admit that I liked Windows XP and thought it was a nice upgrade from previous versions. I ran it for a long time, skipping Vista, before moving to Windows 7 when it was released. Most of the early days of SQLServerCentral were run from an XP workstation, with SQL Server personal and lots of text editors helping me manage the site. I’m somewhat sad that it’s gone, though I think Win7, and now Win10 were very nice improvements.
    XP isn’t likely gone, as some companies will continue to run it and I expect some ATMs, some kiosk displays, and other embedded applications will show that XP start screen on occasion. That’s not unlike the way that some of us might connect to an SQL instance and be surprised to see a single digit number in the major version space. In fact, a lot of you might still see a “9”, for SQL Server 2008. I was seeing that until last year when we moved SQLServerCentral to SQL Sever 2017. I expect the more and more organizations will be slowly moving on, perhaps reluctant to upgrade those old versions that work and don’t cost much to maintain.
    Or do they? Is there a big cost on older systems that run and don’t see regular development work? My thought has been that for many internal systems, I need a database platform, and likely an OS, to run for 10 years or more. I know some of the larger enterprises have this view as well. Picking a platform is a major decision and it can be very hard to change directions. Porting takes a lot of resources, as do upgrades, so ensuring a system can handle a load for 10 years makes sense.
    I know many of us would like to regularly switch versions, and one of the great things about my job is that I get to do so. I’ve held onto SQL 2014 and 2016, but I’m slowly moving all my work to SQL Server 2017 and 2019, with the idea that apart from some repro situations, I won’t try to run older versions anymore.
    It seems many people are town about upgrades. Some people prefer the system they know and are comfortable with, since they know what workarounds are needed. They may bemoan the need to learn new skills and change old habits. Others prefer the latest and greatest, quickly adopting new platforms and agonizing over the spend on old systems. No matter which way you feel, it’s likely that you’ll be forced to do a bit of both as the pace of change for SQL Server means many of us will adopt new versions for new work and end up supporting 4 or 5 versions at any one time.
    Steve Jones