Voice of the DBA

Tag: Azure

  • The Azure Carrot

    Microsoft wants more people to move their database workloads to Azure. They are constantly adding new features, capabilities, even tools to help people move databases to some part of Azure. It’s working well, as the latest quarterly report shows tremendous growth in Azure revenue. We don’t know precisely how SQL Server or Azure SQL databases impact this, but I certainly see more and more customers moving relational workloads to the cloud. Some to Azure, some to other providers.

    In an effort to entice people, Microsoft has a promotion that they were running in March, which was asking SQL Server 2012 customers to move to Azure and save with free extended security updates. They reiterated the promotion on July 12, when SQL Server 2012 went out of support. They calculate savings and promote benefits, which might be tangible and desirable to some of you. After all, not having a supported platform isn’t an option for some people.

    I don’t know how I feel about this. While we recently had a security update for SQL Server 2012, I know that it’s an older platform and at this point, it’s 10 years past RTM. It is probably time to upgrade to a newer version, which should be 2019 if you can’t wait and SQL Server 2022 if you can wait until the end of the year. I can’t imagine that your workload from 2012 won’t run on 2019, but it’s possible there are some issues.

    I’ve run older versions of SQL Server that weren’t supported and often didn’t worry. I knew what worked and what didn’t, and after a few years, it was unlikely that anything in the product would break. However, that was in an environment that was not regulated or certified, which wasn’t always the case. I have worked in places where we would upgrade whenever we got to the end of mainstream support as we weren’t interested in paying for extended support.

    I also think that while SQL Server is very mature, and it’s possible that you don’t need new features, you do want a platform that is secure. That means you do want support and fast action if an exploit becomes published. There is also the case that we often want developers working on modern platforms, both because they can take advantage of better language constructs, but also because this helps retention. Try hiring for an all SQL Server 2012 environment v a SQL Server 2019 one. You might find people, but a lot of the talented ones would prefer to work on a modern platform, not a decade old one.

    If you are running SQL Server 2012 (or 2008x) and considering the cloud, check out the promotion. It might help you make a decision on whether to upgrade locally or move to Azure.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Constant Azure Improvement

    This week I noticed a number of changes in the Azure platform from their change feed. Actually, every week has a crazy number of changes across the entire platform, but this week a number of these were related to data and databases. That isn’t always the case, but I saw some interesting items.

    We have zone redundancy for the general purpose tier of Azure SQL database, as well as AAD security principals in the master database and a new DMV for backups. I think Azure SQL Database is a great product, though one that has felt quite limited at times. The use of DTUs at first, the high pause period for Serverless and limited options at lower tiers were always annoying.

    The point at times seemed to be to get customers to place a big financial bet that a higher tier would meet their needs. I had hoped that some of that was just a blip in time as technology improved and more features would be deployed to all tiers. After all, the original Azure SQL database was a key-value store.

    That seems to be coming true as I continue to see improvements to the offerings. Azure SQL Serverless now has a 1 hour pause period minimum instead of the original 6 hour time. Now Cosmos DB has also lowered their autoscale offering. Cosmos DB can be provisioned at a lower entry point for autoscale, at 100RU/s instead of 400RU/s, which might get more people to experiment with a PoC over a bit of time and use the database. I know I’m thinking of trying to mess with some data on the platform. I don’t think I’d need 100 RU/s, much less 400, but I can better live the cost for a few months.

    Hyperscale has been an interesting version of Azure SQL Database. I know quite a few customers that aren’t sure if they need it, but they considered experimenting with it. The problem was that once you choose Hyperscale, you can’t go back to another tier. That’s quite a disincentive to experimentation, which seems to be part of the philosophy of choosing the cloud. There is now a preview of the migration of Hyperscale back to the general purpose tier. I’m guessing it took a little time for the Azure team to implement this, which is why there was a restriction, but it looks like that limit will go away.

    The cloud is an amazing place, as I’ve written before. It enables things to happen at a pace, and with ease that I’ve never seen in a data center without a lot of prep work. This week it seems that the Azure cloud advanced quite a bit in the area that I care about, the data platform space.

    There was also the general availability release of the Azure SQL Migration extension for ADS.

    Steve Jones

  • My Favorite Data Platform Feature

    At SQL Bits a few weeks back there was a community keynote on Friday. Ben Weissman and Rob Sewell put this together as a fun way of involving others. They invited a number of people to each take 5 minutes and talk about their favorite part of the data platform, which encompasses a wide variety of technologies and products. There were a few people who chose Purview, Azure networking, and quite a few ways of using Power BI.

    Me? I choose the Create button in Azure. Really, Azure itself because of the amazing array of things that you can deploy in minutes. It’s stunning to me to think back to the mid-2000s, when I first saw the Azure platform at TechEd and a Microsoft employee demo’d a key-value table, literally a two-column table. They were trying to sell this as a great way to deploy lookup data to mobile sales applications at scale. I was less than impressed.

    This year I deployed a Synapse workspace in less than 10 minutes. Maybe less than 5. I know little about Synapse, and the idea of building a data warehouse and analytics platform, loading data, and running queries isn’t something I’d think I could do in 10 minutes, but I did it. I’ve found containers to be similarly useful in quickly getting something up and running, but even they require Docker or some other software installed and configured. In Azure, I can set things up in minutes.

    In the last year I’ve set up numerable resources, from databases to web apps to a Kubernetes install. All in minutes, which is quicker than I could do something on my local system except for creating databases. If I had prepped things or had more knowledge that might not be the case, but the lack of needing to prep or learn a lot to get something set up is amazing in and of itself. This is especially true when I want to experiment with a technology or a new solution. I can quickly build an array of systems with much less effort than on-premises.

    I don’t know how many of you use the cloud at work, but I find it amazing. It’s one part of the data platform from Microsoft that I really appreciate, especially as I’ve tried to work with MongoDB, Redis, and other technologies at home. The cloud makes things easier, even with databases.

    Today, I’m wondering what your favorite part of the data platform might be. Is it something in SQL Server? Maybe a related technology? Perhaps something else that helps you work with data these days. Let us know what data platform thing is exciting to your career.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • ETL v ELT

    This is part of a series on my preparation for the DP-900 exam. This is the Microsoft Azure Data Fundamentals, part of a number of certification paths. You can read various posts I’ve created as part of this learning experience.

    I don’t have an ELT tag, and I’m not likely to make one. I tend to think of ETL as loading data somewhere, even though I know it means more.

    The important concepts for DP-900 here are that ELT is becoming more important and you need to understand what this means. I’ll cover these concepts, but also give a little overlap with where the different Azure services fit in with this.

    ETL

    For most of my career, the pattern for loading data was Extract-Transform-Load. In this pattern we:

    • grab data from a source
    • make changes to clean/change/etc.
    • write to a target (or sink)

    It’s how tools like SSIS work. They connect a source to a target and have a bunch of tasks or transforms in the middle that change the data in some way.

    This is a good pattern for getting the work done when the target system is just built for querying data, such as a data warehouse. It is also good when you need to scrub some data, perhaps for privacy reasons.

    This isn’t a good pattern when you are trying to load data quickly as the transform process takes time.

    ELT

    This is the new way of doing things. I this patter we Extract-Load-Transform, though really, it’s not a pattern that quite makes sense in that the process of moving the data just moves it.

    Here we:

    • grab data from a source
    • write it to a target

    Where’s the transform? Well, that happens on the target, often when someone queries the data. Modern analytic systems, like Snowflake and Synapse, can work with vast quantities of data, often stored in a data lake or blob system, and consume that with powerful computational capabilities. There could be some minor re-shaping of the data on write, but that’s not the idea.

    This is good when you might not read all the data. Why process (transform) what isn’t being read. Before you complain that you should know what is used, none of us know if all our data is being used. Unless we write crappy SELECT * code with no WHERE clauses.

    This is also good when we need to work at speed and privacy isn’t a concern. It’s great for the known formats of files sent to us, as the target system can project a table on top of a ser of files.

    ELT seems to be the current future direction of many analytical and warehouse systems.