Voice of the DBA

Category: Editorial

  • SQL Server vNext

    We now know when the next version of SQL Server is coming. At Ignite last week, Microsoft announced SQL Server 2022, which comes nearly 3 years after the last version, SQL Server 2019. Apparently, the pandemic’s effects include delays in product development as it’s been nearly three years since SQL Server 2019. The new version is in preview, with a few new features announced. There’s also a video version from the Ignite site, but if you want a quick summary of things, read on.

    There were a few items that get me a bit excited about the changes. The first is that we get some query improvements with regard to parameter sniffing in stored procedures. This has been a constant problem for many databases and workloads for years. This is because only one plan has been kept in the cache. That changes in SQL Server 2022, as the instance can now keep multiple plans around. It will be interesting to see if this helps many customers.

    The Query Store was an interesting edition to SQL Server and plenty of people have found it valuable. However, its use was optional and you had to enable it. In SQL Server 2022, Query Store is now on by default and there is support for read replicas as well. This change is there to enhance the intelligent query processing (IQP) work that has been growing in each new version. This new version gets MAXDOP and Cardinality Estimator being incorporated into the feedback loop. IQP isn’t perfect, and it might cause you some issues with certain workloads, but many of the changes made in the last few versions do seem to be helping customers.

    There was a quick note in the Ignite video about multi-write replication is now available. Not true peer to peer, but this version should automate the last write wins rule, based on UTC timing. I’ve been skeptical of using peer-to-peer replication in the past, mostly because of the need to code conflict scenarios, but a last-writer wins scenario might work well for some applications. If you can accept this rule, and you have good resources for propagating data, this might be something you can use.

    Blockchain is coming to SQL Server. There is a Ledger feature that provides an immutable ledger of data changes. This ensures that data integrity is maintained with full auditability. Trusted storage is needed to ensure this meets all audit and regulatory requirements, though if the data has been tampered with, you might only know something happened. It’s not perfect, but it does provide the ability to prove there is trust in the data you are watching. I suspect this will be a feature like TDE that auditors want enabled, even if it isn’t perfect. I do wonder what this will do to storage costs and complexity.

    Azure Purview is being integrated into SQL Server 2022. The big takeaway here is that you can scan on-premises data for free. Now I don’t know if this means you can use your data for free in any way, but I do think that this does start to make governance easier. I think that’s a big deal for all of us in the future if companies were to better know what data is risky and work on protecting it. Normalizing the way we look at data governance is a big step forward. This certainly is a long way from most applications using the “sa” account.

    One of the challenges I’ve seen in many companies is dealing with reports. Often companies try to run complex queries on their OLTP system, or they spend a lot of time and effort to build ETL structures. Synapse Link provides automatic change feeds from a SQL Server to a SQL pool in Synapse. This is like an automatic replication of the table(s) into your data warehouse. It’s been available for CosmosDB, but now it comes to SQL Server. Given the cost of development and managing ETL jobs, I could see some organizations upgrading to SQL Server 2022 just for this feature. If it works well and quickly.

    Disaster Recovery (DR) is something I’ve always been concerned about. I’ve had too many issues over time to treat this lightly. In this new version, you can set Azure SQL Managed Instance as a target for a DR recovery, with failover to the cloud through a distributed availability group. That’s has been possible before, though now this seems easier than ever (as it should be). The failover is a good feature, but a lot of us wouldn’t necessarily want to live in the cloud, so the big announcement here is that you can restore the backup from MI back to an on-premises SQL Server 2022 instance. We can restore back from the cloud to our own systems, which has been a hassle for a long time.

    Now, this isn’t failback. I have no idea if logs can be backed up and restored, or if this means a lot of downtime to leave the cloud if you have a big database. From the demo’s, it doesn’t look practical to backup and restore a database of any size to fail back. My guess is this is really a first cut, but the ability to move from MI back on-premises is a huge step forward. If for no other reason than you can backup things for dev work inside your organization. That’s a big win for a lot of customers I know, at least it is if Azure SQL DB and Azure SQL MI are close in functionality from the inside-the-database perspective.

    There are other enhancements as well. The cloud offerings from Microsoft gain some scale, which is good. Most of us don’t have huge systems, but more scale usually trickles down with better pricing for the mid levels. Azure Arc grows, allowing you to run Azure SQL MI and Hyperscale PostgreSQL on your own hardware. You’ll pay something to Microsoft, but having these systems stand up, keeping data local, and having them managed easily is a big deal. I expect that more companies will consider things like Azure SQL DB on their own hardware over time, especially as standing up and managing a Kubernetes cluster becomes easier. This feels like the early days before vSphere when we struggled to manage lots of virtual machines. I suspect we will find better ways to deploy and manage container orchestrators on-premises over time.

    It feels like a long time since we talked about SQL Server. Certainly the last 18 months have felt like the world is frozen and not much has happened. I know I’ve done some things, but it’s been a strange feeling of suspension for much of life. For me, the announcement of a new version of SQL Server brings some excitement. It feels as though the world is moving forward, and I’m looking forward to learning and writing about the future. I’m also hoping to get the chance to deliver some presentations on something new in SQL Server in 2022.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • What I Learned at Community Summits

    I went to the first  Community Summit in 1999. This was one of the first conferences of my career, and I was excited to have one focused on SQL Server and data. I met Kalen Delaney and shook her hand after having her educate me about how tempdb worked in SQL Server 7. That event started a 20-year journey of traveling to subsequent Summits nearly every year since. Over the years, I have a lot of memories from Summits, mostly of people that I’ve met and talked with. Much of the time I’ve been at any of the Summits was spent on networking and business purposes more than learning, but I have picked up a few things along the way.

    Early on Brian Knight and I delivered a presentation in a debate format. We were talking about the pros and cons of using identities. It was an enjoyable session and one where we learned a few things. A developer from Microsoft was in the audience and helped clarify some of the specific numbers related to limitations and scaling issues with using the feature.

    When SQL Server 2005 was ready for release, I still remember going to watch Donald Farmer talk about Integration Services, a radical change from the way that Data Transformation Services used to work. It was quite an eye opening session for me.

    Early in the time when I was working with DevOps and databases, I watched a talk on how you might apply some of the techniques with Reporting Services to implement CI and ensure your report changes would actually work correctly. It was a hack, but I think that a number of the technologies that Microsoft has added to SQL Server aren’t designed with version control and automated code review, which is a shame. However, that’s one of the great things about Summits. You can learn how someone else solved a problem.

    There are many other memories from the event where someone taught me a bit more about SQL Server. Being in the same place as so many smart, talented, inspiring data professionals was magical. That was one of the big reasons that my boss wanted to see the conference continue. Nowhere else in the world brings so many people together.

    You can still register for the event next week. It’s virtual this year, and completely free. No cost, and even if you get busy next week with work, you have access to the content after the conference, so register today. It’s a few minutes, and you’ll get the chance to learn from some amazing speakers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Security Bug or Handy Feature

    There are plenty of times that I want to share something with another person. This could be a link, a slice of data, or maybe the view of some page I’ve seen. Many applications make this easy, either on the Internet, or inside a company in Slack or Teams.

    However, there are lots of times that I might want to share some data publicly, allowing anyone to get to it. That’s a place where no shortage of data breaches have taken place, a problem we still grapple with today. It’s not a “make-a-decision-once” problem either. I could set up some data for access to specific people using permissions. Then someone later changes to anonymous public access, mistaking some dataset that is sensitive for one that can be exposed. Likewise, I could have a dataset that I have set to be publicly available change, with sensitive data added later. The individuals adding the data might not be aware that this particular set is being shared without any controls.

    I ran across this piece about PowerApps, noting that datasets can be configured for anonymous access if a list doesn’t have table permissions enabled. That’s an issue, as often enabling something looks like work. Many people often seek to avoid work and just complete a task as quickly and easily as possible. In many cases, they may not bother with enabling table permissions. Fortunately Microsoft later enabled permissions by default, but there are still cases of data owners exposing sensitive data.

    Misconfiguring access is a big problem overall, and the best solution I see is to ensure data is classified and tagged. We can then build applications that have policies set on the way they handle data. If data were classified as PII or sensitive, then an app (like Power Apps), could refuse to allow anonymous access to be configured. Classifying data, however, is a tedious, boring, awful job. Even if this is easy to do across time, it’s not a job anyone wants. However, this is part of the data lifecycle, and I don’t know that we will get better data security and limit the exposure of data until we have a way to easily classify and tag data that allows applications to make decisions on how to use this data.

    My employer makes a product to help here, and I’ve been pleasantly surprised that more and more customers are looking at doing this. However, we then need to ensure that applications can use these classifications  in an actionable way to protect data.

    Microsoft has talked about enhancing that the TDS protocol and their software to read and use read classification data, which is lightly gathered in SQL Server. They have a Purview product, which also helps, but the best solution, in my mind, is an open API. This would allow for connections to any classification service. Developers and admins could submit classifications for database structures or even files in real time (and hopefully programmatically). Applications could access this data and then determine what controls should be applied. Ideally, they would also refuse access if data wasn’t classified.

    I don’t know that we’ll get here, but I do see some movement from a variety of vendors here, usually limited to the database space. Hopefully this will continue to grow over time and prevent some of the silly data breaches we have where someone mis-configures a data store and allows anyone to access it.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Burning Out

    I haven’t burned out, but I’ve been close. There was a time a few years back when I was working a lot, traveling a lot, trying to watch activities with my kids, making time for my wife, and more. While I have had stretches were I feel like that (like the last month), usually I have calm periods that balance my out. Across about 18 months, I realized I hadn’t gotten enough breaks and worked with my boss to lighten the load.

    During the last 18 months under the pandemic, I’ve had a few busy times, and some stress, but overall, I’ve managed to cope well. My coping tips have helped, and I’ve made a conscious effort to demand less of myself and slow down. It helps that I work for a UK company that values balance and tries to ensure employees work hard, but not too hard. While I  sometimes struggle and get very busy, I do get to balance that out and relax at times.

    That hasn’t been the case for everyone this past year. There is a blog about developer burnout, noting that a lot of developers shouldered a larger workload during the pandemic. I think the move to remote work is stressful, and it can be hard to adjust to expectations, or know what we should expect. There is often a feeling when you work remotely that you need to do more, because no one else can see how hard you are working. There is also a temptation to take some of the time that you used to spend commuting and get a few “extra” things done each day.

    Many managers also haven’t known how to adapt to remote work and can add to stress with either more work, higher expectations, or a lack of awareness of how employees feel. They may unknowingly, or purposefully, make things worse. Add to all of this the challenges of managing kids, juggling noise in your workspace, and finding space to work with others in the house. It’s no wonder many people felt some burnout in the last year.

    The article gives some stages of burnout, a way to measure yourself, and some strategies for coping. While a lot of these may feel like common sense, when we are overwhelmed and busy, we often forget common sense. It’s easy to dismiss the effectiveness of simple strategies. It’s also easy to underestimate just how much better you will feel by taking even small steps to combat burnout. Even things you think might make a tiny difference can help.

    I know there are some very poor work environments and awful managers. If you are in these situations, I’d urge you to look hard to find a new position somewhere, anywhere. Even another bad job will give you a change of scenery and can help in the short term. Even in the poor jobs, however, I’ve often had others that felt the same way in my team or in a related team. Talking, sharing, and bonding with others in a similar situation can help, if for no other reason than you can share, vent, and empathize with each other.

    Burnout is a real problem among technology workers. It’s not a personal failure, but it is something that you have to recognize in your situation. It is also important to make changes and find ways to reduce the stress of your situation over time. There aren’t usually quick fixes, but there are ways to change your life to better handle the situation.

    If you are struggling, please reach out to friends, family, or others and get help. Things can get better.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.