CosmosDB and Consistency

Posted on January 17, 2022 by way0utwest

This is part of a series on my preparation for the DP-900 exam. This is the Microsoft Azure Data Fundamentals, part of a number of certification paths. You can read various posts I’ve created as part of this learning experience.

Update: Added a few things for studying for DP-900

I was doing a little work with CosmosDB recently, and there were a few things that surprised me about the platform.  I’m also not 100% sure I completely understand these, but I’m slowly expanding my knowledge.

I had known there were various consistency levels, but hadn’t ever really looked at them in detail. There is a page on MS Docs that has some animated gifs that help explain the levels. I think the images are useful, and I’m going to try and understand what this means from the perspective of a relational person.

Strong Consistency

This means the replicas remain in sync, and the clients always see the same data, without running into uncommitted or partial writes. This is similar to what SQL Server provides by default for one database.

This also prevents multi-region writes. This is a synchronous commit across regions, essentially.

Bounded Staleness

In this case replicas might be slightly behind the one where writes have occurred. In essence, a delay. I think of this as snapshot replication, where we might set a time between updates.

CosmosDB is not using this, and it’s better about keeping things up to date with a certain number of versions or time between updates, but you set upper bounds on how stale info is. I wish we could do this with replication in SQL Server.

Essentially, this is consistency, but with a delay across regions.

Session Consistency

This one seems slightly tricky. I’m sure if I read it and watch the image 100 more times it will be easy, but this really means that within a session, I get consistency. It also looks like this is for reads and writes in the region. I believe this is the default.

I don’t quite know what this means for SQL Server. It’s likely something like async AGs? Not sure what I think here.

Consistent Prefix

This level allows some dirty reads, so it gets into that area that makes me, as a relational person, uncomfortable. At least we never see out of order writes. From the visual, this looks like there could be various delays between regions. Feels like normal replication to me.

Eventual

This is likely what many people think about with non-relational platforms, and it worries them. There’s no guarantee of write order, or when you will see the updates in any particular region.

For some problem domains, this is really bad. For many that I’ve worked in, it’s fine. As long as the delays aren’t too long, this is tolerable for many applications.

It still makes me slightly nervous.

Summary

Dealing with the nitty gritty of consistency, across many copies, and many clients, is weird. It’s complex, and hard to think about for a human. However, I am glad there are some choices, allowing applications to pick what suits them.

Posted in Blog | Tagged , , | Comments Off on CosmosDB and Consistency

It Isn’t Worth Automating

Posted on January 17, 2022 by way0utwest

Every year I’m responsible for a portion of the Database Weekly newsletters. I used to do half, but right now Kathi, Grant, and I are on a rotation. Our boss puts out a schedule each year, which tells me the weeks for which I am responsible. Each January I then set up a recurring appointment for Mon/Wed/Fri on my weeks to remind me to curate articles and write an editorial.

Each year I also think about whether or not I can automate this. Last year I spent a few hours looking into the Exchange APIs for creating appointments, at the C# code, and PoSh (which is a nightmare). I spent a few hours, got frustrated, and gave up, just manually creating appointments.

This year I took a slightly better approach and tried to figure out how long it takes to create an appointment. I got a stopwatch and went through the process. Each recurring appointment takes about 30 sec to create and (a repetitive) 15 clicks. I have 17 weeks, so this is about 9 minutes (rounding) to get this done.

That doesn’t seem worth automating. In a decade, I’d spent 90 minutes doing this. I’m not sure I could automate it in 90 minutes. I might learn a new skill, but I don’t do much calendaring, and honestly, I’m not sure I could do something I’d think would be reliable, portable to others, or even useful in other ways.

Throughout my career I’ve had a lot of tasks like this as a DBA or developer. They are often short tasks, things I do fairly rarely, and are not tightly defined. Often I just do a little manual work rather than spend too much time trying to automate things. In my life, I’ve found similar tasks, where I’m not sure the time spent building automation or the maintenance requirements over time, would be worth the effort.

I am, however, a big proponent of automation, and I think there are many places where the time spent is worth the effort. Periodically cleaning log files from servers, running data loads of know files, DevOps CI and CD processes, and more are worth the effort. Many people need these processes running often, and the work to get automation running becomes a time saver.

Not everything needs to be automated, or even should be. Certainly, when the manual time or effort requirements are small, it pays to do a little analysis. If you disrupt the way humans do their work, then be sure your solution saves time or effort. As technologists, we often reach for the scripting/programming tools to solve problems, but be sure that you include a little thought before you start coding to be sure the process is worth automating.

Steve Jones

 

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | 1 Comment

Daily Coping 17 Jan 2022

Posted on January 17, 2022 by way0utwest

I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.

Today’s tip is to get moving; do something active today.

I’m on a gym kick. My Jan 3 coping tip was to make a resolution for the month, and I picked exercise. I strained a calf muscle over New Years, but started moving again on the 4th.

I missed a day, but I’ve been doing a good job since. Mostly inside as there has been wind and some cooler weather, so I’m doing a bit more than usual today. It’s a lifting day, but I’m going to add in 30 minutes on the bike to push my heart and lungs a bit.

Posted in Blog | Tagged , , | Comments Off on Daily Coping 17 Jan 2022

A New Language

Posted on January 14, 2022 by way0utwest

In my life, I’ve mostly dealt with people that speak English. I tried Latin in high school and Japanese in university, but I wouldn’t say I learned a lot. This past year I’ve been working on Japanese (konichi wa) and French (Bonjour) on a regular basis, but I’m still fairly uni-lingual. At least compared to many of the people I’ve met in my travels that speak multiple languages. I’m mostly just trying to read a bit, and that’s quite hard.

In my career, I haven’t met many technologists that only know one programming language. Even the SQL professionals know a little C# or a scripting language. Quite a few people know a bit about many languages, and some are very competent. I used to think it would be good to master one language before moving on, but I find that trying different languages can give you an appreciation for others, or even an appreciation for your primary one. Comparing and contrasting code might even help you learn faster.

These days many of us are being asked to handle more tasks, and we may be required to support or work with code that others have written. As a result, I think it makes some sense to have a basic knowledge of other languages that you might encounter. I can often look at code in different languages and get an idea of what’s happening. I might even think about fixing simple bugs if needed.

I saw a Data Exposed edition recently with Hamish Watson that was titled KQL: The Next Query Language You Need to Learn. I had heard of it, but never used it. After watching the episode I played with it a little. While I don’t know how important it is, I do appreciate I may run into it and will likely spend a small amount of time writing some queries and familiarizing myself with some of the common aggregates and other types of structures.

Today I’m curious, which do you think you would learn as a next language? Not that you’d give up your current work in T-SQL or something else, but what else might be good for your career, or perhaps be exciting to you personally?

I’ve enjoyed doing some work in Python and PowerShell over the last year, and time spent in R before that helped me appreciate those languages. There are places I’d choose to work in those over T-SQL, especially for data not already in a SQL Server.

Take a moment and leave a comment: what new language would you like to learn and why?

Steve Jones

 

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | 4 Comments