Voice of the DBA

Author: way0utwest

The Journey to DevOps

Netflix is one of those companies that I find amazing. I was an early subscriber to their mail DVD service, and thought it was amazing how they processed both orders and physical objects with technology. As they pivoted to streaming, I continued to be impressed with their technology growth, from the chaos monkey to their DevOps deployments. They have been an organization often looked to as a model for other technology companies.

I certainly think there are things to learn from Netflix, as they’ve scaled and build quite a resilient system. They aren’t necessarily worth copying, however, as the problem domain they solve is both narrow and also quite different than that many of us work in. If someone can’t watch a movie, it’s annoying, but they can pick another one. If a customer can’t transfer money, communicate with another user in an app, or schedule a ride, it’s a bigger deal for other problem domains.

Still, Netflix didn’t build this system overnight. They didn’t come up with amazing DevOps techniques for building and deploying their software from the beginning. It’s been a journey, and they talk about some of the full cycle developer challenges in a recent blog post. This looks at one team’s journey across 6 years, from 2012 to this year. The piece is an interesting read, and it’s not advocating for their particular approach, but rather trying to explain the value that they received from moving to a DevOps model, where they have a group that must run what they build.

As I try to help customers and clients move to Database DevOps models, I see lots of similarities to what’s in this post. As we look to optimize the entire software development life cycle, this requires a changing of roles and closer cooperation between groups. As I look at the evolution of Netflix, using a centralized group to build tools and then ensuring you have a better staffed development team that works to both build and support their software, ensuring clients get the value (or features) they need quickly.

I’ve worked in orgs that did this in groups, and for the developers to be involved in operations is an eye opening experience. Developers will learn ensure they think about their design and test more because they don’t like getting woken up. They listen Operations and learn about the ways in which they can better build a system that works. They start to realize “works” is a feature, perhaps the most important one.

I’ve also found my role as a DBA can facilitate DevOps. I’m often between development and Operations, with a foot in both camps. I help developers build better tools and techniques to work with databases. I understand the impact on production databases, meaning I can help ensure that deployments are smoother, and we avoid risky changes. We build indexes early, install primary keys, test queries for performance, and more. The DBA is the one of the ways in which DevOps can grow in an organization, if they work with the developers instead of against them.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Libsyn.

June 5, 2018
Prepping for Mastering Index Tuning with Brent Ozar Unlimited
I got the chance to sign up for a few courses from the Brent Ozar Unlimited team and took it. I struggle to find time for training, between work commitments, travel, and family time, it’s hard to fit in a day or two. I’ve almost given up on going to in-person classes mostly because of the time involved.

I’m trying this online, both to learn something and see how the experience is. I’ve watched Brent present many times, and I think he does a good job of explaining concepts. While I suspect there’s a good amount of things in this course that I’ll already know, my experience is that it’s both good to get reminded of concepts from a different point of view as well as get some tips/tricks that I might not know. I’m always amazed how I might learn one small thing in a beginner session that I never knew.

Signup

The signup is simple. Go to the class page and add this to your cart and pay for it. The process is simple, and you get a good agenda as well as a schedule. I really appreciate the schedule, including the breakdown into what’s covered during different blocks.

There are multiple dates/times to choose from, but this is online during the workday, Eastern US time, so plan accordingly. For me, this means a slightly early morning, but since I can turn up the speakers and walk to the coffee machine, this works out well.

There are other classes on their home page, including Database DevOps, PowerShell, SSIS, and Data Science. There are plenty of more mainstream DBA/developer type SQL Server classes as well.

Prep

A couple weeks before the class, I got one of those automated emails from GoToWebinar. I’m used to receiving 3-4 of these before an event, so I promptly ignored it.

Then I got a second one a few days later, and I decided to look at this one. I managed to actually read this section:

“To get ready, go here:
https://BrentOzar.com/xxxxxx
And read the “0.1 Before the Class” module. There’s a ton of prep work and instructions in there for you, including a few videos to watch to prep, and you can download the slides to follow along. ”

I removed the URL, since I’m not sure this is public, but in any case, I clicked the link to find a description of the class, in usual well-written Brent style. There was also a series of links on the side, for various slides and labs.

The Before the Class is a page with 4 videos from Brent that are supposed to prep you and give you some background. It’s 69 minutes, so I set aside some time and got moving.

And I ran out of time. I watched 3 of the videos, but ended up not getting the fourth one done before the class because of family and travel commitments. I should have tried to watch more in short periods, but I kept putting things off. I know a lot of this stuff, but if you’re new to the class and want to learn this, spend the time to get some basics down.

Testing Equipment

I didn’t bother, mostly because I use GoToWebinar every few weeks, between my work or watching other things. As a result, I know my PC works. If you don’t do that, make sure you set things up.

Contact and Notifications

I went back and looked. I got these notifications:
- A confirmation of the purchase
- A 2 week notices that asked me to go to the class page, and also get started on Slack.
- A 2 week (minus 1 day) email that asked me to read the “0.1 Before the Class” module.
- Two 4 day notices, a day apart (one was a 3 day notice). Not sure if this was correct, but these were roughly the same.
- An email the day before the class with a VM URL and credentials
- A “start in 1 hour” notice
That might seem like a lot, but I think this is pretty good. I might miss one of the advance notices in a busy world, so it’s nice to get the second one. I could glance at it and delete it if I had read the previous one.

There’s a good amount of prep stuff out there, and Brent has each module of his class recorded, so I could have gotten ahead and watched some of the class ahead of time. I could have also played with labs a bit, though he warns you IN CAPS to be careful about not breaking some things.

Overall

The process before the class was well done. There is a good amount to help me prepare, but I won’t know until I get to the class if the prep was good enough.

Looking forward to starting. Actually, but the time this publishes, I might have started, so we’ll see.
June 5, 2018
A Broken Linux VM

I have no idea if this is the best thing, but I went to boot up a Linux VM today and got this:

My Linux skills are rusty, but this looks like a disk error. I rebooted and found this:

A search on Stack showed this page: https://askubuntu.com/questions/697190/fsck-error-on-boot-dev-sda6-unexpected-inconsistency-run-fsck-manually

I ran the utility, and answered yes to about 20-30 items. I have no idea what to do, but I figured this was worth a try. Once this completed, I rebooted again.

The boot sequence in text took awhile, with a “start job” running about about 2 minutes on the disk device. While 2 minutes isn’t that long, it’s an eternity when you are hoping something works.

Eventually it completed and I found this:

Good news, as I wasn’t looking forward to rebuilding the machine. It’s just a lab machine, but I need to check something that day, and so I needed it. While installing Ubuntu and SQL Server is fairly quick, it’s still time.

Fortunately things appear to be working and I can connect from SSMS to the server, so I’m guessing things are mostly working. Fingers crossed.

June 4, 2018
Will Coding Be Less Important?

How many of you can read Greek? I can decipher the alphabet and pronounce words, but that’s about it. I thought about this after running across an essay that says that coding will eventually be as important to most people as ancient Greek, which is to say, hardly important at all. The author seems to suggest that eventually we’ll tell systems what we want and they’ll build algorithms. I guess that’s what we do in code, but the impression I have from the piece is that we’ll use natural language to inform some AI system that will actually produce the code.

Possibly, but let’s set that aside for a minute. Assume that we could have an AI that removed the need for us to code the corporate CRUD systems or most of the websites we have. Could such a system be adapted to manage data?

Possibly, and maybe more consistently than we can. Certainly plugging in RTO/RPO values and HA needs might allow an AI like system to schedule backups, build up or tear down new nodes in a cluster or Availability Group, and certainly respond to DBCC errors, reindex systems, and more. I certainly could see that we could request a system to do X and lots of the background details would be handled.

There are some other thoughts about this piece that I tend to agree with. The ability to easily specify the way in which a system can work, for the average or even above average user, isn’t coming soon, if ever. The thing is, I’ll take solace in something that I truly believe.

Writing the code isn’t the hard part.

Sure, syntax can be challenging, and working out the mechanics of how modules interact, and building MAKE-like scripts, and deploying changes are difficult. We forget about the differences in environments and configuration, and I bet better static analysis and AI might help here. The thing that’s really hard, that is likely to still exist even if we specify the way something works is that it’s really, really hard to be specific.

We struggle to express the exact way that systems should work. We struggle sometimes to communicate what “a discount for more than 10 pieces” means. Is that a discount at 10? At 11? what if there are 20 ordered and 10 returned? Do we remove the discount or pro-rate the return? We can solve the issue, but so often, we don’t know there’s an issue until the system gets into production. Changes in rules/regulations? We don’t often know about them, much less know how to express those changes in plain language.

Even if we could get by without writing code, the expression of “how” something works likely requires a lot of detail, a meticulous person, and endless patience to get the machine to understand. Something many developers do now. I suspect the same thing would be necessary for database development. Even if we could specify the result set, would we really trust the machine to query the source and not come up with multiple ways to get this result set, which might not be valid for another?

Call my cynical, but coding isn’t going away, even if the syntax changes to English (or some other spoken tongue).

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 5.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

June 4, 2018