Voice of the DBA

Author: way0utwest

  • Upgrades are Hard

    I think the people that run StackOverflow are pretty sharp. They’ve built a well performing site, they’ve worked on some useful open source projects, and they think through their projects. They still have issues with upgrades.

    Taryn Pratt, from Stack Exchange, has written a few nice posts about her experiences upgrading the Stack Overflow databases. The first one was last year, where their multiple server AG environment upgraded from SQL Server 2012 to SQL Server 2017. Just recently she published another post on their Windows Server upgrade, from 2012 to 2016.

    If you’ve never done a complex upgrade, these posts are worth reading. Don’t second guess Taryn, but rather, just read as if you are following along. You have hindsight now, but in the middle of planning this, you will learn about things that might cause you issues. In the Windows upgrade, one of the interesting issues is a VM vs. physical machine issue with drivers. To me, this might be one reason to never bother with anything other than a VM, even if it’s the only one on the machine. A lightweight version of Hyper-V or Xen doesn’t eat much in the way of resources, but can provide some separation from these issues.

    One other thing to note is that you really need your runbook. I constantly see people asking for a checklist for how to upgrade, and there are good general steps to follow, but your environment likely needs to have a custom runbook that covers your situation. The Stack environment is complex, but even I was surprised with 35 pages of steps and notes.

    As with most plans, this one had issues when it was finally implemented. I think Mike Tyson sums it up nicely, which is why you practice your move. It’s also why we can’t necessarily upgrade every year. There’s a reason many companies still have old versions in production (Thanks, Brent Ozar).

    Plan, practice, test, repeat again, and then be prepared to think on your feet. That’s if you upgrade your systems. It’s time consuming and expensive, and I can see why a lot of companies have looked at cloud services, like Managed Instances. Reducing the time and cost to change your OS and/or SQL version is something we should all be thinking about.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Production Kubernetes

    Still learning more about Kubernetes. I’ve been working through the 50 days of Kubernetes (K8s). Let’s keep exploring

    The API is the security boundary

    Everything happens with the Kubernetes API. Therefore, we want some RBAC here for the roles of individuals, as well as processes, like CI/CD pipelines. We want to ensure we have validation, security scans, etc..

    Things will go wrong

    In addition, we need some monitoring on the cluster. We need this on any system, but with scale and complexity of containers in Kubernetes, we need something in place.

    In addition, you want to test failover, which is likely between clusters. Containers do make this easier, but it’s something we want to ensure is a part of our process. This means practice. We know practice matters in most places, but we often don’t practice well in IT.

    Scaling

    There is planning to think about a large application, or maybe large for your environment means thinking about global access and ensuring that you have a way for your app to easily redirect across clusters or regions. There are also data challenges, as Brendon notes, with things like databases. It’s an opportunity to push CosmosDB, but in the relational world, we know how to handle this.

    Kubernetes will make AGs and HA easier.

  • Salaries in Job Ads

    Hiring at Basecamp is interesting. I caught this interview and think it’s a good look at a process most of us have gone through, but perhaps not thought about. Maybe we ought to try and not start out our relationship as employer/employee with a “I got one over on you”.

    Of course, many companies are not interested in treating employees well and just assume they’ll hire/fire/turnover often. Makes sense in some places (McDonalds type work), but less sense in our industry.

    I do think that we hide salaries too much in our industry. Certainly this aids in favoritism and entitlement for works with seniority. I’d really prefer to see salary ranges published for jobs, both internally and in the description. And not some huge range, but put something like a junior DBA from US$35,000-50,000. Or whatever range you think is appropriate.

    This would do a few things for me. First, it ensures that candidates that want to work for that range will apply. It also ensures you don’t waste time with a developer that wants to move to DBA work, but really needs $70,000 for their financial commitments. This also helps you determine if you’re range is correct for the market.

    I am a big proponent of disclosing more info up front and finding a good match between employer and employee.

  • The 2019 Home Lab

    Many of us won’t grow and expand our career completely at our jobs. While some employers will provide training in the form of funds to buy books, take a class, or learn in some other way. I think most employers will understand that you are learning at work and need time to experiment and practice some skills. That isn’t necessarily going to be enough time for you to learn and practice everything at work. Many of us end up doing some normal work at home, and we’ll likely need to do some career growth at home as well.

    That means some sort of resources that are available to work with the data platform. These days I’d recommend a lab of some sort in Azure or AWS, but I know many of you are worried about costs and uncomfortable with building a lab on a service that might require a monthly charge. I think you can easily get away with a minimal or zero monthly charge, but I do understand your concerns.

    That means a home lab, and I wanted to ask this week what you have and need in your lab. I’ve asked in the past, but it’s 2019, so let’s revisit the question. I wrote about this at Quora, but what would you do today if you were starting? Or what is your current lab?

    For me, I have a desktop machine at home that really powers my lab. I run VMWare, but if I were more cost conscious, I’d just use Hyper-V or Virtual Box for full machine work. I think containers are the future and I’d suggest you start learning about them, but for now, have a virtualization method of setting up a machine and installing SQL Server. There are evaluation versions you can run for Windows, but certainly you can also just install SQL Server on your system. Developer edition is free, so no excuses here.

    While some might want complex setups for HA, DR, and other testing, for most of us, a developer edition, a little disk space, and some time will provide a lab where we can get things done. There are lots of Stairway series here at SQLServerCentral, tons of articles here and at Simple Talk that you can follow along with, and many questions in our forums that you might read the first post and try to answer on your own.

    If you have a lab, let us know how you set it up today. If you don’t, maybe this is a good weekend to start thinking about how to set one up.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.