I think the people that run StackOverflow are pretty sharp. They’ve built a well performing site, they’ve worked on some useful open source projects, and they think through their projects. They still have issues with upgrades.
Taryn Pratt, from Stack Exchange, has written a few nice posts about her experiences upgrading the Stack Overflow databases. The first one was last year, where their multiple server AG environment upgraded from SQL Server 2012 to SQL Server 2017. Just recently she published another post on their Windows Server upgrade, from 2012 to 2016.
If you’ve never done a complex upgrade, these posts are worth reading. Don’t second guess Taryn, but rather, just read as if you are following along. You have hindsight now, but in the middle of planning this, you will learn about things that might cause you issues. In the Windows upgrade, one of the interesting issues is a VM vs. physical machine issue with drivers. To me, this might be one reason to never bother with anything other than a VM, even if it’s the only one on the machine. A lightweight version of Hyper-V or Xen doesn’t eat much in the way of resources, but can provide some separation from these issues.
One other thing to note is that you really need your runbook. I constantly see people asking for a checklist for how to upgrade, and there are good general steps to follow, but your environment likely needs to have a custom runbook that covers your situation. The Stack environment is complex, but even I was surprised with 35 pages of steps and notes.
As with most plans, this one had issues when it was finally implemented. I think Mike Tyson sums it up nicely, which is why you practice your move. It’s also why we can’t necessarily upgrade every year. There’s a reason many companies still have old versions in production (Thanks, Brent Ozar).
Plan, practice, test, repeat again, and then be prepared to think on your feet. That’s if you upgrade your systems. It’s time consuming and expensive, and I can see why a lot of companies have looked at cloud services, like Managed Instances. Reducing the time and cost to change your OS and/or SQL version is something we should all be thinking about.