The cloud has been a controversial concept for much of its existence. While the idea has been around for many decades, AWS started selling IT services in 2006, with Azure following suit in 2008. Since then, the use of cloud services has grown tremendously. While some applications and organizations have embraced the idea from the beginning. I found many of you at SQL Server Central were very hesitant at first. I guess some of you are still skeptical about the value of a production database in a public cloud.
From the beginning, I’ve felt that cloud computing has a place in the world, but in a way that is more appropriate for some situations than others. In terms of database (and maybe compute services), if you have a very well-known and predictable workload, the cloud can be very expensive. It might still be a good choice, but I think it often isn’t. If you have a variable or growing workload, then the cloud might serve you better than trying to keep up with new hardware in your own data center.
Bluesky has had a tremendous amount of growth since its founding. Twitter invested in this as a distributed project and when Elon Musk purchased the site, many users moved away. A lot of them went to Bluesky, which had to deal with a quickly changing workload. They started in AWS, but eventually decided to move to an on-premises setup.
Why? One would think their continued growth would mean AWS (or another cloud) would be a natural fit. However, they hired someone that provided an analysis showing they could invest in their own hardware, overprovision what they needed for growth, and keep up with the demands as they had developed a fairly accurate method of forecasting future needs. The savings in purchasing their own hardware allowed them to buy more than they needed and handle short-term spikes.
To be clear, this doesn’t mean the cloud is worse for most or even many organizations. Bluesky knows they need to continue to invest in hardware, and they are prepared to keep adding resources. They also architected a distributed system that still allows them to scale into AWS if needed in the short term. I don’t know many organizations that would prioritize those things alongside the rest of their business. Most of us do a poor job of forecasting load. Even if we do, often the difficulties of purchasing new resources mean that we can struggle to meet increased demands.
The companies that have moved to the cloud with success, and those that have left the cloud with success, are those that measure, monitor, and make appropriate decisions based on operational data, not opinions and feelings. They aren’t afraid to make a decision one way or the other, choosing what’s best for the organization, not what someone wants to do or thinks will be better.
The cloud might be better for you, or it might be worse, but you ought to have a way to measure and analyze the options. You also need a talented staff that isn’t afraid to try new things and adapt their architecture to take advantage of modern hardware and software. Too many of us aren’t as flexible as Bluesky and might not have the success they have, in or out of the cloud.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Voice of the DBA 