Voice of the DBA

Tag: Cloud Computing

  • Things I’ve Learned About the Cloud

    I do a little cloud work, but mostly I end up working with customers that are trying to build and migrate their systems to the cloud. The transition to cloud-first took place quicker than I expected, though certainly the pandemic of the last 3 years has continued to accelerate the moves by many organizations. While not all systems will move, for better or worse, many organizations consider the cloud for everything and then decide to accept or reject the notion for individual cases. That’s the cloud-first world I see.

    At a recent event, I had numerous people telling me that their groups were mandated to move many systems to the cloud in a lift-and-shift fashion, and many of the technologists weren’t happy. That’s a conversation for another day, but listening to them and others has taught me a few things. Some of these are experiences I’ve learned myself, and some are things from others, nuggets gained from their hard experiences.

    First, lift-and-shift appears to be a way to get started in the cloud. Some people find this to be an end-goal when they can get systems running without ever worrying about hardware or facilities. While costs can be higher, flexibility and tax considerations can make this worth the effort. Others see this as a first step to moving towards Platform-as-a-Service (PaaS) products and rewriting software, and life-and-shift is a good way to approach this process, but that leads me to the second nugget.

    Everything takes longer to get settled in the cloud than you expect. Not that the cloud is slower to build and use resources. The opposite is certainly true as provisioning anything is fast, and upgrades (or downgrades) are incredibly flexible and quick. However, making the move, getting systems running to your satisfaction, decommissioning old systems, and more take longer. It seems most life-and-shift projects end up taking much longer than estimated, and not all of this is the massive data transfers needed to move data assets. Often it’s humans that can slow the process, whether through debate on decisions or just getting comfortable with cloud resources through testing. Things take longer.

    Maybe the last big thing that I’ve seen affect many companies is that we have to put better software development techniques and infrastructure configuration in place. The connection to resources is more complex and tenuous than it is on a LAN. We need to better architect software, really architect it as we were told to do so early in our careers with better error handling and retry logic. The cloud is very secure when you set it up right, but you no longer have the castle with high walls that you grew used to in a data center. Instead, you have a series of people and systems that are disparately connected and you need to ensure you protect each one along with the connections between them. That requires lots of configuration work and standards, and likely Infrastructure-as-code. Plus there are new tools to learn and the habit to build of using command line interfaces.

    I love the cloud. In many ways, I prefer it over building and managing things on premise. Not for everything, and not for things where I might lose productivity with slow connections. I certainly don’t want developer laptops or workstations in the cloud, but having other systems there can remove a lot of the hassles of managing assets that we purchase, track, upgrade, etc. I don’t know I’d ever want to buy another physical server in a company I owned, but I also know that at some scales, it might still make sense.

    The other thing I’ve learned about the cloud is there are a lot of unknown unknowns, as well as plenty of known unknowns. Tackling the cloud require a staff that wants to grow and learn with the platform, as well as one that knows when to use the resources they know best and don’t add complexity or novelty just because someone wants to try something new. The cloud is worth considering, especially as it seems physical offices and data centers are becoming rarer and rarer in many organizations.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Cloud Databases

    Most of us are used to a database that lives on a server somewhere. It might be in our data center or a VM that exists somewhere, but it’s really an on-premises type of infrastructure. Even if the VM is in AWS or Azure, this is a single system on a server that we control. We can add HA capabilities to this system, but the model is the same as if the database were on our development workstation.

    Note: this doesn’t matter if this is an RDBMS like SQL Server or PostgreSQL or a NoSQL type system, such as MongoDB or Neo4J.

    That’s a comfortable system, and it’s how we’ve built many applications over the years that support our business processes. Many of us default to architecting and thinking about our applications with this model.

    These days there are cloud databases, which can be used in this model, but which also have other capabilities. I ran across a piece that noted cloud databases ought to be a part of our modern tech stack, and I tend to agree. These days the need to be more flexible, available, and secure are important for applications. An on-premises database might not meet these needs.

    A cloud database is typically a system designed to be more of a PaaS service, with replication, built-in HA, scaling, and more. Azure SQL Database might be considered cloud-native, but I think it lacks some of the features that we might take advantage of in CosmosDB, Couchbase, or CockroachDB. Things like the availability from various types of connections, quick and easy failover and sync from different regions around the world, and low latency for users anywhere.

    A few examples are in the article, and I see these types of requirements coming about more and more often as our clients start connecting from different devices in different locales, and at scales that are harder to handle with a single on-premises (or clustered) database system. While you might be able to handle the workload, is it worth it for you to manage some of the infrastructure and administration that a cloud vendor can do instead?

    That is why I think you ought to learn about and understand where a cloud database might be useful. While not all of us have clients all over the world needing real-time access from various devices and applications, we might grow into that need. Whether it’s clients that are employees or customers, there are increasing demands for flexible ways to handle the growing workloads on our databases. You want to be prepared for when you might exceed the capabilities of the platform you’re used to,  that way you can decide when adopting a cloud database might make sense.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • New Cloud Licensing Limits

    Licensing is always a complex discussion with SQL Server. Depending on the way you run it, the edition, the version, whether you’ve purchased Software Assurance, and more, you might struggle to ensure you are in compliance with Microsoft’s terms.

    This can be even more complex when you move to the cloud, as sometimes your setup could include a license, and sometimes it doesn’t. If you lift-and-shift to cloud VMs, Azure lets you bring your existing licenses with the Azure Hybrid Benefit, which is tempting for organizations that want to get out of the data center business and move their workload to the cloud.

    However, that isn’t necessarily the same for all clouds. While Amazon lists various ways you can bring your licenses to AWS, Microsoft hasn’t felt that was legal. They’ve noted in 2019 that their “cloud partners” couldn’t support workloads on non-SA licenses. I can’t quite determine if SA licenses can be used, as lots of information out there is conflicting. Microsoft’s FAQ seems to support the idea that you need dedicated hosts, not just any shared VM.

    Not all cloud providers count, however. This week Microsoft noted that some cloud providers can let you run your workload by bringing your licenses. These are smaller providers in Europe who filed complaints. Likely these providers can’t offer the scale or pricing of GCP, AWS or Alibaba, so they aren’t as big a revenue threat as these providers.

    I don’t know what I think is the right thing, as once I’ve purchased a license, why does the location or ownership of the VM count? I ought to be able to run the software within the same guidelines in Rackspace, EC2 or GCP just as I would on-premises. However, such is the strange world of software licensing.

    If you deal with licensing, I’m sorry, but also, you should note that there are rules and restrictions when you move to the cloud. That’s something I see increasingly happening in many organizations, so educate yourself so that you can make the right recommendation. And when you get ready to purchase licenses, think about how SA not only affects upgrades, but also the flexibility of migration.

    Steve Jones

  • The Limited Cloud

    The cloud is truly someone else’s computer, and it’s not limitless. From the perspective of the individual, it might seem like you could scale infinitely if you need more boxes, storage, or networking, but ultimately the cloud vendor (Azure, AWS, GCP, etc.) needs to have spare machines around. Many of them do, and that works well at the scales in which they operate.

    To a point.

    Azure is having capacity issues in some of its data centers. I’m sure that all of the major cloud providers buy lots of hardware constantly, but there have been shortages during the last year, which has meant for Azure that DCs are running out of capacity. At least for the rest of 2022, which is likely driven by both a limited supply of new machines and the growth of cloud usage by their customers. I know I’ve seen continued growth in companies moving to the cloud for various services, including their data services.

    I haven’t seen reports of large AWS issues, though they do have plenty of instances of insufficient capacity errors and solutions. GCP has similar reports.

    Truly, the cloud is the cloud vendor’s computer. It’s also the cloud vendor’s infrastructure with lots of tooling to help you manage software configured networking and storage in addition to compute engines. It is limited in capacity, especially when lots of customers want to use these resources in one particular data center. That capacity is usually quite high, but there is definitely a limit.

    Does this mean that the cloud isn’t better than your data center? That’s an “it depends” questions. The cloud is great in many ways, and far superior to a lot of private data centers (or colocations facilities) that I’ve worked in or seen. I think the cloud is fantastic, but it’s not perfect. There are limits and issues at times, and you should be ready to work within or around those limits.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.