Tag: Cloud Computing

  • Cloud vs On-premises

    The cloud has been a controversial concept for much of its existence. While the idea has been around for many decades, AWS started selling IT services in 2006, with Azure following suit in 2008. Since then, the use of cloud services has grown tremendously. While some applications and organizations have embraced the idea from the beginning. I found many of you at SQL Server Central were very hesitant at first. I guess some of you are still skeptical about the value of a production database in a public cloud.

    From the beginning, I’ve felt that cloud computing has a place in the world, but in a way that is more appropriate for some situations than others. In terms of database (and maybe compute services), if you have a very well-known and predictable workload, the cloud can be very expensive. It might still be a good choice, but I think it often isn’t. If you have a variable or growing workload, then the cloud might serve you better than trying to keep up with new hardware in your own data center.

    Bluesky has had a tremendous amount of growth since its founding. Twitter invested in this as a distributed project and when Elon Musk purchased the site, many users moved away. A lot of them went to Bluesky, which had to deal with a quickly changing workload. They started in AWS, but eventually decided to move to an on-premises setup.

    Why? One would think their continued growth would mean AWS (or another cloud) would be a natural fit. However, they hired someone that provided an analysis showing they could invest in their own hardware, overprovision what they needed for growth, and keep up with the demands as they had developed a fairly accurate method of forecasting future needs. The savings in purchasing their own hardware allowed them to buy more than they needed and handle short-term spikes.

    To be clear, this doesn’t mean the cloud is worse for most or even many organizations. Bluesky knows they need to continue to invest in hardware, and they are prepared to keep adding resources. They also architected a distributed system that still allows them to scale into AWS if needed in the short term. I don’t know many organizations that would prioritize those things alongside the rest of their business. Most of us do a poor job of forecasting load. Even if we do, often the difficulties of purchasing new resources mean that we can struggle to meet increased demands.

    The companies that have moved to the cloud with success, and those that have left the cloud with success, are those that measure, monitor, and make appropriate decisions based on operational data, not opinions and feelings. They aren’t afraid to make a decision one way or the other, choosing what’s best for the organization, not what someone wants to do or thinks will be better.

    The cloud might be better for you, or it might be worse, but you ought to have a way to measure and analyze the options. You also need a talented staff that isn’t afraid to try new things and adapt their architecture to take advantage of modern hardware and software. Too many of us aren’t as flexible as Bluesky and might not have the success they have, in or out of the cloud.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Cloud Transformation

    In the last few months, I’ve been traveling around at a few of the Redgate Summits (one more in NYC coming) running panels on cloud journeys. I’ve had industry experts, both technical and managerial, discussing their approaches and journeys with advice and caveats for others. It can often be more than just migrating systems, so a lot of people have started to talk about cloud transformation.

    However, in some cases, this is just a migration. A lot of companies just lift-and-shift their databases into the cloud, along with various other services. While this is a quick way to get into the cloud, it isn’t much of a transformation. If you review and right-size the resources you’ve provisioned, maybe there is a bit of a transformation, but not a lot.

    Instead, the idea recommended by most vendors and consultants is to transform your software to work in the cloud. This might be moving to containers, to using more cloud-native services, or re-architecting your software to embrace to way cloud vendors provide services. Often this is a major project, though it can provide advantages over time with cost savings, more efficient code, and better-trained developers.

    That last one is key, as a lot of the advantages of the cloud require your developers to write better code and re-think how they interact with data services. Code costs money, and poor code costs more money. This is also true on-premises, but it’s more true (and more visible) in the cloud.

    I was surprised at how many companies had embraced the cloud and how many had seen savings. I met quite a few companies that had moved their databases (including large enterprises) 100% into the cloud. They’ve seen savings, but they also rigorously audit resources and the provisioned sizes.

    The cloud isn’t for every organization, and not for all workloads. There are more than a few companies that have struggled to achieve the results they want in the cloud, be these performance or cost measures. I suspect some organizations will never move fully to the cloud, and may never move databases. However, it is a tool that we technical people ought to understand and learn why we recommend for or against a move.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Kubernetes is Cool, But …

    Kubernetes is cool, and I think it’s really useful in helping us scale and manage multiple systems easily in a fault-tolerant way. Actually, I don’t think Kubernetes per se is important itself; more it seems that the idea of some orchestration engine to manage containers and systems is what really matters. As a side note, there are other orchestrators such as Mesos, OpenShift, and Nomad.

    However, do we need to know Kubernetes to use it for databases? This is a data platform newsletter, and most of us work with databases in some way. I do see more databases moving to the cloud, and a few moving to containers. I was thinking about this when I saw a Simple Talk article on Kubernetes for Complete Beginners. It’s a basic article that looks at what the platform consists of, how it works, and how to set up a mini Kubernetes platform on your system. It’s well written and interesting, but …

    Do we need to know anything about it? Are we running databases in containers, or will we? I think it’s possible that we might run any of our databases in containers. They are like lightweight VMs and there isn’t a reason why we wouldn’t run a database in a container. With external storage, of course, which gives you a cluster-like environment where your storage moves to a new node if the first one fails. That’s a good use case. Deploying consistent environments quickly is a good use case. Using Kubernetes to manage the containers is great, but …

    I don’t think we need to know much about Kubernetes. I don’t think most of us should run it and should outsource any container orchestration to the cloud if we decide to implement database containers. These orchestration engines are quite complex today, and there is a lot of expertise needed to manage them. I don’t know that expertise is worth trying to find, train, and retain for most organizations. We should just outsource the container management to someone else.

    We might need to know how we change the configuration of some resources, but that’s minor knowledge, and really, I suspect that outsourced K8S (shorthand for Kubernetes) will have GUI tools that let you easily pick and choose the CPUs, memory, etc. and then an export of the JSON or YAML or whatever is needed for the config. Most of us likely need the skills to export, save (in a VCS) the files, and then submit them to the cluster.

    A few years ago I went through a bunch of courses and reading material on Kubernetes. I set up some small clusters, I experimented with pods, I even was excited to think about managing containers for various services. What I discovered is that Kubernetes is complex, hard, and something I want someone else to run. Once I set up a cluster in Azure I thought I’d never want to do this on-premises again.

    Much like email. I have run email servers, but …

    I’d like to never run one again, which is how I feel about Kubernetes.

    I think containers have proven more complex and harder to work with than many people thought. I know there are plenty of people using them, but it’s a minority. I see many more organizations still building monoliths, or microservices that run as processes, or client-server apps. Not that many people are excited and using containers. That may change, and if you go to the cloud, containers give you portability that many other solutions don’t, so I’d recommend them there. However, they are still a bit immature, and hard to manage. I think it will be a while before we see lots of databases on containers.

    Even if we do, I’m not sure we need to learn Kubernetes as database people.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • The DBAccountant

    One of the things that I’ve found with cloud computing services is that the people involved with managing these resources sometimes get asked to become financial accountants.

    I saw an interesting post from SQL Rod asking about this new task as something all of us might need to consider a part of our job in the modern world. He asks if we are Techouncants or Accountechs? This isn’t in the sense of being the Financial DBA trying to manage costs in the cloud, though that is part of his post. It’s more about making smart financial decisions. Certainly, as more workloads move to the cloud, and they are for many of us, we likely need to keep an eye on costs, usage, and tuning.

    However, there are other choices. When I had to spec servers, we wanted to get something with room to grow, but not the top-of-the-line most expensive option. I wanted something engineered well for the next few years, and certainly not under-engineered. Usually because adding resources was hard in those days.

    Virtualization has made things easier, but often there are still some limits. Still, the goal is matching resources to workload with room to burst if needed. At the same time, are there other choices we can make? Rod asks about choosing the appropriate edition, the HA/DR strategy, tooling, and more. Can we be efficient with our use of money, time, and results, trying to neither overengineer nor under-engineer?

    The whole post has a lot of “it depends” in it, as the decisions on what you should choose are something you have to think about and arrive at a decision that balances costs with other factors. It can be clearer often to choose one thing over another, but it’s not always crystal clear which choice is better. I like his hardware analogy in the post. I’m not a fan of just throwing hardware at a situation, but in some cases, that’s the best solution.

    The cloud makes it easier to see where costs are. As a Redgate customer put it recently, “tuning queries becomes more important in the cloud as we can see exactly what each one costs.” That can be true for your SQL Server VMs, but it’s often not seen as important when the costs are a one-time spend for your hardware. However, Jeff Moden might argue that you should be tuning queries even without chargebacks or a cloud bill. It will save time for your users, which is always valuable. I agree, since I think we have no shortage of work and time is our most valuable resource. Save it when you can for the most people.

    That often means a big part of being a DBAccountant is being able to show, lead, or somehow get others to write better code early. Ensure everyone (or automated tests) knows how to view execution plans, learns to use them, and tries to write more efficient code the first time.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.