Category: Editorial

There is a skill that I think DBAs and sysadmins will need to develop: cloud cost analysis. I’ve thought this was important for quite a few years, and I’ve been (unsuccessfully) lobbying for cost information to be gathered and analyzed in Redgate Monitor. Hopefully, this work will get done soon, as I see more companies asking their technical people to provide analysis and justification of the resources being billed for in the cloud.

Basecamp analyzed its costs in 2023 and decided it could save money by leaving the cloud. I’ve seen other companies decide they were saving money in the cloud. Many, however, are likely unsure of the total return they get compared to the costs of cloud computing. I have seen some posts (like this one) that try to help you get a handle on your costs, but there is often a lot of complexity in cloud costs when multiple departments have different accounts (AWS) or subscriptions (Azure) with a provider.

In many ways, I think the large cloud vendors haven’t really considered how to support large enterprises that need a lot of resources, a lot of different administrators, and a wide variety of ways to both secure those resources as well as aggregate billing. I know we have an Azure account I didn’t create, where I have very limited rights to do things, but I can access some resources. However, my group has an AWS account that appears to be sending us billing statements instead of our corporate finance group.

What can be even more difficult to understand is that applications using the cloud might want a complete picture of billing, but various different technical groups might be responsible for the different parts of the system. Data services could be managed separately from web servers or application servers, with different groups creating, configuring, and destroying them as the needs change. Billing, however, be something accountants would want to separate by function/area/application, not group. Knowing all data services for all databases cost $xx/month is different than knowing the retail website (containing web servers, databases, and networking charges) costs $yy/month.

This is far different from the days when one group was responsible for most of our computing resources. Often we had a group of IT staffers who managed hardware, ordered it, and could link costs of different machines to a specific application or department. Now we often don’t have a central group who is even aware of all the resources that the company has provisioned.

I suspect this will mean that many more technical people will not only be asked to account for the cloud resources being used, but also split out those costs in different ways, allowing people in finance departments to aggregate the different costs from the various technical groups. I don’t know who will actually track network resources, but I suspect applications will often have this data included with the servers or services that access public networks.

Controlling costs and carefully removing unnecessary or unused resources is going to be a continual problem in the cloud. I suspect that quite a few data professionals are going to be integral to helping manage this, especially for dev/test systems that are easily forgotten about over time.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

This week I saw an announcement from Azure that you can set up auto-renew subscriptions for your reservations for your resources. This allows you to set up auto-renew for reservations to prevent them from not renewing and being charged at the pay-as-you-go rates. This reminds me of the old Columbia House deals, which I had for both cassettes and CDs. For some time these were good deals for me, but over time, I’d lose track of the subscription, forget to return things, and end up purchasing items I didn’t want.

An auto-renewal is likely both a good idea and a bad one, as I suspect that many organizations lose track of their renewal dates when they have an estate of any size. Administrators or finance people will come and go, and they might not know the history of why we reserved something, like a VM, and could forget to renew these. Often we are still using resources, so renewals make sense.

However, we also stop using resources or don’t use them at the same capacity levels, so perhaps we want to not renew. The docs for this feature note that the renewal creates a new reservation, and this setting can be changed at any time. However, some of the other docs aren’t clear to me. The descriptions read as though someone has an idea about how this works, and they don’t quite clearly detail the workings of the system for readers that are not familiar with this feature.

As an example, the docs say if you enable renewal more than 30 days before expiration, then  you get an email explaining renewal costs. However, with automatic renewal, do you get the email? When are costs disclosed or detailed? Maybe more importantly, the doc notes the price may change between when you lock the renewal price and when the renew time occurs. Does this mean you pay the higher price? Are you notified each year?

Maybe more importantly, if we create resources at different times, can we somehow get renewals dates to align? Whether this comes to finance people or IT people, at some point with a big cloud presence, you might have renewals coming every month or week. Who would be able to track this stuff appropriately if it’s a constant item to check? I can see plenty of people starting to ignore these notices if they come too often. There are other issues as well, such as a SKU being different (or deprecated) over time. Will someone figure out how to upgrade to a newer SKU? If so, will they remember to set auto-renewal? Does this transition to the new SKU?

The documentation is woefully incomplete, though I’d be happy to submit PRs for edits. I just don’t know what the answers are, and I suspect that whoever built this feature might not have thought through all the scenarios. Certainly the individual writing the documentation didn’t.

The cloud is complex, and billing is complex. Tracking and managing this is going to be partially a technical role, and I suspect many of us won’t know what the implications are for the choices we make. I also suspect we’ll get notifications for resources that we have no knowledge of and will need to research more about the item. We’ll become more of a financial DBA over time, which may or may not be the job we want.

Steve Jones