Tag: Cloud Computing

Cloud costs are high and growing. Some orgs think they’re out of control and are trying to limit spend. Some orgs are looking to leave the cloud. A lot of IT spend over the years has been seen as a cost center, with many executives trying to limit the growth or spend, even while they aim for digital transformations of their businesses. Throughout my career, it’s been interesting seeing the tension of groups trying to take advantage of technology and the finance departments trying to manage costs.

The cloud brings some of the same debates/arguments/concerns to the forefront. Partially because of scale, as we can add cloud resources much quicker than we can with a CapEx purchase. Partially because we’ve also often lost some control over budgeting with the move to OpEx and the subscription paradigm.

However, the bigger problem is that we get a true chargeback for all the resources we use. This is done in a way that exceeds what I used to see with mainframes. In those days each department got a monthly report of their usage, applied against their budget. In the cloud world, finance gets a real bill they have to pay, and the chargeback is not a made-up number for internal use.

How can you reduce the cost in the cloud for your database (or all data) resources? The easy answer to start is to get rid of those resources that aren’t being used. How many databases have been created for dev/test and aren’t actively used? That’s often an easy chore, and one that could be scripted. Though I’d want to include backups of anything I get rid of that might be reused. That might include dev systems rarely used. Keeping a backup that I can use to restore to a new resource later is always a good idea.

In line with this, you might look for underutilized instances, especially for VMs or expensive services like Managed Instances. There can be a cost overhead for these systems, and if every application/department/etc. uses a separate system, there might be cost savings from consolidation. However, don’t just move databases. Ensure you aren’t creating a security risk or overloading the system with moves. That requires some analysis of the current systems and buy-in from those with security concerns.

However, the biggest win in reducing cloud cost is to performance tune your code. This is something many DBAs request on a regular basis, but often there isn’t any push to write better code, or utilize better patterns of querying that might reduce the amount of CPU, memory, and network traffic. Learn how to write better performing queries, limit the data returned (no SELECT *), watching out for datatype conversions, and using other techniques that people write about at SQL Server Central every day. We constantly answer questions and publish articles on better ways to write code, so invest the time to ensure your teams can and do write better code. Make sure you train your staff and they use their training.

Lastly, you are in the cloud because you’re willing to change how you work. Don’t be afraid of changing your existing code. If you refactor and break something, change it back. The cloud makes this easy, so take advantage of that and keep improving your code.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Recently, I got a bill from Azure. That’s not an unusual thing for many of you, but for me it was a surprise because it said I was late paying. I’ve had a number of services running, and I thought at first that I had left something running too long, like a VM. As I checked, most of the things were paused, even the expensive ones like a Synapse workspace. Instead, I found that my free credits were not being applied. Fortunately, I had changed credit cards or I might have been billed for a few months before I noticed.

This was a change in how Microsoft managed benefits, which is fine. I opened a support call and someone helped me, but it took several days to get a response. I was slightly worried about the bills, so I decided to audit the things I had running.

It was amazing how many little things had been created or added as I’ve worked in Azure. Little experiments, even some targeted demos didn’t consist of just a database or a website. There were storage accounts, network endpoints, analytics and log services, and more. Quite a few resources had been auto-created when I deployed something else and had auto-generated names and I wasn’t sure where they were being used. Even in places where I had tried to group things into resource groups, I wasn’t sure of the purpose of each item.

And that’s just me. Imagine a team of developers or infrastructure people being asked to create things, spin them up for a purpose or after getting a request, and then having some staff turnover. There would likely be dozens or hundreds of resources, which might be poorly or inconsistently named, tagged, or grouped. After this I can see why organizations often feel they’ve lost control of the cloud if they allow too many people to create their own resources.

It was a sobering experience for me, someone trying to stay under $150/mo of costs for various projects. The number of experiments and deployed services I no longer needed was astounding. A good late-summer cleaning taught me to be a little more organized with my resource groups, and to ensure I’m auditing what I have running every few months.

Something most organizations should do, but likely struggle to complete effectively in any way.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

There are many, many choices for cloud database services these days. I would hope everyone is aware of the various IaaS options in public clouds with EC2, Azure VMs, GCP Compute Engine, and others. These are often the easiest way to move your workload, but you’ve really just moved a VM from one place to another (likely more expensive) place.

For managed databases, there are lots of choices, but you might not be aware of your options. I ran across an article that discusses the various flavors of managed databases in the big three public clouds for SQL Server. In the piece, there is a section that talks about when a managed database makes sense. I like that it discloses the development on a managed service is expensive.

The problem I have is that I know lots of companies that struggle when they don’t have a development environment that matches product. Invariably developers will use something in their local SQL Server/MySQL/Oracle/PostgreSQL database that doesn’t work in a managed service and causes no shortage of pain. Containers can help, but they’re not always available.

I won’t delve into details, but the article lightly looks at RDS (Amazon), Azure SQL DB and Managed Instance (Microsoft), and Cloud SQL (Google). There are pros and cons for all of them, but to me, the Google option is interesting. It gives you an instance (with SSIS/SSRS) and a proxy, but it does have limit you to instance restores. Same for RDS, which is a limitation I’d be worried about. Often I have an issue with a database, really often with just a table. Restoring a whole instance instead of a db might be a delay I wouldn’t want to deal with. I hope this changes over time, as not recovering a database from an instance is a big hole to me.

The comparison of hardware resources and performance seems to show Azure has the highest capacities, but as the article notes, if you approach the maximums, the cost is very high. I tend to agree there, and I think this is one of the things that gets lost when companies consider the cloud for workloads. They look at the maximums with an eye towards growth if they need it. Rarely, however, do they compute the increased cost if they’ve under-provisioned resources. There’s also the concern that trying to match CPU/RAM/disk from on-premises to the cloud isn’t easy. This often isn’t the same type of comparison that most of are used to with local machines.

The main thing I find too many people forgetting is that moving to the cloud is a new architecture. You likely need to rework/refactor/rewrite your application to work better. Part of that work is reducing the amount and frequency of data queries. Without changing those things, you likely will end up spending a lot more money than you expect. Possibly without getting the workload response you expect.

The cloud can work well for your application and database, but it’s not easy and it’s not quick.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.