Tag: Cloud Computing

  • Ten Database Commandments

    I love the Notorious B.I.G. His music isn’t for everyone, but I enjoy it. This piece takes one of his songs (NSFW – Ten Crack Commandments) and applies the list to databases. Each item from the song is listed, along with the way it might be changed for database work. The list is from Ottertune, which is a service designed to help you optimize your use of AWS RDS and Aurora databases, with AI. If you believe their hype, their service will get better performance and lower costs for you.

    In any case, with regard to databases, I suspect that the author is having some fun with the rules, and a few of them made me smile. They are cloud-related, but a few will apply to any environment.

    The first one is about budgets, noting you should not disclose your budget, which is in general a good rule whenever you are buying something. Don’t disclose information, and someone working for a vendor of tools, I think that most budget conversations aren’t something technical people should worry about. Decide what value you get from some tool/service/process/etc. and if there are time savings. Then let someone else decide if the ROI is worth the cost.

    Rule 2 is a stretch, but I do think that looking at new tools can be helpful. The caveat here is that many people, don’t often learn enough about existing tools to understand if they will solve issues. There are a lot of things you can do with many tools, so don’t just look for some magical new tool. Learn to use your tools well, and if you want to try a new one, keep in mind you will need to invest some time both learning about it and practicing some skills before you really know if it’s helpful. The other thing to remember is every new tool increases the bar for new staff. Try to limit the number of tools you use to those you need.

    Security is important and rule 3 is remembering not to give out more permissions than needed, especially to developers. I love this quote: “You don’t want randos at your organization logging in and running queries that they found on Stack Overflow.”

    A number of the other rules have to do with performance, which is certainly Ottertune’s business, but these are good basic practices that I see organizations not following. Too often management wants to save money and technical people make decisions that compromise performance. Lots of people overload systems, and then they can’t handle the workload well. I get that many CFOs and others want to see high CPU usage, which means they’re using the resources they pay for, but databases can be very bursty with resource usage. At the same time, you don’t want to over-provision resources because you are scared. Learn to find a balance that doesn’t waste money, but handles your workload well.

    Perhaps one of the best things about the cloud is that if we do make bad decisions, we can often provision more resources. It can take time to move to a new tier of service, so don’t expect instance response up or down, but if you need to double the CPUs for month-end processing, you can likely plan for the time to scale up your database instance and then scale down again when all the work is complete.

    It’s a fun article, and if you’re looking for some fun Notorious B.I.G.-related database content, grab a cup of coffee when you need a break and a smile.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • Flexing for the Cloud

    Many of us in technology regularly experiment with new tools and technology. We often do this to adopt new skills when we need to solve a problem or we have inherited some code that needs enhancement. The cloud is in many ways no different than things we’ve done before, but the way we do them, or maybe the way the cloud handles some things, means that we need to change how we view a task or perform our jobs.

    Migrating to the cloud is something that many of us have had to tackle at some point. I’ve seen a good amount of lift-and-shift, including many companies who just mandated moves to the cloud without much planning. As mentioned in this article, that was cloud migration 1.0. Really, I think this was something that’s been going on for years, and continues to occur today. Lots of management are seduced by the promises of the cloud to make their organization’s IT systems better, so they move everything without thinking.

    Then many run into cloud migration 2.0, or maybe that’s coming. I see plenty of customers with things in the cloud they complain about much in the same way they did on-premises. Nothing has changed with regards to how their systems work, and they are likely spending more money on the same services they had on-premises. With the same people. They didn’t plan well, didn’t provision well, and they haven’t changed anything but their costs.

    Cloud migration 3.0 is what many tech professionals would like to see. Evaluate apps, decide which benefit from the cloud, and leave the rest alone. Leave them on-premises in a familiar environment. That’s hybrid, but that’s fine.

    What many tech professionals worry about are their jobs, so they don’t often consider where the cloud can provide benefits from some apps. The flexibility and scalability are undeniable. You just need to ensure that you understand where the cloud fits, where cloud-native can benefit your org, and then make sensible recommendations.

    No matter how your organization approaches the cloud, likely you’ll have engineers from the provider or third-party consultants involved. Do what I do with people I’ve hired: ask a lot of questions. You’ll learn something, flex your knowledge, and even if you never work in the cloud yourself, you’ll be better prepared to make recommendations and evaluate future choices.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • The Cost of the Cloud

    Basecamp (formerly 37 Signals) is quitting the cloud. One of the founders gives some reasons, and he had some detail in a tweet on what they’ve spent in the cloud the last few years. Over USD$3mm on various services, though their costs in search seem very high. I don’t, and haven’t, run as busy a business as they do, so I don’t know if they’ve truly done a good job architecting things and setting up services. They say they have, though I’d expect everyone to say and think that.

    However, I’ll assume they are correct and they can’t optimize things any more than they are currently. Their decision makes some sense, and I agree with it. I’ve been surprised at the growth of the cloud, both in size and how quickly people are moving to the cloud. I’ve also been saying for years that if you have a steady or known workload, the cloud is likely very expensive.

    Maybe that’s worth it for your organization. Not dealing with physical resources, maybe having slightly less staff, maybe less CapEx vs. OpEx. Those are decisions for management and finance people. For most of us, the cloud both simplifies some tasks and makes others more complex. Provisioning, testing out Proof-of-concepts, and scaling are easy. Identity protocols, gaining (and keeping) knowledge of how various options work (networking, storage, etc.) , and keeping track of resources become more complex. Not to mention the world constantly shifting under your feet as cloud providers change how their platforms work.

    There are costs in both hard dollars (or your currency of choice) and in the time your staff spends dealing with a new way of doing business. The calculation of whether this is a cost that makes sense is very dependent on your situation. I have customers that love the cloud and others that hate it. The value they get varies dramatically and some would never go back to data centers while others are ready to work long hours to leave the cloud. Overall the sentiment is the cloud is great, but like many decisions made by management there are particulars that baffle the technical staff.

    The one thing I have learned about the cloud is that it takes a different sort of mentality from staff than on-premises resources. We have to learn to spin things up and down, scaling as needed. We need to better understand budgets and not look at costs as though they were personal expenses. We also need to be flexible with resources, understanding that machines that are idle are not sunk costs; they are ongoing costs.

    The cloud is amazing, and I think it is very useful in lots of situations, but a blanket move to the cloud can be expensive. Make sure that everyone involved in moving to the cloud understands that.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Things I’ve Learned About the Cloud

    I do a little cloud work, but mostly I end up working with customers that are trying to build and migrate their systems to the cloud. The transition to cloud-first took place quicker than I expected, though certainly the pandemic of the last 3 years has continued to accelerate the moves by many organizations. While not all systems will move, for better or worse, many organizations consider the cloud for everything and then decide to accept or reject the notion for individual cases. That’s the cloud-first world I see.

    At a recent event, I had numerous people telling me that their groups were mandated to move many systems to the cloud in a lift-and-shift fashion, and many of the technologists weren’t happy. That’s a conversation for another day, but listening to them and others has taught me a few things. Some of these are experiences I’ve learned myself, and some are things from others, nuggets gained from their hard experiences.

    First, lift-and-shift appears to be a way to get started in the cloud. Some people find this to be an end-goal when they can get systems running without ever worrying about hardware or facilities. While costs can be higher, flexibility and tax considerations can make this worth the effort. Others see this as a first step to moving towards Platform-as-a-Service (PaaS) products and rewriting software, and life-and-shift is a good way to approach this process, but that leads me to the second nugget.

    Everything takes longer to get settled in the cloud than you expect. Not that the cloud is slower to build and use resources. The opposite is certainly true as provisioning anything is fast, and upgrades (or downgrades) are incredibly flexible and quick. However, making the move, getting systems running to your satisfaction, decommissioning old systems, and more take longer. It seems most life-and-shift projects end up taking much longer than estimated, and not all of this is the massive data transfers needed to move data assets. Often it’s humans that can slow the process, whether through debate on decisions or just getting comfortable with cloud resources through testing. Things take longer.

    Maybe the last big thing that I’ve seen affect many companies is that we have to put better software development techniques and infrastructure configuration in place. The connection to resources is more complex and tenuous than it is on a LAN. We need to better architect software, really architect it as we were told to do so early in our careers with better error handling and retry logic. The cloud is very secure when you set it up right, but you no longer have the castle with high walls that you grew used to in a data center. Instead, you have a series of people and systems that are disparately connected and you need to ensure you protect each one along with the connections between them. That requires lots of configuration work and standards, and likely Infrastructure-as-code. Plus there are new tools to learn and the habit to build of using command line interfaces.

    I love the cloud. In many ways, I prefer it over building and managing things on premise. Not for everything, and not for things where I might lose productivity with slow connections. I certainly don’t want developer laptops or workstations in the cloud, but having other systems there can remove a lot of the hassles of managing assets that we purchase, track, upgrade, etc. I don’t know I’d ever want to buy another physical server in a company I owned, but I also know that at some scales, it might still make sense.

    The other thing I’ve learned about the cloud is there are a lot of unknown unknowns, as well as plenty of known unknowns. Tackling the cloud require a staff that wants to grow and learn with the platform, as well as one that knows when to use the resources they know best and don’t add complexity or novelty just because someone wants to try something new. The cloud is worth considering, especially as it seems physical offices and data centers are becoming rarer and rarer in many organizations.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.