Voice of the DBA

Tag: Cloud Computing

  • A View of the Cloud

    Amazon has been an amazing digital company in the last twenty years. I remember making my first order from them, unsure of whether the online bookstore would work better than browsing locally, or if I’d even get my books. I watched them transform into a great shipping company that could sell anything, to a digital reading company with the Kindle, and even an amazing cloud hosting provider with world class programming platforms. At each stage, Amazon has grown to become even more efficient and seductive, slowly gaining more of my business over the years.

    Whether you like the business model and practices of Amazon or not, part of the reason Amazon has become a successful company is that they have an incredible software development process and great developers, both of which have produced a software stack that is very impressive. Apart from their web site, which is impressive, they host their Amazon Web Services, which is a dizzying array of services that can be purchased by anyone, at any time, and get some software up and running quickly. Even if you look at their free tier, it’s very impressive in the number and type of services, including database access.

    I ran across a piece on the 10 year anniversary of AWS that contains some software development lessons from Werner Vogels, CTO of Amazon. It’s a good list, and while some might not apply to your particular environment in the same way, it’s a good list of principles to follow. Certainly planning to evolve and to handle failures is something that most of us need to consider. Including security from the ground up, however, is one thing that many of us know, but don’t have good patterns, practices, or habits to follow. Even those companies that work in security conscious areas often leave security concerns until they are well into, or even finished, with their core software development.

    The one area that I think is going to become more important for all of us in the future is understanding resource usage. Certainly we’ve needed to know some baselines and expected growth in the past to plan for upgrades and future needs. However, with the move to VMs and cloud services, how many of us really know what resource usage we would require for peak workloads and when. I suspect most people don’t really know how to map their instance resource usage across different virtual systems, much less evaluate whether their system would be cost effective in a cloud scenario. A better understanding of what resources we really use is something that more and more organizations are going to expect from their technical professionals. I only hope we get some tools to help us here as this is a complex topic.

    The move to cloud services is going to happen for many of us. We might not move all our data, or even much of it, but as we experiment with new platforms, as companies work to avoid capital expenditures, and as the platforms and services mature, more of us will end up with some cloud platform in a portion of our organization. During your journey, or even while you still work with on-premise systems, I’d keep the principles in Mr. Vogels’ article in mind. They’ll help you to build better software in the future.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Relentless Cloud

    The march to the cloud is inevitable. No, I don’t think all your services will be run from a cloud service, but I do think that some of them will be. Maybe 10% for your company, maybe 90%, but likely somewhere in between. In the efforts to reduce costs, and maybe more likely, the idea of shifting costs to expenses, I would bet that most organiztions will take serious looks at the cloud in the next decade for a significant portion of all new services. They might not move most applications, but they’ll think about it.

    I saw a note recently on Geekwire that Azure is estimated to exceed  Amazon Web Services (AWS) by 2019. Most of us might not care about the race between these two tech giants to build large cloud infrastructures, but their capital expenditure is based on the idea that your organizations will want to use some of those services. They have good reasons to think this with surveys and conversations with many of their customers.

    In the absence of changing tax policy in many nations, it also makes more sense to make move costs to expense, rather than capital expenditures. And that’s the way things are going, at least according to what CIOs are saying. More and more applications and services will be moving to the cloud in the next few years. Up to 30% from the current 14% of applications today. As I’ve seen more and more entreprises, public and private, moving mail and web services to outsourcing providers like O365, I also think we’ll see other applications moving.

    I don’t think every application is suited to an outsourced area, but I do think there’s a case to be made for many of them, including databases. For those with privacy or security issues, cloud providers are working to mitigate the issues. Amazon has built separate data centers for government services, and I believe Azure is (or has) done so as well. The challenges that exist can be solved, and many of the risks or concerns will be mitigated, though you’ll still need to evaluate your situation on a case by case basis, which will change over time.

    My recommendation: learn about the cloud.  Really learn, don’t just assume you know. It will be good for your career.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Mevio .

  • The New Data Warehouse Choice

    I was listening to the SQL Data Partners podcast the other day with BI expert, Tim Mitchell, and the opening question was “Is the on-premises Data Warehouse dead?” Tim is a friend, so I tuned in knowing he has some good thoughts on the topic. It’s an interesting listen, and one you might enjoy if you’re at all interested in data warehousing and related topics. Spoiler alert, Tim says no, on-premise isn’t dead, but he does point out some interesting things about the Azure SQL Data Warehouse (ASDW) and similar offerings.

    One of the more interesting comments Tim made was about a health care company he worked for. They had an end of month process that heavily taxed their systems. If they didn’t need that peak level of processing, Tim noted that the cost of their large data warehouse architecture would be that halved. That need to scale up dramatically can be a big savings in moving to a cloud based system, where you can pay for a much lower level of performance most of the time and increase your scale at particular times.

    I know this is feasible as I worked with a similar situation. My employer purchased a very large system for our end of month and end of quarter closing load. Fortunately, we had an AIX machine that contained its own hypervisor. At the time (2001), we purchased a 32 processor server, with the idea that only 18 CPUs (and a slice of RAM) were running our financial systems most of the month. We had QA, development, and other guests on the same hardware. During the month closing, we would shut down some VMs and dedicate most of the processors to the finance system for a few days to handle the load. What’s more, the IBM machine actually contained additional CPUs that we could “rent” from IBM for a few hours if we really needed them.

    That’s what data warehouses in the cloud can do for you. Certainly the decision to move to a cloud architecture is more complex than just having the scale up power of ASDW or Amazon’s Redshift. The ability to load into the system, the development challenges, the tax implications, and more will impact the decision. I think the workload characteristics are also important. If you don’t have a highly variable, or large peak, workload, then the cloud might make less sense. If you don’t have any sort of data center, then maybe the cloud makes more sense.

    I do think, however, that the decision to implement a new data warehouse isn’t a simple one, and the cloud is a viable choice. The platforms are becoming more capable all the time, with more tools and scale options, as well as better performance guarantees. Many of the tools used to analyze data in a warehouse are more important than the underlying platform, with Excel, Tableau, Power BI, and more easily connecting to any data warehouse platform, in the cloud or on-premise.

    This means that we will end up managing more disparate systems over time, especially in larger organizations where some groups will adopt cloud systems while others stick with on-premise installations. Certainly if you are a person that works with a data warehouse, you might want to build a small POC on Azure SQL Data Warehouse and see what you think about its capabilities. At least then you’ll be able to add some educated and intelligent thoughts to the discussion when the question comes up inside your organization.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.3MB) podcast or subscribe to the feed at iTunes and Mevio .

  • Going On-Prem?

    When I started in this business, every company owned their own servers, and had their own data center. Now, some of those data centers were closets (including that for one state government Senate). Some were offices-converted-to-computer rooms, rarely with separate air conditioning until I requested it. Some were actual data center rooms, though usually for larger companies. The idea of putting your computing resources off-site was seen as ludicrous.

    As time passed, the idea of using a co-location facility grew. In fact, SQLServerCentral went from a couple machines in a residential basement to a locked rack at a facility in Denver. Many companies started to use third party data centers, and in the late 90s and early 2000s, this was common, with facilities springing up. Full time employees will often racked equipment and visited data centers when they needed physical access.

    This changed over time as virtualization grew, and more and more companies started to rent a VM, rather than purchase a physical box. Some still used physical machines, but they rented the machine, allowing a third party to setup and mount the equipment, sometimes even installing the OS and then providing access to the lessee. We’ve come to the place now where we have cloud services that are often just platforms or services, where we have no idea of the underlying equipment. In the case of things like Salesforce, Azure SQL Databases, or Amazon’s RDS, we don’t even care, just asking for a certain level of performance.

    Is this the future? Will we now be asking ourselves if we “want to go on-prem” as the default question? I ran across the blog linked, which is from a SaaS perspective, that is trying to always get customers to use the service (or platform) and never have a local install. I know Microsoft would like to do this, especially for Azure databases as they make more profit on compute services.

    Some of you think this is crazy. In fact, I know plenty of industries that struggle with this from a regulatory standpoint. However, I’d point out that the idea of letting someone else run your email system was seen as crazy 20 years ago. Today many people (myself included) would never think to install Exchange or any other email software. We would always purchase this service from someone else.

    Tim Mitchell had a podcast interview recently talking about whether the on-premise data warehouse is dead. While I think the Azure Data Warehouse has some great advantages, and is worth considering, I’m not sure the local DW is dead. In fact, for lots of tasks I perform, including software Continuous Integration builds (.NET or database), I prefer to have some local resource doing the work. Especially when I can’t predict the number of times I’ll build and that task is easy to scale on local machines.

    I don’t think that we are at the point where we’d put more databases in a cloud service than on premise, but I do think that for new applications, it’s a valid question to ask whether the database and application could be hosted in the cloud.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Mevio .