Voice of the DBA

Tag: Cloud Computing

  • Checking Up on Vendors

    Many years ago I was in the Denver airport, waiting to board a plane. At the time I was consulting with a small startup company and they called me in a panic, having had an issue with their database deployment and needing to restore the previous version. However, they were getting the “The media set has 2 media families but only 1 are provided. All members must be provided.” error. Fortunately, I knew what was wrong and verbally walked them through the process. When I landed, I logged in and verified things were working and that they’d followed my instructions.

    A few years later I was shopping for furniture with my wife one Saturday night and I got a call from a fellow Operations person at JD Edwards. They had some security issue with a SQL Server, and since I was the SME (subject matter expert), they called me for help. Sitting in a comfy lounge chair (which I never purchased), I helped them solve the issue. Later, I verified things were working.

    In these cases, I was using remote hands from trusted individuals at an organization. However, these days, many people call cloud vendors (or other hosting companies) for the same type of remote hands support, but without necessarily having the accountability from that staff. Sure you can check if they solved an issue, but are you sure they didn’t read some data or change anything else?

    It’s a concern, but it’s one that I think can be solved with help from vendors. Microsoft recently announced there’s an audit capability for Azure SQL Servers, allowing customer to determine what actions Microsoft support engineers might take on your system. Since we may not necessarily see what is happening when someone else is working on our PaaS or IaaS systems, it’s good to have something tracking their moves. I don’t know this gathers all queries run, but it certainly should.

    I’ve seen some presentations on how customers have to authorize engineers in VMs with Customer Lockbox, and the technical details are interesting. These are time limited and audited access controls to ensure that customers are aware when a Microsoft employee touches their systems. I don’t know if this is comprehensive enough, but it is a good start.

    Over time, more and more of us have to delegate some work to others, either inside or outside of our organization. We need strong controls and good tools to ensure that we can determine what happened, especially in the event of some incident taking place. Building these controls is important, and really, they should be easily enabled in all our systems. While I don’t know we have a lot of rogue employees accessing things, we do have some, and we have more that make mistakes. Auditing their actions should be something we can easily do.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Cloud Database Growth

    The growth of the cloud is exploding overall, with all of the major vendors showing growing lots of revenue growth. Not all of them are profitable, but they are growing. This shows that customers want to use the cloud, and more of them are migrating all the time.

    I do think that databases are likely migrated less than many other services, though their use is increasing. Plenty of organizations would like to offload some of the overhead of owning and managing hardware, which may be one reason why the growth of IaaS in the cloud is one of the more popular options.

    For those databases that are being used in the cloud, can you guess which ones have the fastest growing usage? Since this is a primarily a SQL Server-based site, you might guess Azure SQL Database, and you’d be close. It’s one of the fastest growing, edged out by PostgreSQL.  That’s one reason we’ve been publishing some PostgreSQL articles; it’s very popular and being used in many organizations that also have SQL Server.

    While we know we can build cheaper systems on premises, and we can better control, or at least predict, performance, the cloud is attractive to many businesses. It’s not perfect, but it is an option that you should be able to knowledgeably talk about when someone brings it up. That means you need to learn a bit about how it works, the costs, the performance, and the differences with an on-premises system. The options change all the time, so don’t rely on your knowledge of the cloud from a few years ago or hearsay. Spend some time asking questions of people using the cloud today.

    You also might consider using VMs if someone wants to migrate to the cloud, as the system will appear to be very similar to the way your on-premises instance runs. The hardware setup is different, and you need to learn about the differences in the hardware, especially how you connect disks and their performance. 

    Knowledge is important, and for many of us, we are looked at as those with knowledge about these different platforms and technologies. That means we need to spend some time learning, and likely a bit practicing to understand the impact so that we can present a good argument about when it may or may not be appropriate to use a cloud database.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A Cloud Database for Statistics

    Baseball is an interesting game from a data perspective, with lots of numbers being tracked, and lots being generated every year. I used to have a sample database for demos that I used, since it was fun to run various numerical queries on the data. I should set that back up.

    Recently I saw that an analysis site, FanGraphs, was adopting MariaBD, but in a cloud version of the database. They gather a lot of statistics, more than most places publish. Not only do they have the various aggregates from games, but they have tracking for things like the velocity of pitches thrown. Add that to odds, projections, and more, and this is a lot of data.

    For a fan, that’s a lot. For database people, maybe not so much. They are projecting a million records each season for pitches, which might be the largest data set. However, for a database, even with 100 years of baseball, 100mm rows isn’t that large. There can be, however, lots of queries on this data.

    The founder used to manage the database himself, and has been on MariaDB for a long time. He started on Windows, but has been looking to outsource some of the administration. They left dedicated servers to move to a vendor running on the Google Cloud Platform (GCP). Now they are looking at data warehousing and other options to continue with deeper data analysis options.

    The move to the cloud, removing some of the headaches and hassles of managing servers is something many executives think about. Certainly it isn’t cheaper than buying your own machines, but with the cost of people, benefits, and the inflexibility of being limited by past decisions, I get why companies do this. Especially those that aren’t so focused on the technology, but are more interested in what technology can enable them to do.

    Many of us working might find opportunities to work with data at a company like this, without the need to actually manage the systems. I can imagine for a data analyst or developer that enjoys baseball, this might be a fun type of challenge. Learn to apply technology at a company that doesn’t really care about the software itself, but wants to use it to build something new and exciting for their customers.

    Steve Jones

  • The Cloudy Future in 2021

    Ignite took place recently, and the keynote from Satya Nadella was interesting, and if you have time, interesting to watch. If not, there’s an excerpt in written form. In any case, I like his passion and excitement about the future. I’m slightly sad that he’s not on stage somewhere, and instead in a studio of some sort, but that’s the world today.

    There were five pillars of the keynote which related to the future of the cloud. One of these was the idea that we are at “peak centralized computing” now, but we need to get more decentralized. The idea here, I think, is that we will have more of the computing services taking place outside of data centers and server rooms, and in more devices. We already see some of this, with mobile devices, cars, sensors, and more handling some computing loads.

    The Edge is something Microsoft continues to push, and I think the adoption of this paradigm will continue, with some pre-processing or filtering occurring in devices or intermediate servers, and then less, or limited, data being loaded into our database platforms. I also expect that we’ll see more data loaded into a database somewhere, and less ETL into another platform and more just in time querying from a client or other data store.

    I was also glad that trust was a pillar, though I think this was an attempt to corral a number of disparate ideas into one topic. However, the idea of data protection, and individual rights such as privacy, are important, and I do think that Microsoft recognizes this. Even if they are just trying to build a product to profit, I appreciate the work they are doing in this area, adding new capabilities to allow companies to better control, limit access, and protect data in flexible ways. Now, if we can just get the companies to respect the individual’s data they hold, I think the world will be better.

    Lots of what Microsoft presents in their keynotes and conferences is aspirational, with limited experience or understanding of whether customers really will adopt these ideas. Every year I see some visions that don’t ever get widely used across the next year and aren’t mentioned the next year. However, some of their ideas are good, and some will evolve in different ways to help improve our world. This year, I loved seeing the remote ways HoloLens 2 is being used.

    I never know what will catch on, but I do like seeing the visions that others have for our world.

    Steve Jones