Not too many data professionals are working with databases in the cloud, and while I think it’s a platform that will grow, it’s not suited for all environments. One thing all of us need, however, is bigger, better, and faster storage for our data. Our data volumes are growing, sometimes surprising us with the sheer number of bits our queries must go through, and access times for storage are often a bottleneck in our systems.
There have been some interesting advances in how storage is connected to our systems, and there are some even more interesting ways in which the large cloud vendors are tackling the storage issues they face. I read a fascinating piece on the various storage systems used by Google, Amazon, and Azure. It talks about the various architectures being used, and there are some innovative ideas being implemented in order to ensure there is a high level of reliability and performance for these systems.
Should you be interested in storage architectures as a data professional? I think that having a basic idea of the options out there will give you the chance to understand the options your storage people (or vendors) may recommend in the future, and have some idea of the suitability of these systems for your databases. I don’t think you need to understand the detailed implementations, but knowing the pro and cons of these systems, as well as the experiences of other companies, may help you detect how much fluff is being presented to you.
There’s one part at the end of the piece that caught my eye: “as data volumes continue to go up for “big data” applications, server memory is becoming “the new disk” and file systems are becoming where the log for application activity gets stored—”the new tape.” That’s interesting. Maybe we’ll start to do more in stream processing, ala Streaminsight, on our data and keep more aggregations and analysis in memory, reducing the need for disk access.
The Voice of the DBA Podcasts
My apologies. The hosting provider for the podcasts is having issues, so we do not have them available today.