Voice of the DBA

Category: Editorial

  • Tiered Storage

    Storage, storage, storage

    One of the things that I saw demonstrated at my very first PDC conference in 1998 was the addition of Hierarchical Storage Management to Windows servers. This was in the Windows NT 4 era, and I was investigating a variety of ways that we could potentially handle a large number of fax and scanned images for my company. We eventually implemented a RW optical jukebox to help us manage our large collection of images and it worked very well in helping us manage costs.

    I saw this great article recently on tiered storage that uses a database system as an example of how you can potentially improve the performance of your system and manage costs by using different types of storage. It’s worth the read, and talks about a fictional example of how we might use SSDs for frequently accessed data and SATA HDDs for cold, less accessed data.

    As we collect and store more and more data, I think that we will find that we are storing lots of data that we potentially access very infrequently.  If that is the case, then we ought to be considering different types of storage that can handle the needs of that particular set of data while also managing costs. I have always struggled with my budgets for database servers, trying to manage CPU, RAM, and disk costs and find a balance among them. If I can potentially use different costs of disk storage to gain more RAM or CPU power, it’s a trade-off that I would have often made.

    The trick with tiered storage is knowing your data and the access patterns. That means better understanding your queries and access patterns, and that requires better knowledge of how SQL Server works. You should learn to query DMVs and read performance metrics and apply that knowledge to your own systems. Those skills might just help you improve performance in a very cost effective way, an accomplishment that is worth bringing up in your annual review.

    Steve Jones

    The Voice of the DBA Podcasts

  • An Impressive SQL Server

    What would impress you?

    Recently I heard about one SQL Server with 1TB of RAM and another one that was handling over 10,000 transactions a second. I read a note about a 1.1PB database, built on SQL Server to hold data from telescopes. That particular data warehouse is producing 1.4TB a day. I know it’s mostly binary data, but that’s a lot of 1s and 0s being stored in SQL Server.

    Over the years I’ve heard regular complaints and concerns from executives about SQL Server scalability. Those complaints have dwindled, but I think there are people in the relational database world that still question SQL Server’s ability to handle high volumes or data sets.

    This Friday, I wanted to see what might change their mind, or maybe what might make you view SQL Server differently. While I suspect most of you are SQL Server fans, that doesn’t mean that you don’t have concerns at times about how far you can push SQL Server. The question this week is:

    What company would impress you if SQL Server backed their main systems?

    This isn’t scientific, and I’m not defining which systems impress you. It could be their web site, their supply chain management, their largest data warehouse, or anything else. What company talking about using SQL Server in one of their systems would impress you? And which system would you like to see backed by SQL Server?

    When I think about a lot of data, high volumes and rates, I think of a few things. The stock market, sports, and Amazon. If Amazon switched their website and ordering systems to SQL Server, I’d be impressed. If Major League Baseball, or the National Football League were to run all their statistical systems on SQL Server, I’d be impressed. With all the fantasy sports fans out there, the gathering and processing live data for real time calculations of performance, would be impressive.

    I’ve left out other companies. Wal-Mart using SQL Server for supply chain management would be impressive. UPS managing packages on SQL Server would be something. I’m sure have your own ideas, and let us know this week.

    Steve Jones

    The Voice of the DBA Podcasts

  • The Titanic Cloud

    Nothing is Unsinkable

    Is a cloud provider unsinkable? This piece from CNN shows an image of the Titanic as it talks about Amazon’s recent Amazon Web Services outage, which took down parts of many sites, including Reddit, FourSquare and the New York Times.

    While I think Amazon has done an amazing job of building technology well, they are no different than any large provider. Google has had issues with their services and Hotmail recently had a failure that lost data for some people. Salesforce.com has gone down and Oracle’s “unbreakable” moniker has proven to be untrue, and the unsinkable Titanic wasn’t the first large endeavor to fail.

    What’s interesting to me is that AWS wasn’t down for me. I didn’t use any of the services mentioned at the time of the outage, so as far as I was concerned, nothing happened. That’s the luck of the draw and as we move to a more connected world, with our systems reaching people all across the globe, we have a large variety of customers who will never notice our issue. The flip side is that it’s more likely at least someone will notice any of our issues.

    There isn’t a foolproof way to build a computer system that runs 100% of the time. As much as every large provider in the “cloud” will try to convince you otherwise, there will be issues. If the cloud is right for you, and fits your business, make sure that you have good backup plans in place, consider architecting your systems to work across different providers, and ensure that you always, always, always have good backups.

    More than ever I think this mantra applies to people managing data in the cloud: good backup, good resume. You need one of them.

    Steve Jones

  • Cascading Human Error

    One mistake could cascade

    I caught this short piece on the Amazon EC2 outageand while it does highlight a potential problem in the world of the cloud, there was one part of it that really stuck out to me. This quote in particular was one that has me worried about the future: “It’s the kind of error an operator could makes as a wrong choice on a menu or the entry of the name of the last network worked on instead of the one needed. In short, it was a human error that’s all too likely to occur with anyone momentarily preoccupied with the price of mangoes or a flare up with a spouse. “ That’s a little scary, and while it’s not necessarily a reason to avoid the cloud, it is potentially a reason to avoid the extremely large companies like Amazon and Microsoft. They have built these large infrastructures, using lots of standardization and hopefully, automation. In the places where humans do have to type commands or make configuration changes, it’s possible that they make a mistake. While that can happen in any size company, when it happens for Amazon, Microsoft, or Google, a large number of people can be affected. In many ways I think we have a problem with extremely large, standardized, and highly dependent infrastructures. While being able to deploy a configuration change to a 1,000 servers at once or reboot 800 for a patch is cool, it’s also potentially a problem. We used to manage nearly 1,000 Windows servers and patch the large majority at one time with SMS, but we also held our collective breaths whenever hundreds of them rebooted at once. One mistake, and it makes for a very long night. And a very long day when you are explaining the issues to management. I don’t think we will ever eliminate human error, but we can definitely minimize it. Strong QA processes should require automated deployments and ensure the exact same deployment run on QA is run in production. If we can stick to that type of process, regardless of the delays it might create, we can minimize lots of human error. Steve Jones

    The Voice of the DBA Podcasts