Voice of the DBA

Category: Editorial

  • Learn to Use Filegroups

    In SQL Server, filegroups are a management technique that I don’t see many people using. It’s amazing how many people ask questions about filegroups on the discussion forums, often unsure of how they fit into a well architected SQL Server. I have tended to use filegroups mostly as a space management technique, when I need to add more disks to my server, but they can be used in many more places.

    We continue to grow our data sizes all the time. While many databases are still measured in the single digits of gigabytes (or smaller), it is fairly common to find many database servers with over a terabyte of disk space. Our disks grow larger and larger, but it seems that data grows faster than disks, requiring larger storage subsystems all the time.

    While our storage grows larger, the tolerance for delays shrinks and demands for better performance increase. That means that data professionals need to be more cognizant of not only how their code is written, but also how they design storage. Tiering storage is one idea that I think has merit, but one that requires some planning.

    In SQL Server, we can’t split a table across filegroups. Or can we? We can partition a table (Enterprise Edition and higher), which can allow us to manage performance and storage appropriately. There is also the recommended practice of only having system objects in the primary partition and using separate filegroups for user data. That allows you to bring a partial database online, again, in Enterprise Edition only, while you restore different filegroups.

    This isn’t the first thing I would recommend you learn about SQL Server, but as you advance your knowledge, you should better understand when and how filegroups can help you. You will use them at some point and being comfortable with a filegroup restore is one of the skills that separates the accidental DBA from the data professional.

    Steve Jones

    The Voice of the DBA Podcasts

  • The Poor Soul

    Poor Souls

    I recently had someone post this after I made a comment about someone taking responsibility for their database server.

    “So Steve, what would you recommend then for those poor souls?  As being one of them it drives me nuts to constantly be told I can’t get more training as it’s not my true job position yet I’m responsible for making the databases work. “

    I’ve been in this spot a few times, in and out of IT. I’ve been tossed into cooking or bartending jobs without training and had to learn quickly how to do the job, and I’ve had the same thing happen in IT. In a few cases the companies knew it was a bad situation and they eventually got me training, in others they didn’t. Here’s the advice I have for you.

    Ultimately you are responsible for the job. Tough love, but you’re being paid for that job, so you are responsible.  That means you have to learn how the technology in your environment works and how to solve the problems you have.

    First, get your resume up to date. Make sure it is ready for submission, and you are prepared to get fired every week.  Keep an eye on the job market and save some extra money, because to me, the financial security for my family comes first.

    Second, learn to restore data and then make sure you have backups in place. This is secondary because if you have a failure quick, you want to be ready to get a new job. But ultimately no matter what breaks or doesn’t work, getting data back first is crucial.

    As you go through all of this, you might be fighting fires. So while you practice restores or document the environment, you might be trying to fix things and asking questions of others, but invest the time to get yourself into a solid position.

    I’d also talk to my boss regularly. Every time I found a place I didn’t know something, I’d make a note and let my boss know this is a hole. Maybe I can learn it, maybe I can’t, but I could use help. That might be the best ROI for conferences. Go, make friends, get contacts that can fill your knowledge holes. Or find consultants you can call.

    Steve Jones

    The Voice of the DBA Podcasts

  • Dropping a Row

    Could you drop a row in your system?

    In a scalability discussion, I saw this comment: Companies like Google or FaceBook manage a lot of data, but it’s not held the same degree of scrutiny. For example, if FaceBook dropped 1 out of 1,000 random guestbook posts, would anyone notice? At the end of the day would they even care enough for it to make national headline news?

    How many companies would accept a random 1 out of 1,000 dropped data entry row? Or an update that didn’t take? Most management in companies I’ve worked for wouldn’t even want to think about accepting that level data loss.

    Ultimately I think this points out the difference between some of the non-RDBMS platforms  that can accept some data loss. Even Google, as amazing as their results are and with lots of redundancy, aren’t held to some large standard of data integrity. If two of us search for the same term at the same time and get different results, is that an issue? Or to put it another way, if the CFO and CEO both run reports at the same time, can they differ in their results?

    For most of us, the answer is that the results cannot differ. While I think most of the NoSQL and other non-RDBMS architectures have a lot of effort put into ensuring that data gets hardened on a node when it is updated, there can be a lack of consistency between nodes. A node could lag behind others or even fail before synchronization with other nodes. That is a concern in any system that looks to scale out to a large number of servers, and an even larger concern for data whose integrity is critical.

    An amazing level of thought has gone into SQL Server to provide extremely high levels of data integrity. Every time I think I’ve found a problem or hole in the product, it seems someone at SQLskills explains the reason behind the architecture. The answer usually makes perfect sense to me and has me wondering what else I will learn in one of their Immersion training weeks. Hopefully I’ll get to one soon.

    There are definitely places where you might accept dropped rows. Information published on intranets, an application recording vacation requests, and any other number of small non-critical systems. SQL Server is not a good fit for all database applications, but for those that use it, you can be sure that none of your rows will be dropped.

    Steve Jones

    The Voice of the DBA Podcasts

  • Tiered Storage

    Storage, storage, storage

    One of the things that I saw demonstrated at my very first PDC conference in 1998 was the addition of Hierarchical Storage Management to Windows servers. This was in the Windows NT 4 era, and I was investigating a variety of ways that we could potentially handle a large number of fax and scanned images for my company. We eventually implemented a RW optical jukebox to help us manage our large collection of images and it worked very well in helping us manage costs.

    I saw this great article recently on tiered storage that uses a database system as an example of how you can potentially improve the performance of your system and manage costs by using different types of storage. It’s worth the read, and talks about a fictional example of how we might use SSDs for frequently accessed data and SATA HDDs for cold, less accessed data.

    As we collect and store more and more data, I think that we will find that we are storing lots of data that we potentially access very infrequently.  If that is the case, then we ought to be considering different types of storage that can handle the needs of that particular set of data while also managing costs. I have always struggled with my budgets for database servers, trying to manage CPU, RAM, and disk costs and find a balance among them. If I can potentially use different costs of disk storage to gain more RAM or CPU power, it’s a trade-off that I would have often made.

    The trick with tiered storage is knowing your data and the access patterns. That means better understanding your queries and access patterns, and that requires better knowledge of how SQL Server works. You should learn to query DMVs and read performance metrics and apply that knowledge to your own systems. Those skills might just help you improve performance in a very cost effective way, an accomplishment that is worth bringing up in your annual review.

    Steve Jones

    The Voice of the DBA Podcasts