Voice of the DBA

Tag: Database Weekly

  • The Challenges of Space

    It wasn’t that long ago in my career when a survey showed the vast majority of SQL Server databases were < 10GB in size. These days I’m sure the average has risen as there are some very, very large SQL Server databases. I know a few people managing PB sized systems, which certainly will move the average upa bit. It’s likely, however, that the majority of databases are still very small, powering smaller applications with < 100GB.

    However almost every databases continues to grow, and as systems grow, new challenges come into play. While we can still store a 100GB backup file on a thumb drive or SD card, it’s still a significant amount of data to write, which can take more time than our clients would like. Archivals, backups, restore, all tend to come from non-flash media, so larger systems certainly mean longer RTOs if we have issues, even if we’re just restoring a copy of a database to extract some deleted information.

    However there can be more fundamental problems. I ran across a post from the CSS Engineers at Microsoft that talks about exhausting NTFS resources as more and more writes occur on a filesystem. I’d never heard of this, or expected it to be an issue, but apparently it’s occurring more often. I suspect even the NTFS developers hadn’t considered the scales to which we probably stress the file system daily. What is interesting is that this can occur with DBCC operations, database snapshots, and even bcp operations. There are workarounds, and certainly I’m sure that Microsoft has people working hard to increase the capacities of ReFS, which I expect will take over from NTFS, just as NTFS has largely displaced FAT.

    As we grow to larger and larger scales of data sizes, with multi-terabytes on laptops and petabytes on servers, I think we’ll need to adapt a number of our techniques and operations in the future. We might end up needing different types of backups (anyone considering lookup table backups or daily backups of a set of rows, like orders?), as well as the possibility of continuing to spread the load of our server across multiple drives. I just hope that Microsoft recognizes this and looks to ensure mount points work smoothly across all of their technologies and features. At some point, 26 drive letters won’t be enough for the majority of us.

    Steve Jones

  • Awareness

    How often have you been caught off guard by some issue with one of your databases? I would guess most people have received a phone call, email, or trouble ticket that alerts them to the fact that something has gone wrong. A few of you might have even had an irate customer or manager wander over to your desk and loudly inform you of an issue. Those are not great times, and most of us would prefer to find out about problems before others bring them to our attention.

    This past week was the 66th T-SQL Tuesday blog party. If you haven’t heard of it, I would encourage you to look each month for the announcement on Twitter and participate. It’s a great chance to share some knowledge, and showcase your own experiences or viewpoint on a particular topic. I’d actually encourage all of you to go back and write posts about the past topics at some point. It’s a great way to challenge yourself to document your own experiences in a concrete way.

    The theme this month was monitoring (thanks to Catherine Wilhelmsen b | t), and this is exactly the way that you build awareness in your environment. Monitoring allows you to ensure that you, as the DBA, learn about issues, or potential issues, first. Monitoring doesn’t solve your problems; that’s up to you to do, but it can help you proactively detect issues early and perhaps mitigage them before users notice them.

    The blog entries this month cover all sorts of issues that you might think about watching for. The 825 errors in your log, metadata about your backups, jobs, and more. There’s a variety of posts that cover all sorts of home grown systems, as well as noting that third party products work well. You have a huge variety of choices, from building your own tools and processes or purchasing something like SQL Monitor from Red Gate. However no matter what your budget, resources, time, desire, and more, I’d highly recommend that you really think about ensuring that you are monitoring your systems in a way that lets you know how things are working.

    What to do when they’re working poorly, that’s up to you. Hopefully the information we publish at SQLServerCentral, and the links we send out every week from Database Weekly, can help you learn to be better at your job.

  • The Devil’s in the Details

    This week I was scanning through a number of SQL Server posts (in between working on April Fool’s Jokes) and a couple of them caught my eye. They dealt with simple subjects, but subjects whose details are important. I’ve often found people have had performance issues in SQL Server because of simple misunderstandings of how the system works, or because not enough weight is given to the impact of small details.
    Kimberly Tripp (of SQLskills) wrote a basic post on parameters, variables, and literals, which would think most programmers would understand. However I realize more and more that lots of people that write code in the world taught themselves. That’s amazing, but it does mean that people have many different holes in their knowledge. What I might consider basic or common knowledge might be something that another person never had been exposed to. In any case, Kimberly does a fantastic job of laying out the differences in these three concepts, how they work side by side, and how the choice can dramatically impact performance of your code. Read this one.
    The other post this week was a fairly simple post on data conversions by Rob Sheldon. Again, precedence and conversions are something I dealt with when writing C code, and you do not want to get things wrong there. However many people might not have the same background and wouldn’t understand by varchar and nvarchar wouldn’t be a basic, simple conversion with little cost. Or why the type of conversion might result in far different results than you expect. Since we often test with one set of data and our application work with a (much) wider set of data, this one is a must read as well.
    There are so many things you should consider when writing code it’s really a constant and regular effort to improve your skills in this business. You won’t learn everything, or learn something every day, but pick things up and practice with them. Learn to build better code, one technique at a time, and cement that knowledge with actual implementation in your work.
    You’ve got at least two new things to look at this week, so pick one and try to ensure your code next week considers these details.
    Steve Jones

  • Which Problem Are You Solving?

    One of the problems I’ve seen for years in computing is people seeing the forest or the trees. Not that either is good or bad, but far too often I’ve run across people that only see one of these, meaning they examine problems from a high level perspective, or a very low level perspective. While either of those can help you solve your issues, there will be a problem or a time when you need to examine the situation from another perspective. Knowing when to look at a higher or lower level can be invaluable in troubleshooting.

    This past week I ran across a great post from Grant Fritchey called Understand the True Source of Problems. In this post, Grant talks about some strange DBA “best practices” that were implemented to solve problems, but aren’t necessarily the best of anything. It’s an interesting list, and you might read it for entertainment, though I hope none of you are following these ideals at your place of employment.

    However the thing I liked about this post is that in the beginning Grant begins with an old doctor joke, which is still funny to me, but it often is the approache taken by people trying to solve problems. They just avoid the issue, and never solve it. That can work in the short term, but I’ve seen this build up over time to the point where people are afraid to make any changes to a system or code because of the fear of causing some unknown issue.

    I don’t think that every issue needs to be solve to the root cause. There are plenty of times something goes sideways on one of my computers and a reboot fixes the issue. I’m fine with that, but if it occurs more than two or three times, I do need to solve the problem. I need to be able to dig deeper to diagnose what resources are failing, what performance issue exists, or maybe what line of code is broken. Perhaps I need to step back and examine if some other part of my system, like the disk or network, is causing an issue that appears to be isolated to code. In other words, I need to be able to look at the wider environment (the forest) or dive into the particulars of a process (the trees).

    Knowing how to strike this balance is a bit of an art, and it’s what really makes the best problem solvers experts. I can’t teach you that part of computing, but I can teach you this. When you don’t think you’ve completely solved the issue, stop for a minute and take a wider or narrower view. Or maybe both. Look for what you might not have thought of. If you’re like me, there are probably plenty of possibilities that come to mind when you stop and re-examine the issue. Then work your way through those items, discarding the ones that aren’t relevant. Sometimes opening your mind a bit is the best way to find the exact cause of an issue.

    Steve Jones