Voice of the DBA

Tag: administration

  • Reserved Space

    Many of us that work with SQL Server are familiar with the idea of reserving space that our system might need later. This is normal operation in SQL Server, where we pre-allocate space to data and log files, knowing that SQL Server will manage and use this space over time. This can often be a strange concept for newcomers to the platform, who expect our database behaves like a text file, only using the space it needs for data. Choosing the amount of space to reserve is an art that many of us both struggle to articulate and explain to newcomers. It’s also a problem that we often solve by guessing high and implementing monitoring.

    Lots of people worry about storage, even in this era of TB sized drives. I ran across an article noting that Windows is reserving 7GB of space for updates, which makes sense to me. I’ve had a few updates for smaller devices, like mobiles, fail because of space. I think the idea of having some space reserved for temp storage and then using that for updates is a good one. I also recognize that I’m in a different place than others. Most of my devices have plenty of free space. Well, not my phone, but that’s a separate issue.

    From the comments of the article, it seems that many people feel as I do. Reserving the space is a good idea. There are plenty of complaints about older, small storage devices, and I feel bad for those people. I’ve own a 32GB Windows device and over time it’s a pain. Those devices likely need to have some sort of network/cloud storage to be usable, so There are also no shortage of the “Windows is already too large” complaints, but to me, those are separate discussions.

    Ensuring enough space is around for normal operation is one thing. Keeping extra around for emergencies and critical functions is another. I’ve been a big fan of placeholders on drives, especially database servers. I keep a few on all my drives, both on desktops and laptops. If I run low on space and really need some, I can make the decision to remove these without thought. Something I can’t always do with other files.

    Like the DBCC SHRINKFILE, which lets us get out of situations where we’ve made a mistake in allocation, I’d like all OSes to include some options that remove older files, such as updates, temp files, etc. that we might not need, but could make a difference in our space usage. I know modern Android helps me when I’m low on space, and would like to see that in Windos. My other hope is that  we get quantum, PB sized storage devices that can fit in our phone. I can’t ever remember anyone on Star Trek running out of storage space on their drives. I look forward to the day in my life when that won’t happen to me.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 6.3MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Mitigate Issues Early

    This is a great postmortem from Basecamp with a detailed explanation of a problem and how they are hoping to avoid issues with their service. Basecamp is a Software as a Service tool from 37 Signals that had an outage recently. Actually, the service was up and could be read, but new items couldn’t be added and existing items couldn’t be changed. That somewhat defeats the purpose of the tool. 37 Signals recognizes this and notes that they’re calling this downtime and not trying to get pedantic about the state of the service.

    I have sympathy here because I’ve gotten the same error in SQL Sever. I’ve reached the point where I had 2,147,483,647 items in a table. Any additional items caused an error, which some of you might guess. This is the largest positive INT value and if you’ve started at 0 and incremented all new rows, when you reach this mark, you can’t add more data. There are a few fixes for this, including moving to negative numbers or changing to a bigint type.

    The staff at 37 Signals knew this was an issue and had actually set a new default for Basecamp that used a bigint in MySQL and PostgreSQL. What they didn’t do is actually make the changes in their hosted Basecamp database because they thought they had more time. I’ve seen that before, often when an application runs at a higher workload than expected for a few years and DBAs aren’t paying attention. When staff least expects it, the INT will reach the max value an clients start complaining. I’ve had to respond to an urgent phone call about this in a few companies.

    If you use INT values that are being incremented in some standard way, I’d likely set some alert process that triggers when the value gets to 2 billion or so, depending on how fast you use the numbers. SQL Monitor can do this with a custom metric, and I’m sure other software can as well, including SQL Agent. Having some monitoring in place will help you plan and be prepared to address issues before they cause problems.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Moving to Query Store

    In SQL Server 2016, Microsoft introduced the Query Data Store (QDS) as a tool that would capture data about the execution of queries inside of your database. This was a project that had been in the works for a number of years, and one that many of us that were bound by NDA agreements had been following. We were excited by the chance to actually gather some information on the.

    Are you using Query Store? You should be, as this tool will become more valuable over time. I know that there are potential overhead issues (3-5% for most people, but possibly larger). I would argue that the potential for better performance and understanding of our systems outweighs the overhead. After all, if we’re unwilling to devote some resources to measuring our systems, how do we really know what to improve?

    We upgraded the SQLServerCentral servers to SQL Server 2017 this year (2018), and I’ve been wanting to enable the Query Store. I’ve been slightly hesitant with over 75,000 air miles and 5,000 driving miles on the road since the upgrade. Being distracted and out of my routine isn’t the best way to document and carefully observe the effects of a change. Not to mention concerns over data leakage for a company bound by the GDPR. After a little discussion and debate, and my schedule slowing, I’m looking to change that soon.

    I don’t expect that a lot of improvement at SQLServerCentral from changing this, as our third party forums and much of the internal code is batch SQL, and quite a bit generated on the fly. However, there are some stored procedures, and I might be very wrong. While we’re over provisioned with resources to avoid any performance problems, I do expect that we’ll learn a few things. I hope we find places to better tune code, and with some documentation of the process, hopefully some of you out there might spot things our team doesn’t.

    If you’ve got stories of the QDS working well or not well, let us know. Certainly let Microsoft know as well. The QDS is a major part of the SQL Server platform improving in the future and there are enhancements in SQL Server 2019. While I don’t know that the QDS and some of the automatic tuning features remove the need for a data professional to watch a system, I’d like to think they do provide opportunities and insight for how we might better structure and develop applications, as well as help us find better patterns that are useful in our initial database coding.

    If you’ve got stories, Erin Stellato wants to know (and she has a few in the post). If you’re concerned about overhead, read her other post. If you’re confused, we’re working on some articles to help you learn more. Give the QDS a try, especially if you’ve got some less critical systems. Part of our job is learning how to use new tools, and this is one that ought to be on most DBAs ToDo list.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • New Database Options

    I saw recently that Azure SQL Database is getting a few more Database Scoped Options for that platform. These are intended to give more control over the way in which the engine behaves, without requiring each database on a server to function the same way. I expect to get to the on-premises product at some point, where they’ll be even more useful as we often might want different behavior for different contexts on an instance.

    While there are advantages to managing all databases in an instance in the same way, I do think that more and more we consolidate databases at times and it’s better to have additional control when needed at the database level. This week, I wonder if there are things that you wish you would have been able to specify for each individual database.

    What options would you want to see added at the database level? 

    I think that many of the options we’ve been given in current versions, as well as the newer ones appearing in SQL Server 2019 are a good start. I don’t know which instance level settings I might want here, but I certainly would like to see newer capabilities at the database level. It would be nice to see the capabilities for jobs and alerts to be set at the database level. Even if this were a part of the Agent subsystem, having the ability to keep these jobs within a database and have the agent read them would be useful.

    Moving more capabilities to the database level gives us more flexibility in separating the workloads for different applications. With the movement of the platform code, and many customers, to Azure SQL Database where the system requires less dependence on an instance, it makes sense to start including more options at the database level. I would guess that at some point most of the settings that we need for manage a system will be included and set at the database level.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and Libsyn.