Voice of the DBA

Tag: administration

Monday Monitor Tips: Looking Back in Time

Often we find out about a problem reported by a customer after the incident has passed. This might be from a trouble ticket or even an email that we didn’t see until a period of time has passed.

How can we look back at the activity of a server in the past? This post looks how a DBA can time travel back to a situation that occurred in the past.

This is part of a series of posts on Redgate Monitor. Click to see the other posts

Time Traveling

Let’s imagine I get a ticket that said there was a problem at 2:15am from a user running a process. I didn’t get to this at 2am, but at 9:15am when I receive it, I need to look back at what was happening.

If I pick a server in Redgate Monitor, I’ll see the view below. This is of the staging02 server on monitor.red-gate.com. By default, this shows me the last hour of activity on the server.

In the upper right corner, I can see the time frame selected on the left (below) and the amount of time. I’ve selected the drop down, and there are many other choices. I also see the metric time at the top, just in case, I’ve started to mess with other values.

Note: there is a calendar control to the left that can go back to previous days if you don’t want to use the time duration drop down.

In this case, let’s jump to the last 12 hours. If I select that, you can see my display changes a bit, zoomed out to show 12 hours not 1. The four charts below haven’t changed, however.

Most of the top chart has a darker background, except for a portion at the far right, which has a white background. This white background part is the focus window, and it determines what the 4 graphs below show, as well as the query information and other data.

This is set to 1 hour, but I can expand it. If I drag the box on the left side of this further to the left, I can expand the amount of time shown. If you look below, I’ve expanded this to 7:37am as the start.

I can also slide this. I’ll slide this to the left to cover to 2:00am-3:00am part of the graph. Now I see different views below in the four graphs.

In this case, I now can focus on the 2:00am issue. I see an annotation that there was a Flyway deployment at 2:00am. You can see the annotation zoomed in with the tooltip when I hover the mouse on this icon.

I can scroll down to the query area, and I see the top queries, of which there were just a few.

The top one has a lot of duration, and if I expand it, I can see the query history. Note there was a query plan change just after 2:00, when my deployment occurred. The duration went up and then started to slightly drop. I see another plan change at 2:40am, and if I were to look back at the top, I’d see a second deployment from Flyway at that time.

I don’t quite know what changed in the deployment, but I’d start looking here to see if this affected my query.

Summary

The focus window in the overview for an instance allows you to set the time frame in which you see data related to that instance. This lets you time travel back to look at the server as it existed in the past. The amount of time you can travel back depends on your data retention settings, which we’ll examine in another tip.

Hopefully this gives you a quick tip on how you can focus your efforts to a relevant period of time when you get an issue to review.

Redgate Monitor is a world class monitoring solution for your database estate. Download a trial today and see how it can help you manage your estate more efficiently.

March 24, 2025
Tracking Table Sizes

Managing a production database can be a challenge for many reasons. However storage growth has been one I see often as causing issues. From full logs to filled disks, it seems that we can struggle with dynamic workloads where we have no control over who can add data to systems.

Many DBAs have some sort of alert set on data files or disks, but even with those alerts, an unexpected load or runaway query could still fill a disk. I know I’ve been called back into the office because some business person decided to load an unexpected amount of data. In fact, more than a few times someone thought their load didn’t work for some reason and repeated loaded a large amount of data, filling a disk or log file.

Recently, I had a customer ask about tracking table sizes daily in their databases. They were looking to watch table growth. Another person dismissed this as not valuable, which was interesting to me. I’ve done this in the past, and I’ve found it valuable. No knowing the size every day, but having an idea of the growth factor. I didn’t keep this data long, usually a week rolling older data off as I added more. That was enough for me to trend how quickly a table was growing.

This also allowed me to set an alert if there was unusual growth for a table, and often track down a potential issue quickly. Either I’d realize our workload changed and tables were growing faster, or I could debug a sudden growth issue down to some sort of data load. Often I could catch an issue before I received an alert (or phone call).

I wonder how many of you track table sizes and if you find it beneficial. Is this something that helps you better understand your system or is the total size of the database good enough.

Or maybe you have so much storage allocated you don’t worry about space. Lucky you.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

March 7, 2025
Extended Event Comfort

I saw an interesting thread recently in the SQL Server Community Slack where someone posted about extended events (XE). They were asking about whether XE would have a problem with a situation. The problem wasn’t so interesting, but a quote from one of the responders was. The quote was:

The best time to have learned Extended Events was ten years ago. The second best time is today.

I love that, and I tend to agree. If you need to trace what is happening inside your SQL Server, you need to learn how to capture information with Extended Events. That’s the best way to dig into the details of how queries affect your system.

It’s also hard. I know that whenever I need to use it, which is rare, I have to dig through some articles and docs to understand what thing I need to do. Even having some scripts hasn’t helped because it’s a sufficiently complex system that unless I use it regularly, I forget how all the filters, targets, events, etc. work.

On one hand, I think it’s amazing, and on the other, it’s too hard to use. Even when I try the Extended Events profiler, it’s so different from Profiler that I find myself getting frustrated at times trying to dig through the information.

I am curious how many of you think XE is easy to configure and if you use it often. What are the places it works well? For those of you that don’t use XE or haven’t learned, why not? Do you not have to trace what’s happening with queries in some detail? Or do you have another way that you dive deep into your system? Or do you not have the need?

If you do want to learn more, we have a short Stairway Series on Extended Events to help you get started, as well as a few other articles. If you’re an expert, we’d love a few more on using XE in specific situations.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

February 10, 2025
How Do You Patch 100 Database Servers?

In most of the organizations I’ve worked for or consulted with, patching was always a challenge. Patching hasn’t usually been given a priority and is often skipped when operations staff is busy. This has resulted in lots of un-patched, or slowly patched systems. I assume this is one reason Microsoft continues to release RTM-GDR patches because some people won’t patch at all unless there are critical fixes.

I also know that much of IT management sees patching systems like patching parking lots. Needs to be done, but tomorrow, after we do other important work today.

Patching isn’t easy, in fact, Allan Hirt says it was never easy, but these days we don’t get the downtime over a weekend to patch, and there is a desire to patch security issues immediately because of the potential reputational (more likely) or regulatory (less likely) risks. Also, we often need to patch dozens, if not hundreds or thousands, of systems.

So is there a way that most organizations do this? It’s interesting in the piece above that Allan notes that most of us have technical debt, and this debt consists of more than just code and systems. It’s also people, budget, politics, and more. This is even more of an issue if you didn’t write the software. Applications often limit patches or upgrades, and it can be hard enough to get applications certified on new platforms when you control everything. If you purchased something from a vendor, you might be bound by their timelines not your own.

So how do you patch a lot of systems? There are lots of ideas and potential solutions. I’m sure Allan covered some recently in his session at the PASS Data Community Summit. For me, this boils down to building part of a process, using it, evaluating it, and then rolling it out wider. This might mean I need two processes because some systems will lag behind others for various reasons. I’d probably spend a year or two slowly adding to this process and getting it better, all the while ensuring I used automation as much as humanly possible to process notifications, approvals, and actually deploy code.

Start small, experiment, test, evaluate, make changes. Always codifying things that I can. It’s a method that has worked for a long time.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

November 18, 2024