Voice of the DBA

Category: Editorial

  • The Complexity of Metrics

    Monitoring your SQL Server instances is important to ensure you can meet your SLAs. Availability, performance, reliability, quality, whatever you care about, it’s important that whoever is responsible is looking at how the database is performing. At Redgate, we have multiple teams working on SQL Monitor to enhance and grow it to meet your needs.

    A short while ago there was an internal conversation recently about page life expectancy. We’ve had some customers ask about this and setting alerts to watch this value. Our developers and sales engineers asked for a few thoughts from Grant and others on how to respond. There are a variety of opinions, some saying monitor it, some saying don’t bother.

    I think both pieces of advice have merit, which is to say that this isn’t a metric that you can look at in isolation. There is no value of PLE that is good or bad, or that says x is wrong or y is right. There is both a subtlety and a complexity to understanding what PLE is telling you about your system. If PLE is growing, you have to look deeper. If it’s falling, same thing. If it suddenly drops, there are multiple possible causes, and you need to examine other things. However, in many cases, this isn’t an actionable metric, but one that provides context about what might be happening in the database when combined with other values you monitor.

    This certainly isn’t a metric that you want to set an alert on because it can rise or fall and many times the change isn’t indicative of an acute problem.

    This is just one metric of many that are available in SQL Server, and knowing which ones to monitor is something good administrators learn. They know that very few values they instrument have a good or bad value, and often the rate of change needs to be combined with the actual reading to determine if there is a problem. We also often want to know if a high (or low) reading appears for an extended period of time. Having 100% CPU being used for 3 minutes likely isn’t an issue. If it lasts for 3 hours, I might feel differently.

    Metrics have more complexity than just having a range in which we ignore them and a limit at which we alert people. They are intended to be combined with each other, with observations by clients, and with the experience of looking at past observations over time. Our systems often develop patterns, and we don’t get too concerned about any values when the pattern repeats. It’s when something new happens and someone complains that we dig in to determine if there is a problem or the start of a new pattern.

    We definitely need monitoring of our database metrics, but we also need to understand why values move and the implications of them doing so. That’s something which isn’t as simple as setting alert for each one based on some value we think should never be exceeded.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Speakers Are Normal People

    The #SQLFamily is amazing, at least I think it is. Like many families, it’s welcoming, supportive, and comforting. It’s also maddening, frustrating, and exacerbating at times. Like most families, or at least the ones I know, it’s not perfect, but it’s what we have and at the end of the day, most of us get along with each other.

    It’s also an open group of people. In general, we welcome people with open arms and smiles. Those of us that are more visible or prominent are willing to listen to, help, and support anyone. I was overjoyed during the recent PASS Data Community Summit, where I had the chance to see so many people that I haven’t seen in person in 2-3 years. I met many other interesting people for the first time and enjoyed the experience.

    Not everyone feels the same way. I loved seeing Kimberly Tripp and Paul Randal for the first time in years and was honored to share the stage with them for the Community Keynote. I enjoyed the time we spent together, but afterwards Paul wrote about some people not feeling included or welcome.

    I understand that feeling. In many ways, much of adult life can mimic teenage years in high school with cliques and pettiness. I won’t pretend that doesn’t happen in the #SQLFamily, but I find it quite minimized, especially compared with many other communities of which I’ve been a part in the past. I haven’t seen the higher profile speakers and leaders in the SQL Community dismiss someone for asking a question or expressing an opinion. I have, however, seen that in other communities.

    It can be intimidating to walk up to someone that you don’t know who you might feel is famous or well known. It can be intimidating to just walk up to a group of people who are talking when they appear to know each other. I have that feeling at times even today, so I appreciate feeling like an outsider. In the Summit keynote, I talked about the thrill in meeting Kalen Delaney in 1999 and shaking her hand. I was nervous and intimidated to ask her a question after her presentation. At the time I hadn’t delivered a talk in front of anyone outside of school environments and was a fairly introverted geek. It was hard to step up and make that effort, but I’m glad I did.

    As Paul writes, anyone is welcome in the #SQLFamily. Anyone can join. You don’t have to come shake a hand or say hi, but I’m happy when you do. Will we be best friends right away? Probably not. Will we go out to dinner that night? Maybe. I’ve certainly met attendees at events and then had dinner with them. I know plenty of other speakers who have as well.

    Many of the people who speak, organize, and write/blog/tweet/etc. in our industry are friends. We do value time with each other, and that can feel like a club, but it’s not. We enjoy seeing each other and want to catch up, like any group of friends. However, we are also welcoming of newcomers, so feel free to introduce yourself.

    Ultimately those of us who engage in these highly visible, extroverted acts are often just like the rest of you. We’re a mix of people that are mostly introverted, with a few extroverts thrown in. Some speakers are very smart and talented, some are more like me: we know enough to get the things done that we’re asked to do.  Some of us love to go out and sing karaoke until all hours of the night and others prefer a small dinner or a little tabletop gaming in a quiet environment.

    My encouragement to get people to meet others, network, set up events or meetups, and more isn’t to try and convince any of you to join the cool kids club. It’s not to get you to change who you are.

    It’s to help you find your tribe. To find your kind of people.

    I’d love to greet all of you with a hug at events, call you by name, and go out to dinner with you. I can’t because there isn’t enough time in the day. And quite frankly, I really, really value my alone time. What I want more than anything is for you to be successful, find a great job or career you relish, and for you to enjoy spending time with those you enjoy. Whoever and wherever that is. That takes some effort, but it’s worth the energy involved.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Losing Track of Data

    I saw this article a few months ago, which talks about engineers at Facebook not knowing where their customers’ personal data is stored. The engineers were being questioned in a legal matter, where they were asked to definitively state where all personal PII data for any human was stored by Facebook. Their answer was that they didn’t think anyone in the company would be able to answer that question.

    Facebook has been controversial over the years, and plenty of people dislike the way the company conducts business. I noticed no shortage of data people (and many others) commenting on this situation, saying that Facebook should be shut down because they don’t know where data is being stored.

    However, I don’t agree. In working with lots of customers, on all aspects of how they handle, process, and manage data, I expect this to be a problem in many organizations. Whether large or small, whether they have few or many software engineers, it is highly possible that there isn’t a good list of where personal data is being stored. As we work with customers to classify data with SQL Data Catalog, that process takes a long time, and very often the system administrators or developers who undertake take the task are unaware of all the places where data is stored.

    That’s just in relational databases, ignoring all the Excel spreadsheets, text exports, mail merge operations, and uploads to services for mailing, analysis, or something else. Very often the control of personal data is fragmented among groups, with there being few efforts made to coherently manage a customer’s data.

    The world has adopted computing at an incredibly fast pace, often by people with little knowledge or forethought of the implications of gathering and processing data. In many cases, probably most cases, there is no overriding strategy. Just like with applications slapped together quickly, we find data being gathered and stored based on the requirements and demands of business people, with no planning for management or archival, and often not even with any security requirements.

    I liked the GDPR as a step forward, asking companies to not only handle data appropriately, but remove it when not needed, not use it without consent, and to be able to keep track and delete it if not necessary. I don’t know that this has been successful, but it has changed handling practices in some organizations. At least in responsible organizations, and many of them have had to track down personal data to delete it. I’m not sure they know where it all is, but I at least assume they know where all of the data about a person is in their various relational stores.

    As a technical person, do you know where all data is stored about a customer? Are you sure you know where marketing has been keeping information and what other mailing, analysis, reporting, CRM, etc. systems they’ve put data? Any idea how many copies the operations group keeps? Test systems, QA, UAT, and others? What about test data sets, are they sanitized? Perhaps legal or finance has gotten extracts of data to reconcile their systems.

    Tracking down all data can be hard, and I’m not surprised Facebook struggles. I would guess engineers in many organizations would have similar answers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Challenge of Deleting Data

    We collect a lot of data in our databases. Not as much in bytes as a lot of the video/audio/TikTok/Instagram sites, but still enough that many of us are constantly adding storage to our systems. All this data is not only a challenge to manage, but it also means that we are regularly dealing with query tuning issues. Better code, indexes, and more become regular challenges with large volumes of data.

    I am a big fan of trying to reduce the data you manage where possible. Archive, delete, remove older data, do something. This not only makes your systems easier to manage and improves performance, but it reduces your risk. Any PII data you have that might store is an ongoing risk in the event of a data breach. I don’t pretend this is easy to do in any way, but it’s a good idea.

    If you can remove data (or must because of a regulation like the GDPR), how do you ensure that data is deleted? Most of us know how to submit a DELETE statement, but that just removes the data from an online system. What if you restored or recovered this database tomorrow, would you remember to delete the data again? What about losing a copy of the data or log backup? What about older dev/test systems that were refreshed from production? The data might be in there. If you work through the possible problems, deleting data from a system isn’t as simple as you might expect.

    This might be even more complex in the age of cloud computing, where we don’t control the hardware for primary systems, or for backups. There is an article on deleting data in the cloud that talks about the government standards that require that you not only delete data, but that you overwrite the physical hardware to ensure it can’t be recovered. This still doesn’t address backup systems, but it does help to clarify that many of us might start to demand cloud vendors not only de-allocate the disks we use (or the backup storage), but they also overwrite the storage with zeros.

    Data security and the risks of not taking this seriously is becoming a bigger issue all the time. I don’t know that poor security will cause your organization to fail, but there can be significant costs and possibly reduced employment opportunities. While you might not want to be overly paranoid or concerned about every possible issue, it is worth asking questions of vendors, working through likely scenarios, and trying to quantify risk.

    More and more systems are regularly under attack from malicious groups, which means we want to minimize simple mistakes, reduce human error, and limit the exposure we have from the data we have by storing only the data we need.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.