Voice of the DBA

Tag: administration

  • What is the Future of Monitoring?

    I read Tom LaRock’s post on the future of monitoring, looking at the rise of machine learning and complex algorithms to evaluate systems. Tom notes that the data input into a system is crucial in determining whether the system can produce a prediction or output that has value. Since the humans that decide what data is included may not choose wisely, the system won’t necessarily reduce the time required to manage a system.

    I tend to agree. From my conversations with those people having success with machine learning systems, the data preparation is the most critical (and difficult) step. Shortcut the cleansing and organizing, or make poor choices of what data to include and you won’t likely get actionable results.

    However, I have a different view of monitoring. After my watching of the Darpa Grand Challenge (editorial on this recently), I think the future of monitoring is perhaps actually making changes to code in-line. Perhaps with approval from humans, but I suspect that we can train some process to understand how particular batches can cause issues. We can certainly set filters that might note index changes on large tables could be problematic and should be investigated before an issue arises. We can teach a system to recognize code deployments, perhaps even roll back certain changes if the application fails. We certainly could have a machine learning system watching index usage and query plans to recommend indexes in a real time manner, perhaps even turning on and off on-line rebuilds.

    I think there is possibility, but where I’d really like to see advanced monitoring is not in production environments. I’d like to see better systems that can watch development, helping suggest or rewrite SQL before it’s deployed. Perhaps applications can warning developers of potential performance or security issues. Such systems could help us in building more consistency into our applications. We can have systems that help our developers code better.

    As our databases and software become more critical to the functioning of most of our businesses, we certainly need more reliable and robust development practices.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn

  • The Danger of xp_cmdshell

    Securing a computer is a challenge. There are all sorts of potential issues in every platform, and ensuring safety for your data can be less a reflection of your ability and more the good fortune there isn’t a focused effort to attack your systems. However, we certainly also face issues with inside users, many of which may make mistakes that are accidental more than malicious. It’s for these reasons that we look for secure by default applications and a reduced surface area for any system.

    Many people refuse to turn on xp_cmdshell as an option for scripting in SQL Server. This is disabled by default, and quite a few DBAs are glad of this setting. However, there are plenty of people that think xp_cmdshell isn’t a big security risk. There are certainly ways to mitigate the usage by non-privileged users, and this can be a tool that is very handy for accomplishing work without a lot of development time.

    This week, as security issues become more important to us all, I’m curious how you feel.

    Do you think xp_cmdshell is dangerous?

    I have to admit that I’m torn. I don’t think this inherently dangerous. It does open up some attack vectors, but the last few versions of SQL Server have allowed some limitations, so I would enable this if needed to solve some issues without too many concerns. However, I wonder if many of you feel the same way.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and Mevio .

  • The Copy Cat Poll

    This editorial was originally published on Aug 3, 2012. It is being re-run as Steve is on vacation.

    One of the interesting facts I saw a few years ago talked about storage in enterprise environments. There was research that showed many enterprise applications had 6 or 7 copies of their large databases inside the organization. In addition to the production copy, there were many other copies in use, resulting in an explosion of growth. That wasn’t surprising, and it was one of the drivers for implementing compression in many databases.

    While the cost of storage is constantly coming down, it’s still expensive for enterprise class hardware, especially in a large SAN device. Today I wanted to ask those of you that work on real world systems to make a quick count of your own system, and let us know. I can’t decide if 6 copies of a production database is high, or low.

    How many copies, on average, of your production databases are in your company?

    I suppose you could count backups as a copy, since it’s disk space usage and you have to pay for it. If you count backups, let us know, but I’m thinking just about the test systems, development systems, HA or DR systems that might receive copies of the data. Some of those secondary systems might be in use for other purposes, such as reporting from readable secondaries in an AlwaysOn scenario. Whether they are or not, they are still copies of your database.

    I used to think that four or five copies would be a lot, but with the advances in technology that allow different DR options, and the cheap local storage available on today’s desktops and laptops, I wonder if seven or eight copies might be more accurate.

    Take a count today; you might surprise yourself with the results.

    Steve Jones

  • What’s Your Smoke Test?

    Many DBAs and operational staff regularly stress over software deployments to production systems. Even when the administrator has built and tested the deployment scripts, there is still a nagging fear that something will be missed, incurring downtime for systems. I think automation, building an ALM And DLM process, as well practicing deployments in other environments is the way to alleviate concerns, but that’s a discussion for another day.

    However, I had a question this week that is related to the deployment process. I’m curious, do you smoke test your production deployments?

    Perhaps the first question is do you know what a smoke test is? If you don’t, maybe you want to read a bit about smoke tests. If you know about smoke tests, do you have anything more than testing if a server or service is running?

    If you don’t, that’s fine. In fact, throughout most of my career our smoke tests consisted of checking that we could connect to a server or database. In fact, in most cases, we did this manually. We’d deploy some change or patch, then see if we could connect to the server. Sometimes a person would connect with an application, but not much more than logging in. Since many apps were fairly simple client-server applications, this was usually sufficient, but there were times we had problems with not all changes being deployed. A few times we didn’t realize this for days.

    If you deploy changes, there should be some small things that you check as an method of validating that your systems are working correctly. There may be initial checks that machines are actually running. If those checks fail, there’s no point in going further. However, if the basic connections work, perhaps you should have a few critical functions that can be called and ensure that basic functionality of the application is there. Or that your new changes actually are deployed. If you have complex, interconnected systems, maybe you want a few smoke tests that you can automate and quickly ensure everything is running as expected and communicating.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and Mevio