Voice of the DBA

Category: Blog

  • Capacity Planning – Monitor and Extrapolate

    I wrote a post recently on capacity planning, and then thought of one more thing. In addition to planning for the future for new systems, you also need to regularly monitor your existing ones.

    Planning for the future, IMHO, isn’t something you want to do when you have a crisis, like running out of disk space. A good DBA is proactive, and looking to monitor resource usage and extrapolate out regularly to get an idea of

    1. when you’re overwhelm your current hardware
    2. how big to go on the next upgrade.

    Nothing worse than your boss complaining about performance, you recommending an upgrade and then having no idea what to ask for. You also don’t want to get caught in emergency situations when nothing has changes in your workload, other than you weren’t paying attention.

    I used to do this in a few ways, and there are numerous ones today. You can use a product like SQL Monitor, you can use the MDW from Microsoft if you have EE, you can use other products or roll your own. I have done all of these, except MDW, and they have all worked well.

    In terms of memory, I haven’t done this in some time, so I’ll let you get details from another source on how to monitor the usage over time on the modern versions. Here’s a good thread to get you started.

    In terms of CPU, I tend to monitor the raw CPU percentage over time, using a broad average across a whole day, watching the peaks. I would trend this out for 3-4 months, and see if it is increasing. Since CPUs can be hard to upgrade, you want to know this well in advance.

    Disk Space

    This is the big one, and quite embarrassing when you can’t capacity plan here. There are numerous ways to do this, but here’s what I’ve done.

    I monitor backup size every day and store it in a small administrative database. If I move the database, I move the backup information to the new instance as well. You can query the msdb tables for the sizes.

    SELECT DATEPART( yyyy, backup_finish_date) 'backup year' , DATEPART( mm, backup_finish_date) 'backup month' , DATEPART( dd, backup_finish_date) 'backup day' , database_name
, SUM(backup_size) 'backup_size' , type FROM msdb.dbo.backupset
 GROUP BY database_name
 , type , DATEPART( yyyy, backup_finish_date) , DATEPART( mm, backup_finish_date) , DATEPART( dd, backup_finish_date) ORDER BY DATEPART( yyyy, backup_finish_date) , DATEPART( mm, backup_finish_date) , DATEPART( dd, backup_finish_date) , database_name

    There’s a quick query, and you can perform some summaries of sizes by types of backup, convert to GB, etc. I used to run a similar query every day and do three things:

    • store the results
    • compare the last backup to the one before it.
    • extrapolate out the growth of backups to reach the disk size

    Obviously I wanted to track the results, and I would monthly create an old average of the sizes from last year so I wasn’t storing crazy amount of data.

    I compared the backup each day with the previous one to catch weird things. If the backups differed in size by more than 20%, I raised a flag. This was to catch changes in process, or potential errors from data loads or imports. Anything that changes 20% a day can be a problem, so this allowed me to find problems quickly.

    Lastly, I aggregated all backups on the instance, extrapolating out the growth across weeks, and estimating when I’d fill the disk. When this was inside of 2 months, I knew I better start asking for more space.

  • Ethics and Power

    Power corrupts.

    A famous quote, and it’s one of the the most true quotes I’ve seen in my life. When someone has power, they develop a sense of importance, their decisions seem to be more “correct” and their arrogance grows.

    It’s a sign of being human, and we have try to build in checks and balances into our governments and corporations to limit the abuses of power. We’re not any more perfect than our systems, and we continue to see our systems, and people, fall down.

    Being elected to a position confers power to an individual. It gives them responsibility, but it also seems to dull them in some way. The elected individual finds it harder to make the “right” decision, to stand up in an ethical way over time. It becomes common to justify their choices when they shouldn’t need to. A good ethical, moral decision shouldn’t need much justification; it should stand on its own, It’s the rare individual that can stand up for their beliefs in the face of criticism and follow their moral compass.

    The organization that I was proud to join a decade ago as a professional working with SQL Server has completely fallen down for me in many ways, and the leaders elected to positions have time and time again chosen to set their ethics aside for various reasons.

    “Let it not be said that no one cared, that no one objected once its realized that our liberties and wealth are in jeopardy. – Ron Paul “

    We don’t have wealth, or perhaps even liberties involved with our professional organization, but we do have pride, or we should have pride, and once again I have none.

    As the appointments were announced for the board of directors, I was not surprised, but saddened. An election was completed recently, with six people deciding to run, to devote their time and resources to serving the community.  Three were elected, by a narrow margin, and there were two resignations. With the election so recent, with so many others declining to volunteer for the board, one would think that it would be easy to appoint the fourth and fifth highest vote recipients to the board.

    How easy would that be? These were people approved by the nomination committee, validated by the community and voted for. Apparently the board felt that the nomination committee and the community must not have good judgment.

    Sri Sridharan, a friend, a volunteer, and an energetic worker on behalf of the Dallas community was not chosen. He has organized three SQL Saturdays, and was instrumental in bringing the SQL Rally to Dallas. He decided to run for the board of directors, something that few have done. Regardless of the qualifications of others, there is no argument that Sri is qualified for the board.

    UPDATE (2012/1/16): To be clear, I do not argue that James Rowland-Jones or Kendal Van Dyke are qualified, and I have no complaint with their qualifications.

    Apparently the board, led by its President, does not agree, and decided that someone the community wanted less, and someone that declined to run for the board, were better choices. I have no complaints against those appointed, but they did not deserve the position more so than Sri.

    I can only look at this as an ethical lapse by the board, taking their personal feelings towards a candidate into consideration over qualifications and community support. All those directors that voted for these appointments fall into one of the failings of elected officials, and they certainly do not represent me.

    It’s a sad day, one that has me viewing the organization as one that lacks professionalism, and simply an Association for SQL Server, without the P in front.

    Sri, my friend, my apologies to you as a member of the SQL Server community, and I can only empathize with you, having been in a similar situation.

  • Bloopers from Dec 2012

    The raw ones from podcasts in late Nov and Dec 2011.

    Dec 2011 Bloopers

  • Unlimited Vacation

    Unlimited Vacation sounds good, and maybe it works, but I wonder how much vacation people really take at Red Frog. My guess is that it’s not far off from what most progressive companies give, something like 20 days a year.

    It’s a good perk to have, and I’m sure it doesn’t get abused since the people that abuse it get let go. The people that don’t abuse it might even shortchange themselves. I know it sounds like a company would have people taking 40, 50, maybe 100 days a year off, but the reality is that if someone does that you find yourself in one of two situations:

    • their work isn’t getting done and you fire them
    • their work is getting done and you ignore it.

    There are plenty of managers that might feel they should be getting more work from someone that can take 100 days a year off and still get their work done, but what type of attitude is that? You hire someone, expect them to get xx things done in a year for yyy dollars. If they get xx things done in 2/3 year while the person next to them takes a year, why complain? If others complain, tell them to just get their work done quicker (or learn how to work more efficiently).

    I get over 20 days, plus holidays, and I struggle to take it. I’m essentially in the same boat as the Red Frog employees in that I manage my own schedule, and can take off whenever I want if my work is done. I rarely take sick days, since I can work at home when I’m sick. I have a grinding, daily job that requires regular effort, so I can’t plan on a month project, take off 2 days in the middle and catch up later. I essentially run a newspaper, every day.

    However I could take unlimited vacation. I could potentially get my work done in 3 days every week, keeping enough items scheduled, to take two days a week off. I’m just not sure that it would be vacation since I’d be stressed during those 3 days.

    If you are a professional, I think most places will work with you to get the vacation you need, whether it’s tracked, booked, managed, or not. The key, in my opinion, is to set a schedule that works for you, and a set amount of work that justifies your salary and makes both you and your employer happy.