Voice of the DBA

Tag: software development

  • T-SQL Tuesday #51 – Place Your Bets

    tsqltuesdayIt’s T-SQL Tuesday time again and Jason Brimhall is hosting this month’s event. He used to live in Las Vegas, and with that theme in mind, he calls for you to Place Your Bets.  That’s a time when someone has gambled, or risked something on an application, venture, process that relates to SQL Server? When have you done something that caused an issue you could have avoided?

    It’s an interesting topic, and I’m looking forward to reading what people write.

    T-SQL Tuesday is a monthly event, the second Tuesday of the month, started by Adam Machanic (blog|twitter). You can participate by setting up a blog and writing on the month’s topic. Check Twitter for the #tsql2sday hash tag or look for the blog postings with a search.

    If you want to host, contact Adam.

    The Big Bet

    I didn’t place this bet, but a bunch of application designers and management did. I came in to the company late, and realized we had issues, but wasn’t able to effect change in any short order.

    I worked for a large software company. Large as in thousands of employees, and certainly hundreds of developers. Most were engaged in writing the software we sold, but there were plenty of people engaged in managing our internal systems and writing software that we were using internally. One of these was our internal Intranet site for employees.

    This was a large project, replacing a number of applications, and designed to be a central point of information where employees could turn and various departments could publish information. Being a software company, we knew that it’s worth paying for software that’s written and we chose a framework that would cover many of our needs, but allow us to customize parts of the site for use by different departments. All in all, a CMS-type system that would fit our needs.

    The project started before I did, and no one consulted me or my group, as the production DBAs, as to the architecture, hardware spec, or anything else. It’s not that we should have, or would have, led the way, but we could have offered opinions about how things might perform.

    The day of the roll out came. I was told which day it was and never participated in any testing. The global policy was changed to set everyone’s default browser to the intranet, and I came into work expecting to find a new site where I’d see information posted, resulting in (I hoped) less emails from various departments. Not long after I arrived, I was pulled out of a meeting because of performance problems.

    I had no idea what to look at on the site, but we found CPU pegged and very high I/O on the server. I decided to run a short Profiler trace from my machine to get an idea of what the workload was and what we might look to run. I found fairly simple queries, lots of SELECT * to a few tables. What was interesting was that the queries were scrolling rather quickly on the screen. I found that the default trace values in SQL Server 2000 were storing around 25MB/sec worth of data on my local hard drive.

    In 2001, this was a fairly large load on a SQL Server 2000 instance. When we checked the tables and calculated result sets, we found very little data from each query. There were missing indexes, but even adding those to tables that were a few hundred rows of data didn’t help. The problem was simply that so many employees, each hitting the server by default, with a series of sidebars on the web pages, each of which created its own connection, ran its own query, and ran lots of them on each page, was simply overloading the hardware.

    If was bad design, poorly tested, and not well thought through. Even simple caching mechanisms as Brent Ozar has talked about would have dramatically removed a lot of the load. In the end, those weren’t really needed. What we found was that the majority of the information being returned by queries changed less than once a week. A few weeks of development time removed most of those queries and had them replaced with static XML files that were loaded by the application for display, and could be updated from the database by an administrator.

    Ultimately the site worked well, and we did get less emails from various departments since announcements could be made on the Intranet, which most of us learned to check once or twice a day.

  • Just use Version Control

    There are free systems out there. If you have no budget, and want to get started, download one of these:

    There are good ones to pay for as well, and some of them have other features and integration you may like:

    There are plenty more. Please, please, please, start using Version Control. There are compelling reasons from many successful developers, including those that want your database code under control.

    There are ways to do this, but please pick one. Version control is a must for professional software developers. That means if you write code, and get paid for it, use version control. If you write code, and don’t get paid for it, value your time and use VCS anyway.

  • Too Much Choice

    As this piece shows, more information can be bad. As drivers get more choices, options, and opportunities to interact with information, they may not be driving as well as they otherwise might. Restaurants and retailers have seen something similar. Too much information either prevents decisions or slows the process. I know when I go to a restaurant that has too large a menu, I often find myself taking a long time to decide on an item.

    I think this can certainly be a problem for our clients in technology, as we discuss specifications and requirements with them. If we allow clients too many choices, and too many options, it can be hard for them to make a decision, much less understand the implications of their choices. As software professionals, we need to give clients two or three options, but recommend one and explain in a fairly simple manner what the advantages and disadvantages of each solution are.

    Too much choice can also be a problem for us in technology. If we have too many architectural options, we can debate entirely too long. If we have too many requirements, we may be unable to get work done if we can’t focus or distill them all down. If we have too many concerns, we may not find an effective solution because we’re looking for a perfect one. The latter is a situation I find all too often with developers that want a perfect, 100% solution when a 90% one will work fine because the edge cases so rarely occur.

    The exceptions to this, both in my restaurant experience, and in research, are when the person already knows what they want, or they have some preconceived idea. That can be good for us in technology, as we can move quickly when we have some idea of what is needed. However this is a double edged sword as it can lead us to work with tunnel vision, not thinking about the alternatives or possibilities we may face.

    In general I think more information is better than less, but we can’t let the abundance paralyze us from moving forward and accomplishing work.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

  • Uncontrolled Code

    Wow, Excel really sucks. To be fair, that isn’t just Excel, but any spreadsheet that’s used to make decisions in business (or government) and hasn’t had proper auditing or checking. Spreadsheets are amazingly powerful and useful, and so ubiquitous that Microsoft gave up on a separate BI tool and built one into Excel.

    I ran across this story on a serious Excel issue that cost a lot of money in the London Whale case. There are a lot of comments with the post as well as a discussion on Hacker News as well. I also found this piece on the amount of spreadsheet errors that exist in the world, based on a number of studies. It’s dry reading, and also scary.

    As I read through the thoughts and posts, I’m somewhat stunned that we don’t have many, many more business problems because of a spreadsheet being used as an application without any auditing. Perhaps we actually do and just don’t know about it. Perhaps the reason you didn’t get a bonus last year at your company was because of a spreadsheet error. There could have been either an error that caused a poor business decision, or one that incorrectly calculated bonus payouts.

    It’s a mess, and I honestly don’t know what to do. A more rigid structure in building applications that are checked, rechecked, audited that can prevent the miscopying of formulas is a great idea, but in the real world, we know it takes too long. Perhaps more time is a good thing and businesses should slow down, especially financial businesses, but I can’t see that happening either. Perhaps we need more tools that handle precedents and dependents. Personally, I’d like to see some VCS hooks built into Excel as well.

    The entire process of building applications with spreadsheets reminds me of a race to the bottom, where companies take more and more shortcuts and chances, just because they think other companies are doing the same thing. Ultimately I don’t think we can fix this, but we can try to make a difference by producing software quicker, and pointing out the errors in spreadsheets that become too important. We can meet somewhere in the middle between a full application development lifecycle and total ad hoc spreadsheet based tracking and processes.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Mevio . feed

    The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.