Voice of the DBA

Category: Editorial

  • Inception

    This editorial was originally published on Sep 7, 2010. It is being re-run as Steve is on vacation.

    I recently went to see the movie Inception on the recommendation of my wife. She had taken one of my sons to see it and loved it, so she convinced me to go. She actually enjoyed it so much that she went back a third time with another son to see it again.

    The movie is interesting, and deals with the possibility of planting an idea in someone’s head through a dream. It’s a very well done movie, and I recommend it if you haven’t seen it. It’s definitely science fiction (I hope) and not something I think is possible, but it is a very well done movie and gets you to consider the possibility.

    It’s Friday, and no, I’m not asking you what you think about the movie. I’m not looking for more movie recommendations, and I’m not asking if you think inception is possible. Instead, I want to ask you a more basic question, looking for how you find inspiration.

    How do you get your ideas?

    We all work with SQL Server, we solve problems, and we have to write code. More and more I see the work we do in building technological solutions as art, based in science. We must be creative in how we come up with a solution,  and then use best practices and proven techniques to implement the solution.

    Most of us are at work when we respond, so the answers can be based on how you develop solutions in code. But if you have a way that helps you think, a technique that focuses you on any problem, let us know this Friday. Maybe you’ll help someone else find a way to think more creatively about the problems they face.

    Steve Jones

     

  • Too Large a Workload?

    I often hear the technical staff in many organizations complaining about the amount of work they are tasked with managing. Plenty of technology professionals view their jobs are stressful, and I’ve had no shortage of people say they would never recommend their job, or even this industry, to others as a career choice.

    That’s unsurprising to me, which is a little sad. I’ve felt that we face a dilemma in this industry. Our skills are easy to acquire, with any of us able to learn new skills and gaining knowledge as quickly as we are able. Often, our ability to learn is limited only by the time and effort we put into understanding a skill, not any external factor.

    However that also means that our employers seem to expect that we can learn without training, quickly, on our own, and that we can instantly be experts that make computer systems do our bidding. When we can’t respond quickly enough, or as quickly as we did for the last request, our managers feel that we’re just being lazy this week.

    Perhaps we do respond quickly, and get tasked with even more work. This might not be the case for most of you, or even many of you. I’m sure plenty of you are in situations where your managers understand the complexity of your jobs, but I wanted to ask about your situation:

    Do you have too large a workload at your current job?

    By this I mean, do you feel you’re not able to get enough work done in a reasonable work week and you feel pressure to work longer hours. Are your managers asking you to do more than you think they should? Or do they appreciate that, like many endeavors, working with technology takes time.

    Certainly I think many of us could work more efficiently by automating tasks, but depending on the variety and complexity of tasks, as well as the amount of time you can invest in completing future work quicker, you might be stuck in situations that just require more hours. If that’s the case, I hope your workload isn’t too large, but let us know.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.3MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • You Better Learn to Work at Scale

    Early in my career, I worked on a large Novell Netware installation. We bought a server that was about 1/4 the size of a modern rack, which contained a number of blade slots each holding some type of hardware: CPUs, disk adapters, network adapters, and even disk drives. The disk slots held ~30MB drives, which were large for that time. Both in capacity, and physical size. Each of these monsters weighed a pound or more, and felt like it was built to withstand a fall from a table top.

    That’s one reason that I’m amazed by the 64GB storage in my phone and stunned by the 500GB mSata card I recently bought. The storage capacities we have available to us today, in small companies and startups, are truly stunning to us as data professionals, but not to the people that have the space available. It seems on a regular basis people easily fill up the space they have and clamor for more. Whether that’s valuable data or not, I do know that much of that data still requires management and takes effort from us as data professionals.

    We’re not going to get more help, so as our databases grow, we need to find ways to manage more data, of disparate forms. Not just rows in tables, but Excel files, binary files of all sorts, XML extracts, Office documents, Hadoop hives and who knows what else. We need to be better at working with varying patterns of data, and perhaps files, at scale in order to continue to efficiently manage our workloads.

    Just like system administrators that are starting to deal with hundreds or thousands of virtual instances, we have to be able to write code that helps us manage data in large quantities. I like PowerShell for more and more tasks, but without a doubt, writing efficient T-SQL that works with sets, learning SSIS, package design patterns, and more will become the skills that differentiate the valuable and capable DBAs from those that just keep the lights on.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

  • Normal Data

    I thought this was an interesting post: Your data is not normal. It talks about the fact that we often assume some normal distribution when working with data. We seem to think we can extrapolate, often in a linear fashion, based on past experience. We often use functions and write queries that tend to think of normality as having some standard distribution.

    However that’s not often the case. It’s not necessarily even the best way to begin looking at data for patterns. We might need to apply a number of different types of algorithms to a set of data to see what types of analysis might be appropriate. Not sure what I mean? Let me give you a few examples.

    As a speaker, I often get feedback from my sessions as a series of forms where people have rated the session on a scale and possibly included comments. Many speakers receive the data in the same format, but I’ve often seen speakers get one form that rates them poorly, or has a complaint. The speaker will then view that one item as somehow representative of the session. This despite the fact that they might have ten forms with high ratings. They’re somehow assigning more weight to the negative rating then the all the others. There’s perhaps some value here in doing so, since there might be a legitimate complaint, but often we need to discard this one form as an outlier.

    As another example, I worked with an import firm that was trying to estimate future purchases based on past ones for ordering purposes. Initially they looked at an average of the last three months (rolling) to place future orders. However I pointed out that we had some seasonal fluctuations in our business. There were a few times a year that the rolling average would dramatically under or over estimate the product we needed. As we dug deeper into our historical data, we learned that a better estimate required some level of complexity in our formulas, but we also realized we needed to allow for human overrides when other information was available. That was a fun project, but one that was quite complex when we analyzed the data.

    There are numerous other examples I could give, but the point is this. Learn about your data. Dig into patterns, and don’t assume that normal for one set of data looks anything like normal for other sets.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and LibSyn. feed