Voice of the DBA

Category: Editorial

  • Server Hardware or Coffee?

    I actually chuckled out loud in my office recently. I was reading a piece about Expensify and how they learned some lessons from their database architecture over time. It’s a good piece, but at one point the author talks about two Dell servers. There’s a dual core 2.8Ghz CPU with 500GB of storage for US$700. This could be upgraded to an 8 core, 3.7GHz CPU with 64GB of RAM and 10TB of storage for around US$3,200. However, the author says don’t be cheap. Why?

    That’s less than the company spends on coffee each month.

    I thought that line was funny, and it proves a point. It can be easy to think about the cost of hardware as being high because it’s a large ticket item for most of us. When you view it in the context of the scale of business, then it’s cheap. If any of us spent $3000 a month on food, we might not think $3000 for a server is expensive. When you look at the cost of your labor that might need to babysit an underpowered system, a few more thousand dollars seem like a bargain for memory or CPU resources.

    The author also notes that while EC2 charges for managing your systems, it’s still a steal, and for many of us, capacity isn’t a big issue. We do find that one single system runs most of our workload. If it doesn’t, then we could buy more, or a second system, for a relatively low cost. At least, compared to the rest of our business.

    There certainly are some of you that have a large workload, or you have spent substantially more on server hardware. I think many of my systems have been tens of thousands of dollars, but there are good points to be made. Today’s computers are very powerful, and even though we have more data, a single server is likely able to handle many workloads.

    If we write good code.

    That’s a big caveat, but training your staff to code better, having them spend time learning to query hierarchies more efficiently or quickly splitting strings, can pay off with much happier servers and customers. While I am a fan of using hardware to avoid spending too much time tuning queries, I also think continuously improving the skills of your development staff is much more important and might be the best IT investment you can make. Assuming, of course, that you treat them well and they enjoy working for your management staff.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Generating Our Own Bad Data

    I’ve worked in a few different industries and for a number of companies throughout my life. In almost every position, employees were given goals to improve the performance of the company. Often these were revenue-related goals that were supposed to drive additional sales. The one thing I learned when these goals directly related to sales was that there would always be unintended consequences that might skew business in a way that management didn’t expect. This is especially true when the rewards for meeting goals lead to direct compensation for employees. In fact, additional compensation has always led some percentage of the staff to work towards improving their own pay, even with fraudulent or unethical activity. Hopefully this has usually been a small percentage of people, but perhaps that’s not always the case.

    What does this have to do with data and databases? I recently read about fraudulent activity at Wells Fargo, where a sales policy caused a number of problems. Employees used real customer data to open fake accounts in order to meet sales quotas, generate fees, and earn bonuses. While certainly criminal and immoral, this also shows what I’ve believed for a long time. While computers allow work to take place very quickly, computers also allow mistakes to be made at incredible rates. While these weren’t mistakes, the computing systems allowed this fraudulent data activity to take place at a large scale. One that resulted in not only fraudulent charges to customers, but fraudulent reporting to investors, unearned bonuses paid out, likely unearned promotions, and who knows what else.

    To be fair, this type of criminal behavior could have happened before the age of computers, and likely could have lasted just as long with employees forging paper documents to split accounts, open credit lines, or any other activity. However, I’d hope that computer systems would have detected this sooner. Too detect the bad data, customers really need to be tied into some sort of reporting system that can audit their accounts, that can examine credit bureaus, and warn them if anyone is making unauthorized changes.

    We have seen similar unintended consequences when companies have offered bug bounties to developers. More bugs appear, and not surprisingly, more bugs are fixed, with more payouts ensuing. I’d like to think that most people would be driven to be professional, but time and time again we find enough of a percentage of employees aren’t, and the programs are doomed to failure. We need to do better, though I’m not sure what would help.

    Protecting data from insiders and outsiders is our responsibility as data professionals. However, insiders can cause more problems, which is why I would hope that the auditing and transparency of data platforms would be a priority for vendors across the next decade. In my mind, not enough attention has been paid to these areas, and our systems are woefully immature in these areas.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Forever Technology War

    The Forever War is a classic science fiction novel. I’d highly recommend it if you enjoy that genre, but even if you aren’t a science fiction fan, this is a story that can really make you think. The gist of the story is that as various parts of our military travel faster than light, we might encounter foes (or other units of our own military) that are substantially ahead of, or behind us, in technology. All because of the nature of time travel. Recently I ran across a few situations that reminded me of this story.

    The first item was at home, where my son’s computer acted up. He likes Windows 7 and has stuck by it. However an update caused boot corruption and he wasn’t able to repair the system. In the process of trying to get repair things, we dropped into the command line to run chkdsk and bootrec. He wasn’t sure how those worked, which makes sense. He’s 17 and most of his life has been spent in GUI OS’s. However, once we realized he needed to reinstall, I told him to copy things off his boot drive to another drive. I showed him a bit of moving around the command line, making directories, listing contents, copying files. As I was walking away, he asked, “how do I make a folder?”

    A folder. Not a directory. We’ve had not only a technology shift, but also a nomenclature change in computing. It had never occurred to me that my son wouldn’t know basic command line calls. As a side note, he greatly appreciated the /? parameter to learn more about commands.

    The second situation was at a conference, where I was talking about continuous integration and deployment. A person asked about SSIS packages, and how would you move these with a tool like TeamCity or VSTS. I asked how they’d do this manually, and said that we would just program that. This person asked how they’d program the steps they take in SSDT. My answer was to use the command line, but this person had no idea there was a command line. Their whole career had been spent in GUI systems and they assumed that was the only way. Hint, pretty much everything can be run from a command line.

    Those situations showed me that we have a technology shift in place. There are things that some of us have grown up with, used as a base to learn more, and we can assume that others have similar paths. After all, wouldn’t you need to know about x to learn about y?

    Not necessarily. The longer I am in this business, the more I see that the experiences and knowledge of different people can be vastly different. A person that knows how some technology, say T-SQL and SQL Server security might have no idea how replication works. Or they might know AlwaysOn well, but couldn’t implement an Extended Event session. There are gaps and holes in the knowledge that each of us has, and assuming that others have similar understanding of any particular item can cause us problems when we try to work together.

    Understanding, transparency, sharing knowledge, and appreciation of diverse experiences can help us all learn, and work together better. If a situation seems strange or a person is confused, perhaps a few questions on their frame of reference of the situation is a better response than assuming they are incompetent or telling them to RTM.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Wasting Time

    This editorial was originally published on Oct 11, 2012. It is being re-run as Steve is on vacation.

    I ran across this infographic on wasting time at work. From the title I was thinking this would point out the ways in which people avoid work, and perhaps it does, but it’s really geared towards showing us the framework and structure that many people have for work. I agree with the first two sections of the page that show email and meetings waste a lot of time. I certainly think they have in many jobs for me, especially when they are used as “catch-all” techniques for including everyone that is remotely relevant to an issue.

    Interruptions at work are hard to quantify as a problem. They definitely can be, but stopping by someone’s office to ask a question or take a quick break can be a way to recharge yourself between long periods of concentration. The issues come into play when the other person is in the middle of focusing on a task and you force a context switch on them. Recovering from the interruption can take time, time that’s often wasted as a person tries to remember exactly what they were focusing on.

    Meetings certainly interrupt the day, but since they are often planned in advance, you can be mentally prepared for the break. It makes me wonder if there wouldn’t be some benefit to scheduling some “open time” in your day where you plan on taking a break. Others that needed a minute of your time would know to come find you at that time.

    Time is very valuable, one of the most valuable resources we have at work. Management should be aware of this and working to limit interruptions, whether through email, meetings or anything else that prevents work from getting done. The fewer meetings and emails you require of your developers, the more time they have to work on the tasks they are being paid to complete.

    Steve Jones