Voice of the DBA

Tag: Business Intelligence

  • Hard Data v Gut Feel

    basic instinct picture
    Instincts are great, but they need data behind them

    One of the mantras that has guided Google through so many of its product development routines is the fact that the data matters. If you can show that there is data to support a decision, it is more likely to be well received than one based on the experience, instinct, or hunches of any person.

    Often in the past I wished that the companies I worked for had operated in a similar manner. Many times I’ve had managers decide to pursue some course of action based on something other than data. At times I’ve even had managers ask if we could rework queries or reports to support their position. It wasn’t entirely unreasonable to question the queries we had written as we may not have approached the problem in the best way, but asking for evidence to support a pre-determined result felt wrong to me.

    Companies spend thousands, or millions, of dollars every year on all sorts of resources to collect, manage, and analyze an ever growing set of data points. Many of us find our jobs revolving around data, looking for new ways to analyze data and drive better decisions. At the same time, many companies struggle with decision makers that trust their own instincts more than the data reports they review.

    There isn’t a lot we can do as data professionals, other than continue to refine the models we build and work more closely to convince managers to tune our models, using the actual data before and after the decisions to analyze the results. We can also champion the ideas that the data is more objective, and should be a significant part of the decision process, while allowing for some input by the people that have a wealth of experience in our particular industry. We want their gut feel, but we don’t want to ignore the data.

    As with most things in business, communication and compromise help to drive us forward to better decisions in the future.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Social Data Analysis

    Tufte Poster
    A great visualization from an Edward Tufte seminar.

    Today we have an editorial originally published on June 24, 2007. It is being republished as Steve is traveling.

    I haven’t been a big proponent of BI as a technology that I think will catch on and become the focal point for many businesses. It’s not that it doesn’t work, but it’s expensive, it’s complex, and it requires a long term investment. While I’m not sure what to do about the last two with today’s IT folks and executive management, I do know that Microsoft and others are helping with the first one. So when I was this item about social data analysis using a new tool from IBM, I became a little intrigued. It’s an interesting idea for the amateur BI person that may be looking for an easy way to examine their data. It’s certainly better from the manual calculations I did in Economics to do regression analysis of data. We often didn’t even have graphics. That is if you don’t count my pencil and graph paper.

    This is a good outgrowth of the open source ideas of programming. It’s often said that many eyes on a program in the community model, produce a better piece of software. After all, developers work harder if they know hundreds or thousands of people will look through their code and more bugs are likely to be found because more people are working on the problem. I think that’s what IBM is trying to do here: get many people to examine data.

    While the sample visualizations tend to look like something just thrown together by people, I can see as graduate students, government officials, and others looking at lots of public data, and data that doesn’t necessarily need to be secure, you could possibly get some real help on problems.

    And maybe the most important thing, this gets a lot of people using the same tools. From what I’ve seen of many BI tools, they’re expensive and there are lot of choices, so trying to facilitate a wide reaching data analysis program could be hard, even among universities, each of whom might have their own software.

    While I’m not sure about BI catching on as a mainstream, every company has it, technology, I do think it’s very cool and it’s application to a wide range of problems is likely to give us some new solutions.

  • Real Time Dangers

    There can be tremendous volatility in short term data.

    There seems to be a quest to move closer and closer to real time decision making. Gather data, analyze it, and make decisions instantly, preferably with the help oif expert systems. That makes some sense, and as shown in the article, it can allow analysts to respond to events very quickly, performing verification, fraud checks, or just about anything you can think of.

    It’s a good goal, and it can definitely help many companies make more informed decisions at any point in time. However there are problems as well. Sometimes short term data can fundamentally distort the picture of reality. Some of our large stock market meltdowns are the result of automated systems, perhaps not so much expert systems, as very quick reacting systems that might overvalue the last few pieces of data and make decisions that are less than optimal.

    We cannot program systems to handle every situation, nor can we even give enough guidance to inexperienced humans that might be involved in the workflow. Instead we ought to recognize that short term data might not represent longer trends and ensure that we have people looking over the data across a longer timeline before any important decisions are made.

    Too often it seems we build systems, assuming that more data, delivered quicker, is the way to prevent poor business decisions. We might easily overwhelm other systems, or people with too much data, delivered too quickly, or used to inform decisions too quickly. Real time systems can provide many benefits, but their use should be tempered with this saying I have long believed: computers give us the power to make mistakes quicker than ever before.

    Steve Jones

    The Voice of the DBA Podcasts

  • The HR Scorecard

    Dundas controls make building interesting dashboards easy

    Are you looking for a BI project in your company to get some experience with? I’ve got an idea that you might try tackling as a side project, or even an informal project within the business.

    One of the problems that many corporations face today is a regular turnover of employees. That causes a loss of productivity, moral suffers, and projects may run late. This might be because of poor working conditions, poor management, or some other factor. While you might not be able to change that, perhaps you can help bring some visibility to potential problems.

    Human Resource departments are similar to IT departments in that both of them are a cost to the organization without generating revenue. These are overhead departments, and often receive limited budgets and assistance to accomplish anything beyond the bare minimum. These departments can add value, and the good ones help companies in many ways, reducing the costs that might otherwise occur.

    I have rarely seen HR departments with deep analysis tools available to them that might help them function more efficiently. If you want to tackle a BI project, think about building a detailed scorecard or set of KPIs for the HR department that might help them better understand what is happening in the company. Watching for the impact of vacations, turnover, etc. and aggregating exit interview data might help HR improve the conditions at the company, perhaps even for you.

    I’d consider working on an HR BI project, even during some off hours if you have an interest in this area. I’m sure the HR people will appreciate it, and they might be the ones that are in the best position to help you with your career down the road.

    Steve Jones

    The Voice of the DBA Podcasts