There can be tremendous volatility in short term data.

There seems to be a quest to move closer and closer to real time decision making. Gather data, analyze it, and make decisions instantly, preferably with the help oif expert systems. That makes some sense, and as shown in the article, it can allow analysts to respond to events very quickly, performing verification, fraud checks, or just about anything you can think of.

It’s a good goal, and it can definitely help many companies make more informed decisions at any point in time. However there are problems as well. Sometimes short term data can fundamentally distort the picture of reality. Some of our large stock market meltdowns are the result of automated systems, perhaps not so much expert systems, as very quick reacting systems that might overvalue the last few pieces of data and make decisions that are less than optimal.

We cannot program systems to handle every situation, nor can we even give enough guidance to inexperienced humans that might be involved in the workflow. Instead we ought to recognize that short term data might not represent longer trends and ensure that we have people looking over the data across a longer timeline before any important decisions are made.

Too often it seems we build systems, assuming that more data, delivered quicker, is the way to prevent poor business decisions. We might easily overwhelm other systems, or people with too much data, delivered too quickly, or used to inform decisions too quickly. Real time systems can provide many benefits, but their use should be tempered with this saying I have long believed: computers give us the power to make mistakes quicker than ever before.

Steve Jones

The Voice of the DBA Podcasts

• Watch the Windows Media Podcast – 16.5MB WMV 
• Watch the iPod Video Podcast – 12.8MB MP4 
• Watch the MP3 Audio Podcast – 3.1MB MP3