Tag: databases

On one of my recent trips, there was a delay in leaving the gate. There were two passengers, both of whom had the same seat assignment on their boarding pass. You’d think there would be some sort of primary key that prevented this situation from occurring, but with the constant reassignment and movement of passengers at the gate, I can understand this happening. We print passes at a point in time, but since data can be revised, and new passes printed, there potentially would be a conflict.

However what came to light as we waited was both disconcerting and puzzling to me. The real problem was that one of the passengers was supposed to be on a later flight to the same city.

Apart from the security implications, it seems to me that the ability of someone to get through the scanner near the jetway with a boarding pass for the wrong flight implies a broken data model. I can only guess that the scanner is reading a passenger’s name, destination, and seat number. Or that the flight data model is broken and there are two flights on the same day that share a flight number. The journey was short, so I suppose this airplane could have actually been making two round trips in a day.

I can’t image the complexity and problems that programming an airline’s flight system must bring to the data models and applications. With the constant shuffling of people, flights, and seats, I’m regularly surprised that it works smoothly most of the time. As my airline has modernized their systems, I’m even amazed at how much data they disclose on monitors to passengers waiting for upgrades or standby flights, and how quickly my mobile app updates with new data when there’s a change.

I’d never experienced someone getting on the wrong flight before, and was surprised to find their software allowed it. However given that so much of the airline industry relies on systems that were developed decades ago, perhaps I shouldn’t be surprised.

Steve Jones

The Voice of the DBA Podcasts

We publish three versions of the podcast each day for you to enjoy.

Due to some technical issues, there is no podcast today, but we will return with a new episode tomorrow.

Big data is a hot topic these days. Like the “cloud” terminology we’ve been hearing about for the last few years, there isn’t a good definition of what “big data” really is. The best one I’ve seen so far is data that “doesn’t fit in Excel,” which I like. So many people perform their analysis on a spreadsheet of sorts, that if the data doesn’t fit inside their edition of Excel, they’d probably consider it big.

The problem with big data, however, is that it while it contains more information, it can also contain more irrelevant information. That’s noted in this piece on small data (from Brent Ozar, PLF), where the author states the signal to noise ration may be decreased when you examine very large data sets. You may find that there are correlations that appear to causations. With enough data, with enough things to examine, you can often start seeing patterns that aren’t really there. These ghost patterns can lead you to draw incorrect, or at least less correct, conclusions if you do not investigate further and test your ideas on portions of your data set.

Some of you might have noticed fractal patterns like this:

This is a well known Mandelbrot Set pattern. However if we were to zoom in on this picture, we’d find that the patterns repeat over and over again. What holds true for the largest image we have holds true inside smaller sections. The pattern repeats.

The same thing can happen with patterns in business. We may see a pattern in a large set of data, but we should verify that it also holds true for subsections of the same data set before we make a decision based on that pattern.

Steve Jones

The Voice of the DBA Podcasts

We publish three versions of the podcast each day for you to enjoy.

• Watch the Windows Media Podcast – 16.1MB WMV 
• Watch the iPod Video Podcast – 18.2MB MP4 
• Listen to the MP3 Audio Podcast – 3.6MB MP3 

Can you own data? In one sense, that’s what the arguments before the US Supreme Court over a Myriad Genetics patent are about. The company has patents for a few genes, which in and of itself, are really just data about a particular organism. There are a lot of technical issues here, and I’m not completely sure how I feel about this topic or who I’d like to see prevail in the case. I do think, however, that this does have lots of implications for how we might view other data in the future if the patent is upheld.

As data professionals, we usually think of the mechanics of dealing with the various pieces of data in our organizations. We must protect data, ensure it’s integrity, make it available and more. A large part of our careers is spent manipulating or managing the bits and bytes stored in databases. We must extract information out, and often need to understand what the data represents to do this, but most of us haven’t been exposed to the issues of ownership or rights with data. I think that will increasingly change as government rules, regulations, and laws scramble to catch up with the digital world.

Digital ownership is a very thorny subject, complex in its breadth and depth. We’ve seen technology companies fight against some laws, like CISPA, while supporting others. Often our government officials don’t understand the complexity of the topics, and I’m not sure that we in the technology business explain the the situation well. Even very intelligent people often don’t convey the implications of these complex topics, many of which often don’t have good analogies in the analog world.

Ultimately I think we will find our jobs as data professionals becoming more and more complex as we must not only solve problems and manage data, but we will also end up being constrained by the legalities governing the data.

Steve Jones

The Voice of the DBA Podcasts

We publish three versions of the podcast each day for you to enjoy.

• Watch the Windows Media Podcast – 15.8MB WMV 
• Watch the iPod Video Podcast – 19.4MB MP4 
• Listen to the MP3 Audio Podcast – 3.9MB MP3