Voice of the DBA

Tag: data analysis

  • Tracking Professional Athletes

    I used to read about technology to allow computers to track a single person’s movements from video footage. We’ve seen this shown in Hollywood movies, where casinos can take a picture of a person and backtrack all their previous movements throughout the day. I have no idea if this is possible, but if it is, it’s a scary proposition.

    However in specialized, controlled circumstances, we can track people very well. The NBA (with SportVU) is tracking all player’s movements and gathering more data that can be analyzed to better evaluate player’s performances. I expect in the next year or two, players will start to review this analysis and learn how to better adapt to the situations on the court.

    The NFL is also starting to use sensors to track their players, but with different goals. They are monitoring workloads, trying to ensure the health and peak performance of players. It remains to be seen how else they might use this data, but in the linked article, there are perhaps more implications for us as data professionals.

    Can you imagine more tracking data, for any movable object available? People, machinery, who knows what else will be tracked, and what else we might need to analyze. I thought spatial data had tremendous possibilities when I first saw it introduced in SQL Server 2005. However across the last decade I think we’ve barely scratched the surface of what location data might mean for applications, using it mostly for mapping locations and routes. If this data does become useful, that means that learning to aggregate, trace, and analyze location might just need to be a valuable, if not core, skill set for the DBA and developer in the future. If you want to get started, maybe run through this article on your system and see what you think.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • The Machines are Learning

    One of the technologies that Microsoft is promoting heavily is machine learning. This has rapidly gone from a technology that I heard little about to being in multiple keynotes that I’ve seen at large conferences this year. It almost seems that I can’t go a week without seeing some sort of machine learning article, announcement, or reference.

    How applicable is machine learning for most of us? I’m not sure, but banks have certainly taken advantage of machine learning to reduce their risks associated with fraud and their systems work well. In fact, given the ways in which I travel, I’m amazed that I don’t get more calls about fraud related to my card use. With me in a new state almost every month, on a very irregular schedule, sometimes buying computer supplies far from home, I get called by a bank once or twice a year.

    Across the ten or so calls I’ve gotten in the last few years, only one questioned legitimate purchases. The rest caught fraud on the same day that someone tried to use my card without my knowledge. That’s a very successful rate of both true positives, and a very, very low rate of false positives. I’m impressed.

    If you haven’t played with machine learning, Microsoft has made it easy to give a try in Azure. You don’t even need to put in a credit card to get up and working with machine learning. Whether it’s applicable to your industry or not, you’ll have to decide, but I do think that lots of tedious analysis that humans do now could be better done by a machine.

    At least after one of us humans has configured the algorithms and trained the machine to recognize patterns. And, of course, with one of us technologists periodically tuning the system to work better  and monitoring the analysis as data changes.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.6MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Changing Scales and Creating Disappointment

    I got an email recently that notified me that session feedback from the Summit was available for my talk. I’d had a lot of people in the room, and was curious how things went. I think the session was OK, a little off on time, a few too many questions I tried to answer, and perhaps a bumpy flow.

    However when I got my scores, I had a 2.85 for the session overall, with various aspects of the talk being rated from 2.5 to 2.9.

    Well, I sucked.

    That was my first thought. I’ve been getting rated, and evaluating speakers on a 5 point scale for quite a few years at PASS events. I was surprised, and disappointed, and then a bit embarrassed that I hadn’t delivered a good talk at the Summit. Since I hadn’t delivered that talk in public anywhere prior to the conference, I thought I had made a big mistake. Apparently my practice that week in my hotel room had been for naught.

    However then I saw this note in another email: One of the changes this year was to move from a five point rating scale to a three-point scale.

    Hmmm, I missed that in my email somewhere, and didn’t notice this as I filled out a couple of session evaluations.

    I don’t think there’s anything wrong with changing the scale. Personally I like the 3 point scale, but it wasn’t a change I noticed. The first communication with speaker feedback didn’t mention this.

    Scale matters. Many of us know that by manipulating scales, we can make data look different, We can prove a point that might not be supported by a different presentation of the data on another scale.

    Our clients and business users come to know and expect the various ways we present data. They will start to internalize scales and interpret data based on their expectations. We can change scales, but we need to make it clear and visible that we have changed scales.

    Personally I would have appreciated the results being reported as:

    Overall Session Score: 2.85/3.00

    instead of

    Overall Session Score: 2.85

    That little extra information can mean a lot. Keep this in mind as you make fundamental changes to the way you present data.

  • Baseball Analytics

    I love baseball. In fact, for the last ten years I’ve participated in an adult league each summer, playing 12-20 games each year against other 40 year olds. While we play the same game with the same rules as the professionals, we have a much different outlook on the game, with many of us enjoying the competition and distraction, but often viewing avoiding injury as a higher priority than winning.

    Baseball is an interesting game as the professional season is so long and for over a century there have been a huge amount of statistical data captured about each team. Prior to the fantasy sports popularity explosion, people had leagues that competed against each other solely based on this statistical data. I even wrote a program with a friend on an Apple II that allowed us to set up teams of random players, read in statistics, and then compete against each other using pitching v batting statistics, just showing us text results.

    I ran across an interesting post from a consultant that examined a hypothesis he had about winning baseball teams playing better at the end of the season. Using a free data set, the R language, and a Jupyter Notebook, he was able to perform an analysis that proved he was right. Whether or not you agree with him, I would guess that this analysis didn’t take that long to complete, and was fun as well.

    With R coming to SQL Server and the increasing call from organizations to perform more complex analysis on our data, I think this is a great way to get started. In fact, I’m thinking that I should load up some of the datasets myself and see what kinds of things I can prove or disprove about my knowledge of the game. If nothing else, looking at baseball stats is probably a fun way to brush up my statistics skills.

    BTW: Congratulations to the Kansas City Royals for their World Series win (and to the #sqlfamily in KC).

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn.