Voice of the DBA

Tag: data analysis

T-SQL Tuesday #75–Power BI

This month’s host is Jorge Seggara, the @sqlchicken, who works for Microsoft. A busy schedule caused a slight delay, so we’re posting the third Tuesday of this month, but that is OK. This is a great topic for T-SQL Tuesday.

Power BI Data

While Power BI is a great visualization tool, you can’t do anything without data. That means you need to find data, which is both easy and hard. Easy if you’re working within your own organization on a specific project. Slightly more complex if you want to look at data out in the world.

However I saw this in a talk last year and I was amazed. This is the type of thing I’ve written before, and it’s cumbersome and problematic. I would think that SSIS would have made things this simple years ago.

I love sports, and wanted to play with some sports statistics awhile back. Finding good data is tough, at least in a format like CSV, that you can easily import. However Power BI makes this easy. Start up the desktop and you’ll see this:

Right away Power BI wants to get data. Click on this and the Get Data dialog opens, with lots of choices.

However if you pick “Other”, you’ll see one more that I love. Web.

Click this. You get asked for a URL. Any URL.

I happen to have one handy. After the win for Denver in Super Bowl 50, I thought I’d look back at Mr. Manning’s career.

I take that URL and drop it in the dialog.

Once I click OK, this will analyze the URL for tables of data. In this case, I get quite a few.

Now, I can click each one to see what data this is. This isn’t what I want

But this is.

I now click “Edit” at the bottom to clean my data. I could just load it, but there are a few issues.

I see all the data in the designer, and I have lots of options for working with this data.

First, since I’m going to do a comparison, let me rename the table.

Next, I see the steps below the name. I’ll add more steps, but I’ll do this in the designer GUI. First, let me remove the last row, which is a career summary.

In this case, I’m only removing one row.

Now, I want to remove a couple columns. In my case, I don’t care about a few of the data items, so I’ll pull them away. I can right click a column or choose “Remove Colums” in the ribbon. Either way, I get rid of QBR and Team.

Now I’ve got a nice year by year summary of Peyton Manning’s career. When I close and apply the query, my data is loaded into a data set for use by my Dashboard. I can then repeat this, and I’ll have two sets of data.

And, here’s my PowerBI Dashboard. It’s not terribly useful, or interactive, but it’s got data from the web that I didn’t have to copy or move.

https://app.powerbi.com/view?r=eyJrIjoiMWNmYzBiYjUtMTU3Yi00NWFhLWFiZjQtNTY0NzY4NDRkZTJmIiwidCI6IjY2NjBkOGZkLTJjNmItNDg0Mi1iZmZmLTcxOTY1YzE2NTczYSIsImMiOjN9

February 16, 2016
Tracking Professional Athletes

I used to read about technology to allow computers to track a single person’s movements from video footage. We’ve seen this shown in Hollywood movies, where casinos can take a picture of a person and backtrack all their previous movements throughout the day. I have no idea if this is possible, but if it is, it’s a scary proposition.

However in specialized, controlled circumstances, we can track people very well. The NBA (with SportVU) is tracking all player’s movements and gathering more data that can be analyzed to better evaluate player’s performances. I expect in the next year or two, players will start to review this analysis and learn how to better adapt to the situations on the court.

The NFL is also starting to use sensors to track their players, but with different goals. They are monitoring workloads, trying to ensure the health and peak performance of players. It remains to be seen how else they might use this data, but in the linked article, there are perhaps more implications for us as data professionals.

Can you imagine more tracking data, for any movable object available? People, machinery, who knows what else will be tracked, and what else we might need to analyze. I thought spatial data had tremendous possibilities when I first saw it introduced in SQL Server 2005. However across the last decade I think we’ve barely scratched the surface of what location data might mean for applications, using it mostly for mapping locations and routes. If this data does become useful, that means that learning to aggregate, trace, and analyze location might just need to be a valuable, if not core, skill set for the DBA and developer in the future. If you want to get started, maybe run through this article on your system and see what you think.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

January 14, 2016
The Machines are Learning

One of the technologies that Microsoft is promoting heavily is machine learning. This has rapidly gone from a technology that I heard little about to being in multiple keynotes that I’ve seen at large conferences this year. It almost seems that I can’t go a week without seeing some sort of machine learning article, announcement, or reference.

How applicable is machine learning for most of us? I’m not sure, but banks have certainly taken advantage of machine learning to reduce their risks associated with fraud and their systems work well. In fact, given the ways in which I travel, I’m amazed that I don’t get more calls about fraud related to my card use. With me in a new state almost every month, on a very irregular schedule, sometimes buying computer supplies far from home, I get called by a bank once or twice a year.

Across the ten or so calls I’ve gotten in the last few years, only one questioned legitimate purchases. The rest caught fraud on the same day that someone tried to use my card without my knowledge. That’s a very successful rate of both true positives, and a very, very low rate of false positives. I’m impressed.

If you haven’t played with machine learning, Microsoft has made it easy to give a try in Azure. You don’t even need to put in a credit card to get up and working with machine learning. Whether it’s applicable to your industry or not, you’ll have to decide, but I do think that lots of tedious analysis that humans do now could be better done by a machine.

At least after one of us humans has configured the algorithms and trained the machine to recognize patterns. And, of course, with one of us technologists periodically tuning the system to work better and monitoring the analysis as data changes.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.6MB) podcast or subscribe to the feed at iTunes and LibSyn.

December 17, 2015
Changing Scales and Creating Disappointment

I got an email recently that notified me that session feedback from the Summit was available for my talk. I’d had a lot of people in the room, and was curious how things went. I think the session was OK, a little off on time, a few too many questions I tried to answer, and perhaps a bumpy flow.

However when I got my scores, I had a 2.85 for the session overall, with various aspects of the talk being rated from 2.5 to 2.9.

Well, I sucked.

That was my first thought. I’ve been getting rated, and evaluating speakers on a 5 point scale for quite a few years at PASS events. I was surprised, and disappointed, and then a bit embarrassed that I hadn’t delivered a good talk at the Summit. Since I hadn’t delivered that talk in public anywhere prior to the conference, I thought I had made a big mistake. Apparently my practice that week in my hotel room had been for naught.

However then I saw this note in another email: One of the changes this year was to move from a five point rating scale to a three-point scale.

Hmmm, I missed that in my email somewhere, and didn’t notice this as I filled out a couple of session evaluations.

I don’t think there’s anything wrong with changing the scale. Personally I like the 3 point scale, but it wasn’t a change I noticed. The first communication with speaker feedback didn’t mention this.

Scale matters. Many of us know that by manipulating scales, we can make data look different, We can prove a point that might not be supported by a different presentation of the data on another scale.

Our clients and business users come to know and expect the various ways we present data. They will start to internalize scales and interpret data based on their expectations. We can change scales, but we need to make it clear and visible that we have changed scales.

Personally I would have appreciated the results being reported as:

Overall Session Score: 2.85/3.00

instead of

Overall Session Score: 2.85

That little extra information can mean a lot. Keep this in mind as you make fundamental changes to the way you present data.

December 1, 2015