Nate Silver is a “data guy.” He has made a career of analyzing data, and doing it well. His site, FiveThirtyEight, has become very popular in the last few years after some of their political predictions were surprisingly accurate. Now the site publishes analyses of sports, politics, economics, and more.
Mr. Silver works with a lot of data, as does the rest of his company, and arguably, he’s one of the people that understands just what “big data” is and isn’t. We could argue about what the term means, but no matter what “big” is to you, Mr. Silver has some thoughts about the challenges we all face with larger data sets.
In an interview, he noted that storage and access are always issues. Many of are us know this because we are charged with managing data. However he also talks about bias, false positives, and complexity being problems with an analysis that isn’t performed well. It’s a short piece, and I wish there were more depth since many of these topics could be the subject of a book all by themselves.
While many of us aren’t in charge of performing the analysis, we do often work with those that do. We assist others in writing the queries or assembling the data. Our skill in helping perform analysis, understanding the meaning of data stored in our systems, and learning how to separate the signal from the noise will be talents that your employer will appreciate.