Blog

Big Data Perception vs. Reality: Is it Value or Noise?

Posted by Gary Corrigan

Find me on:

Jun 3, 2014 8:16:10 AM

Great data points that stand out on their own (out of context) can certainly look impressive and convincing enough for decision-makers to make a bold move. The problem is, in the big data universe, there are plenty of those data points if you look closely enough. Do all of those big data findings equate to prime business opportunities? Nate Silver—one of the foremost statisticians, predictors, and vocal big data experts—says no. There is a need to discern from all of the noise that big data brings and logically assess the information that is in front of you. As Silver puts it, businesses need to stop “cherry-picking the results they want to see.”

So how should businesses go about running the most optimal predictive models and analytics to uncover the truth about their data?

For starters, organizations can’t let their data speak for itself. They need to put their own human intelligence around the results to flesh out a viable decision. Big data and analytics solutions are certainly powerful enough to get this process headed in the right direction. But these tools can’t be cure-alls.

In Silver’s words (from his book, The Signal and the Noise): “The numbers have no way of speaking. We speak for them. We imbue them with meaning… [W]e may construe them in self-serving ways that are detached from their objective reality.”

The last point is perhaps the most significant—and certainly an alert to decision-makers that rely solely on data without applying meaning to it. Analysts need to pinpoint the biases, the anomalies, and the outliers that can skew the picture. With that proactive approach, they can isolate the findings that are worthy of further dissection, and remove the noise.

The other crucial component to deriving value from big data is making meaningful correlations. Within these massive, raw data sets, there are simply too many data points that match up. Proper correlations can help narrow the field. Also, taking an active approach to associating causality makes a big difference.

Ultimately, as Silver has pointed out, big data can help determine probabilities – not certainties. With that caveat in mind, businesses can gain an edge using big data. We will dive into this topic even further in our ongoing series of analytics blogs.
   

Blog Search

Subscribe to Email Updates

Follow Me