Does big data have us ‘fooled by randomness’?

Being surrounded by data makes it easy to see the noise rather than the signal, and the trees rather than the forest, writes Andre Mouton in USA Today.

“Nassim Nicholas Taleb achieved notoriety with several books written before the housing crisis, criticizing the financial industry for putting so much faith in its predictions. He argued in Fooled by Randomness that there are problems with our attempts to understand the past, and even larger issues when we use it to predict the future. Those criticisms turned out to be justified.

“Big data” is allowing more industries to try their hand at fortunetelling. With social media and portable devices, we can watch society just like traders watch the stock market. People can be measured, quantified, modeled. As we enter this brave new world, it’s worth considering some of Taleb’s points and seeing how they might apply to big data.”

He concludes: “If businesses and governments see [big data] as a tool for self-measurement, they’ll find it useful. If they see it as a way to “crack the code,” or quantify human nature, or predict the unpredictable, they’re probably fooling themselves.”