Steamrolled by Big Data

“Some problems do genuinely lend themselves to Big Data solutions,” writes Gary Marcus in The New Yorker.

“But not every problem fits those criteria; unpredictability, complexity, and abrupt shifts over time can lead even the largest data astray. Big Data is a powerful tool for inferring correlations, not a magic wand for inferring causality. The field has thus far apparently yielded only modestly improved weather prediction, and had little, if any, impact on challenges such as getting computers to program themselves.” […]

“The more complex a problem is, and the more particular instances differ from those that came before, the less likely Big Data is to be a sure thing.

In the years to come, scientists and engineers will develop a clearer picture of the circumstances in which Big Data can and can’t make a big difference; for now, hype needs to be tempered with caution and a sensitivity to when humans should and should not remain in the loop. As Alexei Efros, one of the leaders in applying Big Data to machine vision, put it, Big Data is “a fickle, coy mistress,” inviting, yet not without risk.”