Five ways to fix statistics

As debate rumbles on about how and how much poor statistics is to blame for poor reproducibility, Nature asked influential statisticians – Jeff Leek, Blakeley B. McShane, Andrew Gelman, David Colquhoun, Michèle B. Nuijten & Steven N. Goodman – to recommend one change to improve science. The common theme? The problem is not our maths, but ourselves.

Jeff Leek (associate professor of biostatistics at Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland) says: “To use statistics well, researchers must study how scientists analyse and interpret data and then apply that information to prevent cognitive mistakes. […] We need to appreciate that data analysis is not purely computational and algorithmic – it is a human behaviour. […] The first step is to observe: what do people do now, and how do they report it?

My colleagues and I are doing this and taking the next step: running controlled experiments on how people handle specific analytical challenges in our massive online open courses3.

We need more observational studies and randomized trials – more epidemiology on how people collect, manipulate, analyse, communicate and consume data. We can then use this evidence to improve training programmes for researchers and the public. As cheap, abundant and noisy data inundate analyses, this is our only hope for robust information.