When experts get it wrong, where do we go for answers?

Recently two economists, Ken Rogoff and Carmen Reinhart, saw their influential study on national debt implode when a graduate student examined their work and discovered a spreadsheet error. The study had gotten lots of press for its claims about how much debt it is safe for a nation to have, and its debunking inspired NPR's Planet Money team to ask a larger question: how trustworthy are economics studies in general? If you listen to the podcast, you might be left wondering what it even means to be an expert in a field. If the experts are so vulnerable to error in their own fields of study, where do we turn when we need solid answers? What hope do we have of ever knowing something for sure?

On the one hand, this is how scientific and academic research work: each researcher tests earlier work and tries to correct the errors of those who have gone before so that the human race can move toward a truer vision of reality. But on the other hand, stories like this remind us how scarily how wrong we can be. Few of our errors are as widely publicized (and thereby corrected) as Rogoff and Reinhart's has been. And some errors cost lives: the Planet Money team mentions one study that claimed that, for each execution of a murderer, 18 fewer murders occurred--when in fact the correlation was the other way around, and each execution corresponded to an increase of 18 murders. Such flawed studies lead to flawed policies and laws.

The blog Retraction Watch does perform the important work of reporting on journals' retractions, which might otherwise go unnoticed. But retractions sometimes come years after publication, once dozens of researchers have already based their own new work on a problematic study. 

How much faith do you have in the top academics in your field? In other fields? Do you have more faith in the hard sciences, the social sciences, or the humanities? How much do you trust your own conclusions? 

Posted on June 19, 2013 .