Is Peer Review Broken?

Submitted by Aspergillus Administrator on 27 September 2013

The centuries-old process of assessing the worth of a piece of research prior to it being published is coming under attack and calls for reassessment. All research papers submitted to a scientific journal since the 1650’s have been subjected to examination by experts in the field of research that the paper addresses.

This is undertaken to ensure that the paper offers something new but also try to ensure that when someone (usually several people) who is familiar with that type of research has time to look deeply into the data presented, they cannot find and major faults or omissions. Only then is the paper published.

This gives us a minimum standard of trust in the results of that paper. If this system goes wrong and a ‘rogue’ paper slips through that paper can still be questioned and withdrawn if found to be at fault. Peer review has sustained scientific research for nearly 350 years and the results are difficult to argue with – science has achieved so much that it would not have been able to do without ensuring the bedrocks of the process are of high quality.

 It is however not a perfect system. In a small field there may be only a few people who could thoroughly review an article. This leads to an increase in the probability that the reviewer will know the author of the paper personally (the authors are never told who their reviewers are) and thus rumours of foul play begin to circulate as friends may mutually support each others work. This is not usually a problem of fraud (but fraud does happen measured at 0.3% of all papers) but accusations that a paper has been published in a higher ranked journal than it deserved on merit. Occasionally we hear of a suspicion from a disgruntled researcher who has had lengthy problems getting a paper published and in the meantime someone else has published the same thing or misused their early access to an original work in some other way – perhaps a reviewer has read their paper and stolen their ideas? – always very difficult to prove either way.

In our modern world the system comes under new pressures. This talk outlines some of them:

Full discussion here