Can offering choice to researchers reduce researcher bias?

High-quality peer review has never been more important to validate the scientific evidence we publish. With research integrity making headlines regularly, especially since the COVID pandemic, publishers and reviewers play a critical role in ensuring robust, reliable research is published and flawed work is not.

The backbone for maintaining integrity is peer review, where experts in the same field assess the quality of research. The process is designed to ensure high standards, improve promising work and weed out problematic papers, but there are some flaws in the process.

peer review deficiencies

A well-documented problem is peer review bias. Whether intentional or not, it interferes with fair judgment based on things like gender, name, nationality, affiliation, or career status. To counteract this, new peer review approaches are being introduced and tested for their effectiveness.

One such approach is doubly anonymous peer review, which obscures the identities of both authors and reviewers. The Institute of Physics was the first STM publisher to offer doubly anonymous peer review for all of our proprietary journals. The move is part of our commitment to combat inequality in the scholarly publishing process. We decided to implement double anonymity on a voluntary basis to give our authors choice and to help us investigate the effectiveness of this approach as a tool against bias. We hope the authors will see the benefits of judging research on merit rather than factors such as the prestige of the institution they work for.

The initial results are encouraging, in fact, Nobel Laureate Novoselov recently anonymized his manuscript when submitting his work to one of our journals, demonstrating his belief in the publication system and confidence in the quality of the research, rather than relying on their established reputation . But the question is, if we don’t enforce double anonymity and just encourage it, do we see any reduction in bias from peer reviews at all? Is there even prejudice against doubly anonymous articles when reviewers assume that the author is hiding his identity for a specific reason? To put this question to the test in publication, we partnered with researchers at the University of Michigan and provided data on our journey to doubly anonymous peer review.

Level the playing field

The researchers looked at submissions from a large number of researchers – 390,000 authors and 168,000 reviewers from around the world. The results of the study are encouraging, showing that offering double anonymous peer review increases the likelihood of positive referee recommendations for low-prestige authors by 2.4% and by 1.8% and 1.8% for medium- and high-citation authors, respectively % lowers .

But the most exciting finding was the fact that the policy has the greatest impact on reducing prestige bias in final paper decisions, increasing the acceptance of low-prestige authors by 5.6% while increasing them by 4.6% and 2 .2% for medium and high citations lowers authors respectively In short, the double anonymous peer review levels the playing field. We need to look at the impact on other types of prejudice, e.g. gender and geography, although the early results look intriguing as researchers from some parts of the world are twice as likely to accept their work using the double anonymity method.

Giving researchers choice, rather than forcing them, offers some of the benefits of hard-to-implement policies at a vastly reduced cost. And while the results are encouraging, we believe that duplicate anonymous peer reviews are just one method of supporting greater research integrity. As a sector, we need to work together to develop best practices and to support and educate researchers in their reasoning and the consequences of their publishing decisions.