Impact Science

Peer review is integral to scientific research and scholarly publishing, but the system is fraught with challenges, which often brings into question the quality and reliability of reviewer reports. Editors may often find it difficult to get suitable reviewers for every submitted manuscript, which can be further exacerbated by the increasing number of submissions. A potential solution is expanding the reviewer network. However, building and maintaining a reliable pool of reviewers and regularly monitoring them can often be challenging — with some instances of poor-quality peer review slipping through the ranks. Several publishers have introduced peer review training, which is a positive step towards ensuring that their review standards align with those of the journals and publishers as well as shaping the peer review system to be more robust and reliable. Here, we discuss five key elements that every publisher should consider in their peer review training models.

1. Ethics and conflicts of interest

Most journals expect their reviewers to adhere to Committee on Publication Ethics (COPE) guidelines and may also have their own set of guidelines typically covering reviewer responsibilities to practice confidentiality, objectivity, and diligence. A comprehensive module that covers the importance of honoring these ethical guidelines can help reviewers better adhere to them. Typically, journal editors avoid inviting reviewers with obvious conflicts of interest; however, some reviewers may have conflicting interests that may be unknown to editors. Careful consideration of conflicts of interest on the invited reviewer’s part is crucial, as they can compromise the objectivity of the report. The policies around conflicts of interest may differ depending on the journal or publisher and providing a comprehensive module on identifying such instances can help prevent delays and maintain integrity of scholarly publishing. For instance, PLoS One has a checklist to help reviewers identify any conflicts of interest. Incorporating such resources can provide a clear guide to reviewers.

2. Sensitivity training and addressing unconscious bias

Sensitivity training is an important yet often overlooked component of peer review training. The importance of sensitivity training is further apparent from a study that reported 50% authors receiving at least one unprofessional review, with some comments personally targeting a scientist and being too harsh without offering constructive criticism. Such comments can negatively impact authors, particularly those in marginalized or underrepresented groups. Training reviewers to adhere strictly to codes of conduct can avoid unprofessional comments, eventually reducing the need for oversight and scrutiny of reviewer reports required at the editorial level.

While reviewers are expected to point out the shortcomings of a manuscript, they may lack training in identifying and dealing with any implicit or explicit bias. Bias in peer review can be unconscious, including age, gender, more favorable view of manuscripts with positive findings, personal or professional relationships with authors’ or affiliated institutions, and financial interest in the presented work. A thorough understanding of unconscious bias, and its effects on the peer review process, can help the development of approaches to mitigate it. An example of such an attempt is the Center for Scientific Review’s training, targeted towards mitigating common biases in the peer review process. It incorporates interactive exercises and real-life examples of strategies to intervene in cases of bias.

3. Areas to focus on while reviewing a manuscript – how to dissect a manuscript

The peer review process typically encompasses assessing the importance, originality, strength and weaknesses of the work, and analysis and presentation of data. A good review would require reviewers to dissect a manuscript’s components and evaluate them individually and as a whole. Researchers often learn to review manuscripts by experience and mentoring, rather than through formal training. A well-constructed training program that explains how to break down a manuscript to evaluate it objectively and critically can help researchers gain an in-depth understanding of this critical task. Additional resources like checklists can help reviewers cover all bases and present a well-rounded assessment.

4. Writing peer review reports

Peer review reports are expected to provide well-corroborated feedback along with recommendations about a manuscript. One of the modules in the training can be dedicated to explaining to potential reviewers how to write a peer review report. This can cover how to provide detailed comments while maintaining ethical and professional standards. Providing examples and elaborating the dos and don’ts can further help reviewers while drafting their reports. Training reviewers to write effective peer review reports can help speed along the final decision and revision processes.

5. Recognition for reviewers

While peer review is considered one of the best ways to safeguard the quality of scientific publications, peer reviewers often feel underappreciated and unrecognized for the role they play. Incorporating peer review training certification would provide a formal nod of appreciation to the contribution of peer reviewers. Echoing this is a survey of Nature reviewers, wherein 44% of respondents said recognition coming from publishers or editors would be viewed as most valuable. Several publishers like Elsevier, Springer Nature, Wiley, and the Royal Society of Chemistry have taken initiatives to offer recognition to reviewers. Some forms of recognition include offering awards and rating reviewers’ profiles.

Despite its challenges, peer review has stood the test of time and integrating peer reviewer training can be a critical step towards making the process more robust, creating a pool of qualified reviewers adhering to scholarly publishing standards, and improving the overall peer review process. Several publishers, organizations, and academic societies like Wolters Kluwer, the Genetic Association of America, and the American Chemical Society have peer review training programs to coach researchers. As peer review training becomes more popular, well-designed programs can contribute to improving scholarly publishing and maintaining scientific integrity.