Annual Meeting Report

Peer Review Innovation

Moderator:
Alison O’Connell
Marketing Manager
Aries Systems Corporation
North Andover, Massachusetts

Speakers:
Allison Leung
Editorial Manager
Public Library of Science (PLOS)
San Francisco, California

Liz Bal
Associate Publisher
BioMed Central
London, United Kingdom

Carissa Gilman
Managing Editor
American Cancer Society
Atlanta, Georgia

Adam Nicely
Manager, Editorial Solutions
Wolters Kluwer, Health, Research, and Practice
Philadelphia, Pennsylvania

Reporter:
Kelly Newton
Content Production Manager
American Diabetes Association
Washington, DC

With time to journal publication taking 6–18 months, during which work is under embargo, preprints allow for an earlier claim of priority and an informal review and revision cycle through reader comments. PLOS believes journals and preprints can benefit one another, with journals continuing to provide formal review and services like marketing and discovery platforms. Allison Leung covered some of the ways PLOS is encouraging preprint use through partnership with bioRxiv. Authors can submit directly and easily to PLOS journals through the preprint server, and the preprint is linked to the version of record through Crossref at final publication. Editors and readers are therefore given access to the entire history of an article, and preprint readers are directed to the publisher’s site. PLOS Genetics has also hired editors to recruit submissions from bioRxiv, promising a full review to authors.

Positive results are often used to justify poor methods, while null results are rarely submitted at all, and therefore go unpublished. Liz Bal described a workflow developed for BMC Psychology called “results-free review” designed to prioritize study quality over results in editorial decisions. If authors opt in, results and discussion sections are stripped from the manuscript file used in initial review. The full paper is only reviewed after an “in-principle acceptance”; at that point, the paper can be rejected only if the results deviate unjustifiably from the methods. As of May 2017, 6 results-free articles had first decisions and 14 were undergoing review, so Bal offered only preliminary observations. The acceptance rate was similar to that of manuscripts undergoing standard peer review, but reviews were faster, and there was a perception of greater focus on transparency. Despite enthusiasm for the experiment, the opt-in rate is lower than hoped for, and BioMed Central (BMC) is currently focused on making the new process as attractive and seamless as possible for authors. Once the feasibility phase of the pilot is complete, BMC plans to launch a full randomized-controlled trial involving other journals to determine whether results-free review actually reduces publication bias.

Peer Review Innovation panel

The final two speakers both focused on ways to increase reviewer recognition. Some feedback from reviewers at Cancer suggested that journal-based rewards were less important than broad community recognition, particularly regarding career or tenure decisions. In response, Carissa Gilman reported, Cancer created the Outstanding Reviewer Recognition Program for reviewers who met predefined criteria for speed, quality, and annual number of reviews. These top reviewers not only received personal thank-you emails from section editors and discounts on open access fees, but also letters describing their contributions were sent to their deans and department heads.

The American Cancer Society also participates in Publons, a platform of verified reviewer profiles. If they choose to participate, reviewers must complete a score sheet for each review; the information is automatically sent to Publons and, upon publication, regardless of whether they recommended acceptance, reviewers receive links to the final articles. The publisher can change privacy settings on a per-journal basis, discover suggested reviewers, and track rejected manuscripts and reviewer demographics—including other journals reviewed for.

The final speaker, Adam Nicely, covered another strategy to add value to manuscript review and retain reviewers currently in place at 79 of Wolters Kluwer’s (WK) journals. Already an accredited provider of continuing medical education (CME) by the American Nurses Credentialing Center and the Accreditation Council for Continuing Medical Education, WK offers CME credits for completing reviews. Upon completion of a review, reviewers answer four questions about their experience within the Editorial Manager peer review workflow; the editor then evaluates the responses and determines whether to grant credit for that review. While there are a few complications to the process—such as there being no clear criteria for granting credit and some annoyance for reviewers who do not want credit and are still required to complete the post-review evaluation—Nicely reported that it is overall a successful and popular option.