Peer review is the cornerstone of quality assurance within the scholarly publishing process. To have no peer review would result in letting loose on the world treatises from honest and self-serving researchers alike. Faulty methodologies, inaccurate calculations, fraudalent data, outright plagiarism, and excessive hyperbole would go unchecked. However from its beginnings, which can be traced back over 350 years, we have yet to devise anything more perfect. We still grapple with whether it is useful and can be performed in a nonbiased and valid manner. Yes, peer review is an imperfect system, but like democracy, it’s the least bad system we’ve got.
Peer review is meant to not merely separate wheat and chaff but to focus attention on ideas beneficial for the development of a line of inquiry. Scholarly research depends on the circulation of new approaches to sustain advancement in not just traditional areas of study but in the ever-increasing number of interdisciplinary fields. The challenges to the current peer review system’s ability to achieve either the promise of accuracy or of fairness have only become more, rather than less, frequent.
There are signs of progress being made to improve the quality of peer review. Prior to the adoption of appeals processes, authors had little recourse to challenge biased reviews whether generated by inexperienced or less-than-competent reviewers or based on conflicts of interest from competitors or “the old guard.” Improvements such as the education and training of reviewers, the application and continuous development of software programs, and new approaches such as post-publication and open review have increased each scholarly community’s confidence to challenge past and even present approaches. Recent attention and mandates from funders and the general public’s awareness of the role played by the peer review process in the larger context of policy have added to this sense of whether mere alterations will suffice or whether systemic change will be inevitable.
With a nod to Peer Review Weeks present and past, this column begins with recent 2020 articles and ends with resources from the Society for Scholarly Publishing’s The Scholarly Kitchen, spanning a decade. The goal is to present attitudes and research about the successes and problems resulting from changing or maintaining the status quo. These may assist you and your organization in assessing your current operation and perhaps implementing changes to achieve the best peer review process for your publications.
Publish, profit, predate, perish and peer review
Fish P. Univ World News. 2020.
https://www.universityworldnews.com/post.php?story=20201005070134522
What can we do to improve peer review in NLP?
Rogers A, Augenstein I. 2020.
https://arxiv.org/pdf/2010.03863.pdf
An editorial and technical journey into Post Publication Peer Review (PPPR)
Roment L. 2020.
https://hal.inria.fr/hal-02960535
Transparent Peer Review at Wiley: Two years on what have we learnt?
Moylan E, Junge K, Oman C, Morris E, Graf C. 2020.
https://d197for5662m48.cloudfront.net/documents/publicationstatus/48666/preprint_pdf/2d105e8e318671b2c0f0839bb5f092be.pdf
Peer review declaration
J Phys: Conf Series. 2020.
https://iopscience.iop.org/article/10.1088/1742-6596/1644/1/011003/pdf
Exploring academic publishing for Peer Review Week
September 21–25 was Peer Review Week, an annual celebration of peer review, for which the 2020 theme was “trust.” In this list, Technology Networks highlights several important aspects of the scientific publication process to mark the event.
Campbell M. Tech Networks. 2020.
https://www.technologynetworks.com/tn/lists/exploring-academic-publishing-for-peer-review-week-340866
IOP Publishing pledges to implement double-blind peer review
Banks M. Phys World. 2020.
https://physicsworld.com/a/iop-publishing-pledges-to-implement-double-blind-peer-review/
A parallel pandemic: the crush of covid-19 publications tests the capacity of scientific publishing
Kurth T, Piccininni M, Loder EW, Rohmann JL. BMJ Opinion. 2020.
https://blogs.bmj.com/bmj/2020/05/26/a-parallel-pandemic-the-crush-of-covid-19-publications-tests-the-capacity-of-scientific-publishing/
Stanford researchers discuss the benefits – and perils – of science without peer review
Science moving forward without traditional forms of peer review could shorten the path to solutions – but it also increases the chances that low-quality science gets overhyped.
Kubota T. Stanford News. 2020.
https://news.stanford.edu/2020/04/06/open-science-era-covid-19/
External tests of peer review validity via impact measures
Gallo SA, Glisson SR. Front Res Metr Anal. 2018;3:1–22.
https://doi.org/10.3389/frma.2018.00022
Toward new indicators of a journal’s manuscript peer review process
Moed HF. Front Res Metr Anal. 2016;1:1–5.
https://doi.org/10.3389/frma.2016.00005
Emerging trends in peer review—a survey
Walker R, da Silva PR. Front. Neurosci. 2015;9:1–169.
https://doi.org/10.3389/fnins.2015.00169
Who’s afraid of peer review?
Bohannon J. Science. 2013;342(6154):60–65.
https://science.sciencemag.org/content/342/6154/60/
Beyond open access: visions for open evaluation of scientific papers by post-publication peer review
Kriegeskorte N, Deca D. Front Comp Neourosci. 2012.
https://www.frontiersin.org/research-topics/137/beyond-open-access-visions-for-open-evaluation-of-scientific-papers-by-post-publication-peer-review
Bias in peer review
Lee CJ, Sugimoto CR, Zhang G. Cronin B. J Am Soc Inform Sci Technol. 2012;64(1):2–17.
https://doi.org/10.1002/asi.22784
The history of the peer-review process
Spier R. Trends Biotechnol. 2002;20(8):P357–P358.
https://doi.org/10.1016/S0167-7799(02)01985-6
The process of peer review of scientific manuscripts
Williams E. JAMA. 1988;260(12):1761–1768.
https://jamanetwork.com/journals/jama/article-abstract/374151
Parallel peer review at Cell Press: an interview with Deborah Sweet
Vines T. Scholarly Kitchen. 2020.
https://scholarlykitchen.sspnet.org/2020/10/15/parallel-peer-review-at-cell-press-an-interview-with-deborah-sweet/?informz=1
The Scholarly Kitchen Archives
Peer Review posts August 31, 2010 to October 1, 2020
https://scholarlykitchen.sspnet.org/category/peer-review/
AIP Publishing implements ORCID reviewer recognition service
https://publishing.aip.org/about/news/aip-publishing-implements-orcid-reviewer-recognition-service/
*A person who indulges in and desires information gathering and interpretation. The term was introduced in 2006 by neuroscientists Irving Biederman and Edward Vessel.