Opinion

As a Reviewer, Each Review is Yours, and Sometimes It Should Be Everyone’s

Download Article

Scientists periodically ask on social media whether they may post their review of a published paper, considering the review process was confidential and anonymous. Is it necessary to ask the paper’s authors for permission to post the review? Should the journal give a green light? Frequently, publishers and editors don’t even know the answers or have clear policies. However, copyright law is straightforward in this case: the reviewer is the owner of the review. Unless the copyright is explicitly reassigned to the journal, the reviewer owns the content of the review and, therefore, has the right to share it.

In September 2015, the Committee on Publication Ethics (COPE) online forum had a discussion around “Who ‘owns’ peer reviews?” with much informative commentary. Certainly, many will hesitate to publish their review because of the understanding it was an anonymous and confidential process. And even when copyright supports the legal publishing of a peer review, is sharing this review a breach of expectations, norms, and responsibilities?

Confidentiality

There are a number of reasons why confidential peer review is standard practice. First of all, scientists do not want to be scooped on their work; if a competing group discovers the submission, they can try to rush their publication. Second, disclosing the existence of the submission to a particular journal could harm its future publication chances if that journal rejects the manuscript.

At the same time, the significance of confidentiality is diminishing as the popularity of preprints grows. Moreover, after a paper is published, the confidentiality of the manuscript itself is a moot point.

Anonymity

The vast majority of research journals practice single-blind peer review. That is, the identity of the reviewers is hidden to readers and to authors, while the names of the authors are known. The purpose of this type of policy is to protect reviewers from retribution from authors and to ensure the review is honest and complete.

Interestingly, one study suggests this anonymity actually makes no difference for the quality and may in fact make the reviews worse. Reviews themselves aside, open review has the potential to improve the publishing process and the progress of science. This is one reason why many scientists, including me, sign their reviews, and a number of journals encourage or even require open non-anonymous reviews (e.g., EMBO Press, F1000Research, The BMJ, Nature Communications, PeerJ, eLife, and Plant Cell).

Again, after a paper’s publication, anonymity of the submission and review process is no longer as important since the review has been considered and the editorial decision was made. The review can be posted openly on PubMed Commons or anonymously on PubPeer, and many journal websites make it possible to share the review as a comment directly on the article itself, whether the paper was reviewed for that journal or another one.

The Ethics of Publishing a Review

The biggest barrier to disclosing peer reviews after publication may have nothing to do with the legality, confidentiality, and anonymity issues. Is it a show of disrespect to the colleagues who authored the original papers to publish your review? In the journals that practice open review by default, authors and reviewers have a clear understanding. But what are the ethical considerations around sharing your review if the original authors don’t expect you to do so?

Here, I repost my previous editorial, slightly revised, on this topic.

Want to Be Ethical in Science? Speak Up.

What is the etiquette for disclosing an anonymous review you wrote? Because all of us have the natural sense that an anonymous review is supposed to stay anonymous, this is not a trivial question. Even I, an advocate of non-anonymous open peer review, see the problem of going public with something that was written in private at the request of an editor, with the understanding from the author and the journal that the review is both confidential and anonymous.

I tracked an excellent and extensive discussion of this on Twitter, I listened, and I have been thinking hard about it for a few hours on an airplane. And the more I think about this, the more I believe it is deeply wrong to keep silent about a paper you reviewed and think is flawed. Yes, we have the responsibility to the author of the paper to be civil. But that author–reviewer contract stands in direct conflict with our responsibility to the scientific community.

Suppose I got the stimulus-triggered acquisition of pluripotency (STAP) paper to review. The STAP technique was a major publication that could not be reproduced by anyone and was retracted. Suppose I saw through it and outlined the fundamental flaws. Then the journal editor decided to publish it anyway. Can I make my review public? By keeping silent, I am honoring a single author and disrespecting a world of researchers. By keeping silent, I am letting students and postdoctoral researchers waste months or years chasing smoke. I have a responsibility to the countless scientists following up on the published work. I have a responsibility to science and society. I think my duties to science, society, and the world of researchers override my oath to the author.

Now we come to the question of the appropriate means of responding. Many whose work is questioned fire back with, “This is unprecedented! Why didn’t you contact me first? Why didn’t you write to the editor? Where is your civility? You didn’t follow the etiquette!”

But if regular publishing is hard, publishing a rebuttal is an agonizing ordeal. In a pair of online polls, I asked how often scientists get rejected on any given paper, compared to rejections for a paper challenging previously published results. The frequency of getting three or more rejections jumps from 14% to 37% (Fig.). By arguing for the via-editor/journal way, we are placing an extraordinary burden on the scientist raising the question.

Figure. Results from an informal Twitter poll by the author, @lteytelman.

Yes, we should be civil and focus on the science rather than personal motivations. We can’t accuse authors of misconduct unless we have clear proof of it. We don’t need to be aggressive or rude. But we have to be honest and we have to encourage open and critical post-publication discussion. In discussions of this, I often see very different opinions on the “good” and “bad” ways to have post-publication discussion: Alerting the editor and trying to publish the rebuttal in the original journal is okay in some cases, but elsewhere it’s not. Publishing elsewhere is okay if it’s a journal, but not okay if it isn’t peer reviewed. Publishing a critique on a blog is okay, but disclosing your review on PubPeer is out of line.

These lines seem entirely arbitrary to me. As an example, my paper was ripped to shreds by another scientist-colleague in a publication. No one had reached out to me. I found the critique accidentally through my PubChase recommendations. I responded on PubMed Commons and on PubPeer. At least my response went to the corresponding author via PubPeer.

There is no rhyme or reason to personal opinions of what is and isn’t ethical for post-publication disputes. Post-publication commenting is certainly not traditional, but 30 years ago, there existed no web, no blogs, PubPeer, Twitter, and so forth. New traditions have been enabled today by technology. By asking people not to criticize, however, we undermine science. We hurt scientists. We hurt ourselves.

We must realize the criticisms are not about us—they are about the science. We have to learn to criticize each other’s work in a firm but civil manner. That can be done any way you like, whether through Twitter, the original journal, PubPeer, your blog, PubMed Commons, or other ways. While some are going to be mean and uncivil no matter where, I hope most can be respectful and focus on the science. Regardless, I think science is best served by scientists embracing post-publication critique.

Update: While reviewing this article prior to publication, Jonathan Schultz, Deputy Editor of Science Editor, asked the following astute question:

What is the benefit of acknowledging that the criticism posted on a published article originated as a *confidential* review? And why not simply reword the criticism to be a true post-publication review of the published article (which is fine)?

I can only think of two reasons to announce that you were a reviewer and to repost the original review as is. One, as some type of admonishment of the journal for publishing the article (in spite of a recommendation by that same reviewer that the manuscript not be published). Two, to draw attention to a dubious change made by the authors after the manuscript was reviewed. Even in these cases, I think it’s best to at first go through a journal’s official process and for a reviewer to only post these unedited reviews in the cases where the journal has failed to act.

LT:  I agree. What is important is sharing your review to help other scientists; how and where you do it is not nearly as critical.

Lenny Teytelman is the founder of www.protocols.io.