Interactive Peer Review: What’s In It for Reviewers?

This is the fourth post in a series on new, interactive methods of peer review and their advantages—and disadvantages—for early-career scientists. You may want to read part 1, “How Interactive Peer Review Works, ” part 2, “Advantages for Authors,” and part 3, “For Authors, Potential Downsides.”

There are two ways that a scientist can participate in the peer-review process, as an author and as a reviewer. Being a reviewer is a good experience for early-career scientists, and a more open, interactive peer-review process offers some advantages and some disadvantages. What are they?

Interactive and public review gives the opportunity to shortly/specifically comment on a paper without the burden of a full evaluation.

More recognition. One thing that “is broken in the current review process is that it requires an extraordinary amount of labor on the part of the peer reviewers involved, who remain anonymous [and] do the work for free,” says Kathleen Fitzpatrick, director of scholarly communication for the Modern Language Association and a visiting professor at New York University in New York City. Fitzpatrick studies how networked communication technologies affect scholarship. “When … this kind of review takes place in an open environment in a conversational process, we can begin to give credit to reviewers.”

While the Frontiers journals do not make the individual contributions of reviewers public (they share them only with authors), they do offer reviewers exposure, says Henry Markram, a neuroscientist at the Swiss Federal Institute of Technology in Lausanne, who is the co-founder and co-executive manager of Frontiers. They do so first by naming them in the publication and second by giving them the opportunity to publish a joint statement to accompany the article they have reviewed. Frontiers also publishes reviewer profiles on their Web site.

Networking opportunity. Especially when it is done in public, interactive reviewing can be a substantive networking opportunity, helping early-career scientists make useful connections. “Particularly for junior scholars, this kind of reviewing in an open space can be extremely important, as … generally one engages with a community of scientists,” Fitzpatrick says. This engagement, she says, can help reviewers build a reputation among other scientists.

Weighing in. At some journals—those that are open to the whole scientific community like the European Geosciences Union’s (EGU’s) Atmospheric Chemistry and Physics (ACP)—scientists can contribute even if they are not official reviewers. “Interactive and public review gives the opportunity to shortly/specifically comment on a paper without the burden of a full evaluation,” writes Davide Zanchettin, a postdoctoral researcher at the Max Planck Institute for Meteorology in Hamburg, Germany, in an e-mail to Science Careers. Zanchettin has co-authored two papers that were submitted to different EGU interactive-review and open-access journals. On one occasion, he corrected the interpretation of some of his previous findings in an ACP discussion paper. “I think this was good for the authors (they welcomed my suggestion) and for me, since I made them aware of my work, and I did it in an effective way.”

ACP initiator and Chief Executive Editor Ulrich Pöschl of the Max Planck Institute for Chemistry in Mainz, Germany, argued in July 2012 that journals with public peer review, like ACP, tend to receive manuscripts that are better prepared and so need less work. “[A]nticipation of public peer review and discussion deters authors from submitting low-quality manuscripts and, thus, relieves editors and referees from spending too much time on deficient submissions,” he wrote.

Keeping up with the literature. To intelligently participate in a public discussion, whether as an assigned reviewer or as an interested participant, you must first have insight into the field to be able to recognize what is and isn’t important. On one side, that means staying up to date on the literature, which takes up more and more time as new journals proliferate. But on the other side, it is a highly worthwhile activity and even essential. Participating in online discussions can help because you get to see, in real time, what other scientists see as key advances and subjects of disagreement. “For the readers the main advantage is the access to the newest studies, with opportunity to track the discussion ‘live,’ ” Zanchettin writes.

The risks of challenging established scientists. This is potentially the only outright disadvantage to participating as a reviewer in an open review process. It can be dangerous for vulnerable early-career scientists to criticize the work of top scientists when their names and comments are disclosed. On the other hand, if you’re too deferential and choose to overlook shortcomings, that can hurt you, too, especially in a public venue. “There are junior scholars who are nervous about presenting criticism of the work of more senior scholars in public,” Fitzpatrick says.

But Fitzpatrick believes that there are ways to engage in this process so that it doesn’t become damaging or risky. “The thing to do is to maintain a positive, professional, collegial, supportive mode of working with the material of anyone, … to help support them through the process of improving their work rather than thinking about this as … a process of negative, critical, potentially risky engagement.”

Series conclusion: deciding where to publish

Should early-career scientists publish in journals with interactive peer review? Opinions are mixed. There are, as Science Careers has noted, advantages and disadvantages. Perhaps the biggest disadvantage is simply that the process is new and evolving and its impact on careers remains largely untested. Still, “Overall … the pros largely overwhelm the cons for authors,” Zanchettin writes in his e-mail. But in any case, when choosing where to submit an article, “I wouldn’t put the type of review process at the top of my list” of criteria, he adds. “If one has something really groundbreaking, then definitely he/she should first try Nature or Science or similar.” Otherwise, scientists should submit to the journal that best fits the paper, he advises.

In an e-mail to Science Careers, Rosemary Redfield, a microbiologist at the University of British Columbia, Vancouver, in Canada advises caution. “I guess I’d try it (and recommend it to others) first for a low-importance manuscript, and see how it goes.”

Gender Equality in European Research

A Downstream Pathway into Teaching