Interactive Peer Review: For Authors, Potential Downsides

This is the third post in a series on new, interactive methods of peer review and their advantages—and disadvantages—for early-career scientists. You may want to read part 1, “How Interactive Peer Review Works,” and part 2, “Advantages for Authors.”

Interactive peer review, especially when it is done in public, can be a difficult and risky exercise. Why?

Premature exposure of your work. In journals like Electronic Transactions on Artificial Intelligence (ETAI) and Atmospheric Chemistry and Physics (ACP), your work becomes public in pretty much the form that it was in when you submitted it, prior to the thorough vetting that a closed review process typically provides. In journals with public peer review, that process, in which embarrassing, if trivial, errors may be discovered, “does not occur anymore ‘behind the scenes,’ ” writes Davide Zanchettin, a postdoctoral researcher at the Max Planck Institute for Meteorology in Hamburg, Germany, who co-authored two papers submitted to two different interactive-review and open access journals of the European Geosciences Union, in an e-mail toCareers. “Somehow I feel this lowers the confidence when you submit, since, for instance, rejection of a paper will be visible forever.” Also, that exposure—the fact that the work has already been made public—makes it “more difficult to resubmit the study in a revised version to a different journal.”

Public criticism. “Our professional reputations are very much at stake here, so authors and reviewers are very cautious about participating,” writes Rosemary Redfield, a microbiologist at the University of British Columbia, Vancouver, in Canada, in an e-mail to Science Careers. “Reviewing/being reviewed is a very perilous business for scientists even under the present system. We’re not at all comfortable with losing control over what critical comments might become public.”

Erik Sandewall, a now-retired computer scientist from Linköping University in Sweden who launched ETAI in 1997, believes the risk is manageable. “What people will observe is not how many times you are criticized but how well you respond to a criticism,” he says. “So don’t be afraid to have a discussion,” he adds. “You should keep your cool, and you should realize that the people who give you the criticism are not necessarily right, and [other] people will not think that they are necessarily right.”

Time. Kathleen Fitzpatrick, director of scholarly communication for the Modern Language Association and a visiting professor at New York University in New York City who studies how networked communication technologies affect scholarship, found in her experiments that, for authors, interactive peer review is “more work than a traditional review process,” she says. Before they get involved, young scholars should make sure that they understand the process and what will be expected of them.

An ongoing experiment. The biggest advocates for open peer review are those who believe that traditional peer review is broken and are looking for ways to fix it—so a lot of new ideas are flying around and a lot of experiments are occurring. In her experiments, “a couple of [the authors] did have concerns about whether their deans, for instance, were going to understand that this was a peer review process that had all of the authority of traditional peer review,” Fitzpatrick says. So, before you publish in one of these journals, it’s a good idea to talk about it with your dean and department chair. 

Journal reputation. For scientists who are seeking to be hired, tenured, or promoted, the status of the journals that they publish in is often vitally important. But many interactive-review journals are very young, which shows in their impact factors. “What could be difficult for a young scientist at the beginning of their career … is that some of the early journals don’t have an impact factor,” says Henry Markram, a neuroscientist at the Swiss Federal Institute of Technology in Lausanne who is also the co-founder and co-executive manager of Frontiers. Establishing an impact factor can take 2 to 3 years. Nonetheless, “there are journals with open peer-review processes that have extremely high impact factors, so it’s entirely possible to do this kind of work and maintain all those traditional markers of prestige,” Fitzpatrick says. Examples include ACP and some of the Frontiers journals.

Not all the new experiments have good intentions. Jeffrey Beall, a librarian at the University of Colorado, Denver, offers a comprehensive list of potentially predatory scholarly open-access publishers, as Redfield points out. These are journals and publishers that, in various ways, don’t adhere to accepted ethical and editorial practices (see box, below). “The use of new forms of peer review by some borderline/predatory online-only publishers diminishes these forms’ credibility,” she writes.

On his Web site, Jeffrey Beall offers a list he calls “Criteria for Determining Predatory Open-Access Publishers.” Here are some items from the list, telltale signs that something isn’t right:

We have one more installment planned in our short series on new, interactive peer-review protocols, which we’ll publish in the coming days. Stay tuned.

Opening industryacademic partnerships

Looking Broadly to Find the Best