Earth System Observer

My peer-reviewed publication record isn’t long yet, but I have participated in the anonymous peer-review process in a number of different scientific journals. I’ve read some great science that I thought was ready to be published only to see it rejected at the editor level of the journal. I’ve read grant proposals (how scientists fund their research but also how they gain recognition well beyond their immediate sphere of colleagues) that I would recommend and support, but that are rejected. I’ve read other manuscripts that I reject repeatedly to the point that I tell the editor asking me to review the article yet again that I simply will not read that article anymore. Science is built on recognizing incremental progress towards a solution, and more rarely on that great leap forward. Underneath it all, however, is the peer-review process.

Peer-review is the gateway to publications in technical journals, acceptance into conferences and workshops, and earning money to conduct research through grant proposals from funding agencies like NSF and NASA (among others). All professors hired to do research face these basic metrics when their professional activities are reviewed. Did they publish? Were they awarded a grant? Did they attend conferences? Other than organizing travel around teaching (and family), the last one is the easiest professional activity to participate in. I have never been rejected from a conference, for example. Conferences also carry the least weight when professional activities are reviewed.

So it comes back to publications and grants – two activities that are strongly dependent on peer-review. When I began reviewing articles submitted for publication as a graduate student, I couldn’t believe that the process of review was anonymous only from the reviewer side. Right away, I thought the review should be anonymous from both sides – the reviewer should not know who the authors (usually there are multiple) are and the author(s) should not know who reviewed the paper. Imagine a 4th year PhD student receiving a paper to review from a prominent journal that is co-authored by a scientist who has 100+ publications (a large number in climate and related sciences – usually meaningful in the sense that this person has done some serious work. Check out James Hansen’s peer-reviewed publication list) What kind of bias does the reputation/track-record of that author introduce into the review process for that poor graduate student? The same issue of a heavy hitter can play out the other way too. Imagine that prominent scientist is reviewing the second paper you’ve ever published and they’ve never heard your name or anything about your research. They might (might!) more easily dismiss your work than they would of a fellow heavy-hitter. The question of bias is hard to answer quantitatively. All that being said, I have had some great informal interactions with reviewers of my papers who voluntarily disclosed their identity. And even without knowing the reviewer’s name, their comments have almost always helped to improve my paper in some way. Still, bias is a nasty beast and scientists are human. That link goes to a paper published last year that concludes

The dearth of women within academic science reflects a significant wasted opportunity to benefit from the capabilities of our best potential scientists, whether male or female. Although women have begun to enter some science fields in greater numbers, their mere increased presence is not evidence of the absence of bias. Rather, some women may persist in academic science despite the damaging effects of unintended gender bias on the part of faculty. Similarly, it is not yet possible to conclude that the preferences for other fields and lifestyle choices that lead many women to leave academic science (even after obtaining advanced degrees) are not themselves influenced by experiences of bias, at least to some degree. To the extent that faculty gender bias impedes women’s full participation in science, it may undercut not only academic meritocracy, but also the expansion of the scientific workforce needed for the next decade’s advancement of national competitiveness.

Maybe the tone of that conclusion is reflected in this “letter” to Harvard University published recently in the Washington Post. I’m not convinced that the letter is really effective, but the point I took from it was that bias has consequences that may not be evident right away. Clearly, bias (intended, but more likely unintended as the PNAS article concludes) is, as I said before, a nasty beast.

Getting back to peer review and how it’s susceptible to bias, the simplest solution is that both sides remain anonymous and this is known as a double-blind peer-review. Finally, the journal Nature, which has a long history and is internationally-recognized and is so large it is now a publishing group with a number of disciplinary publications, has decided to include the option for double-blind peer-review in the journals Nature Geosciences and Nature Climate Change. This move is in my mind really important and I hope it is successful. Nature Geosciences published this editorial. The key point in this editorial is

In a reader survey last year (Nature Geosci. 5, 585; 2012), three-quarters of respondents were supportive of double-blind peer review, with only 16% unconvinced. Interestingly, those who might benefit did not preferentially support a double-blind process: the ratios of males to females, established scientists to young researchers, and people from western countries to scientists elsewhere in the world, were all very similar (down to a per cent or so) between supporters of double-blind peer review and the entire group of respondents.

Towards the future we go!