How Not to be an Asshole: Peer Review Edition

I am somewhat late to the “here’s everything that’s wrong with the peer review process” party. This is in part because I have spent the last several weeks busting my ass to complete revisions requested by a reviewer of an article I submitted quite literally years ago. Following the Wordsworthian injunction that the “spontaneous overflow of powerful feelings” should be “recollected in tranquility,” I didn’t want to write about peer review when I was feeling Hulk-smash-ragey about the peer review process. But I’m feeling much better now. Since peer reviewers are so kind as to provide us all with feedback on our work, it seems only right that we should reciprocate by providing some suggestions for them. So, to the gatekeepers in my readership (I think there might be one of you, maybe even two!), I humbly submit my suggestions for revision of peer review practice.

Try to respond in a timely fashion.

I understand that peer review is generally thankless work; it is unremunerated, and at best it earns you some favors from the editor and an entry in the “service” column of an application for promotion. Mostly, it’s just service you do to the scholarly community because you have an investment in that community–this is very benevolent of you, and you probably deserve more thanks than you get. But a quick look at the editorial boards of all the big journals indicates that most of the people doing peer review work do so as tenured faculty: you folk have lots of pressures on your time, but you also have a position of security. By contrast, a lot of us who send essays out to journals and edited collections do so with a big fat clock ticking over our heads. For graduate students and adjuncts looking for full-time employment, it’s the job-search clock, which ticks down the minutes until the next market cycle. For these, the difference between having an article accepted for publication or not before the CVs go out to search committees can mean the difference between having their application advanced to the interview pool or not. For junior faculty, that ticking you hear is the sound of the tenure clock. If we don’t have enough publications when the buzzer sounds, we get to start looking for a new careers. So when an article lands on your desk, it would be extremely helpful to us if you could read it and return it with feedback ASAP.

We can debate what a reasonable timeframe would be for returning essays to their authors, but we can probably all get behind that notion that, if you’re measuring in half-years or years, then it’s taking too long. I am not the only person I know to have had an article sit on someone’s desk for over two years, and that’s really not okay, both for the reasons I just mentioned and because sitting on an essay for that long shows serious disrepect towards effort and time the author put in to writing the work in the first place. The problem here isn’t that it takes such a long time to read and comment on a 25-page article; the problem is that the work of reviewing doesn’t get prioritzed by the reviewers in this situation, which says to the author, “Your work is not important to me.” Again, there are reasons why the work of reviewing might get de-prioritized that I mentioned above. But if you take on the work, then you should commit to doing the work. It’s okay if you’re too busy: just decline the invitation to review, and let the editor find someone who isn’t, at present, swamped. The fact is that the author of the article is also, more likely than not, a tremendously busy person with similar if not more pressures on their time. The difference is that, when you accept the invitation to review an essay, that author is now depending on you for their work work to mean anything. It’s simple professional courtesy to hit your deadline and respond in a timely fashion. And if by (likely) chance that author is lower than you on the academic totem-pole, then anything less than a timely response is a failure of stewardship for your profession, insofar as you are potentially shutting the door in junior scholars’ faces without giving them the opportunity to succeed.

Help, don’t hurt

I remember one reader’s report I received that stated, “The author is clearly unaware of the large body of scholarship on X.” This was somewhat surprising to me, because I’d actively looked for things written on X, but hadn’t found much, and so I assumed it was a peripheral issue rather than something a lot of scholars talked about. The problem with the reviewer’s comment from my perspective–which is to say from the perspective of someone who sincerely wanted to make this the best essay I could–was that the comment was both dismissive and unhelpful. The phrasing was condescending insofar as it was a claim about my ignorance, rather than about an omission in the essay (my favorite on this score was one that said I had “only a light grasp” of the scholarship in the field), and it offered me no direction for fixing the problem. I’d already searched on this topic and found nothing; now I was being told that there was, indeed, a “large body” of stuff somewhere out there, but I was given no guidance in how to find it. By contrast, the most useful reader’s reports I have received–ones that eventually led to publication–made recommendations rather than accusations. The first reader for my book noted that the draft wasn’t addressing a body of scholarship that I hadn’t realized existed, and recommended that I look into work by a couple of specific scholars. Those recommendations helped me find other material on the subject as well, so that I was able to address the omission in the draft substantively, and, I think, produce a much stronger piece of work. Instead of calling me deficient and leaving it at that, my book’s reader told me what was missing, then pointed the way where I might find what I needed.

If the point of peer review were just to say “yay” or “nay” to pieces of scholarship, we wouldn’t bother with reader’s reports: readers would just give a thumbs-up or thumbs-down, and that journal would give the author a yes or no, without further comment. The fact that we do reports that get passed to the author at all suggests that part of the purpose of peer review is to help authors improve their work. If that’s the case, then when you are writing up a report you should ask yourself about every comment you make, “Will this help?” You have to justify your decision regarding the decision to publish or not, but if the answer is “no,” then providing guidance for how the essay could be brought up to snuff should be more of a priority than making sure the author knows you think they’re stupid. We do this all the time commenting on drafts of student essays. The point isn’t to tear the student down; where there are problems, we point out those problems and offer suggestions for how to address them–and we don’t (or we shouldn’t) say, “Your work is woefully inadequate” without further comment.

I’ll be honest, though: when, after trying to research a topic as thoroughly as I can, I see a reader’s report that alludes to a “large body of scholarship” I’ve missed without mentioning a single scholar or scholarly work that belongs to that body, I’m suspicious. It reads a little bit too much like undergraduate appeals to “what everyone knows,” and I have occasionally wondered if this “large body of scholarship” actually existed, or whether the reader were simply half-remembering a couple of essays they’d read that mentioned the topic. At best, this kind of criticism is lazy, a gesture with no substance. At worst, it’s intellectually dishonest. If you can’t come up with one citation off the top of your head, you might need to rethink whether this body of scholarship is as large and as important as you think it is.

Not everything is about you

I considered titling this section, “Don’t be a narcissist.” There’s a lot of different bad reviewer behaviors that fall under this category. Perhaps the most obnoxious–and most dangerous to the profession–is the one that doesn’t distinguish between the tasks of deciding whether a submitted work is good on its own merits and whether the reviewer actually likes what the work has to say. My very first reader’s report was also the most vicious I’ve received in my (still relatively short) career, because it came from a reader who seemed not to be able to tell the difference between an argument and a personal attack. The reader opened by saying that the essay was well-written, that it was adequately grounded in the relevant scholarship, and that the argument was well-structured and well-reasoned. And then they recommended that it not be published. I reread the report as many times as I could stomach, trying to figure out why they didn’t recommend publication, but all I could come up with was that the reader didn’t like my argument. Mostly, the report just criticized the politics of the position, and at one point the reader weirdly compared me–not the essay, but me–to George W. Bush. This went on for three pages, single-spaced. Luckily for me, the editor over-ruled the reader and gave me a revise-resubmit. I had to go a second round with the reader, but eventually they conceded with the comment, “perhaps I am just worn down from engaging with a well-crafted argument with which I vehemently disagree.”

This reader wanted to stand between my article and publication not because my article was a bad piece of scholarship–they admitted more than once that it was a good piece of scholarship–but because they didn’t like what I had to say. If this is the criterion all readers use for deciding whether something will be published, then there will be no innovation, no opportunity for argument or critique, and no advancement. We will simply clone ourselves. Which, as at least one person has pointed out, is pretty much what the humanities are already doing, suggesting that my experience with my first reader is not anomalous.

One of my favorite (by which I mean least favorite) comments I’ve ever seen from a reviewer was on a report a close friend received, which told him that, “If we accept the author’s position, then that closes down other avenues of interpretation.” Well, yes, that is what an argument–any argument worth the name–does: it makes the case for one way or set of ways of reading, and in doing so implicitly argues against others. What the reader really meant was that my friend’s position argued against the reader’s preferred way of reading the text, and the reader didn’t like that (so the reader recommended not publishing the piece; it got published in a better journal with no substantial changes later). This kind of gatekeeper is more like the bouncer manning the velvet rope at an exclusive club: they only let through the “right” kind of people, which is to say those who fit the tastes and preferences of gatekeepers, rather than those who are simply producing good scholarship. This is obviously not good for the profession of academic inquiry.

If you’re reviewing an essay submitted for publication, please try to remember that not everything is about you. If your preferred position on a topic you care about is being assailed, that’s not a personal attack on you and your career; it’s just an attempt to create a different way of looking at the problem. Similarly, not everything you read has to cite you, your friends, and everything you’ve read recently. After doing a Twitter poll of ridiculous responses from peer reviewers, Rebecca Schuman provided a succinct summary of what she saw:

So many readers’ reports can be boiled down to: “Why wasn’t this article exactly the one I would have written?” (Or: “Why wasn’t I cited enough?”)

This accurately describes my own experiences with peer review, and I suspect I'm not the only one. Asking authors to cite your own work more is just tacky, unless you’ve written the definitive treatise on the topic (and by “definitive” I mean others have recognized it as such–you don’t get to give yourself that designation). You may have written something on the topic, but you really shouldn’t treat your position as gatekeeper as your own little citation generator.

More often, though, reviewers seem to think that any work they review should have the same frame of reference they do. It’s one thing to read an essay and say, “Well, I’ve just read this other essay that seems like it might be useful to the author–the author might want to take a look at that.” It’s quite another to say, “The author neglects the work of [person I was just reading]; this is necessary to cite.” About a year ago, a reviewer told me it was “necessary to cite” an essay that was (1) at best tangentially related to all the main claims of my essay, and (2) NOT EVEN IN PRINT YET. This speaks to an argument I made long ago about over-citation; but it also suggests a mind that isn’t distinguishing between the reviewer’s work and ideas and the author’s. Unsurprisingly, this same reviewer also wanted me to cite their own work several times, including an essay that had just come out in print. When I went to look at the new essay, lo and behold, my reviewer had cited this “necessary” study that was still in manuscript. My reviewer wanted me to reproduce their own research.

Look, I get that we all work necessarily from our own frame of reference, drawing on our own bodies of knowledge to judge things. And I get that it’s a lot easier to see the faults in an argument we disagree with than in one that we agree with. But if part of the purpose of academic inquiry is the disinterested pursuit of truth, then at the very least those of us serving as gatekeepers have a responsibility to try to be aware of the limits of our perspectives and biases and to correct for them as best we can. If you’re reviewing a work and you want to say “no” or “make these revisions,” ask yourself first where you stand on the topic being discussed, and consider seriously whether your position is potentially skewing your recommendations. Ask yourself whether your recommendations are enforcing conformity with pre-existing research. Ask yourself also whether these recommentations are designed to help improve the work or to hurt the author of the work. And, for the love of all that’s holy, don’t take three years to do it.