5 November 2008

Lazy reviewers and their effect on my blood pressure

Today, I responded to a second round of reviews on a paper I'd written.  

Reviewers vary.... some are helpful, insightful, and even help to produce a better standard of work than would have been possible without their comments.  I think I have received one review like this in my admittedly short publishing career.  Other reviewers are insane, or at least produce rude, spiteful, obstructive "reviews" that contain more ad hominem remarks than comments relating to the actual work.  Most reviewers lie somewhere in between these two extremes; thankfully closer to the helpful than sociopathetic end of the spectrum..

For the paper I dealt with today, I had a reviewer from a different category: the lazy "I can't be bothered reading your manuscript in full so I thrash it with sweeping statements" reviewer.  This individual's first review can be summed up as saying our conclusions are flawed because:
  1. Results in [related but fundamentally different paradigm] mean there is no way to rule out [alternative theory]
  2. Results could be explained by [jargon term I've never heard of, by reference that doesn't exist in any number of alternative spellings and years]
  3. Findings [for condition A] could be due to [explanation that does not fit for condition B]
These are the kind of reviews that make me grind my teeth as I read.  I have no problem replying to specific criticisms, but the vague, careless nature of these points made them almost unanswerable.  However, answer them we did, though because the journal imposes a tight word limit on the article we put most of our discussion in a detailed response letter rather than the manuscript.

Then arrived more reviews, where this individual (now the only one who isn't content to see the article published) had a single comment:
  1. The authors haven't addressed any of my concerns.
So... my question is this: why are such people repeatedly invited to review?  If someone can't state their concerns about an article clearly enough to enable an answer, why are their comments given any weight?  If a reviewer doesn't even read authors' responses to their criticisms, why is their vote allowed to obstruct a paper for the second time?

I have spent far longer working on this paper than it really deserves as it's only a moderately-interesting-topic in a moderately-good-journal (yes, I did think of submitting it elsewhere but I wasn't expecting these multiple rounds of reviews).  The experience has made me add two more items to my list of When I'm an Action Editor:

"I will not allow the comments of a lazy reviewer to be the lone voice of obstruction"

"I will blacklist lazy reviewers and not ask for their comments unless every other reviewer in the field has already said no"

No comments: