Tuesday, February 17, 2009

Better than refereeing fees

A few days ago, I received an invitation to join a facebook group that demands that all journals should follow JHEP to pay their referees. So far, I did not sign up.

I am convinced the refereeing system has many flaws. But the pay of referees is not one of them. I don't know how much JHEP pays, I have heard the figure of 30$ per paper. What's that? Refereeing a string theory paper is a job that requires a specialist with a broad academic background. So, you would expect an hourly rate that is well above 100$ (judging for example the rate that lawyers demand). That means, by paying this specialist 30$ I expect that the refereeing takes him less than 20 minutes (including typing the report and uploading it to a web page). But that's exactly the problem with the refereeing system: The value you can add by refereeing a paper in 20 minutes negligible. You have to spend significantly more time with the paper to have a more significant opinion than you have after one minute of seeing the authors' names, reading the abstract and flipping through the pages.

On the other hand, referees are already paid for their refereeing: That's part of an academics job, and he/she already gets a salary from the university. That should already cover the refereeing as it is part of the job like it is to attend seminars and to discuss with other scientists.

The problem with the refereeing system really is that too often too little attention is given to the actual paper. Everybody knows first hand examples of excellent papers that were rejected for stupid reasons. On the other hand, there is a lot of very low quality stuff that gets printed, the Bogdanovs' papers and the El Nashie[no link so far] story being only the most prominent examples.

Of course, the refereeing process is most likely the only value that publishers add to a paper when it is promoted from a freely available preprint on arxiv.org to a published article. And we (that is our employers through their libraries) pay enormous sums for this more and more demanding justification. And giving this justification gets harder and harder with every b.s. paper that appears in print.

The flaw with the "you are already paid" argument is of course that refereeing is invisible and besides your obligation as a scientist there is little incentive to do a good job. Nobody (except maybe the editor) sees it and there is no reward, not even an idealistic one.

There is however one simple improvement that would be trivial to implement. I learned this from Vijay Balasubramanian a few years ago and I am convinced it should be introduced immediately: If a paper gets accepted, the identity of the referee should be published together with the paper while a referee that rejects a paper should stay anonymous.

This would give an incentive to do good work as a referee. If the paper's value is low and you still accepted it because you did not properly read it you will receive shame while if the paper is good people can see you put some effort into it. Keeping the identities of rejectors hidden of course prevents referees from accepting papers because of fear of any kind of "revenge" from the authors.

I am sure the quality of the refereeing process would increase significantly if this were implemented. Thus, I would urge you to support the publication of accepting referees names in the next discussion of the flaws of the refereeing system I am sure you will take part in over the next few weeks!

Update: Some brave soul has collected all the El Naschie stuff that seems to have disappeared from the web.

6 comments:

Anonymous said...

There is a rather weird attitude in academia that one should do lots of stuff for free. There is no reward at all in refereeing so why do we do it at all? Everyone is happy with the arXiv. Its the journals that tell us refereeing adds value so if it adds value then they should pay us. I also recently got asked to write a popular article (for free) I won't be doing that either. I also got asked to referee a grant application unpaid, I won't be doing that either. One can literally spend all ones time doing extraneous stuff. By charging it will hopfully make people take us less for granted.
The rest of the world seems to work based on monetary reward why can't we.

David B

Robert said...

If it's not yet part of your job the pay it should be proper. It takes several hours to seriously review a paper and many more to write one (even a popular one). How much do you value an hour of your time?

Robert said...

And of course, when I argue for zero pay it goes without mention that the commercial journals charge absolutely hilarious prices for their subscriptions. Totally unjustified. People don't send papers to high price journals!!!

Anonymous said...

But anonymous rejection leaves open the way for abusively negative referees, and will further discourage unusual new ideas since referees will be afraid to let through anything that they don't already entirely understand.

The better idea is to publish, of make available, the review text itself, both in case of rejection and acceptance.

About those journal fees: There is a campaign to refuse to review papers unpaid if the publisher works for profit. Now Physical Review is non-profit, being run by APS, but all Elsevier journals are obscenely profituous. I think this is reasonable...

Anonymous said...

I would say that in the best of the worlds, peer review should not be a hassle. If the paper is worth and the reviewer works in the field, critically reading the paper IS part of the reviewer's academic work, either he is actually reviewing the paper or not. This of course depends on the skill of the editor to able to match the paper with an interested reviewer. On the other hand, the writing of the reviewer's report could be considered a proper extra job and should be payed accordingly. $100 an hour for doing this sounds reasonable.

Anonymous said...

... there is a lot of very low quality stuff that gets printed, the Bogdanovs' papers...

Didn't Motl write "L'équation Bogdanov" promoting his thoughts about the Bogdanovs' work? I think this book stands as a towering emblem to Motl's buffoonery. I still wonder why people take this guy, now only surviving for the sake of his blog, seriously at all. Sometimes I think publishing houses should be working with scientists to prevent such nonsense to be published, and feed the layman with confusion concerning fundamental theoretical physics.