I get asked to review journal papers almost every day, and I have to say no to almost all of them. I know it is hard to find reviewers, but many of these requests indicate very lazy editors. So to all the editors out there looking for reviewers, here is some advice.
- Never ask someone who is an editor for another journal. I am handling about 500 submissions per year for the International Journal of Forecasting, and about 10 per year for the Journal of Statistical Software. There is very little time left to review for other journals. You are much better off identifying someone early in their career, within 10 years of finishing their PhD. They have more time, fewer requests, and are often looking to build an academic reputation.
- Look at the key papers cited in the submission, especially the recent ones, and then check the web sites of their authors. Find someone who is currently working in the area. For multi-authored papers, figure out which author was the PhD student, who was the professor, etc. If there was a post-doc involved, ask him/her.
- If that fails, do a Google Scholar search for an author who has written on the same topic recently. That is, in the last 2-3 years, not 10 years ago.
- If possible, ask someone who has recently authored a paper in your journal. They owe you one.
- Ask someone you know rather than a stranger. They are much more likely to say yes. If you don’t know many people you shouldn’t be an editor.
Every week I reject some papers submitted to the International Journal of Forecasting, without sending the papers off to associate editors or reviewers. Here are five of the most common reasons for rejection. Continue reading →
I’ve been an editor of JSS for the last few years, and as a result I tend to get email from people asking me about publishing papers describing R packages in JSS. So for all those wondering, here are some general comments. Continue reading →
I sent this rejection letter this morning about a paper submitted to the International Journal of Forecasting.
I am writing to you regarding manuscript ????? entitled “xxxxxxxxxxxx” which you submitted to the International Journal of Forecasting.
It so happens that I am aware that this paper was previously reviewed for the YYYYYYY journal. It seems that you have not bothered to make any of the changes recommended by the reviewers of your submission to YYYYYYY. Just submitting the same paper to another journal is extremely poor practice, and I am disappointed that you have taken this path. Reviewers spend a great deal of time providing comments, and it is disrespectful to ignore them. I don’t expect you to do everything they say, but I would expect some of their comments to be helpful.
I am unwilling to consider the paper further for the International Journal of Forecasting. Read the previous reviews to know why. And before you submit the paper to a new journal, take the time to consider the reviews you have already been given.
(Editor-in-Chief, International Journal of Forecasting)
I have written on this issue before. The peer-review system requires people to donate considerable amounts of time to writing reviews. In general, they do a great job and provide helpful comments. So it really annoys me when authors treat the system as a game with the aim to get a paper accepted with minimal work, and with no interest in learning from feedback.
The nature of research is that other people are probably working on similar ideas to you, and it is possible that someone will beat you to publishing them. Continue reading →
I’ve just finished another reviewer report for a journal, and yet again I’ve had to make comments about reading the literature. It’s not difficult. Before you write a paper, read what other people have done. A simple search on Google scholar will usually do the trick. And before you submit a paper, check again that you haven’t missed anything important.
The paper I reviewed today did not cite a single reference from either of the two most active research groups in the area in the last ten years. Any search on the topic would have turned up about a dozen papers from these two groups alone.
I don’t mind if papers miss a reference or two, especially if they have been published in an obscure outlet. But I will recommend a straight reject if a paper hasn’t cited any of the most important papers from the last five years. Part of a researcher’s task is to engage with what has already been done, and show how any new ideas differ from or extend on previous work.
I’ve written briefly on this before. For an excellent and more detailed discussion of what is involved, there is a series of excellent posts on Pat Thomson’s blog:
- Refereeing a journal article part 1: reading
- Refereeing a journal article part 2: making a recommendation
- Refereeing a journal article part 3: writing the feedback
If every reviewer followed her advice, my life as an editor would be much easier, and the quality of research would be improved.
I regularly get emails from people wanting to be referees for the International Journal of Forecasting, usually with an accompanying CV. This is not how the process works.
Referees are almost always selected because they have previously written papers on a similar topic to the manuscript under review. If you want to be a referee, then write good papers and get them published in scholarly journals. Very quickly you will be invited to referee papers in the same journals. But until you have demonstrated your own research skills, no editor is going to trust you to assess the research of someone else.
It’s not a good idea to annoy the referees of your paper. They make recommendations to the editor about your work and it is best to keep them happy. There is an interesting discussion on stats.stackexchange.com on this subject. This inspired my own list below.
- Explain what you’ve done clearly, avoiding unnecessary jargon.
- Don’t claim your paper contributes more than it actually does. (I refereed a paper this week where the author claimed to have invented principal component analysis!)
- Ensure all figures have clear captions and labels.
- Include citations to the referee’s own work. Obviously you don’t know who is going to referee your paper, but you should aim to cite the main work in the area. It places your work in context, and keeps the referees happy if they are the authors.
- Make sure the cited papers say what you think they say. Sight what you cite!
- Include proper citations for all software packages. If you are unsure how to cite an R package, try the command
- Never plagiarise from other papers — not even sentence fragments. Use your own words. I’ve refereed a thesis which had slabs taken from my own lecture notes including the typos.
- Don’t plagiarise from your own papers. Either reference your earlier work, or provide a summary in new words.
- Provide enough detail so your work can be replicated. Where possible, provide the data and code. Make sure the code works.
- When responding to referee reports, make sure you answer everything asked of you. (See my earlier post “Always listen to reviewers“)
- If you’ve revised the paper based on referees’ comments, then thank them in the acknowledgements section.
For some applied papers, there are specific statistical issues that need attention:
- Give effect sizes with confidence intervals, not just p-values.
- Don’t describe data using the mean and standard deviation without indicating whether the data were more-or-less symmetric and unimodal.
- Don’t split continuous data into groups.
- Make sure your data satisfy the assumptions of the statistical methods used.
More tongue-in-cheek advice is provided by Stratton and Neil (2005), “How to ensure your paper is rejected by the statistical reviewer”. Diabetic Medicine, 22(4), 371-373.
Feel free to add your own suggestions over at stats.stackexchange.com.
This week I was asked to review a paper that I had seen before. It had been submitted to a journal a few months ago and I had written a detailed report describing some problems with the paper, and noting a large number of typos that needed fixing. That journal had rejected the paper, the authors had submitted it to a second journal, and the paper ended up on my desk again for review. I was interested to see what the authors had done about the problems I had described. Alas, nothing had changed. Not even the typos. It was identical to the previous version with every error still there. So I sent the same report off to the second journal advising the editor of the situation.
I’m not sure what the authors imagined would happen. It is not uncommon for a paper to be sent to the same reviewer after it has been rejected by one journal, especially when the field of potential reviewers for some topics is quite small. This paper was on an extension to the automatic time series modelling procedures provided in the forecast package for R. Since I am the author of the forecast package for R, the probability of me being asked to review the paper is approximately one.
In general, always listen to reviewers, even if the paper has been rejected by the journal that sent you the reviews. While you are not obliged to respond to everything in a review when you are not sending a revision back to the same journal, you should think about what has been said and revise the paper accordingly. At least fix the typos! If the same reviewers see it again, they won’t be happy if you’ve ignored them. Also, they may just have some useful comments that will lead to improvements in your paper.