Every week I reject some papers submitted to the International Journal of Forecasting, without sending the papers off to associate editors or reviewers. Here are five of the most common reasons for rejection. Continue reading →
I’ve been an editor of JSS for the last few years, and as a result I tend to get email from people asking me about publishing papers describing R packages in JSS. So for all those wondering, here are some general comments. Continue reading →
I sent this rejection letter this morning about a paper submitted to the International Journal of Forecasting.
I am writing to you regarding manuscript ????? entitled “xxxxxxxxxxxx” which you submitted to the International Journal of Forecasting.
It so happens that I am aware that this paper was previously reviewed for the YYYYYYY journal. It seems that you have not bothered to make any of the changes recommended by the reviewers of your submission to YYYYYYY. Just submitting the same paper to another journal is extremely poor practice, and I am disappointed that you have taken this path. Reviewers spend a great deal of time providing comments, and it is disrespectful to ignore them. I don’t expect you to do everything they say, but I would expect some of their comments to be helpful.
I am unwilling to consider the paper further for the International Journal of Forecasting. Read the previous reviews to know why. And before you submit the paper to a new journal, take the time to consider the reviews you have already been given.
(Editor-in-Chief, International Journal of Forecasting)
I have written on this issue before. The peer-review system requires people to donate considerable amounts of time to writing reviews. In general, they do a great job and provide helpful comments. So it really annoys me when authors treat the system as a game with the aim to get a paper accepted with minimal work, and with no interest in learning from feedback.
The nature of research is that other people are probably working on similar ideas to you, and it is possible that someone will beat you to publishing them. Continue reading →
I’ve just finished another reviewer report for a journal, and yet again I’ve had to make comments about reading the literature. It’s not difficult. Before you write a paper, read what other people have done. A simple search on Google scholar will usually do the trick. And before you submit a paper, check again that you haven’t missed anything important.
The paper I reviewed today did not cite a single reference from either of the two most active research groups in the area in the last ten years. Any search on the topic would have turned up about a dozen papers from these two groups alone.
I don’t mind if papers miss a reference or two, especially if they have been published in an obscure outlet. But I will recommend a straight reject if a paper hasn’t cited any of the most important papers from the last five years. Part of a researcher’s task is to engage with what has already been done, and show how any new ideas differ from or extend on previous work.
- Refereeing a journal article part 1: reading
- Refereeing a journal article part 2: making a recommendation
- Refereeing a journal article part 3: writing the feedback
If every reviewer followed her advice, my life as an editor would be much easier, and the quality of research would be improved.
I regularly get emails from people wanting to be referees for the International Journal of Forecasting, usually with an accompanying CV. This is not how the process works.
Referees are almost always selected because they have previously written papers on a similar topic to the manuscript under review. If you want to be a referee, then write good papers and get them published in scholarly journals. Very quickly you will be invited to referee papers in the same journals. But until you have demonstrated your own research skills, no editor is going to trust you to assess the research of someone else.
It’s not a good idea to annoy the referees of your paper. They make recommendations to the editor about your work and it is best to keep them happy. There is an interesting discussion on stats.stackexchange.com on this subject. This inspired my own list below.
- Explain what you’ve done clearly, avoiding unnecessary jargon.
- Don’t claim your paper contributes more than it actually does. (I refereed a paper this week where the author claimed to have invented principal component analysis!)
- Ensure all figures have clear captions and labels.
- Include citations to the referee’s own work. Obviously you don’t know who is going to referee your paper, but you should aim to cite the main work in the area. It places your work in context, and keeps the referees happy if they are the authors.
- Make sure the cited papers say what you think they say. Sight what you cite!
- Include proper citations for all software packages. If you are unsure how to cite an R package, try the command
- Never plagiarise from other papers — not even sentence fragments. Use your own words. I’ve refereed a thesis which had slabs taken from my own lecture notes including the typos.
- Don’t plagiarise from your own papers. Either reference your earlier work, or provide a summary in new words.
- Provide enough detail so your work can be replicated. Where possible, provide the data and code. Make sure the code works.
- When responding to referee reports, make sure you answer everything asked of you. (See my earlier post “Always listen to reviewers”)
- If you’ve revised the paper based on referees’ comments, then thank them in the acknowledgements section.
For some applied papers, there are specific statistical issues that need attention:
- Give effect sizes with confidence intervals, not just p-values.
- Don’t describe data using the mean and standard deviation without indicating whether the data were more-or-less symmetric and unimodal.
- Don’t split continuous data into groups.
- Make sure your data satisfy the assumptions of the statistical methods used.
More tongue-in-cheek advice is provided by Stratton and Neil (2005), “How to ensure your paper is rejected by the statistical reviewer”. Diabetic Medicine, 22(4), 371–373.
Feel free to add your own suggestions over at stats.stackexchange.com.
This week I was asked to review a paper that I had seen before. It had been submitted to a journal a few months ago and I had written a detailed report describing some problems with the paper, and noting a large number of typos that needed fixing. That journal had rejected the paper, the authors had submitted it to a second journal, and the paper ended up on my desk again for review. I was interested to see what the authors had done about the problems I had described. Alas, nothing had changed. Not even the typos. It was identical to the previous version with every error still there. So I sent the same report off to the second journal advising the editor of the situation.
I’m not sure what the authors imagined would happen. It is not uncommon for a paper to be sent to the same reviewer after it has been rejected by one journal, especially when the field of potential reviewers for some topics is quite small. This paper was on an extension to the automatic time series modelling procedures provided in the forecast package for R. Since I am the author of the forecast package for R, the probability of me being asked to review the paper is approximately one.
In general, always listen to reviewers, even if the paper has been rejected by the journal that sent you the reviews. While you are not obliged to respond to everything in a review when you are not sending a revision back to the same journal, you should think about what has been said and revise the paper accordingly. At least fix the typos! If the same reviewers see it again, they won’t be happy if you’ve ignored them. Also, they may just have some useful comments that will lead to improvements in your paper.
Being on the editorial board of a journal is a lot of work. I’m currently Editor-in-Chief of the International Journal of Forecasting and previously I’ve been Theory & Methods Editor of the Australian & New Zealand Journal of Statistics. Although it is time-consuming and often goes un-noticed, there are some important rewards that make it worthwhile in my opinion.
- You are forced to read carefully a lot of papers in your area of interest. Everyone intends reading the papers published in their area, but this activity often gets neglected for more urgent activities. When you are an editor or associate editor, you have to read the papers, and you have to read them thoroughly. That way you are forced to keep up-to-date with new ideas.
- You become better known in your field. This tends to lead to invitations to speak at conferences, write invited papers, etc.
- You get to shape the discipline to some extent, at least once you are a managing editor. For example, you can arrange a special issue or a review paper on a topic that you think needs addressing.
- It brings with it some prestige.
- You get to see a lot of the latest research before everyone else, although these days, that is becoming less of an issue as most people put their papers online in repositories such as RePEc or arXiv.
I sometimes get emails from people asking to join the editorial board of the IJF (usually accompanied by a resumé which shows the person has little experience in forecasting research). One such email prompted this post today. I will never invite someone to join the IJF editorial board in this way. Please don’t ask. Instead, if you want to join an editorial board, there is no substitute for the following approach:
- write good papers that are published in the relevant journal (if you’ve never published in the IJF you are unlikely to be considered for the editorial board);
- write good papers that are published in other journals on the relevant topic (e.g., publish good forecasting papers in JASA, JRSSB, JF, etc.);
- regularly attend any conferences associated with the journal (that’s the ISF);
- provide high quality referee reports for papers submitted to the journal (almost all our associate editors were people who did a sterling job as referees first);
- work in an area where we need additional associate editors (every year we consider what areas we need new associate editors, and then we find the appropriate people).
Being on an editorial board gives you a position of leadership in your research community. But you don’t get to be a leader until you’ve proven yourself by writing papers, attending conferences, writing referee reports, etc.