What to cite?

This ques­tion comes from a com­ment on another post:

I’ve seen authors cit­ing as many ref­er­ences as pos­si­ble to try to please poten­tial ref­er­ees. Many of those ref­er­ences are low qual­ity papers though. Any gen­eral guid­ance about a typ­i­cal length for the ref­er­ence section?

It depends on the sub­ject and style of the paper. I’ve writ­ten a paper with over 900 cita­tions, but that was a review of time series fore­cast­ing over a 25 year period, and so it had to include a lot of references.

I’ve also writ­ten a paper with just four cita­tions. As it was a com­men­tary, it did not need a lot of con­tex­tual information.

Rather than pro­vide guid­ance on the length of the ref­er­ence sec­tion, I think it is bet­ter to fol­low some gen­eral prin­ci­ples of cita­tion in research. Con­tinue reading →

Nominations for best International Journal of Forecasting paper, 2012–2013

Every two years, the Inter­na­tional Jour­nal of Fore­cast­ing awards a prize for the best paper pub­lished in a two year period. It is now time to iden­tify the best paper pub­lished in the IJF dur­ing 2012 and 2013. There is always about 18 months delay after the pub­li­ca­tion period to allow time for reflec­tion, cita­tions, etc. The prize is US$1000 plus an engraved plaque. Con­tinue reading →

Honoring Herman Stekler

stekler_The first issue of the IJF for 2015 has just been pub­lished, and I’m delighted that it includes a spe­cial sec­tion hon­or­ing Her­man Stek­ler. It includes arti­cles cov­er­ing a range of his fore­cast­ing inter­ests, although not all of them (sports fore­cast­ing is miss­ing). Her­man him­self wrote a paper for it look­ing at “Forecasting—Yesterday, Today and Tomor­row”.

He is in a unique posi­tion to write such a paper as he has been doing fore­cast­ing research longer than any­one else on the planet — his first pub­lished paper on fore­cast­ing appeared in 1959. Her­man is now 82 years old, and is still very active in research. Only a cou­ple of months ago, he wrote to me with some new research ideas he had been think­ing about, ask­ing me for some feed­back. He is also an extra­or­di­nar­ily con­sci­en­tious and care­ful asso­ciate edi­tor of the IJF and a delight to work with. He is truly “a scholar and a gen­tle­man” and I am very happy that we can honor Her­man in this man­ner. Thanks to Tara Sin­clair, Prakash Loun­gani and Fred Joutz for putting this trib­ute together.

We also pub­lished an inter­view with Her­man in the IJF in 2010 which con­tains some infor­ma­tion about his early years, grad­u­ate edu­ca­tion and first aca­d­e­mic jobs.

IJF review papers

Review papers are extremely use­ful for new researchers such as PhD stu­dents, or when you want to learn about a new research field. The Inter­na­tional Jour­nal of Fore­cast­ing pro­duced a whole review issue in 2006, and it con­tains some of the most highly cited papers we have ever pub­lished. Now, begin­ning with the lat­est issue of the jour­nal, we have started pub­lish­ing occa­sional review arti­cles on selected areas of fore­cast­ing. The first two arti­cles are:

  1. Elec­tric­ity price fore­cast­ing: A review of the state-​​of-​​the-​​art with a look into the future by Rafał Weron.
  2. The chal­lenges of pre-​​launch fore­cast­ing of adop­tion time series for new durable prod­ucts by Paul Good­win, Sheik Meeran, and Karima Dyussekeneva.

Both tackle very impor­tant top­ics in fore­cast­ing. Weron’s paper con­tains a com­pre­hen­sive sur­vey of work on elec­tric­ity price fore­cast­ing, coher­ently bring­ing together a large body of diverse research — I think it is the longest paper I have ever approved at 50 pages. Good­win, Meeran and Dyussekeneva review research on new prod­uct fore­cast­ing, a prob­lem every com­pany that pro­duces goods or ser­vices has faced; when there are no his­tor­i­cal data avail­able, how do you fore­cast the sales of your product?

We have a few other review papers in progress, so keep an eye out for them in future issues.


Probabilistic forecasting by Gneiting and Katzfuss (2014)

The IJF is intro­duc­ing occa­sional review papers on areas of fore­cast­ing. We did a whole issue in 2006 review­ing 25 years of research since the Inter­na­tional Insti­tute of Fore­cast­ers was estab­lished. Since then, there has been a lot of new work in appli­ca­tion areas such as call cen­ter fore­cast­ing and elec­tric­ity price fore­cast­ing. In addi­tion, there are areas we did not cover in 2006 includ­ing new prod­uct fore­cast­ing and fore­cast­ing in finance. There have also been method­olog­i­cal and the­o­ret­i­cal devel­op­ments over the last eight years. Con­se­quently, I’ve started invit­ing emi­nent researchers to write sur­vey papers for the journal.

One obvi­ous choice was Tilmann Gneit­ing, who has pro­duced a large body of excel­lent work on prob­a­bilis­tic fore­cast­ing in the last few years. The the­ory of fore­cast­ing was badly in need of devel­op­ment, and Tilmann and his coau­thors have made sev­eral great con­tri­bu­tions in this area. How­ever, when I asked him to write a review he explained that another jour­nal had got in before me, and that the review was already writ­ten. It appeared in the very first vol­ume of the new jour­nal Annual Review of Sta­tis­tics and its Appli­ca­tion: Gneit­ing and Katz­fuss (2014) Prob­a­bilis­tic Fore­cast­ing, pp.125–151.

Hav­ing now read it, I’m both grate­ful for this more acces­si­ble intro­duc­tion to the area, and dis­ap­pointed that it didn’t end up in the Inter­na­tional Jour­nal of Fore­cast­ing. I fore­cast that it will be highly cited (although I won’t cal­cu­late a fore­cast dis­tri­b­u­tion or com­pute a scor­ing func­tion for that).

Also, good luck to the new jour­nal; it looks like it will be very use­ful, and is sure to have a high impact fac­tor given it pub­lishes review articles.

How to get your paper rejected quickly

I sent this rejec­tion let­ter this morn­ing about a paper sub­mit­ted to the Inter­na­tional Jour­nal of Forecasting.


I am writ­ing to you regard­ing man­u­script ????? enti­tled “xxxxxxxxxxxx” which you sub­mit­ted to the Inter­na­tional Jour­nal of Fore­cast­ing.

It so hap­pens that I am aware that this paper was pre­vi­ously reviewed for the YYYYYYY jour­nal. It seems that you have not both­ered to make any of the changes rec­om­mended by the review­ers of your sub­mis­sion to YYYYYYY. Just sub­mit­ting the same paper to another jour­nal is extremely poor prac­tice, and I am dis­ap­pointed that you have taken this path. Review­ers spend a great deal of time pro­vid­ing com­ments, and it is dis­re­spect­ful to ignore them. I don’t expect you to do every­thing they say, but I would expect some of their com­ments to be helpful.

I am unwill­ing to con­sider the paper fur­ther for the Inter­na­tional Jour­nal of Fore­cast­ing. Read the pre­vi­ous reviews to know why. And before you sub­mit the paper to a new jour­nal, take the time to con­sider the reviews you have already been given.


Rob Hyn­d­man
(Editor-​​in-​​Chief, Inter­na­tional Jour­nal of Fore­cast­ing)

I have writ­ten on this issue before. The peer-​​review sys­tem requires peo­ple to donate con­sid­er­able amounts of time to writ­ing reviews. In gen­eral, they do a great job and pro­vide help­ful com­ments. So it really annoys me when authors treat the sys­tem as a game with the aim to get a paper accepted with min­i­mal work, and with no inter­est in learn­ing from feedback.