Nominations for best International Journal of Forecasting paper, 2012–2013

Every two years, the Inter­na­tional Jour­nal of Fore­cast­ing awards a prize for the best paper pub­lished in a two year period. It is now time to iden­tify the best paper pub­lished in the IJF dur­ing 2012 and 2013. There is always about 18 months delay after the pub­li­ca­tion period to allow time for reflec­tion, cita­tions, etc. The prize is US$1000 plus an engraved plaque. Con­tinue reading →

Statistical modelling and analysis of big data

I’m cur­rently attend­ing the one day work­shop on this topic at QUT in Bris­bane. This morn­ing I spoke on “Visu­al­iz­ing and fore­cast­ing big time series data”. My slides are here.

The talks are being streamed.

OVERVIEW

Big data is now endemic in busi­ness, indus­try, gov­ern­ment, envi­ron­men­tal man­age­ment, med­ical sci­ence, social research and so on. One of the com­men­su­rate chal­lenges is how to effec­tively model and analyse these data.

This work­shop will bring together national and inter­na­tional experts in sta­tis­ti­cal mod­el­ling and analy­sis of big data, to share their expe­ri­ences, approaches and opin­ions about future direc­tions in this field.

RSS feeds for statistics and related journals

I’ve now res­ur­rected the col­lec­tion of research jour­nals that I fol­low, and set it up as a shared col­lec­tion in feedly. So any­one can eas­ily sub­scribe to all of the same jour­nals, or select a sub­set of them, to fol­low on feedly. Con­tinue reading →

New R package for electricity forecasting

Shu Fan and I have devel­oped a model for elec­tric­ity demand fore­cast­ing that is now widely used in Aus­tralia for long-​​term fore­cast­ing of peak elec­tric­ity demand. It has become known as the “Monash Elec­tric­ity Fore­cast­ing Model”. We have decided to release an R pack­age that imple­ments our model so that other peo­ple can eas­ily use it. The pack­age is called “MEFM” and is avail­able on github. We will prob­a­bly also put in on CRAN eventually.

The model was first described in  Hyn­d­man and Fan (2010). We are con­tin­u­ally improv­ing it, and the lat­est ver­sion is decribed in the model doc­u­men­ta­tion which will be updated from time to time.

The pack­age is being released under a GPL licence, so any­one can use it. All we ask is that our work is prop­erly cited.

Nat­u­rally, we are not able to pro­vide free tech­ni­cal sup­port, although we wel­come bug reports. We are avail­able to under­take paid con­sult­ing work in elec­tric­ity forecasting.

 

A time series classification contest

Amongst today’s email was one from some­one run­ning a pri­vate com­pe­ti­tion to clas­sify time series. Here are the essen­tial details.

The data are mea­sure­ments from a med­ical diag­nos­tic machine which takes 1 mea­sure­ment every sec­ond, and after 32–1000 sec­onds, the time series must be clas­si­fied into one of two classes. Some pre-​​classified train­ing data is pro­vided. It is not nec­es­sary to clas­sify all the test data, but you do need to have rel­a­tively high accu­racy on what is clas­si­fied. So you could find a sub­set of more eas­ily clas­si­fi­able test time series, and leave the rest of the test data unclas­si­fied. Con­tinue reading →

Honoring Herman Stekler

stekler_The first issue of the IJF for 2015 has just been pub­lished, and I’m delighted that it includes a spe­cial sec­tion hon­or­ing Her­man Stek­ler. It includes arti­cles cov­er­ing a range of his fore­cast­ing inter­ests, although not all of them (sports fore­cast­ing is miss­ing). Her­man him­self wrote a paper for it look­ing at “Forecasting—Yesterday, Today and Tomor­row”.

He is in a unique posi­tion to write such a paper as he has been doing fore­cast­ing research longer than any­one else on the planet — his first pub­lished paper on fore­cast­ing appeared in 1959. Her­man is now 82 years old, and is still very active in research. Only a cou­ple of months ago, he wrote to me with some new research ideas he had been think­ing about, ask­ing me for some feed­back. He is also an extra­or­di­nar­ily con­sci­en­tious and care­ful asso­ciate edi­tor of the IJF and a delight to work with. He is truly “a scholar and a gen­tle­man” and I am very happy that we can honor Her­man in this man­ner. Thanks to Tara Sin­clair, Prakash Loun­gani and Fred Joutz for putting this trib­ute together.

We also pub­lished an inter­view with Her­man in the IJF in 2010 which con­tains some infor­ma­tion about his early years, grad­u­ate edu­ca­tion and first aca­d­e­mic jobs.

Prediction competitions

Com­pe­ti­tions have a long his­tory in fore­cast­ing and pre­dic­tion, and have been instru­men­tal in forc­ing research atten­tion on meth­ods that work well in prac­tice. In the fore­cast­ing com­mu­nity, the M com­pe­ti­tion and M3 com­pe­ti­tion have been par­tic­u­larly influ­en­tial. The data min­ing com­mu­nity have the annual KDD cup which has gen­er­ated atten­tion on a wide range of pre­dic­tion prob­lems and asso­ci­ated meth­ods. Recent KDD cups are hosted on kag­gle.

In my research group meet­ing today, we dis­cussed our (lim­ited) expe­ri­ences in com­pet­ing in some Kag­gle com­pe­ti­tions, and we reviewed the fol­low­ing two papers which describe two pre­dic­tion competitions:

  1. Athana­sopou­los and Hyn­d­man (IJF 2011). The value of feed­back in fore­cast­ing com­pe­ti­tions. [preprint ver­sion]
  2. Roy et al (2013). The Microsoft Aca­d­e­mic Search Dataset and KDD Cup 2013.

Con­tinue reading →