Generating quantile forecasts in R

From today’s email:

I have just fin­ished read­ing a copy of ‘Forecasting:Principles and Prac­tice’ and I have found the book really inter­est­ing. I have par­tic­u­larly enjoyed the case stud­ies and focus on prac­ti­cal applications.

After fin­ish­ing the book I have joined a fore­cast­ing com­pe­ti­tion to put what I’ve learnt to the test. I do have a cou­ple of queries about the fore­cast­ing out­puts required. The out­put required is a quan­tile fore­cast, is this the same as pre­dic­tion inter­vals? Is there any R func­tion to pro­duce quan­tiles from 0 to 99?

If you were able to point me in the right direc­tion regard­ing the above it would be greatly appreciated.

Many Thanks,

Con­tinue reading →

Resources for the FPP book

The FPP resources page has recently been updated with sev­eral new addi­tions including

  • R code for all exam­ples in the book. This was already avail­able within each chap­ter, but the exam­ples have been col­lected into one file per chap­ter to save copy­ing and past­ing the var­i­ous code fragments.
  • Slides from a course on Pre­dic­tive Ana­lyt­ics from the Uni­ver­sity of Sydney.
  • Slides from a course on Eco­nomic Fore­cast­ing from the Uni­ver­sity of Hawaii.

If any one using the book has other mate­r­ial that could be made avail­able, please send them to me. For exam­ple, recorded lec­tures, slides, addi­tional exam­ples, assign­ments, exam ques­tions, solu­tions, etc.

A new candidate for worst figure

Today I read a paper that had been sub­mit­ted to the IJF which included the fol­low­ing figure


along with sev­eral sim­i­lar plots. (Click for a larger ver­sion.) I haven’t seen any­thing this bad for a long time. In fact, I think I would find it very dif­fi­cult to repro­duce using R, or even Excel (which is par­tic­u­larly adept at bad graphics).

A few years ago I pro­duced “Twenty rules for good graph­ics”. I think I need to add a cou­ple of addi­tional rules:

  • Rep­re­sent time changes using lines.
  • Never use fill pat­terns such as cross-​​hatching.

(My orig­i­nal rule #20 said Avoid pie charts.)

It would have been rel­a­tively sim­ple to show these data as six lines on a plot of GDP against time. That would have made it obvi­ous that the Euro­pean GDP was shrink­ing, the GDP of Asia/​Oceania was increas­ing, while other regions of the world were fairly sta­ble. At least I think that is what is hap­pen­ing, but it is very hard to tell from such graph­i­cal obfuscation.

Forecasting with R in WA

On 23–25 Sep­tem­ber, I will be run­ning a 3-​​day work­shop in Perth on “Fore­cast­ing: prin­ci­ples and prac­tice” mostly based on my book of the same name.

Work­shop par­tic­i­pants will be assumed to be famil­iar with basic sta­tis­ti­cal tools such as mul­ti­ple regres­sion, but no knowl­edge of time series or fore­cast­ing will be assumed. Some prior expe­ri­ence in R is highly desirable.

Venue: The Uni­ver­sity Club, Uni­ver­sity of West­ern Aus­tralia, Ned­lands WA.

Day 1:
Fore­cast­ing tools, sea­son­al­ity and trends, expo­nen­tial smoothing.
Day 2:
State space mod­els, sta­tion­ar­ity, trans­for­ma­tions, dif­fer­enc­ing, ARIMA models.
Day 3:
Time series cross-​​validation, dynamic regres­sion, hier­ar­chi­cal fore­cast­ing, non­lin­ear models.

The course will involve a mix­ture of lec­tures and prac­ti­cal ses­sions using R. Each par­tic­i­pant must bring their own lap­top with R installed, along with the fpp pack­age and its dependencies.

For costs and enrol­ment details, go to

GEFCom 2014 energy forecasting competition is underway

GEF­Com 2014 is the most advanced energy fore­cast­ing com­pe­ti­tion ever orga­nized, both in terms of the data involved, and in terms of the way the fore­casts will be evaluated.

So every­one inter­ested in energy fore­cast­ing should head over to the com­pe­ti­tion web­page and start fore­cast­ing: www​.gef​com​.org.

This time, the com­pe­ti­tion is hosted on Crow­d­AN­A­LYTIX rather than Kag­gle.

High­lights of GEFCom2014:

  • An upgraded edi­tion from GEFCom2012
  • Four tracks: elec­tric load, elec­tric­ity price, wind power and solar power forecasting.
  • Prob­a­bilis­tic fore­cast­ing: con­tes­tants are required to sub­mit 99 quan­tiles for each step through­out the fore­cast horizon.
  • Rolling fore­cast­ing: incre­men­tal data sets are being released on weekly basis to fore­cast the next period of interest.
  • Prizes for win­ning teams and insti­tu­tions: up to 3 teams from each track will be rec­og­nized as the win­ning team; top insti­tu­tions with mul­ti­ple well-​​performing teams will be rec­og­nized as the win­ning institutions.
  • Global par­tic­i­pa­tion: 200+ peo­ple from 40+ coun­tries have already signed up the GEFCom2014 inter­est list.

Tao Hong (the main orga­nizer) has a few tips on his blog that you should read before starting.


Visit of Di Cook

Next week, Pro­fes­sor Di Cook from Iowa State Uni­ver­sity is vis­it­ing my research group at Monash Uni­ver­sity. Di is a world leader in data visu­al­iza­tion, and is espe­cially well-​​known for her work on inter­ac­tive graph­ics and the XGobi and GGobi soft­ware. See her book with Deb Swayne for details.

For those want­ing to hear her speak, read on. Con­tinue reading →

Coherent population forecasting using R

This is an exam­ple of how to use the demog­ra­phy pack­age in R for sto­chas­tic pop­u­la­tion fore­cast­ing with coher­ent com­po­nents. It is based on the papers by Hyn­d­man and Booth (IJF 2008) and Hyn­d­man, Booth and Yas­meen (Demog­ra­phy 2013). I will use Aus­tralian data from 1950 to 2009 and fore­cast the next 50 years.

In demog­ra­phy, “coher­ent” fore­casts are where male and females (or other sub-​​groups) do not diverge over time. (Essen­tially, we require the dif­fer­ence between the groups to be sta­tion­ary.) When we wrote the 2008 paper, we did not know how to con­strain the fore­casts to be coher­ent in a func­tional data con­text and so this was not dis­cussed. My later 2013 paper pro­vided a way of impos­ing coher­ence. This blog post shows how to imple­ment both ideas using R. Con­tinue reading →