Stanford seminar

I gave a sem­i­nar at Stan­ford today. Slides are below. It was def­i­nitely the most intim­i­dat­ing audi­ence I’ve faced, with Jerome Fried­man, Trevor Hastie, Brad Efron, Persi Dia­co­nis, Susan Holmes, David Donoho and John Cham­bers all present (and prob­a­bly other famous names I’ve missed).

I’ll be giv­ing essen­tially the same talk at UC Davis on Thurs­day. Con­tinue reading →

Reproducibility in computational research

Jane Fra­zier spoke at our research team meet­ing today on “Repro­ducibil­ity in com­pu­ta­tional research”. We had a very stim­u­lat­ing and lively dis­cus­sion about the issues involved. One inter­est­ing idea was that repro­ducibil­ity is on a scale, and we can all aim to move fur­ther along the scale towards mak­ing our own research more repro­ducible. For example

  • Can you repro­duce your results tomor­row on the same com­puter with the same soft­ware installed?
  • Could some­one else on a dif­fer­ent com­puter repro­duce your results with the same soft­ware installed?
  • Could you repro­duce your results in 3 years time after some of your soft­ware envi­ron­ment may have changed?
  • etc.

Think about what changes you need to make to move one step fur­ther along the repro­ducibil­ity con­tin­uüm, and do it.

Jane’s slides and hand­out are below. Con­tinue reading →

Upcoming talks in California

I’m back in Cal­i­for­nia for the next cou­ple of weeks, and will give the fol­low­ing talk at Stan­ford and UC-​​Davis.

Optimal forecast reconciliation for big time series data

Time series can often be nat­u­rally dis­ag­gre­gated in a hier­ar­chi­cal or grouped struc­ture. For exam­ple, a man­u­fac­tur­ing com­pany can dis­ag­gre­gate total demand for their prod­ucts by coun­try of sale, retail out­let, prod­uct type, pack­age size, and so on. As a result, there can be mil­lions of indi­vid­ual time series to fore­cast at the most dis­ag­gre­gated level, plus addi­tional series to fore­cast at higher lev­els of aggregation.

A com­mon con­straint is that the dis­ag­gre­gated fore­casts need to add up to the fore­casts of the aggre­gated data. This is known as fore­cast rec­on­cil­i­a­tion. I will show that the opti­mal rec­on­cil­i­a­tion method involves fit­ting an ill-​​conditioned lin­ear regres­sion model where the design matrix has one col­umn for each of the series at the most dis­ag­gre­gated level. For prob­lems involv­ing huge num­bers of series, the model is impos­si­ble to esti­mate using stan­dard regres­sion algo­rithms. I will also dis­cuss some fast algo­rithms for imple­ment­ing this model that make it prac­ti­ca­ble for imple­ment­ing in busi­ness contexts.

Stan­ford: 4.30pm, Tues­day 6th Octo­ber.
UCDavis: 4:10pm, Thurs­day 8th October.

International Symposium on Forecasting: Spain 2016

June 19–22, 2016
San­tander, Spain – Palace of La Magdalena

The Inter­na­tional Sym­po­sium on Fore­cast­ing (ISF) is the pre­mier fore­cast­ing con­fer­ence, attract­ing the world’s lead­ing fore­cast­ing researchers, prac­ti­tion­ers, and stu­dents. Through a com­bi­na­tion of keynote speaker pre­sen­ta­tions, aca­d­e­mic ses­sions, work­shops, and social pro­grams, the ISF pro­vides many excel­lent oppor­tu­ni­ties for net­work­ing, learn­ing, and fun.


Greg Allenby, The Ohio State Uni­ver­sity, USA
Todd Clark, Fed­eral Reserve Bank of Cleve­land, USA
José Duato, Poly­tech­nic Uni­ver­sity of Valen­cia, Spain
Robert Fildes, Lan­caster Uni­ver­sity, United King­dom
Edward LeamerUCLA Ander­son, USA
Hen­rik Mad­sen, Tech­ni­cal Uni­ver­sity of Den­mark
Adrian Raftery, Uni­ver­sity of Wash­ing­ton, USA

Important Dates

Invited Ses­sion Pro­pos­als: Jan­u­ary 31 2016
Abstract Sub­mis­sions: March 16 2016
Early Reg­is­tra­tion Ends: May 15 2016

More infor­ma­tion at www​.fore​cast​ers​.org/isf



This is a very dif­fer­ent book from my usual areas of fore­cast­ing and sta­tis­tics. It is a per­sonal mem­oir describ­ing my jour­ney of decon­ver­sion from Christianity.

Until a few years ago, I was reg­u­larly speak­ing at church con­fer­ences inter­na­tion­ally, and my books are still used in Bible classes and Sun­day Schools around the world. I even helped estab­lish an inno­v­a­tive new church, which became a model for sim­i­lar churches in other coun­tries. Even­tu­ally I came to the view that I was mis­taken, and that there was lit­tle or no evi­dence that the Bible was inspired or that God exists. In this book, I reflect on how I was fooled, and why I changed my mind.

Buy a print copy via Cre­ate­Space
Buy a print copy via Ama­zon
Buy an e-​​copy via Google books

Advice to other journal editors

I get asked to review jour­nal papers almost every day, and I have to say no to almost all of them. I know it is hard to find review­ers, but many of these requests indi­cate very lazy edi­tors. So to all the edi­tors out there look­ing for review­ers, here is some advice.

  1. Never ask some­one who is an edi­tor for another jour­nal. I am han­dling about 500 sub­mis­sions per year for the Inter­na­tional Jour­nal of Fore­cast­ing, and about 10 per year for the Jour­nal of Sta­tis­ti­cal Soft­ware. There is very lit­tle time left to review for other jour­nals. You are much bet­ter off iden­ti­fy­ing some­one early in their career, within 10 years of fin­ish­ing their PhD. They have more time, fewer requests, and are often look­ing to build an aca­d­e­mic reputation.
  2. Look at the key papers cited in the sub­mis­sion, espe­cially the recent ones, and then check the web sites of their authors. Find some­one who is cur­rently work­ing in the area. For multi-​​authored papers, fig­ure out which author was the PhD stu­dent, who was the pro­fes­sor, etc. If there was a post-​​doc involved, ask him/​her.
  3. If that fails, do a Google Scholar search for an author who has writ­ten on the same topic recently. That is, in the last 2–3 years, not 10 years ago.
  4. If pos­si­ble, ask some­one who has recently authored a paper in your jour­nal. They owe you one.
  5. Ask some­one you know rather than a stranger. They are much more likely to say yes. If you don’t know many peo­ple you shouldn’t be an editor.

Mathematical annotations on R plots

I’ve always strug­gled with using plotmath via the expression func­tion in R for adding math­e­mat­i­cal nota­tion to axes or leg­ends. For some rea­son, the most obvi­ous way to write some­thing never seems to work for me and I end up using trial and error in a loop with far too many iterations.

So I am very happy to see the new latex2exp pack­age avail­able which trans­lates LaTeX expres­sions into a form suit­able for R graphs. This is going to save me time and frus­tra­tion! Con­tinue reading →

Data Science for Managers (short course)

I am teach­ing part of a short-​​course on Data Sci­ence for Man­agers from 10–12 Octo­ber in Melbourne.

Course Overview

The impact of Data Sci­ence on mod­ern busi­ness is sec­ond only to the intro­duc­tion of com­put­ers. And yet, for many busi­nesses the bar­rier of entry remains too high due to lack of knowhow, organ­i­sa­tional iner­tia, dif­fi­cul­ties in hir­ing the right man­power, an appar­ent need for upfront com­mit­ment, and more.

This course is designed to address these bar­ri­ers, giv­ing the nec­es­sary knowl­edge and skills to flesh out and man­age Data Sci­ence func­tions within your organ­i­sa­tion, tak­ing the anxiety-​​factor out of the Big Data rev­o­lu­tion and demon­strat­ing how data-​​driven decision-​​making can be inte­grated into one’s organ­i­sa­tion to har­ness exist­ing advan­tages and to cre­ate new opportunities.

Assum­ing min­i­mal prior knowl­edge, this course pro­vides com­plete cov­er­age of the key aspects, includ­ing data wran­gling, mod­el­ling and analy­sis, predictive-​​, descrip­tive– and prescriptive-​​analytics, data man­age­ment and cura­tion, stan­dards for data stor­age and analy­sis, the use of struc­tured, semi-​​structured and unstruc­tured data as well as of open pub­lic data, and the data-​​analytic value chain, all cov­ered at a fun­da­men­tal level.

More details avail­able at it​.monash​.edu/​d​a​t​a​-​s​c​ience.

Early-​​bird book­ings close in a few days.