Data Science for Managers (short course)

I am teach­ing part of a short-​​course on Data Sci­ence for Man­agers from 10–12 Octo­ber in Melbourne.

Course Overview

The impact of Data Sci­ence on mod­ern busi­ness is sec­ond only to the intro­duc­tion of com­put­ers. And yet, for many busi­nesses the bar­rier of entry remains too high due to lack of knowhow, organ­i­sa­tional iner­tia, dif­fi­cul­ties in hir­ing the right man­power, an appar­ent need for upfront com­mit­ment, and more.

This course is designed to address these bar­ri­ers, giv­ing the nec­es­sary knowl­edge and skills to flesh out and man­age Data Sci­ence func­tions within your organ­i­sa­tion, tak­ing the anxiety-​​factor out of the Big Data rev­o­lu­tion and demon­strat­ing how data-​​driven decision-​​making can be inte­grated into one’s organ­i­sa­tion to har­ness exist­ing advan­tages and to cre­ate new opportunities.

Assum­ing min­i­mal prior knowl­edge, this course pro­vides com­plete cov­er­age of the key aspects, includ­ing data wran­gling, mod­el­ling and analy­sis, predictive-​​, descrip­tive– and prescriptive-​​analytics, data man­age­ment and cura­tion, stan­dards for data stor­age and analy­sis, the use of struc­tured, semi-​​structured and unstruc­tured data as well as of open pub­lic data, and the data-​​analytic value chain, all cov­ered at a fun­da­men­tal level.

More details avail­able at it​.monash​.edu/​d​a​t​a​-​s​c​ience.

Early-​​bird book­ings close in a few days.


The bias-variance decomposition

This week, I am teach­ing my Busi­ness Ana­lyt­ics class about the bias-​​variance trade-​​off. For some rea­son, the proof is not con­tained in either ESL or ISL, even though it is quite sim­ple. I also dis­cov­ered that the proof cur­rently pro­vided on Wikipedia makes lit­tle sense in places.

So I wrote my own for the class. It is longer than nec­es­sary to ensure there are no jumps that might con­fuse stu­dents.
Con­tinue reading →

Resources for the FPP book

The FPP resources page has recently been updated with sev­eral new addi­tions including

  • R code for all exam­ples in the book. This was already avail­able within each chap­ter, but the exam­ples have been col­lected into one file per chap­ter to save copy­ing and past­ing the var­i­ous code fragments.
  • Slides from a course on Pre­dic­tive Ana­lyt­ics from the Uni­ver­sity of Sydney.
  • Slides from a course on Eco­nomic Fore­cast­ing from the Uni­ver­sity of Hawaii.

If any one using the book has other mate­r­ial that could be made avail­able, please send them to me. For exam­ple, recorded lec­tures, slides, addi­tional exam­ples, assign­ments, exam ques­tions, solu­tions, etc.

Forecasting with R in WA

On 23–25 Sep­tem­ber, I will be run­ning a 3-​​day work­shop in Perth on “Fore­cast­ing: prin­ci­ples and prac­tice” mostly based on my book of the same name.

Work­shop par­tic­i­pants will be assumed to be famil­iar with basic sta­tis­ti­cal tools such as mul­ti­ple regres­sion, but no knowl­edge of time series or fore­cast­ing will be assumed. Some prior expe­ri­ence in R is highly desirable.

Venue: The Uni­ver­sity Club, Uni­ver­sity of West­ern Aus­tralia, Ned­lands WA.

Day 1:
Fore­cast­ing tools, sea­son­al­ity and trends, expo­nen­tial smoothing.
Day 2:
State space mod­els, sta­tion­ar­ity, trans­for­ma­tions, dif­fer­enc­ing, ARIMA models.
Day 3:
Time series cross-​​validation, dynamic regres­sion, hier­ar­chi­cal fore­cast­ing, non­lin­ear models.

The course will involve a mix­ture of lec­tures and prac­ti­cal ses­sions using R. Each par­tic­i­pant must bring their own lap­top with R installed, along with the fpp pack­age and its dependencies.

For costs and enrol­ment details, go to

Student forecasting awards from the IIF

At the IIF annual board meet­ing last month in Rot­ter­dam, I sug­gested that we pro­vide awards to the top stu­dents study­ing fore­cast­ing at uni­ver­sity level around the world, to the tune of $100 plus IIF mem­ber­ship for a year. I’m delighted that the idea met with enthu­si­asm, and that the awards are now avail­able. Even bet­ter, my sec­ond year fore­cast­ing sub­ject has been approved for an award.

The IIF have agreed to fund awards for 20 fore­cast­ing courses to start with. I believe they have already had sev­eral appli­ca­tions, so any other fore­cast­ing lec­tur­ers out there will need to be quick if they want to be part of it.

Creating a handout from beamer slides

I’m about to head off on a speak­ing tour to Europe (more on that in another post) and one of my hosts has asked for my pow­er­point slides so they can print them. They have made two false assump­tions: (1) that I use pow­er­point; (2) that my slides are sta­tic so they can be printed.

Instead, I pro­duced a cut-​​down ver­sion of my beamer slides, leav­ing out some of the ani­ma­tions and other fea­tures that will not print eas­ily. Then I pro­duced a pdf file with sev­eral slides per page. Con­tinue reading →

Interpreting noise

04_03_2_prevWhen watch­ing the TV news, or read­ing news­pa­per com­men­tary, I am fre­quently amazed at the attempts peo­ple make to inter­pret ran­dom noise.

For exam­ple, the lat­est tiny fluc­tu­a­tion in the share price of a major com­pany is attrib­uted to the CEO being ill. When the exchange rate goes up, the TV finance com­men­ta­tor con­fi­dently announces that it is a reac­tion to Chi­nese build­ing con­tracts. No one ever says “The unem­ploy­ment rate has dropped by 0.1% for no appar­ent reason.”

What is going on here is that the com­men­ta­tors are assum­ing we live in a noise-​​free world. They imag­ine that every­thing is explic­a­ble, you just have to find the expla­na­tion. How­ever, the world is noisy — real data are sub­ject to ran­dom fluc­tu­a­tions, and are often also mea­sured inac­cu­rately. So to inter­pret every lit­tle fluc­tu­a­tion is silly and mis­lead­ing. Con­tinue reading →

Fast computation of cross-validation in linear models

The leave-​​one-​​out cross-​​validation sta­tis­tic is given by

    \[\text{CV} = \frac{1}{N} \sum_{i=1}^N e_{[i]}^2,\]

where e_{[i]} = y_i - \hat{y}_{[i]}, ~y_1,\dots,y_N are the obser­va­tions, and \hat{y}_{[i]} is the pre­dicted value obtained when the model is esti­mated with the ith case deleted. This is also some­times known as the PRESS (Pre­dic­tion Resid­ual Sum of Squares) statistic.

It turns out that for lin­ear mod­els, we do not actu­ally have to esti­mate the model N times, once for each omit­ted case. Instead, CV can be com­puted after esti­mat­ing the model once on the com­plete data set. Con­tinue reading →

Highlighting the web

Users of my new online fore­cast­ing book have asked about hav­ing a facil­ity for per­sonal high­light­ing of selected sec­tions, as stu­dents often do with print books. We have plans to make this a built-​​in part of the plat­form, but for now it is pos­si­ble to do it using a sim­ple browser exten­sion. This approach allows any web­site to be high­lighted, so is even more use­ful than if we only had the facil­ity on OTexts​.org.

There are sev­eral pos­si­ble tools avail­able. One of the sim­plest tools that allows both high­light­ing and anno­ta­tions is Diigo. Con­tinue reading →

Interview for the Capital of Statistics

Earo Wang recently inter­viewed me for the Chi­nese web­site Cap­i­tal of Sta­tis­tics. The Eng­lish tran­script of the inter­vew is on Earo’s per­sonal web­site.

This is the third inter­view I’ve done in the last 18 months. The oth­ers were for: