Tourism forecasting competition ends

Hyndsight

And the winners are … Jeremy Howard and Lee C Baker. (See my earlier post for information about the competition.)

Jeremy describes his approach to seasonal time series in a blog post on Kaggle.com. Lee described his approach to annual time series in an earlier post.

A few lessons that come out of this:

  • For data from a single industry, using a global trend (i.e., estimated across all series) can be useful.

  • Combining forecasts is a good idea. (This lesson seems to be re-learned in every forecasting competition!)

  • The MASE can be very sensitive to a few series, and to optimize MASE it is worth concentrating on these. (This is actually not a good message for forecasting overall, as we want good forecasts for all series. Maybe we need to find a metric with similar properties to MASE but with a less skewed distribution.)

  • Outlier removal before forecasting can be effective. (This is an interesting result as outlier removal algorithms used in the M3 competition did not help forecast accuracy.)

Jeremy and Lee receive $500 for their efforts and they have decided to donate their prize money to the Fred Hollows Foundation. $500 will restore vision to 20 people. They will also write up their methods in more detail for the International Journal of Forecasting. I am hopeful that Philip Brierley of team Sali Mali (who did very well in the second stage of the competition) will also write a short explanation of his methods for the IJF.

Thanks to everyone who participated in the competition. Thanks also to Anthony Goldbloom from Kaggle for hosting the competition. Kaggle is a wonderful platform for prediction competitions and I hope it will be used for many more competitions of this type in the future.

comments powered by Disqus