Archive for the ‘Working papers’ Category:

Low-dimensional decomposition, smoothing and forecasting of sparse functional data

Published on 5 June 2014 in Working papers

Alexander Dokumentov and Rob J Hyndman

Abstract: We propose a new generic method ROPES (Regularized Optimization for Prediction and Estimation with Sparse data) for decomposing, smoothing and forecasting two-dimensional sparse data. In some ways, ROPES is similar to Ridge Regression, the LASSO, Principal Component Analysis (PCA) and Maximum-Margin Matrix Factorisation (MMMF). Using this new approach, we propose a practical method of forecasting mortality rates, as well as a new method for interpolating and extrapolating sparse longitudinal data. We also show how to calculate prediction intervals for the resulting estimates.


Fast computation of reconciled forecasts for hierarchical and grouped time series

Published on 5 June 2014 in Working papers

Rob J Hyndman, Alan Lee & Earo Wang

Abstract: We describe some fast algorithms for reconciling large collections of time series forecasts with aggregation constraints. The constraints arise due to the need for forecasts of collections of time series with hierarchical or grouped structures to add up in the same manner as the observed time series. We show that the least squares approach to reconciling hierarchical forecasts can be extended to more general non-hierarchical groups of time series, and that the computations can be handled efficiently by exploiting the structure of the associated design matrix. Our algorithms will reconcile hierarchical forecasts with hierarchies of unlimited size, making forecast reconciliation feasible in business applications involving very large numbers of time series.


Reconciling forecasts for hierarchical and grouped time series

Published on 24 May 2014 in Working papers

Rob J Hyndman and George Athanasopoulos This is an introduction to our approach to forecast reconciliation without using any matrices. The original research is available here: Hyndman, Ahmed, Athanasopoulos and Shang (CSDA, 2011) Athanasopoulos, Ahmed and Hyndman (IJF, 2009) The software is available in the hts package for R with some notes on usage in the vignette. There is also a gentle introduction in my forecasting textbook.


Monash Electricity Forecasting Model

Published on 22 May 2014 in Working papers

Rob J Hyndman and Shu Fan

The model we developed for peak electricity demand forecasting in Hyndman and Fan (2010) is now widely used in practice around Australia, and has undergone many improvements and developments. This document describes the current version of the model. It will be updated from time to time as the model continues to be modified and improved.


“Facts” may still be artefacts, since models can make unrealistic assumptions: statistical methods for the estimation of invasion lag-phases from herbarium data

Published on 2 May 2014 in Working papers

Rob J. Hyndman, Mohsen B. Mesgaran and Roger D. Cousens

We present a new method for estimating the length of the invasion lag phase from simple time series of counts of herbarium records. This is based on annual rather than cumulative data, a generalized linear model incorporating a log link for overall collection effort, and piecewise linear splines. We demonstrate the method on two species representing good and poor data quality, then apply it to two data sets comprising 448 species/region combinations. Significant lags were detected in only 28% and 40% of time series, a much lower level than the 95% and 77% found in previous analyses of the same data. In a case with high quality data, a lag was concluded even though during the “lag” the locations of herbarium collections indicated that it was spreading rapidly at a continental scale. In species with few records, results were sensitive to the way in which zeroes were included. Given the poor representation of herbarium samples in the early stages of invasions and the fact that they do not constitute a structured survey of abundance, we warn against over-reliance on statistical analysis of such data to reach conclusions about the dynamics of invasions.