Monday 26 October 2026, 6:00 PM Eastern Time

Slides

View slides

Open slides in a new tab

NUMBAT seminar

Consistently formatted source is easier to read, review and diff, and linters alert you to problems before they become bugs. I will survey and demonstrate the current generation of fast, opinionated formatters and linters — mostly written in Rust, and usable from the command line or from within Positron (and in some cases from RStudio). I’ll focus on air for R and panache for Quarto and R Markdown. If time, I might also mention jarl for R linting, ruff for Python, tex-fmt and chktex for LaTeX, and vale for prose (checking manuscripts against a style guide). I’ll show how to use TOML configuration and format-on-save, so that your code and documents stay tidy with no ongoing effort.

Abstract

I will discuss a probabilistic approach to anomaly detection based on extreme ‘surprisal values’ aka log scores, equal to minus the log density at each observation. The surprisal approach can be used for any collection of data objects, provided a probability density can be defined on the sample space. It can distinguish anomalies from legitimate observations in a heavy tail, and will identify anomalies that are undetected using methods based on distance measures. I will demonstrate the idea in various real data examples including univariate, multivariate and regression contexts, and when exploring more complicated data objects. I will also briefly outline the underlying theory when the density is known, and when it is estimated using a kernel density estimate. In the latter case, an innovative bandwidth selection method is used based on persistent homology.

Abstract

I will demonstrate how to forecast the age structure of a workforce using demographic growth-balance equations that incorporate functional data components for graduate influx, international mobility, retirement, and mortality. This model adapts the stochastic population framework proposed by Hyndman and Booth (IJF, 2008), substituting the birth component at age 0 with a graduation process at any age, and integrating retirement dynamics alongside a standard mortality process. It accommodates variable graduate influx, international mobility, and aging, through functional data components.

The model is illustrated by forecasting the future age structure of scientific disciplines in Australia, utilising a variety of data sources. Data limitations are addressed through cohort interpolation and constrained penalised regression splines in order to obtain a workable data set. A combination of local and global ARIMA models is employed to address the temporal dynamics in the time series components.

By forecasting age structures in disciplines such as Physics and Astronomy, Mathematical Sciences, Chemical Sciences, Earth Sciences, and Biological Sciences, the model informs strategic workforce planning and policy development.

This forecasting framework not only aids in strategic planning and policy formulation within the scientific community, but also offers a blueprint for other professional sectors seeking to understand workforce dynamics amid demographic shifts.

Talk for WOMBAT 2024

Abstract

Univariate, multivariate, and hierarchical forecasts can all be improved using projections onto linear subspaces, regardless of what forecasting method is used. I will show some theoretical guarantees of this statement, and demonstrate using empirical applications how linear projections can lead to (sometimes dramatic) improvements in forecast accuracy.

The procedure involves creating new time series that are linear combinations of the observed time series. These are then forecast, along with the original series, and all forecasts are adjusted via a projection matrix to ensure they satisfy the linear constraints.

This procedure is now widely used for hierarchical forecasting. It has the potential to revolutionise multivariate forecasting as well.

NUMBAT seminar

I will introduce the vital package which allows analysis of demographic data using tidy tools. The package uses a variation of tsibble objects as the main data class, so all of the infrastructure available for tsibble and tibble objects can also be used with vital objects. Data may include births, deaths, mortality, fertility, population and migration data. Functions for plotting, smoothing, modelling and forecasting data are included. Models include the classical Lee-Carter model as well as functional data models. Future plans include replicating all of the models available in the demography and StMoMo packages. The package is currently available at pkg.robjhyndman.com/vital/ and on CRAN.

Seminar for the OPTIMA ARC training centre

Summary

People have been forecasting for thousands of years. They forecast whether it will rain tomorrow, how much wheat will be harvested, how long it will take for dinner to cook, how many widgets their company will sell next month, what the unemployment figure will be in a year’s time, or how much superannuation they will have when they retire.

Some things are relatively easy to forecast and some are unpredictable. Why is it that we can accurately forecast a solar eclipse in 1000 years, but we have no idea whether Google’s stock price will rise or fall tomorrow? How can we forecast the daily electricity consumption for the next week with remarkable precision, but we cannot forecast daily ambulance call-outs with the same accuracy?

In his presentation, Professor Rob J Hyndman will discuss the conditions we need for predictability, how to measure the uncertainty of our forecasts, and how to evaluate whether we are uncertain enough. Rob will draw on his 30 years of forecasting experience, including forecasting Australia’s health budget for the next few years, forecasting peak electricity demand in 20 years, and producing weekly forecasts of daily COVID-19 cases for all Australian states from 2020-2023.

He will also look at how forecasting has changed over the last 50 years, and what forecasting might look like in the future.

Play video

Download pdf

UseR!2024

14:30, 9 July

I will introduce the vital package which allows analysis of demographic data using tidy tools. The package uses a variation of tsibble objects as the main data class, so all of the infrastructure available for tsibble and tibble objects can also be used with vital objects. Data may include births, deaths, mortality, fertility, population and migration data. Functions for plotting, smoothing, modelling and forecasting data are included. Models include the classical Lee-Carter model as well as functional data models. Future plans include replicating all of the models available in the demography and StMoMo packages. The package is currently available at pkg.robjhyndman.com/vital/ and on CRAN.

International Symposium on Forecasting

When a forecast is very inaccurate, it is sometimes because a poor forecasting model is used, but it can also occur when an unusual observation occurs. I will discuss the latter situation, where a good forecasting model can be used to identify anomalies. The approach taken is to use a probabilistic forecast, and to compute the “density scores” equal to the negative log likelihood of the observations based on the forecast distributions. The density scores provide a measure of how anomalous each observation is, given the forecast density. A large density score indicates that the observation is unlikely, and so is a potential anomaly. On the other hand, typical values will have low density scores. A Generalized Pareto Distribution is fitted to the largest density scores to estimate the probability of each observation being an anomaly. Applications to pharmaceutical scripts and mortality data will be used to illustrate the ideas using the fable and weird R packages.

Suessmilch Lecture, MPIDR

3pm CEST, 26 June 2024

I will introduce the vital package which allows analysis of demographic data using tidy tools. The package uses a variation of tsibble objects as the main data class, so all the infrastructure available for tsibble and tibble objects can also be used with vital objects. Data may include births, deaths, mortality, fertility, population and migration data. Functions for plotting, smoothing, modelling and forecasting data are included. Models include the classical Lee-Carter model as well as functional data models. Future plans include replicating all the models available in the demography and StMoMo packages. The package is available at pkg.robjhyndman.com/vital/ and on CRAN.

Talk given as part of the Monash University FUTURES symposium

Statistical forecasting involves using historical data to predict future outcomes. The goal is to build a stochastic model that captures the underlying patterns in the data, and how those have changed over time, and use it to make predictions about future values. I have developed many statistical forecasting models that are now widely used in diverse domains including retail, energy, demography, and tourism.

The strength of statistical forecasting lies in its ability to provide quantifiable predictions based on historical data, and to include uncertainty in those predictions. Statistical forecasts are typically presented as probability distributions of future outcomes, which can be used to make decisions that account for the uncertainty in the predictions.

Most recently I have been developing tools that allow statistical forecasts to be produced for thousands of related time series, such as sales forecasts for individual products in a retail store, or electricity demand forecasts for small regions in a country. The disaggregated data tends to be noisy and sparse, and the models need to be able to share information across related series to make accurate predictions.

Because of the assumption that the future will continue to evolve similarly to the past, statistical forecasting tends to perform best when: (a) there is substantial historical data; (b) the underlying environment is relatively stable; and (c) the aim is to produce short-term forecasts. When there is limited historical data, or when the data generating process is subject to significant structural changes, statistical forecasting can struggle to make accurate predictions.

I am also interested in using statistical forecasting techniques to identify anomalies, such as sudden changes in the underlying process, or unusual patterns in the data. This can be useful for detecting fraud, monitoring the health of complex systems, or identifying recording errors in data.