Rob J Hyndman

WOMBAT 2026

Sun, 04 Oct 2026 13:00:00 GMT

WOMBAT (Workshop Organised by the Monash Business Analytics Team) is on again, on 30 November – 1 December 2026 in Melbourne. Early bird pricing ends on Sunday 18 October.

The theme this year is “TIME: Timely Insights, Models, and Exploration”. As always, the aim is to bring together analysts from academia, industry and government to learn about and discuss open-source tools for business analytics and data science.

30 November: Tutorials

Eight half-day tutorials across four tracks, including Nick Tierney on targets, Cynthia Huang on Quarto, Mitchell O’Hara-Wild on exploratory time series analysis, and Hadley Wickham on package development with Positron and AI.

1 December: Workshop

I’m giving the keynote, “Spotting the weird ones”, on finding anomalies in data using statistical methods and the weird R package. That is followed by invited talks on time series, AI workflows and industry perspectives.

The workshop is at 750 Collins Street, Docklands.

Registration

Early bird pricing ends on Sunday 18 October.

For the full program, see the WOMBAT 2026 website.

Evaluating time series foundation model claims

Mon, 24 Aug 2026 14:00:00 GMT

I was emailed by a journalist writing an article about time series foundation models. Here is my response. While the questions were about a specific new transformer model, the comments are more widely applicable, so I’ve edited my answer to be more generic.

As background, I would point you to my online forecasting textbook where we have a chapter on foundation models (Hyndman et al., 2026). Another excellent reference on this topic is Christoph Bergmeir’s paper “LLMs and Foundational Models: Not (Yet) as Good as Hoped” (Bergmeir, 2024).

1. How significant a change do time series foundation models represent for the practice of forecasting?

They’re a real shift in how forecasts get produced, but not yet a demonstrated shift in how accurate forecasts are in general. The genuine change is architectural and operational: instead of building, tuning and maintaining a separate model for every series, an organisation can point a single pre-trained model at a new dataset and get a forecast with no dataset-specific training or model selection. That lowers the barrier to entry for forecasting at scale, particularly for teams without in-house statistical expertise.

What it hasn’t yet done is replace the case for careful statistical modelling. The strongest independent critique of the field is Bergmeir (2024) who found that when you check the reported empirical results, the performance is often nowhere near as good as claimed. Often the statistical benchmarks are implemented poorly, or missing altogether, or highly selective. So these foundation models are an important development, but they are not close to being the panacea that many of the companies behind them are claiming.

2. How should organisations compare foundation models with well-tuned statistical or task-specific approaches, particularly considering accuracy and cost?

First, make sure the statistical baseline is actually well implemented. This is the single biggest problem Bergmeir (2024) identifies in the foundation-model literature: papers routinely compare against under-tuned or de-ensembled versions of classical methods, then claim victory. So I would be asking: what exactly were the statistical benchmarks, how were they tuned, and were they given the same advantages (ensembling, proper cross-validation) as in the studies that established them as strong benchmarks in the first place?

Second, evaluate on your own data with proper rolling-origin backtesting, not on a vendor’s leaderboard, or even a public community leaderboard. Public benchmark scores are increasingly unreliable as measures of genuine out-of-sample skill because the data used in comparisons was also used in training. This “leakage” is a growing problem – see Meyer et al. (2025) who found that of 401 datasets used across 22 published time-series foundation models, only 6% had never appeared in any model’s pre-training or fine-tuning corpus. Also, historical series that are not in the training data can be correlated with series that are in the training data due to being subject to the same environmental shocks (e.g., COVID). So even if there is not direct leakage, there can be indirect leakage induced by correlations. Therefore, these comparisons on historical data are not a great guide to what the models will do on new data.

Third, weigh total cost, not just error metrics. Well-implemented statistical models (ETS, ARIMA, Theta, and simple combinations of these) are cheap, transparent, fast, and easy to explain. Foundation models bring inference cost, latency, hosting/vendor dependency, and limited interpretability. Even if the accuracy gain over a well-tuned statistical model is real, the operational cost may not be worth it.

3. What do current benchmarks fail to tell us about how these models will perform in real production settings?

Public leaderboards report an aggregate score across many datasets and domains at once, which can hide wide variance in performance on the specific kind of series a given business actually needs to forecast.
As noted above, they carry a real contamination risk which may render the benchmark unreliable.
Benchmarks rarely report the things that determine whether a model is usable in production: inference latency at the volumes a business needs, hosting and total compute cost, robustness to structural breaks and regime changes, how the model handles exogenous variables and irregular/missing data, and how failures are detected and explained when they occur. A model that tops a MASE leaderboard by a small margin but takes an order of magnitude longer to run, or performs poorly on genuinely new data, may be the wrong choice operationally even if it’s the “best” model on paper.

References

Bergmeir, C. (2024). LLMs and foundational models: Not (yet) as good as hoped. Foresight: The International Journal of Applied Forecasting, 73, 33–38. https://cbergmeir.com/papers/Bergmeir2024LLMs.pdf

Hyndman, R. J., Athanasopoulos, G., Garza, A., Challu, C., Mergenthaler, M., & Olivares, K. G. (2026). Ch 15. Foundation forecasting models. In Forecasting: Principles and practice, the pythonic way. OTexts. http://OTexts.com/fpppy/15-foundation-models.html

Meyer, M., Kaltenpoth, S., Zalipski, K., & Müller, O. (2025). Rethinking evaluation in the era of time series foundation models:(un) known information leakage challenges. https://arxiv.org/abs/2510.13654v3

Researcher portals

Sun, 02 Aug 2026 14:00:00 GMT

I first wrote this post in 26 November 2011. Since then, some services have been discontinued, and new ones started. So I’ve updated it to reflect current offerings.

A researcher portal is a website that attempts to list all the publications of a given researcher, and calculate citation statistics, etc. Every researcher wants their work read and cited, so these websites can be useful tools for getting your work noticed. Conversely, they can be a good way to find new work by researchers in your field.

Google Scholar Citations

Google Scholar itself is an incredible resource covering journal articles, working papers, books, and almost everything else a scholar might cite. Google Scholar Citations uses Google Scholar data to provide information about all the outputs from a researcher. Some examples are Andrew Gelman and Di Cook. My profile is also there. It provides a way of listing your publications, tracking citations to your publications, computing citation metrics, etc. There is very little work in setting up a profile. When I did it, Google had automatically identified all my publications, and it even listed one paper I had forgotten I had written!

My Google Scholar profile (September 2026).

To see the profile of an existing researcher, just search for their name on Google Scholar. If they have made their profile public, it will appear at the top of the search results.

To set up your profile, go to scholar.google.com/citations. You may have to do some editing of the results to merge versions of the same publication, or to correct some errors in the database. I wish Google would use the information provided by these edits to correct their Google Scholar database.

Semantic Scholar

An alternative to Google Scholar is Semantic Scholar. The coverage is not quite as broad as Google Scholar – it misses working papers unless they are on arXiv. On the other hand, there are some useful tools, including AI-generated summaries that are quite good. It is also easy to follow cited papers, and citing papers, along the citation tree. Related papers are also identified, so it is possible to explore further afield than just direct citations. You can follow individual authors, and set up alerts for new citations of specific papers. Citations are classified as “highly influential”, “background”, “methods”, “results”, etc., so you can easily filter papers that have had a big impact, those that apply new methods, etc. One interesting difference is that Semantic Scholar automatically creates personal author pages for each author, without needing each researcher to set them up. Mine is here. My h-index on Semantic Scholar is 75, compared to 92 on Google Scholar, reflecting the broader coverage of Google Scholar.

My Semantic Scholar author page (September 2026).

All the rest

There are several other sites attempting to provide similar services, but none of them come close to Google Scholar or Semantic Scholar in useful features and usability. I’ve set up profiles on all of them, just to see how they work.

Web of Science (previously called Researcher ID): lists 234 of my publications, and gives my h-index as 61. This tends to miss books, R packages, working papers and book chapters, but has most of my journal articles. Links to online versions of the papers are provided. The “Clarivate Highly Cited” Awards are based on these citations.
ResearchGate: lists 364 of my publications (so there must be some double counting). I used to get a lot of unsolicited mail from this site, but that seems to have stopped. In any case, it does nothing that the other sites don’t do better.
Academia.edu: wants me to sign up for a premium account to do anything more than the basic. I had to block all the emails I was receiving from this site. Not interested.

Recommendations

Ignore the emails from ResearcherID, ResearchGate, Academia.edu, etc. Just hit delete and block them.
Set up a page on Google Scholar Citations. It’s not much work and makes your work more visible. It also allows you to track citations which are useful if you apply for promotion.
Try Semantic Scholar. You might find it helpful.
Check that your Semantic Scholar author page is accurate. Spend 10 minutes editing your profile so it gives a little more information about you.

Mathematical genealogy

Wed, 29 Jul 2026 14:00:00 GMT

Having a PhD student is like having a child. I have had many such “children” graduate, and have another two “on the way”. (See here for my offspring.)

Going in the other direction, here is my family tree, compiled from the Mathematics Genealogy Project (MGP). Each person advised the one below them. Hover over a portrait (or tap it) to see who they were.

Many people have two or more advisors, so there are multiple “lines of ascent” (as there are with regular family trees). I am showing two of many possible ancestry lines here, which happened to reconnect in the 1600s.

Jakob ThomasiusUniversität Leipzig, 1643
B. 1622, Leipzig. D. 1684, Leipzig.

Philosopher and philologist at Leipzig, and the teacher both branches share. He supervised Leibniz's 1666 philosophy degree and Otto Mencke's 1665 thesis, which is where the two lines below separate.

Wikipedia

Gottfried LeibnizDr. jur. Universität Altdorf, 1667
B. 1646, Leipzig. D. 1716, Hanover.

German philosopher, mathematician and logician, best known for inventing calculus independently of Newton.

His other advisors were Erhard Weigel at Jena (pictured, professor of mathematics there from 1653), Bartholomäus Schwendendörffer at Altdorf, and Christiaan Huygens in Paris.

Wikipedia
Nicolas MalebrancheOratoire de France, Paris
B. 1638, Paris. D. 1715, Paris.

Oratorian priest and Cartesian philosopher, author of De la recherche de la vérité (1674). He came to mathematics late, after reading Descartes, and his Paris circle carried the new calculus into France.

The MGP records his link to Leibniz to mark intellectual descent, not an actual supervision, so treat this edge as a courtesy.

Wikipedia
Jacob BernoulliDr. hab. Universität Basel, 1684
B. 1654, Basel. D. 1705, Basel.

One of the famous Bernoulli family. His contributions include the law of large numbers in probability theory and work on the convergence of infinite series.

Wikipedia
Johann BernoulliUniversität Basel, 1690, 1694
B. 1667, Basel. D. 1748, Basel.

Jacob's brother, elected a fellow of the academies of Paris, Berlin, London, St Petersburg and Bologna. Known as the "Archimedes of his age", he solved the catenary problem (alongside Leibniz and Huygens), and the rule for limits of functions commonly attributed to his student de l'Hôpital is his.

Wikipedia
Leonhard EulerUniversität Basel, 1726
B. 1707, Basel. D. 1783, St Petersburg.

One of the most prolific mathematicians in history, with over 800 papers covering every branch of mathematics known in his day, an influential series of calculus textbooks, and books on a dozen other fields.

Wikipedia
Joseph LagrangeUniversità di Torino, 1754
B. 1736, Turin. D. 1813, Paris.

Developed the theory of differential equations and produced many new results in number theory. His earlier work on permutations of the roots of polynomial equations is now seen as a forerunner of group theory, and he is remembered in the Lagrange multiplier.

Lagrange was never Euler's student in any formal sense. The MGP lists the relationship as "epistolary correspondence" on the calculus of variations.

Wikipedia
Siméon PoissonÉcole Polytechnique, 1800
B. 1781, Pithiviers. D. 1840, Sceaux.

"Life is only good for two things: to do mathematics and to teach it." Best known for his work on probability, including the Poisson distribution. Laplace was his second advisor.

Wikipedia
Michel ChaslesÉcole Polytechnique, 1814
B. 1793, Épernon. D. 1880, Paris.

Worked on projective and enumerative geometry, and wrote a history of geometric method that shaped how the subject was taught. The Chasles relation for directed segments is his.

Late in life he spent a fortune on thousands of forged letters purportedly from Pascal, Newton and Galileo, and defended their authenticity for years before the forger was tried and convicted.

Wikipedia
Gaston DarbouxUniversité de Paris, 1866
B. 1842, Nîmes. D. 1917, Paris.

Geometer at the Sorbonne who reshaped the differential geometry of surfaces. Anyone who has taught introductory analysis knows him through Darboux sums and the Darboux integral.

Wikipedia
Édouard GoursatUniversité de Paris, 1881
B. 1858, Lanzac. D. 1936, Paris.

Sharpened Cauchy's integral theorem by dropping the continuity assumption on the derivative, giving the Cauchy–Goursat theorem. His three-volume Cours d'analyse mathématique trained a generation of French analysts.

Wikipedia
Georges DarmoisUniversité de Paris, 1921
B. 1888, Éply. D. 1960, Paris.

Started in differential geometry, then turned to statistics, joining the Institut de Statistique de l'Université de Paris (founded by Émile Borel) in 1925 and later becoming its second director. The first statistician in my line: the Darmois–Koopman–Pitman theorem characterises the exponential family, and the Darmois–Skitovich theorem characterises the normal distribution.

Wikipedia
Daniel DuguéUniversité de Paris, 1937
B. 1912. D. 1987.

French probabilist who worked on estimation theory and characteristic functions, and who succeeded Darmois in directing the Paris statistics institute.

Wikipedia
Paul DeheuvelsUniversité Paris VI, 1974
B. 1948. D. 2026

Works on empirical processes, extreme value theory and dependence. His 1979 paper on the empirical dependence function, building on related work by Rüschendorf a few years earlier, gave the empirical copula used in dependence modelling today its lasting form. Elected to the Académie des Sciences.

Wikipedia
Adrian RafteryUniversité Paris VI, 1980
B. 1955, Dublin.

Bayesian model averaging, model-based clustering, and the probabilistic population projections the United Nations now uses. Professor of Statistics and Sociology at the University of Washington.

Wikipedia

Otto MenckeUniversität Leipzig, 1665
B. 1644, Oldenburg. D. 1707, Leipzig.

Professor of moral philosophy at Leipzig, and founder in 1682 of Acta Eruditorum, the first learned journal in Germany. Leibniz published much of his calculus there.

Wikipedia
J. C. WichmannshausenM.A. Universität Leipzig, 1685
B. 1663, Ilsenburg. D. 1727, Wittenberg.

Orientalist and professor at Wittenberg, and Mencke's son-in-law. His dissertation was on divorce under natural law, which makes him the least mathematical of my ancestors.

Wikipedia
Christian August HausenUniversität Halle-Wittenberg, 1713
B. 1693, Dresden. D. 1743, Leipzig.

Professor of mathematics at Leipzig. He built one of the early electrostatic friction machines and studied the sparks it produced.

Wikipedia
A. G. KästnerUniversität Leipzig, 1739
B. 1719, Leipzig. D. 1800, Göttingen.

Professor at Göttingen, prolific textbook writer, historian of mathematics, and a well-known writer of epigrams. Gauss studied at Göttingen during Kästner's time there, though he reportedly found Kästner's lectures too elementary to attend.

Wikipedia
Johann Tobias MayerUniversität Göttingen, 1773
B. 1752, Göttingen. D. 1830, Göttingen.

Physicist and mathematician at Göttingen, and son of the astronomer Tobias Mayer. He wrote on geometry and on the design of angle-measuring instruments.

Wikipedia
Enno Heeren DirksenUniversität Göttingen, 1820
B. 1788, Eilsum. D. 1850, Paris.

Professor in Berlin, where his students included Jacobi, Göpel and Heine. His own work was on the calculus of variations and transcendental analysis.

Wikipedia
Carl Gustav Jacob JacobiUniversität Berlin, 1825
B. 1804, Potsdam. D. 1851, Berlin.

Founded the theory of elliptic functions alongside Abel, and gave us the Jacobian determinant and the Hamilton–Jacobi equation. His advice to students was to invert.

Wikipedia
Wilhelm ScheibnerUniversität Halle-Wittenberg, 1848
B. 1826, Ölsnitz. D. 1908, Leipzig.

Professor at Leipzig, working on analysis and number theory.

Story's other Leipzig advisor was Carl Neumann, who also studied under Jacobi (through Hesse and Richelot), so both routes rejoin here.
William StoryUniversität Leipzig, 1875
B. 1850, Boston. D. 1930, Worcester.

Worked on algebraic problems and was an important player in the development of American mathematics. He helped found the American Journal of Mathematics.

Wikipedia
Solomon LefschetzClark University, 1911
B. 1884, Moscow. D. 1972, Princeton.

Russian-born mathematician who became the main source of the algebraic aspects of topology.

Wikipedia
John TukeyPrinceton University, 1939
B. 1915, New Bedford. D. 2000, New Brunswick.

The most innovative statistician of the 20th century. He invented the box plot and the stem-and-leaf plot, co-invented the fast Fourier transform, and contributed heavily to jackknife estimation and spectral density estimation. He is also credited with the words "software" and "bit".

Wikipedia
David BrillingerPrinceton University, 1961
B. 1937, Toronto.

Prolific researcher with over 200 papers, best known for his work on stochastic processes and time series, especially spectral analysis and earthquakes. His 1975 book Time Series: Data Analysis and Theory has been particularly influential. At the University of California, Berkeley.

Wikipedia
Peter GuttorpUniversity of California, Berkeley, 1980
B. 1949.

Uses stochastic models in hydrology, atmospheric science, geophysics, environmental science and haematology. At the University of Washington.

Wikipedia

Gary GrunwaldUniversity of Washington, 1987
B. 1954.

Gary spent the first part of his career on time series analysis, particularly non-Gaussian time series. Then he worked at the University of Colorado on nutrition, physiology, and cardiovascular health. Guttorp and Raftery jointly supervised his thesis, which is why the tree above has two branches. Website
Rob J HyndmanUniversity of Melbourne, 1993
B. 1967, Melbourne.

Peter Brockwell was also my advisor, and he was supervised by Joe Moyal, but then the line stops --- the MGP does not record who Moyal's advisor was.

Wikipedia

Monthly, quarterly and annual growth rates

Mon, 27 Jul 2026 14:00:00 GMT

I received the following question by email:

I was wondering if mathematical relationships exist between monthly, quarterly, and annual growth rates (percentage changes). For time series data …
1) Is there a mathematical formula to know by how many months the Month-to-Month changes lead turns in this-month-to-the-same-month-one-year-ago changes (annual changes)?
2) Is there a mathematical formula to know by how many quarters the Quarter-to-Quarter changes lead turns in this-quarter-to-the-same-quarter-one-year-ago changes (annual changes)?
Also, regarding simple moving averages/totals:
3) Is there an exact or approximate formula to know by how many months a 3-month simple moving average (or moving total) leads a turn in a 12-month simple moving average (or moving total)?
As an example: Inflation is typically measured with Y/Y percentage changes in the CPI. So, it would be helpful to be able to identify when the 12-month rate of change will likely reach a top or bottom, using the 1-month or 3-months rates of change.

Monthly growth vs year-over-year growth

Suppose we have monthly data . Then the monthly growth rates are given by , and the year-over-year growth rates are given by . A little algebra gives So the year-over-year rate is the compounded product of the last 12 monthly rates. This gives a recurrence relationship: Therefore In other words, the year-over-year rate rises when the current monthly change exceeds the change 12 months ago, and falls when the reverse holds. Figure 1 shows why: moving from to , the 12-month window drops and adds , while the other 11 months are common to both.

Figure 1: The 12-month windows of monthly growth rates used to compute and .

For example, annual inflation peaks when the monthly rates entering the 12-month window fall below the rates dropping out of it. Because year-over-year growth is effectively a 12-month trailing window of monthly growth, month-to-month growth tends to lead year-over-year growth by about half the window length: the centre lag of an equally-weighted -period trailing window is periods, or 5.5 months when .

This centre-lag rule works best when the underlying series changes slowly within the 12-month window, as is typical of series like inflation, and is less reliable for more volatile or noisy series.

Quarterly growth vs year-over-year growth

The quarterly version is exactly analogous: just replace 12 by 4. Since four-quarter growth is a trailing 4-quarter window of quarterly growth, quarter-to-quarter growth leads year-over-year growth by about 1.5 quarters, under the same condition as above.

Log growth rates

Taking logarithms turns the multiplicative relationship above into a linear one, which is more convenient for compounding over several periods, for forecasting, and for series such as price indices, where growth is conventionally expressed in log form.

Define monthly log growth and year-over-year log growth , using and from above. Taking logs of Equation 1 turns the product into a sum: So rises when : the same turning condition as before, now linear rather than multiplicative.

This also gives a forecasting formula: So given a model for future monthly growth, you can forecast the future year-over-year rate in log terms.

The same idea works over any horizon, comparing the most recent 3-month block to the corresponding block a year earlier: So the year-over-year rate has risen over the past three months if the most recent 3-month growth exceeds the same 3-month block a year earlier.

This is exactly how CPI inflation is usually reported: is the price index, the monthly inflation rate, and the standard year-over-year rate quoted in the news.

3-month moving averages vs 12-month moving averages

A trailing simple moving average of is defined as: Then So rises when the newest observation entering the window, , exceeds the oldest observation leaving the window, , and falls when the reverse holds.

As with the growth-rate case, the practical lead between two moving averages of different lengths can be approximated from the centre of each trailing window. A trailing -period average is centred about periods in the past, so a 3-month average is centred about 1 month in the past, while a 12-month average is centred about 5.5 months in the past. So a 3-month moving average tends to lead a 12-month moving average by about months, for the same reason that monthly growth leads year-over-year growth above. As before, this holds as long as the series moves slowly relative to the 12-month window.

Women in Forecasting

Sun, 14 Jun 2026 14:00:00 GMT

The second edition of the Women in Forecasting (WIF) Conference is taking place on 27 June, right before the International Symposium on Forecasting in Montreal Canada.

Women in Forecasting was created to showcase the important contributions women are making to time series forecasting, a field where women remain underrepresented. Beyond the talks, the conference is also an opportunity to build community, foster professional connections, and support mentorship for those entering the field and those advancing in their careers.

I’m delighted to see four of my own former PhD students are invited speakers at the conference, along with Mariana Menchero who was a great help to me in solving numerous technical issues related to the Python edition of FPP.

Date: Saturday, June 27

Venue: Université du Québec à Montréal (The same venue as ISF 2026)

Registration is required for both the in-person venue and on the Zoom.

Forecasting: Principles and Practice, the Pythonic Way

Thu, 04 Jun 2026 14:00:00 GMT

The Python edition of “Forecasting: Principles and Practice” is now available in print.

About a year ago, we did a soft launch of the online edition of “Forecasting: Principles and Practice, the Pythonic Way” (FPPPY). In this latest edition of the FPP book, George Athanasopoulos and I are joined by four additional co-authors: Azul Garza, Cristian Challu, Max Mergenthaler and Kin Olivares. Our new co-authors are Python and forecasting experts, and have all been involved in developing the open-source Nixtla software that is used throughout the book.

Since the online edition was launched, we have made numerous small updates and improvements to the book, ironing out errors, improving the presentation, and simplifying the Python code where possible. Now it is available in print as well!

The first 13 chapters of the Python edition closely follow the corresponding R fpp3 edition, but with code examples in Python rather than R. We have also included two new chapters: Neural networks and Foundation forecasting models, which are dedicated to recent techniques and developments in neural networks applied to forecasting.

As with previous editions, all code and examples are fully reproducible, and exercises are provided for teaching or self-study. The online edition will remain free, and will be updated as needed. The print edition is available for purchase, and we hope it will be a useful resource for students, practitioners, and anyone interested in learning about forecasting with Python.

Podcasts

Wed, 22 Apr 2026 14:00:00 GMT

I’ve been interviewed for several podcasts over the last few years. It’s always fun to talk about my work, and I hope there are enough differences between them to make it interesting for listeners. Here is a full list of them.

Updated: 23 April 2026

Date	Podcast	Episode
23 April 2026	Value Driven Data Science	How giving away your work for free can build your authority as a data scientist
16 April 2026	Value Driven Data Science	Why traditional statistics still matters in the age of AI
26 May 2023	Forecasting Impact	Forecasting software panel
14 March 2022	Faculty.net	Forecasting in social settings
17 November 2021	The Random Sample	Software as a first class research output
24 May 2021	Data Skeptic	Forecasting principles and practice
12 April 2021	Seriously Social	Forecasting the future: the science of prediction
6 February 2021	Forecasting Impact	Rob Hyndman
19 July 2020	The Curious Quant	Forecasting COVID, time series, and why causality doesnt matter as much as you think
27 May 2020	The Random Sample	Forecasting the future & the future of forecasting
9 October 2019	Thought Capital	Forecasts are always wrong (but we need them anyway)

Why traditional statistics still matters in the age of AI

Wed, 15 Apr 2026 14:00:00 GMT

I was interviewed on this topic recently by Dr Genevieve Hayes for her podcast Value Driven Data Science.

Here’s the blurb (not written by me):

Data scientists today are under pressure to adopt the latest tools - machine learning, LLMs, generative AI. But in the rush to embrace what’s new, many are leaving some of the most powerful analytical tools sitting on the shelf. Tools that handle something modern AI largely can’t: uncertainty.

In this episode, Prof. Rob Hyndman joins Dr. Genevieve Hayes to make the case for why rigorous statistical thinking remains indispensable in the age of AI, and what data scientists are giving up when they abandon it.

Check it out if you’re interested. Other podcasts I’ve done are listed here.

Rolling scaled forecast accuracy

Mon, 19 Jan 2026 13:00:00 GMT

When we compute a MASE or RMSSE using a rolling origin, should the scaling factor be recalculated every time?

I’ve been asked this a couple of times, so perhaps it is worth a blog post.

For a simple training/test split, the Mean Absolute Scaled Error (MASE) (Hyndman & Koehler, 2006) is defined as and the Root Mean Squared Scaled Error (RMSSE) is defined as In both cases, for non-seasonal data, where is the seasonal period for seasonal data, and the sum in the numerator is over the test set, while the sum in the denominator is over the training set. The notation means the forecast of given data .

These measures are discussed in my forecasting textbook with George Athanasopoulos. The denominator is a scaling factor, introduced so that you can compare MASE or RMSSE values across series of different units. For example, are the forecasts of widget sales more accurate than the forecasts of electricity demand? If all your forecasts are in the same units, then you don’t need to remove the scale, and it is simpler to just use MAE or RMSE (i.e., only the numerators of the above equations).

Now, the question is, when we are doing time series cross-validation, and computing forecast accuracy over a series of training/test sets with a rolling origin, does it make sense to compute a different scaling factor each time? For example, a cross-validated MASE for -step forecasts is where the first observations form the smallest training set, and subsequent training sets increase one observation at a time.

An alternative approach would be to compute the scaling factor across all available data, rather than calculate it separately for each training set. Then MASE would become

Let’s think about the advantages of this alternative:

It is (slightly) faster. But the denominators are very fast to compute, so this really doesn’t make much difference.
It removes a source of variation from the calculation. The scaled errors will be more variable when the denominator changes with each test set, and that makes it harder to see the difference between forecasting methods.
It uses more data in computing the scaling factor, which reduces the uncertainty in the estimate of the accuracy measure. This is potentially important if , so that some training sets are relatively small compared to the available data.

As for disadvantages:

The measures are no longer true measures of forecast accuracy because the calculation potentially involves future observations. On the face of it, this seems important, but in reality it isn’t. The future observations aren’t affecting the forecasts, only the scaling factor, so there is no leakage involved.
One of my correspondents suggested that it changed the interpretation. I think the interpretability of these scaled measures is over-rated. They can only be interpreted as a ratio of out-of-sample accuracy to in-sample accuracy, and the two are not necessarily even over the same horizons. How the scaling factor is calculated doesn’t really affect the interpretation, because there is not much value in interpretation either way.

So I suggest that computing the scaling factor across all available training data when calculating cross-validated MASE and RMSSE values is a good idea.

That raises the question as to why shouldn’t we use all available data in the denominator when doing a simple training/test split? In that context, the only advantage that is relevant is #3 above, and unless the test set is particularly large, it shouldn’t make much difference. Nevertheless, I can’t see any real problem in using all available data in computing the scaling factor.

References

Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International J Forecasting, 22(4), 679–688. https://robjhyndman.com/publications/another-look-at-measures-of-forecast-accuracy/

forecast package v9

Sun, 11 Jan 2026 13:00:00 GMT

It’s been nearly nine years since the last major release of the forecast package, as we have been focusing on developing fable and related packages as a tidyverse-based alternative for time series forecasting in R.

However, version 9.0 of the forecast package has now been released on CRAN. Thanks to Maximilian Muecke for helping with this release, and for modernising some of the old code.

Here are the main new features and changes.

New forecasting models

The usual workflow with the forecast package is to fit a model to a time series, and then forecast from the model. For example

USAccDeaths |> ets() |> forecast()
USAccDeaths |> auto.arima() |> forecast()

But there have always been a few functions that produce forecasts directly from a time series, without the need to explicitly fit a model first. For example,

USAccDeaths |> snaive()

produces seasonal naive forecasts.

But now you can obtain the same forecasts by first fitting a model and then forecasting from it:

USAccDeaths |> rw_model(lag = 12) |> forecast()

Similarly, you can now fit and forecast from mean models, cubic spline models, theta models, and Croston’s method using the new mean_model(), spline_model(), theta_model(), and croston_model() functions. The existing naive(), snaive(), rwf, meanf(), splinef(), thetaf(), and croston() functions are still available for backward compatibility, but are now merely wrappers around the modelling and forecast functions.

This should make it easier to set up forecasting workflows involving many different models. For example, suppose you wanted to compute forecasts from five different models applied to the USAccDeaths time series.

library(purrr)
model_fns <- list(ets, auto.arima, rw_model, rw_model, mean_model)
args <- vector("list", length(model_fns))
args[[3]] <- list(lag = 12)
args[[4]] <- list(lag = 12, drift = TRUE)
models <- map2(model_fns, args, ~ do.call(.x, c(list(y=USAccDeaths), .y)))
names(models) <- map(models, function(u) u$method)
forecasts <- map(models, forecast)

Note that rw_model is included twice, with different arguments, allowing the calculation of seasonal naive, and seasonal naive with drift, forecasts.

Forecasting with multiple models is easy to do using the fable package (and was one of the key considerations in its design). Now it is also possible to do it using the forecast package, with this more consistent interface.

Missing values in ETS models

I wrote the first version of the ETS code (more than 25 years ago) for a consulting project where there were no missing values, so I didn’t write the code to handle them. But obviously missing values occur a lot in many applications, and I’ve now belatedly added support for them. If there are gaps in a time series, ets() can be applied, and it computes the likelihood and fitted values correctly using all available data. (Previously, it would fit a model to the largest contiguous segment of the data.)

This change has also been made to the ETS() function in the fable package.

Bootstrapped prediction intervals

The default prediction intervals in the forecast package are based on the (usually reasonable) assumption that forecast errors are Gaussian. Bootstrapped prediction intervals allow for non-Gaussian forecast errors. These have long been available using ETS, ARIMA, and other models; they are now also available using ARFIMA and TBATS models.

Other changes

Added parallelization for nnetar().
More consistent handling of the biasadj argument across models.
More consistent interface for simulated and bootstrapped prediction intervals for all models.
Bug fixes and performance improvements.
Documentation improvements.

Future of the forecast package

I would still encourage people to use the fable package for new projects, as it provides a more modern and consistent interface for time series forecasting in R. However, the forecast package will continue to be maintained, with bug fixes and minor improvements as needed.

WAPE: Weighted Absolute Percentage Error

Thu, 07 Aug 2025 14:00:00 GMT

I was recently asked for my view on the so-called “Weighted Absolute Percentage Error” (WAPE).

The WAPE was introduced by Kolassa & Schütz (2007) who called it the MAD/Mean ratio. It is defined as where

is the actual value at time ,
is the forecast value at time

You can think of it as a weighted percentage error by writing it as where the weights are given by

It can also be considered a relative MAE where the comparison method has all forecasts equal to zero. In fact, would give the skill score relative to the forecast of zero for all .

Or you could think of it as like a MASE (Hyndman & Koehler, 2006) but with scaling based on the sum of absolute values on the test set rather than the sum of absolute differences on the training set.

This has some obvious advantages over the MAPE:

The MAPE is undefined when any actual value in the test set is zero. The WAPE is defined even when some actuals are zero. It is only undefined when all the actuals used in the denominator are zero.
Optimising the MAPE does not lead to a sensible point forecasts (Gneiting, 2011), but optimising the WAPE will lead to the median forecast.

However, I think there are a couple of problems that do not seem to have been widely recognized.

The resulting estimate is only consistent when the time series is stationary. So it should not be used with data that has trends, or seasonality, or heteroscedasticity.
It is quite possible to have all actuals in the test set equal to zero, especially with intermittent demand time series and small test sets. Then the denominator is zero, and the WAPE is undefined.

For these reasons, I think the Mean Absolute Scaled Error (MASE) (Hyndman & Koehler, 2006) is a better choice than the WAPE. The MASE is defined as where for non-seasonal data, and is the seasonal period for seasonal data, and the sum in the numerator is over the test set (of size ), and the sum in the denominator is over the training set (of size ). Because the denominator is defined on the training data, not the test data, it avoids the above problems with the WAPE:

It is a consistent estimator provided the series is difference stationary, which is a much weaker condition than stationarity.
The training data is also usually much longer than the test data, so it is much less likely to contain only zeros. In fact, if the training data did contain only zeros, then the obvious forecasts would all be zeros too.

I don’t want to suggest there are no problems with the MASE. Kolassa & Schütz (2007) point out one potential drawback of MASE — when there are structural breaks or outliers in the training data. Also, like the WAPE, optimising the MASE will lead to the median forecast, which is probably not what you want, especially when you have intermittent demand time series.

These days, if I want a scale-free accuracy measure, I prefer the Root Mean Squared Scaled Error (RMSSE): where again the sum in the numerator is over the test set, and the sum in the denominator is over the training set. This has all the advantages (and most disadvantages) of the MASE, but optimising it leads to the mean forecast rather than the median. It also aligns better with how models are estimated. Almost all models are estimated by minimising the sum of squared errors, so it makes sense to evaluate them using squared errors as well.

Point forecast reconciliation also works on means rather than medians, and is optimised using least squares, so it is more natural to evaluate using a squared error measure.

Update on consistency and stationarity

The comments below suggest that I need to explain the consistency issue in more detail. Imagine you are forecasting a time series that is a simple linear trend with some noise. So , where is Gaussian white noise with mean 0 and variance . As a good forecaster, you would fit a linear trend to this series, so your forecasts would be . To keep it simple, let’s ignore the estimation error in this model and set and .

Now let’s generate forecasts for this series over a test set of observations (either the next time periods, or over a cross-validated set). If you use the RMSE to evaluate these forecasts, you obtain which is an estimate of the variance of the noise. As more observations are included in the calculation, the RMSE will converge to . That makes it “consistent” in a statistical sense. Practically, it also means that the expected RMSE does not depend on which observations you use to evaluate the forecasts — the RMSE values will be similar regardless of the test set used in the calculation. That is, if you evaluate the forecasts over the next week, the RMSE will be similar to the RMSE calculated on a cross-validated set.

Suppose, instead, we use the RMSSE to evaluate the forecasts. The RMSSE is equal to the RMSE divided by So the RMSSE is still consistent (it converges to ), and the expected RMSSE does not depend on which observations you use to evaluate the forecasts.

Now, suppose we use the WAPE to evaluate these forecasts. Then we obtain where both sums are over the test set. As increases, the numerator will converge to the mean absolute value of the noise, approximately . But the denominator won’t converge at all as it depends on the trend in the series. So the WAPE is not a consistent estimator of anything.

To see the effect of this, imagine the time series is daily sales, and you produce forecasts each week for the next 7 observations, and evaluate them using the WAPE. Over time, your WAPE values get smaller and smaller, because the denominator is growing larger and larger as the trend accumulates. So you have the illusion of improving accuracy, but your forecasts are really no more accurate than they were at the start.

While I have used a linear trend as the example here, similar problems can occur whenever the underlying time series is not stationary.

References

Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association, 106(494), 746–762.

Kolassa, S., & Schütz, W. (2007). Advantages of the MAD/mean ratio over the MAPE. Foresight: The International Journal of Applied Forecasting, 6, 40–43.

fpp3 package update

Tue, 25 Jun 2024 14:00:00 GMT

The fpp3 package has had its first major update since the book was published.

The fpp3 package is a companion to the book Forecasting: Principles and Practice (3rd edition, Hyndman & Athanasopoulos, OTexts). When you load the package, it loads the data and functions needed for the examples in the book.

library(fpp3)

── Attaching packages ──────────────────────────────────────────── fpp3 1.0.0 ──

✔ tibble      3.2.1     ✔ tsibble     1.1.4
✔ dplyr       1.1.4     ✔ tsibbledata 0.4.1
✔ tidyr       1.3.1     ✔ feasts      0.3.2
✔ lubridate   1.9.3     ✔ fable       0.3.4
✔ ggplot2     3.5.1     ✔ fabletools  0.4.2

── Conflicts ───────────────────────────────────────────────── fpp3_conflicts ──
✖ lubridate::date()    masks base::date()
✖ dplyr::filter()      masks stats::filter()
✖ tsibble::intersect() masks base::intersect()
✖ tsibble::interval()  masks lubridate::interval()
✖ dplyr::lag()         masks stats::lag()
✖ tsibble::setdiff()   masks base::setdiff()
✖ tsibble::union()     masks base::union()

This provides a shorthand way of attaching the packages needed for most time series forecasting tasks, using packages from the tidyverts, just as the tidyverse package does for data manipulation and visualization.

In the most recent update, we have added many new data sets that can be used as examples by instructors, or for users to practice their forecasting skills.

Each of these new data sets were previously used in an exam for the forecasting subjects taught by George Athanasopoulos and me at Monash University. Three of our exams are available online for those who are interested to see how we assess forecasting students.

Each year, we take a new data set, and write the exam around how to analyse the time series. We strongly believe in assessing students on their forecasting skills with real data, rather than using artificial data, or asking technical questions that are not relevant to real-world forecasting. As a result, our exams are quite different from anything we have seen elsewhere.

We also think it is best to use “fresh and local”¹ data sets. So the data are almost always from Australia, and included data up to about a month before the exam was written.

In the latest update to the fpp3 package, we have expanded these data sets to cover more series than we did in the exams, and in most cases updated them to include more recent observations.

The new data sets are:

aus_births: Australian births data
aus_fertility: Australian fertility rates
aus_inbound: Monthly short term (<1 year) visitor arrivals to Australia
aus_migration: Australian migration data
aus_mortality: Australian mortality data
aus_outbound: Monthly short term (<1 year) resident departures in Australia
aus_tobacco: Australian cigarette and tobacco expenditure
aus_vehicle_sales: Australian vehicle sales
melb_walkers: Average daily total pedestrian count in Melbourne
nsw_offences: Monthly offences in NSW
ny_childcare: New York childcare data
otexts_views: OTexts page views

Thanks to Nuwani Palihawadana and Shanika Wickramasuriya for most of the work on this latest update, completed at the NUMBAT hackathon last month.

Footnotes

Thanks to Di Cook for this phrase.↩︎

AIC calculations

Tue, 31 Oct 2023 13:00:00 GMT

The AIC returned by TSLM() is different from that returned by lm(). Why?

I get this question a lot, so I thought it might help to explain some issues with AIC calculation.

First, the equation for the AIC is given by where is the likelihood of the model and is the number of parameters that are estimated (including the error variance). For a linear regression model with iid errors, fitted to observations, the log-likelihood can be written as where is the residual for the th observation. The AIC is then Since we don’t know , we estimate it using the mean squared error (the maximum likelihood estimator), giving where is a constant that depends only on the sample size and not on the model. This constant is often ignored. Thus, different software implementations can lead to different AIC values for the same model, since they may include or exclude the constant .

Now, let’s look at what R returns in a simple case using the lm() function.

set.seed(2023)
library(fpp3)
df <- tibble(
  time = seq(100),
  x = rnorm(100),
  y = x + rnorm(100)
)
fit <- lm(y ~ x, data = df)
AIC(fit)

[1] 275.6267

We can check how this is calculated by computing it ourselves.

mse <- mean(residuals(fit)^2)
n <- length(residuals(fit))
k <- length(fit$coefficients) + 1
# With constant
2*k + n*log(mse) + n*log(2*pi) + n

[1] 275.6267

# Without constant
2*k + n*log(mse)

[1] -8.161047

Clearly, AIC() applied to the output from lm() is using the version with the constant.

Now compare that with what we obtain using the TSLM() function from the fable package.

df |>
  as_tsibble(index = time) |>
  model(TSLM(y ~ x)) |>
  glance() |>
  pull(AIC)

[1] -8.161047

This is the AIC without the constant.

The situation is even more confusing with ARIMA models, and some other model classes, because some functions use approximations to the likelihood, rather than the exact likelihood.

Thus, AIC values can be compared across models fitted using the same functions, but not necessarily when models have been fitted using different functions.

P-values for prediction intervals

Tue, 17 Oct 2023 13:00:00 GMT

Does it make any sense to compute p-values for prediction intervals?

I received this email today:

My team recently used some techniques found in your writings to perform forecasts … Our work has been well received by reviewers, but one commenter asked two questions that I was hoping you may be able to provide insight on.

First, they wanted to know if we could provide P-values for our prediction intervals. In our work, we said, “Observed rates were deemed significantly different from expected rates when they did not fall within the 95% PI.” This same language has been used by others published in the same journal. I am curious to hear your thoughts on giving P-values for these PIs and what the appropriate method for doing so would be (if any).

Second, they asked about making a correction for multiple comparisons. … I believe we could apply a Bonferroni correction to the PIs, but that feels too liberal. Moreover, I am curious if this is even called for given our statement of what is deemed significant and the fact that our prediction interval construction relies on a non-parametric method.

Here is my reply:

I don’t think this makes any sense. A p-value is the probability of obtaining observations at least as extreme as those observed given a null hypothesis. What’s the hypothesis here? In forecasting, we don’t usually have a hypothesis. Instead, we fit a model to the data, and make predictions based on the model. I guess you could make the null hypothesis “The future observations come from the forecast distributions”, and then the p-value for each future time period would be the probability of the tails beyond the observations. But it is well-known that the estimated prediction intervals are almost always too narrow due to them not taking into account all sources of variance. So the size of this test would not be well-calibrated. I think you’re better off pushing back rather than trying to meet the request.
A Bonferroni correction assumes independence between the intervals, and that is not true for PIs from a forecasting model. The future forecast errors are all correlated (with the strength of the correlation depending on the model and the DGP). Usually we just say that these are pointwise PI, and so we expect 5% of observations to fall outside the 95% prediction intervals. It is possible to generate uniform PI, which contain 95% of all future sample paths, but this is a little tricky due to the correlations between horizons. It could be done via simulation – simulate a 1000 future sample paths and compute the envelope that contains 950 of them.

It sounds like the reviewers are only familiar with inferential statistics, and not with predictive modelling. You could point them to Shmueli’s excellent 2010 paper “To explain or predict”, highlighting the differences between the two paradigms.

How NASA didn’t discover the hole in the ozone layer

Thu, 28 Sep 2023 14:00:00 GMT

I am writing a new textbook on anomaly detection. It probably won’t be finished for at least a year, but here is an excerpt.

There is a widespread myth that NASA did not discover the hole in the ozone layer above the Antarctic because they had been throwing away anomalous data that would have revealed it. This is not true, but the real story is also instructive (Pukelsheim 1990; Christie 2001, 2004).

NASA had been collecting satellite data on Antarctic ozone levels using a Total Ozone Mapping Spectrometer (TOMS) since 1979, while British scientists had collected ozone data using ground sensors at the Halley Research Station, on the edge of the Brunt Ice Shelf in Antarctica, since 1957. Figure 1 shows average daily values from the NASA measurements in blue, and from the British observations in orange. There is a clear downward trend in the British data, especially from the late 1970s, which is confirmed with the NASA data. So why wasn’t the “ozone hole” discovered until 1985?

Figure 1: Observations of Antarctic ozone levels since 1957, measured in Dobson units (DU). Observations are mean daily values from October each year. Ground observations are from Halley Research Station, while satellite observations were obtained using a Total Ozone Mapping Spectrometer (TOMS). The satellite data were obtained from Leslie R Lait (NASA), while the Halley ground observations were obtained from Jonathan Shanklin (British Antarctic Survey). The horizontal line shows the threshold of 180 DU, used by NASA to determine when the ozone level was unusually low.

The British scientists had noticed the low ozone values as early as 1981, but it took a few years for the scientists to be convinced that the low values were real and not due to instrument problems, and then there were the usual publication delays. Eventually, the results were published in Farman et al. (1985).

Meanwhile, NASA was flagging observations as anomalous when they were below 180 DU (shown as a horizontal line in Figure 1). As is clear from the figure, this is much lower than any of the plotted points before the early 1980s. However, the 180 threshold was used for the daily measurements, which are much more variable than the monthly averages that are plotted. Occasionally daily observations did fall below 180, and so it was a reasonable threshold for the purpose of identifying instrument problems.

In fact, NASA had checked the unusually low TOMS values obtained before 1985 by comparing them against other available data. But the other data available to them showed ozone values of about 300 DU, so it was assumed that the satellite sensor was malfunctioning. The British Halley data were not available to them, and only after the publication of Farman et al. (1985) did the NASA scientists realise that the TOMS results were accurate.

In 1986, NASA scientists were able to confirm the British finding, also demonstrating that the ozone hole was widespread across the Antarctic (Stolarski et al. 1986).

This example reveals some lessons about anomaly detection:

The NASA threshold of 180 was based on daily data, and was designed to identify instrument problems, not genuine systematic changes in ozone levels. The implicit assumption was that ozone levels varied seasonally, but that otherwise the distribution of observations was stable. All anomaly detection involves some implicit assumptions like this, and it is well to be aware of them.
Sometimes what we think are anomalies are not really anomalies, but the result of incorrect assumptions.
Often smoothing or averaging data will help to reveal issues that are not so obvious from the original data. This reduces the variation in the data, and allows more systematic variation to be uncovered.
Always plot the data. In this case, a graph such as Figure 1 would have revealed the problem in the late 1970s, but it seems no-one was producing plots like this.

References

Christie, M. 2001. The Ozone Layer: A Philosophy of Science Perspective. Cambridge University Press.

Christie, M. 2004. “Data Collection and the Ozone Hole: Too Much of a Good Thing?” History of Meteorology 1: 99–105.

Farman, J C, B G Gardiner, and J D Shanklin. 1985. “Large Losses of Total Ozone in Antarctica Reveal Seasonal ClO/NO Interaction.” Nature 315 (6016): 207–10.

Pukelsheim, F. 1990. “Robustness of Statistical Gossip and the Antarctic Ozone Hole.” The IMS Bulletin 19 (4): 540–45.

Stolarski, R S, A J Krueger, M R Schoeberl, R D McPeters, P A Newman, and J C Alpert. 1986. “Nimbus 7 Satellite Measurements of the Springtime Antarctic Ozone Decrease.” Nature 322 (6082): 808–11.

Forecast model selection

Tue, 26 Sep 2023 14:00:00 GMT

When using a training/test split, or time-series cross-validation, are you choosing a specific model or a model class?

This question arises most time I teach a forecasting workshop, and it was raised again in the following email I received today:

I have a time series that I have split into training and test datasets with an 80%-20% ratio. I fit a series of different models (ETS, BATS, ARIMA, NN etc) to the training data and generate my forecasts from each model. When evaluating the forecasts against the test set I find the model that gives the best outcome is an ARIMA(1,1,1) that was selected using the auto.arima function. My question is this, should I proceed to fit an ARIMA(1,1,1) to the whole data set, or should I use the auto.arima function again which may give me a slightly different (p,d,q) order as there is now an extra 20% of unseen data available to the forecast model? Any guidance would be greatly received.

If you only have one class of model to consider (e.g., only ETS or only ARIMA), then it is easy enough to select the model on all available data using the AIC, and use the selected model to forecast the future. But if you are selecting between model classes, then you need to use either a training/test split, or (preferably) a time-series cross-validation procedure.

If you use time-series cross-validation, then there would usually be different models selected for each training set, and the cross-validated error is a measure of how well the model class works for your data. In that case, there is no single model for the training data, and you are selecting the model class rather than a specific model. This makes it clear that you should then apply the selected model class to all the data, when forecasting beyond the end of the available data. In other words, if you choose ARIMA over ETS, then you would then fit an ARIMA model to all the data, and use that model to forecast the future.

You can think of a simple training/test split as a special case of time-series cross-validation, where there is a single fold. So the same argument applies. That is, you are selecting the model class that works best for your data, and so you should apply that model class to all the data, when forecasting beyond the end of the available data.

This example also illustrates why it is important to use a time-series cross-validation procedure, rather than a simple training/test split. In this case, the ARIMA model was selected because it happened to work best for the particular training/test split that was used. But if a different split had been used, then a different model might have been selected. So the model selection is not stable. By averaging over multiple folds using a time-series cross-validation procedure, you can get a more stable estimate of the model class that works best for your data.

Videos for Forecasting: principles and practice (3rd ed)

Thu, 20 Jul 2023 14:00:00 GMT

Over the past 6 months, George Athanasopoulos and I have added videos to most sections of the 3rd edition of our textbook Forecasting: principles and practice.

We have taught from the book many times, but this year we decided to pre-record short videos for each section. Our students often prefer a video explanation than reading the textbook, and we thought other readers might appreciate hearing from us as well.

These videos are embedded in most sections of the book. So far, we’ve covered the sections that we include in our own courses, but we hope to eventually have videos for all sections. Most of these were done in a single take, so they are sometimes a little rough, but hopefully still useful.

You can view the entire playlist on YouTube.

Degrees of freedom for a Ljung-Box test

Tue, 27 Jun 2023 14:00:00 GMT

The Ljung-Box test is widely used to test for autocorrelation remaining in the residuals after fitting a model to a time series. In this post, I look at the degrees of freedom used in such tests.

The Ljung-Box test

Suppose an ARMA() model is fitted to a time series of length , giving a series of residuals , and let the autocorrelations of this residual series be denoted by The first autocorrelations are used to construct the statistic

This statistic was discussed by Box and Pierce (1970), who argued that if is large, and the model parameters correspond to the true data generating process, then has a distribution with degrees of freedom. Later, Ljung and Box (1978) showed that if the model is correct, but with unknown parameters, then has a distribution with degrees of freedom.

Extensions to other models

These days, the Ljung-Box test is applied to a lot more models than non-seasonal ARMA models, and it is not clear what the degrees of freedom should be for other models. For example:

What if the model includes an intercept term? Should that be included in the degrees of freedom calculation?
What about a seasonal ARIMA model? Do we just count all coefficients?
Or a regression with ARMA errors? Should we include the regression coefficients when computing the degrees of freedom?
Or an ETS model? Do we count just the smoothing parameters, or do we include the states as well, or something else?

Not long ago, I had naively assumed that the correct degrees of freedom would be where is the number of parameters estimated. I am in good company because Andrew Harvey in Harvey (1990, p259) made exactly the same conjecture. That was what was coded in the forecast::checkresiduals() function prior to v8.21, and how the test was applied in Hyndman and Athanasopoulos (2018) and Hyndman and Athanasopoulos (2021) until February 2023. But a recent github discussion with Achim Zeilis convinced me that it is incorrect.

Let’s look at a few examples. For each model, we will simulate 5000 series, each of length 250 observations. For each series, we compute the p-value of a Ljung-Box test with and degrees of freedom, for different values of . Under the null hypothesis of uncorrelated residuals, the values should have a uniform distribution.

Code

library(forecast)
library(ggplot2)
set.seed(0)

# Function to simulate p-values given a DGP model and
# a function to fit the model to a time series
simulate_pvalue <- function(model, fit_fn, l=10) {
  ## simulate series
  if(is.null(model$xreg)) {
    y <- simulate(model, n = 250)
  } else {
    y <- simulate(model, xreg=model$xreg, n=250)
  }
  if(inherits(model, "ets")) {
    # If multiplicative errors, fix non-positive values
    if(model$components[1] == "M")
      y[y <= 0] <- 1e-5
  }
  ## Fit model
  m <- fit_fn(y)
  ## compute p-values for various df
  pv <- purrr::map_vec(0:3, m=m,
    function(x, m) {
      Box.test(residuals(m), lag = l, fitdf = x, type = "Ljung-Box")$p.value
    }
  )
  names(pv) <- paste("K =", 0:3)
  return(pv)
}
# Function to replicate the above function
simulate_pvalues <- function(model, fit_fn, nsim = 5000, l=10) {
  purrr::map_dfr(seq(nsim), function(x) {
    simulate_pvalue(model, fit_fn, l=l)
  })
}
# Histograms of p values
hist_pvalues <- function(pv) {
  pv |>
    tidyr::pivot_longer(cols = seq(NCOL(arima_pvalues))) |>
    ggplot(aes(x = value)) +
    geom_histogram(bins = 30, boundary = 0) +
    facet_grid(. ~ name) +
    labs(title = "P value distributions")
}
# A nice table of the size of the test
table_pvalues <- function(pv) {
  tibble::tibble(`test size` = c(0.01, 0.05, 0.1)) %>%
    dplyr::bind_cols(
      purrr::map_df(pv, function(x) {ecdf(x)(.$`test size`)})
    ) |>
    knitr::kable()
}

ARIMA models with an intercept

We will simulate from an ARIMA(2,0,0) model with a non-zero intercept. For the Ljung-Box test, we will consider . Note that was the original proposal by Box and Pierce (1970), counts only ARMA coefficients, and counts all parameters estimated in the model. The resulting distributions of the p-values are shown below.

Code

model <- Arima(sqrt(lynx), order=c(2,0,0))
fit_fn <- function(y) {
  Arima(y, order = c(2, 0, 0), include.mean = TRUE)
}
arima_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(arima_pvalues)

Code

table_pvalues(arima_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0046	0.0072	0.0124	0.0220
0.05	0.0226	0.0354	0.0534	0.0818
0.10	0.0474	0.0678	0.1016	0.1514

Clearly the one with is better than the alternatives. The table shows the empirical size of the test for different threshold levels. The empirical sizes are closest to the nominal sizes when . So we shouldn’t count the intercept when computing the degrees of freedom.

Seasonal ARIMA model

Next, we will simulate from an ARIMA(0,1,1)(0,1,1) model, often called the “airline model” due to its application to the Air passenger series in Box et al. (2016). In fact, our DGP for the simulations will be a model fitted to the AirPassengers data set. Again, we consider . There are two parameters to be estimated.

Code

model <- Arima(log(AirPassengers), order=c(0,1,1), seasonal=c(0,1,1))
fit_fn <- function(y) {
  Arima(y, order = c(0,1,1), seasonal=c(0,1,1))
}
sarima_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(sarima_pvalues)

Code

table_pvalues(sarima_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0100	0.0170	0.0282	0.0456
0.05	0.0484	0.0684	0.1018	0.1452
0.10	0.0882	0.1218	0.1774	0.2526

Interesting. Although there are two parameters here, the tests with and do better than . I would have expected to be the right choice, but the test with has empirical size about twice the nominal size.

As a guess, perhaps the seasonal parameters aren’t having an effect with . We can test what happens for larger by setting (covering two years), and repeating the exercise.

Code

sarima_pvalues <- simulate_pvalues(model, fit_fn, l=24)
hist_pvalues(sarima_pvalues)

Code

table_pvalues(sarima_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0134	0.0164	0.0216	0.0278
0.05	0.0492	0.0614	0.0748	0.0940
0.10	0.0864	0.1066	0.1312	0.1590

I was expecting to do best there, but not so. is the most uniform, and gives empirical sizes closest to the nominal sizes, with the results getting worse as increases. Perhaps always setting would be a sensible strategy for ARIMA models, even if they contain seasonal components. This needs some theoretical analysis.

Regression with ARMA errors

We will simulate from a linear trend model with AR(1) errors. Here, counts only ARMA coefficients, while counts all parameters estimated. The resulting distributions of the p-values are shown below.

Code

model <- Arima(10 + seq(250)/10 + arima.sim(list(ar=0.7), n=250),
               order = c(1,0,0), xreg = seq(250))
fit_fn <- function(y) {
  Arima(y, order = c(1,0,0), include.constant = TRUE, xreg = seq(250))
}
regarima_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(regarima_pvalues)

Code

table_pvalues(regarima_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0072	0.0104	0.0176	0.0294
0.05	0.0310	0.0532	0.0782	0.1200
0.10	0.0696	0.0996	0.1478	0.2150

The test with looks the most uniform, with the size of the test closest to the nominal values. So only counting ARMA coefficients seems to be correct here.

Regression model

Next, we will consider a linear trend model with iid errors. That is the same as the previous model, but with a simpler error structure.

Code

model <- Arima(10 + seq(250)/10 + rnorm(250),
               order = c(0,0,0), xreg = seq(250))
fit_fn <- function(y) {
  Arima(y, include.constant = TRUE, xreg = seq(250))
}
trend_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(trend_pvalues)

Code

table_pvalues(trend_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0148	0.0214	0.0338	0.0554
0.05	0.0580	0.0844	0.1204	0.1746
0.10	0.1054	0.1478	0.2062	0.2816

The test with looks best. If we think of a regression model as a RegARIMA model with ARIMA(0,0,0) errors, this is consistent with the previous results, setting .

ETS(A,N,N) model

Now let’s try an ETS(A,N,N) model, again using 5000 series each of length 250. If we count only the smoothing parameter, , but if we count all estimated parameters, . The distributions of p-values are shown below.

Code

model <- ets(fma::strikes, "ANN")
fit_fn <- function(y) {
  ets(y, model = "ANN", damped = FALSE)
}
ets_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(ets_pvalues)

Code

table_pvalues(ets_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0064	0.0112	0.0174	0.0316
0.05	0.0334	0.0522	0.0778	0.1196
0.10	0.0702	0.0960	0.1442	0.2098

looks about right. That makes sense as an ETS(A,N,N) model is equivalent to an ARIMA(0,1,1) model.

ETS(M,N,N) model

Next, let’s try an ETS(M,N,N) model, which has no ARIMA equivalent, but which has one smoothing parameter and one initial state to estimate.

Code

model <- ets(fma::strikes, "MNN")
fit_fn <- function(y) {
  ets(y, model = "MNN", damped = FALSE)
}
ets_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(ets_pvalues)

Code

table_pvalues(ets_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0054	0.0084	0.0172	0.0308
0.05	0.0334	0.0548	0.0814	0.1274
0.10	0.0708	0.1062	0.1520	0.2136

Again, appears to be the best.

ETS(A,A,N) model

An ETS(A,A,N) model is equivalent to an ARIMA(0,2,2) model, so I expect this one to need .

Code

model <- ets(fma::strikes, "AAN")
fit_fn <- function(y) {
  ets(y, model = "AAN", damped = FALSE)
}
ets_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(ets_pvalues)

Code

table_pvalues(ets_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0034	0.0062	0.0106	0.0172
0.05	0.0180	0.0308	0.0514	0.0802
0.10	0.0430	0.0658	0.1008	0.1486

This time, my conjecture is correct, and works well.

ETS(A,A,A) model

Finally, we will check a seasonal ETS model

Code

model <- ets(log(AirPassengers), model="AAA", damped=FALSE)
fit_fn <- function(y) {
  ets(y, model = "AAA", damped = FALSE)
}
ets_pvalues <- simulate_pvalues(model, fit_fn)
hist_pvalues(ets_pvalues)

Code

table_pvalues(ets_pvalues)

test size	K = 0	K = 1	K = 2	K = 3
0.01	0.0178	0.0276	0.0396	0.0602
0.05	0.0634	0.0916	0.1360	0.1942
0.10	0.1206	0.1672	0.2244	0.3078

Here there are 3 smoothing parameters, and 13 initial states to estimate. So I was expecting to do best, but it is the worst. Instead, is the best. I’m not sure what to make of this result.

Conclusions

Based only on this empirical evidence:

For ARIMA models, use degrees of freedom.
For seasonal ARIMA models, it appears that also gives the best results.
For regression with ARIMA errors, use degrees of freedom.
For OLS regression, use degrees of freedom.
For non-seasonal ETS models, use the number of smoothing parameters.
For seasonal ETS models, use .

The last two of these appear to be contradictory, and it is not clear why.

It seems like this might be a good project for a PhD student to explore. In particular, can these suggestions based on empirical evidence be supported theoretically? It would also be good to explore other models such as TBATS, ARFIMA, NNETAR, etc.

For now, I might avoid teaching the Ljung-Box test, and just get students to look at the ACF plot of the residuals instead.

Other literature

Kim et al. (2004) shows that for an AR(1) model with ARCH errors.
McLeod and Li (1983) consider the equivalent test applied to autocorrelations of squared residuals, and show that .
Mahdi (2016) discusses a variation on the LB test for seasonal ARIMA models considering only autocorrelations at the seasonal lags.
Several other portmanteau tests (i.e., based on multiple autocorrelations) are available, and perhaps we should be using them and not the older Ljung-Box test. See Mahdi (2021) for some recent developments.

References

Box, George E P, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2016. Time Series Analysis: Forecasting and Control. 5th ed. John Wiley; Sons.

Box, George E P, and David A Pierce. 1970. “Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models.” Journal of the American Statistical Association 65 (332): 1509–26. https://doi.org/10.2307/2284333.

Harvey, Andrew C. 1990. Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge University Press.

Hyndman, Rob J, and George Athanasopoulos. 2018. Forecasting: Principles and Practice. 2nd ed. OTexts. OTexts.org/fpp2.

Hyndman, Rob J, and George Athanasopoulos. 2021. Forecasting: Principles and Practice. 3rd ed. OTexts. OTexts.org/fpp3.

Kim, Eunhee, Jeongcheol Ha, Youngsook Jeon, and Sangyeol Lee. 2004. “Ljung-Box Test in Unit Root AR-ARCH Model.” Communications for Statistical Applications and Methods 11 (2): 323–27. https://doi.org/10.5351/ckss.2004.11.2.323.

Ljung, Greta M, and George E P Box. 1978. “On a Measure of Lack of Fit in Time Series Models.” Biometrika 65 (2): 297–303. https://doi.org/10.1093/biomet/65.2.297.

Mahdi, Esam. 2016. “Portmanteau Test Statistics for Seasonal Serial Correlation in Time Series Models.” SpringerPlus 5 (1): 1485. https://doi.org/10.1186/s40064-016-3167-4.

Mahdi, Esam. 2021. “New Goodness-of-Fit Tests for Time Series Models.” http://arxiv.org/abs/2008.08176.

McLeod, A I, and W K Li. 1983. “Diagnostic Checking ARMA Time Series Models Using Squared-Residual Autocorrelations.” Journal of Time Series Analysis 4 (4): 269–73. https://doi.org/10.1111/j.1467-9892.1983.tb00373.x.

Forecasting workshops in New York and Chicago

Wed, 12 Apr 2023 14:00:00 GMT

I’m giving a 2-day workshop on “Tidy Time Series and Forecasting in R”, first at the New York R Conference in July, and then at the Posit Conference in Chicago in September.

Places are limited, so please book in early.

New York, 11-12 July 2023. Followed by the New York R conference. (Register for the workshop as part of the NYR conference registration.)
Chicago, 17-18 September 2023. Followed by the Posit conference. (Register for the workshop as part of the Posit conference registration.)

The workshop introduces the tidyverts set of packages. Further details about the workshop are here.

Towards socially responsible forecasting: Identifying and typifying forecasting harms

Tue, 28 Jul 2026 14:00:00 GMT

Data-driven organizations around the world routinely use forecasting methods to improve their planning and decision-making capabilities. Although much research exists on the harms resulting from traditional AI and machine learning applications, little has specifically focused on the ethical impact of time series forecasting. Yet forecasting raises unique ethical issues due to the way it is used in different organizational contexts, supports different goals, and involves different data processing, model development, and evaluation pipelines. These differences make it difficult to apply existing AI and machine learning harm taxonomies to real-world forecasting contexts. We leverage 20 interviews with expert industry practitioners and academic researchers to remedy this knowledge gap by cataloguing and analysing under-explored domains, applications, and scenarios where forecasting may cause harm, with the goal of developing a novel taxonomy of forecasting-specific harms. Inspired by the Microsoft Azure taxonomy for responsible innovation, we combined a human-led inductive coding scheme with an AI-driven analysis centered on the extraction of an initial taxonomy harm in forecasting. We then augment the taxonomy with relevant theoretical considerations from ethics and law, such as the distinction between a harm and wrong. The resulting taxonomy is designed to guide researchers and practitioners and encourage ethical reflection on the impact of their decisions during the forecasting process. A secondary objective is to create a research agenda for socially responsible forecasting that focuses on possible forecasting-related measures to mitigate harm. Our work extends the growing literature on AI and machine learning harms by empirically identifying unique forms of harm that may occur in forecasting and linking them with traditional concepts of responsibility from ethics and law.

Cite

@unpublished{harms,
  author = {Bahman Rostami-Tabar and Travis Greene and Galit Shmueli and Rob J Hyndman},
  title  = {Towards socially responsible forecasting: Identifying and typifying forecasting harms},
  url    = {https://robjhyndman.com/publications/harms.html},
  year   = {2025}
}

Winter forecasting of respiratory viruses in Victoria Australia

Wed, 20 May 2026 14:00:00 GMT

Temperate regions of the world, such as southern Australia, often experience increased health burden from respiratory pathogens during winter. The ability to forecast short-term trends in cases of these pathogens is of significant interest to public health. Across the 2024 southern hemisphere winter period, the Australia–Aotearoa Consortium for Epidemic Forecasting and Analytics (ACEFA) ran a pilot respiratory virus forecasting initiative in collaboration with the Victorian Department of Health. Each week from the 9th of May 2024 through to 12th September 2024, the consortium solicited 28-day forecasts of daily case incidence for influenza, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and respiratory syncytial virus (RSV) from multiple research groups. Four component model forecasts were contributed by three different research groups, with a fourth group utilising the component forecasts to generate ensemble forecasts (making a total of six models, four component models and two ensembles). The pilot programme confirmed the feasibility of a multi-pathogen collaborative forecasting hub in the Australia setting. A statistical performance evaluation of contributing forecasts against the observed case data found that performance varied by pathogen and epidemic phase, and the two ensemble models were frequently the top performing models.

Cite

@unpublished{winterforecasting,
  author = {Alec S Henderson and Robert Moss and Adeshina I Adekunle and Henry Ye and Mitchell O’Hara-Wild and Oliver Eales and Katharine L Senior and Ruarai Tobin and Saras M Windecker and Nick Golding and Ellie Robinson and Janet Strachan and Rob J Hyndman and Peter Dawson and James M. McCaw and Emma S McBryde and Freya M Shearer},
  doi    = {10.64898/2026.05.18.26353544},
  title  = {Winter forecasting of respiratory viruses in Victoria Australia},
  year   = {2026}
}

Multivariate reconciliation for hierarchical time series

Sun, 17 May 2026 14:00:00 GMT

Some time series can be hierarchically organized into levels based on certain characteristics, such as geography or other attributes of interest. These series are referred to as hierarchical time series. Typically, forecasts are generated at all levels to ensure coherence, meaning that the forecasts should satisfy the same aggregation constraints as the observed data. Various approaches have been proposed to guarantee this coherence by using a set of base forecasts. The process through which these forecasts are adjusted to become coherent is known as forecast reconciliation. Similar to the univariate case, multivariate time series can also be structured hierarchically. However, all existing approaches are limited to a single variable. As a result, ensuring coherent forecasts requires reconciling each variable separately. However, this process does not account for correlations among multiple variables. To address this limitation, this paper proposes a multivariate reconciliation methodology that ensures coherent forecasts and incorporates relationships among variables. The proposed methodology was tested through numerical simulations, considering distinct scenarios within the series hierarchy and across multiple variables. Additionally, some base forecasting models were evaluated. The methodology was also applied to real employment data of admissions and dismissals in Brazil. The results demonstrated that multivariate reconciliation yielded more accurate outcomes than the other methods considered, both in simulated data and in practical applications.

Cite

@unpublished{mvhts,
  author = {Ana Caroline Pinheiro and Rodrigo de Souza Bulhões and Rob J Hyndman and Paulo Canas Rodrigues},
  title  = {Multivariate reconciliation for hierarchical time series},
  url    = {https://robjhyndman.com/publications/mvhts.html},
  year   = {2026}
}

Ensemble forecasts of COVID-19 activity to support Australia’s pandemic response: 2020–22

Tue, 21 Apr 2026 14:00:00 GMT

During the COVID-19 pandemic, many countries used real-time data analyses, predictive modelling, and COVID-19 case forecasts, to incorporate emerging evidence into their decisions. In Australia, national and jurisdictional public health responses were informed by weekly ensemble forecasts of daily COVID-19 case counts for each of Australia’s eight states and territories, produced by a consortium of researchers under contract with the Australian Government. As members of this consortium, who produced these forecasts at each week, we now retrospectively evaluate approximately 100,000 predictions for daily case counts 1-28 days into the future, generated between July 2020 and December 2022, and report here (a) how the ensemble forecasts supported public health responses; (b) how well the ensemble forecast performed, relative to the forecasts produced by each contributing team; and (c) how we refined our reporting and visualisations to ensure that outputs were interpreted appropriately. Similar to COVID-19 forecasting studies in other countries, we found that the ensemble forecast consistently out-performed the individual model forecasts, and that performance was lowest when there were rapid changes in the epidemiology, such as periods around epidemic peaks. Our consortium’s internal peer-review process allowed us to explain how features of each ensemble forecast related to the design of the individual models, and this helped enable public health stakeholders to interpret the forecasts appropriately. Ultimately, our forecasts provided information that supported public health responses during periods of different policy goals, and over a wide range of epidemic scenarios.

Cite

@article{covidensemble,
  author  = {Robert Moss and Ruarai J Tobin and Mitchell O'Hara-Wild and Adeshina I Adekunle and Dennis Liu and Tobin South and Dylan J Morris and Gerard E Ryan and Tianxiao Hao and Aarathy Babu and Katharine L Senior and James G Wood and Nick Golding and Joshua V Ross and Peter Dawson and Rob J Hyndman and David J Price and James M McCaw and Freya M Shearer},
  doi     = {10.1371/journal.pcbi.1014199},
  journal = {PLOS Computational Biology},
  number  = 4,
  pages   = {e1014199},
  title   = {Ensemble forecasts of {COVID-19} activity to support {Australia}'s pandemic response: 2020--22},
  url     = {https://robjhyndman.com/publications/covid-ensemble.html},
  volume  = 22,
  year    = 2026
}

When lookout sees crackle: Anomaly detection via kernel density estimation

Mon, 23 Mar 2026 13:00:00 GMT

We present an updated version of lookout — an algorithm for detecting anomalies using kernel density estimates with bandwidth based on Rips death diameters — with theoretical guarantees. The kernel density estimator for updated lookout is shown to be consistent, and the proposed multivariate scaling is robust and efficient. We show our updated algorithm performs better than the previous version on diverse examples.

The R package lookout implements this algorithm.

Cite

@unpublished{lookout2,
  author = {Rob J Hyndman and Sevvandi Kandanaarachchi and Katharine Turner},
  title  = {When lookout sees crackle: Anomaly detection via kernel density estimation},
  url    = {http://robjhyndman.com/publications/lookout2.html},
  year   = 2026
}

Anomaly detection using surprisals

Mon, 09 Mar 2026 13:00:00 GMT

Anomaly detection methods are widely used but often rely on ad hoc rules or strong assumptions, and they often focus on tail events, missing “inlier” anomalies that occur in low-density gaps between modes. We propose a unified framework that defines an anomaly as an observation with unusually low probability under a (possibly misspecified) model. For each observation we compute its surprisal (the negative log generalized density) and define an anomaly score as the probability of a surprisal at least as large as that observed. This reduces anomaly detection for complex univariate or multivariate data to estimating the upper tail of a univariate surprisal distribution. We develop two model-robust estimators of these tail probabilities: an empirical estimator based on the observed surprisal distribution and an extreme-value estimator that fits a Generalized Pareto Distribution above a high threshold. For the empirical method we give conditions under which tail ordering is preserved and derive finite-sample confidence guarantees via the Dvoretzky–Kiefer–Wolfowitz inequality. For the GPD method we establish broad tail conditions ensuring classical extreme-value behavior. Simulations and applications to French mortality and Test-cricket data show the approach remains effective under substantial model misspecification.

The R package weird implements the methods described in this paper.

Cite

@techreport{surprisals,
  author      = {Rob J Hyndman and David T Frazier},
  institution = {Department of Econometrics \& Business Statistics, Monash University},
  title       = {Anomaly detection using surprisals},
  type        = {Working paper},
  url         = {http://robjhyndman.com/publications/surprisals.html},
  year        = 2026
}

Forecasting the age structure of the scientific workforce in Australia

Sat, 21 Feb 2026 13:00:00 GMT

Planning for a future workforce requires forecasts of age structure changes to inform policy decisions, particularly related to universities and immigration. We propose a new dynamic statistical model for forecasting the age structure of a workforce. Our approach is inspired by a stochastic model used in population forecasting, replacing births with graduate entry, modelling exits through death and retirement, and including a remainder term that captures migration and career changes. Functional data models are used to model age-specific components, while ARIMA models are used for time series components. Simulation is employed to generate forecast distributions, capturing uncertainty from all components. The approach is illustrated using data on Australia’s scientific workforce, allowing us to forecast the age distribution of various scientific disciplines for the next ten years. This analysis was central to an Australian Academy of Science initiative examining the capability of Australia’s science system and identifying workforce gaps.

Cite

@article{asf,
  author  = {Rob J Hyndman and Khuyen Vanh Nguyen},
  doi     = {10.1111/anzs.70043},
  journal = {Australian \& New Zealand J Statistics},
  number  = 1,
  pages   = {1369-1473},
  title   = {Forecasting the age structure of the scientific workforce in Australia},
  url     = {http://robjhyndman.com/publications/asf},
  volume  = 68,
  year    = 2026
}

ggtime: A grammar of temporal graphics

Wed, 29 Oct 2025 13:00:00 GMT

Visualizing changes over time is fundamental to learning from the past and anticipating the future. However, temporal semantics can be complicated, and existing visualization tools often struggle to accurately represent these complexities. It is common to use bespoke plot helper functions designed to produce specific graphics, due to the absence of flexible general tools that respect temporal semantics. We address this problem by proposing a grammar of temporal graphics, and an associated software implementation, ggtime, that encodes temporal semantics into a declarative grammar for visualizing temporal data. The grammar introduces new composable elements that support visualization across linear, cyclical, quasi-cyclical, and other granularities; standardization of irregular durations; and alignment of time points across different granularities and time zones. It is designed for interoperability with other semantic variables, allowing navigation across the space of visualizations while preserving temporal semantics.

R package: ggtime

Cite

@techreport{ggtime,
  author      = {Cynthia A Huang and Mitchell O'Hara-Wild and Rob J Hyndman and Matthew Kay},
  institution = {Department of Econometrics \& Business Statistics, Monash University},
  title       = {ggtime: A grammar of temporal graphics},
  type        = {Working paper},
  url         = {http://robjhyndman.com/publications/ggtime.html},
  year        = 2025
}

Distortion corrected kernel density estimator on Riemannian manifolds

Sun, 31 Aug 2025 14:00:00 GMT

Manifold learning obtains a low-dimensional representation of an underlying Riemannian manifold supporting high-dimensional data. Kernel density estimates of the low-dimensional embedding with a fixed bandwidth fail to account for the way manifold learning algorithms distort the geometry of the Riemannian manifold. We propose a novel distortion-corrected kernel density estimator (DC-KDE) for any manifold learning embedding, with a bandwidth that depends on the estimated Riemannian metric at each data point. Exploiting the geometric information of the manifold leads to more accurate density estimation, which subsequently could be used for anomaly detection. To compare our proposed estimator with a fixed-bandwidth kernel density estimator, we run two simulations including one with data lying in a 100 dimensional ambient space. We demonstrate that the proposed DC-KDE improves the density estimates as long as the manifold learning embedding is of sufficient quality, and has higher rank correlations with the true manifold density. Further simulation results are provided via a supplementary R shiny app. The proposed method is applied to density estimation in statistical manifolds of electricity usage with the Irish smart meter data.

Cite

@article{kderm,
  author  = {Fan Cheng and Rob J Hyndman and Anastasios Panagiotelis},
  doi     = {10.1080/10618600.2024.2415543},
  journal = {J Computational \& Graphical Statistics},
  number  = 3,
  pages   = {906-922},
  title   = {Distortion corrected kernel density estimator on Riemannian manifolds},
  url     = {https://robjhyndman.com/publications/kderm.html},
  volume  = 34,
  year    = 2025
}

Moving averages

Wed, 18 Jun 2025 14:00:00 GMT

Cite

@incollection{IESS25a,
  author    = {Rob J Hyndman},
  booktitle = {International Encyclopedia of Statistical Science},
  doi       = {10.1007/978-3-662-69359-9_383},
  edition   = {2nd ed},
  editor    = {Miodrag Lovric},
  pages     = {1559-1562},
  publisher = {Springer},
  title     = {Moving averages},
  year      = 2025
}

Business forecasting methods

Wed, 18 Jun 2025 14:00:00 GMT

Cite

@incollection{IESS25c,
  author    = {Rob J Hyndman},
  booktitle = {International Encyclopedia of Statistical Science},
  doi       = {10.1007/978-3-662-69359-9_83},
  edition   = {2nd ed},
  editor    = {Miodrag Lovric},
  pages     = {362-372},
  publisher = {Springer},
  title     = {Business forecasting methods},
  year      = 2025
}

Forecasting overview

Wed, 18 Jun 2025 14:00:00 GMT

Cite

@incollection{IESS25b,
  author    = {Rob J Hyndman},
  booktitle = {International Encyclopedia of Statistical Science},
  doi       = {10.1007/978-3-662-69359-9_234},
  edition   = {2nd ed},
  editor    = {Miodrag Lovric},
  pages     = {964-968},
  publisher = {Springer},
  title     = {Forecasting overview},
  year      = 2025
}

Hierarchical time series forecasting in emergency medical services

Thu, 01 May 2025 14:00:00 GMT

Accurate forecasts of ambulance demand are crucial inputs when planning and deploying staff and fleet. Such demand forecasts are required at national, regional and sub-regional levels, and must take account of the nature of incidents and their priorities. These forecasts are often generated independently by different teams within the organization. As a result, forecasts at different levels may be inconsistent, resulting in conflicting decisions and a lack of coherent coordination in the service. To address this issue, we exploit the hierarchical and grouped structure of the demand time series, and apply forecast reconciliation methods to generate both point and probabilistic forecasts that are coherent and use all the available data at all levels of disaggregation. The methods are applied to daily incident data from an ambulance service in Great Britain, from October 2015 to July 2019, disaggregated by nature of incident, priority, managing health board, and control area. We use an ensemble of forecasting models, and show that the resulting forecasts are better than any individual forecasting model. We validate the forecasting approach using time-series cross-validation.

Cite

@article{fem,
  author  = {Bahman Rostami-Tabar and Rob J Hyndman},
  doi     = {10.1177/10946705241232169},
  journal = {Journal of Service Research},
  number  = 4,
  pages   = {278-295},
  title   = {Hierarchical time series forecasting in emergency medical services},
  url     = {http://robjhyndman.com/publications/fem.html},
  volume  = 28,
  year    = {2025}
}

MSTL: A Seasonal-Trend Decomposition Algorithm for Time Series with Multiple Seasonal Patterns

Wed, 15 Jan 2025 13:00:00 GMT

The decomposition of time series into components is an important task that helps to understand time series and can enable better forecasting. Nowadays, with high sampling rates leading to high-frequency data (such as daily, hourly, or minutely data), many datasets contain time series data that can exhibit multiple seasonal patterns. Although several methods have been proposed to decompose time series better under these circumstances, they are often computationally inefficient or inaccurate. We propose a procedure to decompose time series with multiple seasonal patterns that is suited to a wide range of high-frequency data. The procedure for Multiple Seasonal Trend decomposition (MSTL) introduced in this paper extends the traditional Seasonal-Trend decomposition using Loess (STL) algorithm, allowing the decomposition of time series with multiple seasonal patterns. In our evaluation on synthetic and a perturbed real-world time series dataset, compared to other decomposition benchmarks, MSTL demonstrates competitive results with lower computational cost. The implementation of MSTL is available in the R packages forecast and feasts.

Cite

@article{mstl,
  author  = {Kasun Bandara and Rob J Hyndman and Christoph Bergmeir},
  doi     = {10.1504/IJOR.2022.10048281},
  journal = {International J Operational Research},
  number  = 1,
  title   = {{MSTL}: A Seasonal-Trend Decomposition Algorithm for Time Series with Multiple Seasonal Patterns},
  url     = {http://robjhyndman.com/publications/mstl/},
  volume  = {52},
  year    = 2025
}

Comments on: Exploratory functional data analysis

Tue, 07 Jan 2025 13:00:00 GMT

Commentary on Qu, Dai, Euan, Sun and Genton (2025) Exploratory Functional Data Analysis, TEST, 34, 459-482.

Cite

@article{efda,
  author  = {Rob J Hyndman},
  doi     = {10.1007/s11749-025-00963-z},
  journal = {TEST},
  pages   = {483-487},
  title   = {Comments on: Exploratory functional data analysis},
  volume  = 34,
  year    = 2025
}

Optimal forecast reconciliation with time series selection

Tue, 31 Dec 2024 13:00:00 GMT

Forecast reconciliation ensures forecasts of time series in a hierarchy adhere to aggregation constraints, enabling aligned decision making. While forecast reconciliation can enhance overall accuracy in a hierarchical or grouped structure, it can lead to worse forecasts for certain series, with the greatest gains typically seen in series that originally have poorly performing base forecasts. In practical applications, some series in a structure often produce poor base forecasts due to model misspecification or low forecastability. To mitigate their negative impact, we propose two categories of forecast reconciliation methods that incorporate automatic time series selection based on out-of-sample and in-sample information, respectively. These methods keep “poor” base forecasts unused in forming reconciled forecasts, while adjusting the weights assigned to the remaining series accordingly when generating bottom-level reconciled forecasts. Additionally, our methods ameliorate disparities stemming from varied estimators of the base forecast error covariance matrix, alleviating challenges associated with estimator selection. Empirical evaluations through two simulation studies and applications using Australian labour force and domestic tourism data demonstrate the potential of the proposed methods to exclude series with high scaled forecast errors and show promising results.

Cite

@article{hfselection,
  author  = {Xiaoqian Wang and Rob J Hyndman and Shanika L Wickramasuriya},
  doi     = {10.1016/j.ejor.2024.12.004},
  journal = {European J Operational Research},
  number  = 2,
  pages   = {455-470},
  title   = {Optimal forecast reconciliation with time series selection},
  url     = {http://robjhyndman.com/publications/hf_selection.html},
  volume  = 323,
  year    = 2025
}

Extreme value modelling of feature residuals for anomaly detection in dynamic graphs

Thu, 21 Nov 2024 13:00:00 GMT

Detecting anomalies in a temporal sequence of graphs can be applied is areas such as the detection of accidents in transport networks and cyber attacks in computer networks. Existing methods for detecting abnormal graphs can suffer from multiple limitations, such as high false positive rates as well as difficulties with handling variable-sized graphs and non-trivial temporal dynamics. To address this, we propose a technique where temporal dependencies are explicitly modelled via time series analysis of a large set of pertinent graph features, followed by using residuals to remove the dependencies. Extreme Value Theory is then used to robustly model and classify any remaining extremes, aiming to produce low false positives rates. Comparative evaluations on a multitude of graph instances show that the proposed approach obtains considerably better accuracy than TensorSplat and Laplacian Anomaly Detection.

Cite

@inproceedings{oddnet2,
  author    = {Sevvandi Kandanaarachchi and Conrad Sanderson and Rob J Hyndman},
  booktitle = {2024 11th International Conference on Soft Computing \& Machine Intelligence (ISCMI)},
  doi       = {10.1109/ISCMI63661.2024.10851659},
  pages     = {32-37},
  title     = {Extreme value modelling of feature residuals for anomaly detection in dynamic graphs},
  url       = {https://robjhyndman.com/publications/oddnet2.html},
  year      = 2024
}

Functional data analysis for peak shape forecasting

Fri, 15 Nov 2024 13:00:00 GMT

Peak load forecasting is critical for power system operations and planning. Electric utilities are increasingly adopting peak-shaving strategies as they look for sustainable solutions to improve operational excellence. During the past few decades, hourly load forecasts have been used in practice as the input to inform peak-shaving decisions. The BigDEAL Challenge 2022 first proposed the concept of peak (load) shape forecasting, which showed that a better hourly load forecast may not result in a better peak shape forecast. This paper tackles peak shape forecasting, an under-served aspect of peak load forecasting, which is critical for deploying those peak shaving programs. We adopt functional data analysis in exploring new approaches to peak shape forecasting. Using the data published by ISONE, the experiments show that the peak shape forecasts are more robust when compared with the state-of-the-art hourly forecasting models, reducing the errors by 14–18%. The experiment also demonstrates that no single model excels across all attributes, emphasizing the need to build task-specific models.

Cite

@unpublished{fdashape,
  author = {Shreyashi Shukla and Tao Hong and Rob J Hyndman},
  title  = {Functional data analysis for peak shape forecasting},
  url    = {https://robjhyndman.com/publications/fdashape},
  year   = {2024}
}

Online conformal inference for multi-step time series forecasting

Thu, 17 Oct 2024 13:00:00 GMT

We consider the problem of constructing distribution-free prediction intervals for multi-step time series forecasting, with a focus on the temporal dependencies inherent in multi-step forecast errors. We establish that the optimal -step-ahead forecast errors exhibit serial correlation up to lag under a general non-stationary autoregressive data generating process. To leverage these properties, we propose the Autocorrelated Multi-step Conformal Prediction (AcMCP) method, which effectively incorporates autocorrelations in multi-step forecast errors, resulting in more statistically efficient prediction intervals. This method ensures theoretical long-run coverage guarantees for multi-step prediction intervals, though we note that increased forecasting horizons may exacerbate deviations from the target coverage, particularly in the context of limited sample sizes. Additionally, we extend several easy-to-implement conformal prediction methods, originally designed for single-step forecasting, to accommodate multi-step scenarios. Through empirical evaluations, including simulations and applications to data, we demonstrate that AcMCP achieves coverage that closely aligns with the target within local windows, while providing adaptive prediction intervals that effectively respond to varying conditions.

R package: conformalForecast

Cite

@techreport{cpts,
  author      = {Xiaoqian Wang and Rob J Hyndman},
  institution = {Department of Econometrics \& Business Statistics, Monash University},
  number      = {20-24},
  title       = {Online conformal inference for multi-step time series forecasting},
  type        = {Working paper},
  url         = {http://robjhyndman.com/publications/cpts.html},
  year        = 2024
}

Improving out-of-sample forecasts of stock price indexes with forecast reconciliation and clustering

Thu, 19 Sep 2024 14:00:00 GMT

We propose a novel approach to improving forecasts of stock market indexes by considering common stock prices as hierarchical time series, combining clustering with forecast reconciliation. We propose to group the individual stock price series in various ways including via metadata and using unsupervised learning techniques. The proposed approach is applied to the Dow Jones Industrial Average Index and the Standard & Poor 500 Index and their component stocks, and the results obtained with different grouping approaches are compared. The results demonstrate empirically that the combined use of clustering and reconciliation improves the forecast accuracy of the stock market indexes and their constituents.

Cite

@article{dowhts,
  author  = {Raffaele Mattera and George Athanasopoulos and Rob J Hyndman},
  doi     = {10.1080/14697688.2024.2412687},
  journal = {Quantitative Finance},
  number  = 11,
  pages   = {1641-1667},
  title   = {Improving out-of-sample forecasts of stock price indexes with forecast reconciliation and clustering},
  url     = {https://robjhyndman.com/publications/dow_hts.html},
  volume  = 24,
  year    = 2024
}

Surprises in time series analysis

Sun, 25 Oct 2026 13:00:00 GMT

Part of Time Series Connect webinar series

Monday 26 October 2026, 6:00 PM Eastern Time

Abstract

I will present a statistical framework for identifying anomalies in three time series settings: unusual observations within a single historical series, unusual series within a large collection, and real-time surveillance of incoming data streams.

In each case, anomalies are identified using a probabilistic approach based on ‘surprisal values’ — equal to minus the log (conditional) density of each observation — with extreme value theory used to model the tail of the surprisal distribution and avoid strong parametric assumptions.

The methods will be illustrated using the {weird} package for R, with examples including French and US mortality rates, and pharmaceutical sales.

Slides

Open slides in a new tab

Software

References

Hyndman, Rob J. 2026. That’s Weird: Anomaly Detection Using R. https://OTexts.com/weird.

Hyndman, Rob J, and David T Frazier. 2026. “Anomaly Detection Using Surprisals.” http://robjhyndman.com/publications/surprisals.html.

Hyndman, Rob J, Sevvandi Kandanaarachchi, and Katharine Turner. 2026. “When Lookout Sees Crackle: Anomaly Detection via Kernel Density Estimation.” http://robjhyndman.com/publications/lookout2.html.

Formatting and linting code and documents

Mon, 17 Aug 2026 14:00:00 GMT

NUMBAT seminar

Consistently formatted source is easier to read, review and diff, and linters alert you to problems before they become bugs. I will survey and demonstrate the current generation of fast, opinionated formatters and linters — mostly written in Rust, and usable from the command line or from within Positron (and in some cases from RStudio). I’ll focus on air for R and panache for Quarto and R Markdown. If time, I might also mention jarl for R linting, ruff for Python, tex-fmt and chktex for LaTeX, and vale for prose (checking manuscripts against a style guide). I’ll show how to use TOML configuration and format-on-save, so that your code and documents stay tidy with no ongoing effort.

R

Google style guide
Tidyverse style guide
styler: R code formatting
formatR: R code formatting
air: R code formatting
Jarl: R code linting

Python

Ruff: Python linting and formatting

Quarto and Rmarkdown

Panache: Markdown, quarto and Rmarkdown linting and formatting

LaTeX

latexindent: LaTeX formatting
tex-fmt: LaTeX formatting
chktex: LaTeX linting

Other

Prettier: JavaScript, TypeScript, HTML, CSS, JSON, Markdown
Vale: Prose linting

Slides

Download pdf

Spotting the weird ones

Wed, 05 Aug 2026 14:00:00 GMT

Ihaka Lecture Series 2026

Abstract

Data analysis is about finding the stories hidden in the mass of information that make up a data set. Usually we are interested in understanding the major patterns and the relationships that hold true for most of the data. But sometimes we need to look at the weird observations, the mavericks, the ones that don’t follow the crowd and behave differently. These observations are called ‘anomalies’. Many anomaly detection methods have been developed, but they are often based on ad hoc rules and hidden assumptions, and can lead to misleading conclusions.

Instead, I will describe a principled statistical approach to identifying anomalies in diverse data sets, from simple numerical data to complex high-dimensional data objects. The ideas will be illustrated using the {weird} package for R applied to several data analysis problems, including spotting over-priced wines, identifying errors in the records of the Old Faithful Geyser, and uncovering forgotten epidemics in 19th century France.

Software

Book

Hyndman, Rob J. 2026. That’s Weird: Anomaly Detection Using R. https://OTexts.com/weird.

Papers

Hyndman, Rob J, and David T Frazier. 2026. “Anomaly Detection Using Surprisals.” http://robjhyndman.com/publications/surprisals.html.

Kandanaarachchi, Sevvandi, and Rob J Hyndman. 2022. “Leave-One-Out Kernel Density Estimates for Outlier Detection.” J Computational & Graphical Statistics 31: 586–99. https://robjhyndman.com/publications/lookout/.

Slides

Download pdf

YouTube

Developing good research habits

Mon, 30 Mar 2026 13:00:00 GMT

Download pdf

Anomaly detection using surprisals

Mon, 27 Oct 2025 13:00:00 GMT

Abstract

I will discuss a probabilistic approach to anomaly detection based on extreme ‘surprisal values’ aka log scores, equal to minus the log density at each observation. The surprisal approach can be used for any collection of data objects, provided a probability density can be defined on the sample space. It can distinguish anomalies from legitimate observations in a heavy tail, and will identify anomalies that are undetected using methods based on distance measures. I will demonstrate the idea in various real data examples including univariate, multivariate and regression contexts, and when exploring more complicated data objects. I will also briefly outline the underlying theory when the density is known, and when it is estimated using a kernel density estimate. In the latter case, an innovative bandwidth selection method is used based on persistent homology.

Given at

TopTime Conference, Australian National University, Canberra, Australia: 28 October 2025.
Frontiers of Statistical Inference 2025, MBZUAI, Abu Dhabi, UAE: 26 November 2025.

Slides

Download pdf

Software

References

Hyndman, Rob J. 2026. That’s Weird: Anomaly Detection Using R. https://OTexts.com/weird.

Hyndman, Rob J, and David T Frazier. 2026. “Anomaly Detection Using Surprisals.” http://robjhyndman.com/publications/surprisals.html.

ATAR allocation and participation rates

Sun, 21 Sep 2025 14:00:00 GMT

Download pdf

Efficient reproducible workflows with R

Wed, 02 Jul 2025 14:00:00 GMT

I will discuss and demonstrate several digital tools that underpin the efficiency and reproducibility of a modern research workflow. The workflow will combine the renv package for managing project-specific R dependencies, the targets package to avoid unnecessary computation, and quarto for generating multiple versions of papers for online repositories and journal submissions, ensuring seamless integration of code and text, without duplication. I’ll illustrate the workflow using a recent paper I coauthored on “Forecasting interrupted time series”.

Tools covered

Example paper

Slides

Download pdf

Forecasting the age structure of the scientific workforce in Australia

Sun, 29 Jun 2025 14:00:00 GMT

Abstract

I will demonstrate how to forecast the age structure of a workforce using demographic growth-balance equations that incorporate functional data components for graduate influx, international mobility, retirement, and mortality. This model adapts the stochastic population framework proposed by Hyndman and Booth (IJF, 2008), substituting the birth component at age 0 with a graduation process at any age, and integrating retirement dynamics alongside a standard mortality process. It accommodates variable graduate influx, international mobility, and aging, through functional data components.

The model is illustrated by forecasting the future age structure of scientific disciplines in Australia, utilising a variety of data sources. Data limitations are addressed through cohort interpolation and constrained penalised regression splines in order to obtain a workable data set. A combination of local and global ARIMA models is employed to address the temporal dynamics in the time series components.

By forecasting age structures in disciplines such as Physics and Astronomy, Mathematical Sciences, Chemical Sciences, Earth Sciences, and Biological Sciences, the model informs strategic workforce planning and policy development.

This forecasting framework not only aids in strategic planning and policy formulation within the scientific community, but also offers a blueprint for other professional sectors seeking to understand workforce dynamics amid demographic shifts.

Given at

International Symposium on Forecasting 2025, Beijing, China: 30 June.
Australian Statistics Conference 2025, Perth, Australia: 1-5 December 2025.

Slides

Download pdf

References

Hyndman, Rob J, and Heather Booth. 2008. “Stochastic Population Forecasts Using Functional Data Models for Mortality, Fertility and Migration.” International J Forecasting 24 (3): 323–42. https://robjhyndman.com/publications/stochastic-population-forecasts.

Hyndman, Rob J, and Kelly Vanh Nguyen. 2025. Forecasting the Age Structure of the Scientific Workforce in Australia. Working paper. Department of Econometrics & Business Statistics, Monash University. http://robjhyndman.com/publications/asf.

25 years of open source forecasting software

Wed, 25 Jun 2025 14:00:00 GMT

IIF Workshop on Open Source Forecasting, Beihang University: 25-27 June 2025

Slides

Download pdf

Top 10 most downloaded R packages

R packages for forecast reconciliation

Python packages with statistical forecasting models

Python packages with ML forecasting methods

Python packages for forecast reconciliation

Foundation forecasting models

Forecasting data

Monash time series forecasting repository

Books

Forecasting: principles & practice, 2nd ed, Hyndman & Athanasopoulos, 2018 – using the forecast package for R
Forecasting: principles & practice, 3rd ed, Hyndman & Athanasopoulos, 2021 – using the tsibble, feasts and fable packages for R
Forecasting: principles & practice, the Pythonic way, Hyndman, Athanasopoulos, Garza, Challu, Mergenthaler, Olivares, 2025 – using the Nixtla Python packages
Forecasting and Analytics with the Augmented Dynamic Adaptive Model (ADAM), Svetunkov, 2025 – using the smooth and greybox packages for R

Modern research tools and workflow

Sun, 29 Dec 2024 13:00:00 GMT

I will discuss and demonstrate several digital tools that underpin the efficiency and reproducibility of a modern research workflow.

Tools covered

Citing

Searching

Scripting

Saving

Caching

targets

Writing

Example paper

Slides

Download pdf

An efficient reproducible workflow

Mon, 25 Nov 2024 13:00:00 GMT

Talk given at Research Bazaar Victoria, Clayton, Australia.

Abstract

I will demonstrate the digital tools that underpin the efficiency and reproducibility of my research workflow. Utilizing GitHub as the platform for version control and collaboration, and R as a foundational tool, I leverage the renv package for managing project-specific R dependencies, and the targets package to avoid unnecessary computation. I employ quarto for generating multiple versions of my research papers for online repositories and journal submissions, ensuring seamless integration of code and text, without duplication. I’ll illustrate the workflow using a recent paper I coauthored on “Forecasting interrupted time series”.

Slides

Download pdf

Creating custom quarto templates

Tue, 22 Oct 2024 13:00:00 GMT

Talk for WOMBAT 2024

Abstract

I will describe several quarto templates we have made for use at Monash University to create presentations, working papers, theses, exams, slides, reports, memos, and letters. These help make our outputs look professional and consistent. They are relatively easy to make if you know a little LaTeX (for pdf output) or CSS (for html output). More advanced customization is possible with the help of Lua. I will show how you can easily adapt our quarto templates to other organizations, and how to make your own templates from scratch.

Templates: https://github.com/quarto-monash

Slides

Download pdf

Improving forecasts via subspace projections

Wed, 18 Sep 2024 14:00:00 GMT

Abstract

Univariate, multivariate, and hierarchical forecasts can all be improved using projections onto linear subspaces, regardless of what forecasting method is used. I will show some theoretical guarantees of this statement, and demonstrate using empirical applications how linear projections can lead to (sometimes dramatic) improvements in forecast accuracy.

The procedure involves creating new time series that are linear combinations of the observed time series. These are then forecast, along with the original series, and all forecasts are adjusted via a projection matrix to ensure they satisfy the linear constraints.

This procedure is now widely used for hierarchical forecasting. It has the potential to revolutionise multivariate forecasting as well.

Given at

Curtin University, Perth, Australia: 19 September 2024
International Statistics Conference 2024, Sri Lanka: 28 December 2024
Amazon “Forecasting, Macroeconomics & Finance” seminar, online: 21 February 2025
3rd Vienna Workshop on Economic Forecasting, Institute for Advanced Studies, Vienna, Austria: 6 June 2025
International Forum on Statistics, Renmin University, China: 5 July 2025

Slides (Sri Lanka)

Download pdf

Software

References

Athanasopoulos, George, Rob J Hyndman, Nikolaos Kourentzes, and Fotios Petropoulos. 2017. “Forecasting with Temporal Hierarchies.” European J Operational Research 262 (1): 60–74. https://robjhyndman.com/publications/temporal-hierarchies.

Girolimetto, Daniele, George Athanasopoulos, Tommaso Di Fonzo, and Rob J Hyndman. 2024. “Cross-Temporal Probabilistic Forecast Reconciliation: Methodological and Practical Issues.” International J Forecasting 40 (3): 1134–51. http://robjhyndman.com/publications/ctprob.html.

Hyndman, Rob J, Roman A Ahmed, George Athanasopoulos, and Han Lin Shang. 2011. “Optimal Combination Forecasts for Hierarchical Time Series.” Computational Statistics & Data Analysis 55 (9): 2579–89. https://robjhyndman.com/publications/hierarchical/.

Panagiotelis, Anastasios, Puwasala Gamakumara, George Athanasopoulos, and Rob J Hyndman. 2021. “Forecast Reconciliation: A Geometric View with New Insights on Bias Correction.” International J Forecasting 37 (1): 343–59. https://robjhyndman.com/publications/hierarchical-geometry.

Panagiotelis, Anastasios, Puwasala Gamakumara, George Athanasopoulos, and Rob J Hyndman. 2023. “Probabilistic Forecast Reconciliation: Properties, Evaluation and Score Optimisation.” European J Operational Research 306 (2): 693–706. https://robjhyndman.com/publications/coherentprob.

Wickramasuriya, Shanika L. 2021. “Properties of Point Forecast Reconciliation Approaches.” arXiv Preprint arXiv:2103.11129. https://arxiv.org/abs/2103.11129.

Wickramasuriya, Shanika L, George Athanasopoulos, and Rob J Hyndman. 2019. “Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization.” J American Statistical Association 114 (526): 804–19. https://robjhyndman.com/publications/mint.

Yang, Yangzhuoran Fin, George Athanasopoulos, Rob J Hyndman, and Anastasios Panagiotelis. 2024. “Forecast Linear Augmented Projection (FLAP): A Free Lunch to Reduce Forecast Error Variance.” Working paper. https://robjhyndman.com/publications/flap.html.

vital: Tidy data analysis for demography

Wed, 11 Sep 2024 14:00:00 GMT

NUMBAT seminar

I will introduce the vital package which allows analysis of demographic data using tidy tools. The package uses a variation of tsibble objects as the main data class, so all of the infrastructure available for tsibble and tibble objects can also be used with vital objects. Data may include births, deaths, mortality, fertility, population and migration data. Functions for plotting, smoothing, modelling and forecasting data are included. Models include the classical Lee-Carter model as well as functional data models. Future plans include replicating all of the models available in the demography and StMoMo packages. The package is currently available at pkg.robjhyndman.com/vital/ and on CRAN.

R package

vital

Slides

Download pdf

Forecasting the future and the future of forecasting

Tue, 10 Sep 2024 14:00:00 GMT

Seminar for the OPTIMA ARC training centre

Summary

People have been forecasting for thousands of years. They forecast whether it will rain tomorrow, how much wheat will be harvested, how long it will take for dinner to cook, how many widgets their company will sell next month, what the unemployment figure will be in a year’s time, or how much superannuation they will have when they retire.

Some things are relatively easy to forecast and some are unpredictable. Why is it that we can accurately forecast a solar eclipse in 1000 years, but we have no idea whether Google’s stock price will rise or fall tomorrow? How can we forecast the daily electricity consumption for the next week with remarkable precision, but we cannot forecast daily ambulance call-outs with the same accuracy?

In his presentation, Professor Rob J Hyndman will discuss the conditions we need for predictability, how to measure the uncertainty of our forecasts, and how to evaluate whether we are uncertain enough. Rob will draw on his 30 years of forecasting experience, including forecasting Australia’s health budget for the next few years, forecasting peak electricity demand in 20 years, and producing weekly forecasts of daily COVID-19 cases for all Australian states from 2020-2023.

He will also look at how forecasting has changed over the last 50 years, and what forecasting might look like in the future.

Download pdf

vital: Tidy data analysis for demography

Mon, 08 Jul 2024 14:00:00 GMT

UseR!2024

14:30, 9 July

R package

vital

Slides

Download pdf

Probabilistic forecasts for anomaly detection

Tue, 02 Jul 2024 14:00:00 GMT

International Symposium on Forecasting

When a forecast is very inaccurate, it is sometimes because a poor forecasting model is used, but it can also occur when an unusual observation occurs. I will discuss the latter situation, where a good forecasting model can be used to identify anomalies. The approach taken is to use a probabilistic forecast, and to compute the “density scores” equal to the negative log likelihood of the observations based on the forecast distributions. The density scores provide a measure of how anomalous each observation is, given the forecast density. A large density score indicates that the observation is unlikely, and so is a potential anomaly. On the other hand, typical values will have low density scores. A Generalized Pareto Distribution is fitted to the largest density scores to estimate the probability of each observation being an anomaly. Applications to pharmaceutical scripts and mortality data will be used to illustrate the ideas using the fable and weird R packages.

Slides

Download pdf

Tidy data analysis for demography using R

Tue, 25 Jun 2024 14:00:00 GMT

Suessmilch Lecture, MPIDR

3pm CEST, 26 June 2024

I will introduce the vital package which allows analysis of demographic data using tidy tools. The package uses a variation of tsibble objects as the main data class, so all the infrastructure available for tsibble and tibble objects can also be used with vital objects. Data may include births, deaths, mortality, fertility, population and migration data. Functions for plotting, smoothing, modelling and forecasting data are included. Models include the classical Lee-Carter model as well as functional data models. Future plans include replicating all the models available in the demography and StMoMo packages. The package is available at pkg.robjhyndman.com/vital/ and on CRAN.

R package

vital

Slides

Download pdf

Statistical Forecasting

Mon, 13 May 2024 14:00:00 GMT

Talk given as part of the Monash University FUTURES symposium

Statistical forecasting involves using historical data to predict future outcomes. The goal is to build a stochastic model that captures the underlying patterns in the data, and how those have changed over time, and use it to make predictions about future values. I have developed many statistical forecasting models that are now widely used in diverse domains including retail, energy, demography, and tourism.

The strength of statistical forecasting lies in its ability to provide quantifiable predictions based on historical data, and to include uncertainty in those predictions. Statistical forecasts are typically presented as probability distributions of future outcomes, which can be used to make decisions that account for the uncertainty in the predictions.

Most recently I have been developing tools that allow statistical forecasts to be produced for thousands of related time series, such as sales forecasts for individual products in a retail store, or electricity demand forecasts for small regions in a country. The disaggregated data tends to be noisy and sparse, and the models need to be able to share information across related series to make accurate predictions.

Because of the assumption that the future will continue to evolve similarly to the past, statistical forecasting tends to perform best when: (a) there is substantial historical data; (b) the underlying environment is relatively stable; and (c) the aim is to produce short-term forecasts. When there is limited historical data, or when the data generating process is subject to significant structural changes, statistical forecasting can struggle to make accurate predictions.

I am also interested in using statistical forecasting techniques to identify anomalies, such as sudden changes in the underlying process, or unusual patterns in the data. This can be useful for detecting fraud, monitoring the health of complex systems, or identifying recording errors in data.

Download pdf

Forecasting the future and the future of forecasting

Wed, 07 Feb 2024 13:00:00 GMT

Download pdf