Global energy forecasting competitions

Hyndsight

The 2012 GEFcom competition was a great success with several new innovative forecasting methods introduced. These have been published in the IJF as follows:

These papers will appear in print in the next issue of the journal. The data and code for each method is also available.

GEFCom 2014

The 2014 edition of the competition is now under way with a probabilistic focus. The great thing about these competitions and the associated papers is that they provide a serious empirical comparison of the forecast capability of different algorithms. There are lots of suggested approaches to forecasting energy, but without head-to-head comparisons like this, it is not always clear what is a good method, and what methods can be discarded.

The developers of the best performing methods in the competition will be invited to submit a paper to a special issue of the IJF on probabilistic energy forecasting, to be edited by Tao Hong and Pierre Pinson.

In addition to the regular competition, there is also a competition for students where the best-performing classes can win a prize. See Tao Hong’s blog for further information.

Evaluating forecast distributions

Because this round of the competition is focusing on predicting the whole distribution, and not just a point forecast, we need a method of evaluating the forecasts which takes account of the whole distribution. We have chosen to use quantile scoring.

For each time period throughout the forecast horizon, the participants should provide the $0.01, 0.02, \dots, 0.99$ quantiles — call these $q_1,\dots,q_{99}$, with $q_0=-\infty$ or the natural lower bound, and $q_{100}=\infty$ or the natural upper bound. The full predictive densities composed by these quantile forecasts are to be evaluated by the quantile score calculated through the pinball loss function.

For a quantile forecast $q_a$ with $a/100$ as the target quantile, this score $L$ is defined as:

$$L(q_a, y) = \begin{cases} (1 - a/100) (q_a - y), & \text{if y< q_a};\\ a/100 (y - q_a), & \text{if y\ge q_a}; \end{cases}$$

where $y$ is the observation used for verification, and $a = 1, 2, \dots, 99$.

To evaluate the full predictive densities, this score is then averaged over all target quantiles, from 0.01 to 0.99, for all time periods over all forecast horizons. The lower the score, the better the forecasts are.