Large-scale unusual time series detection

Conference proceedings
Authors

Rob J Hyndman, Earo Wang, Nikolay Laptev

Published

1 June 2015

Publication details

In: Proceedings of the IEEE International Conference on Data Mining. Atlantic City, NJ, USA. 14–17 November 2015

Links

 

It is becoming increasingly common for organizations to collect very large amounts of data over time, and to need to detect unusual or anomalous time series. For example, Yahoo has banks of mail servers that are monitored over time. Many measurements on server performance are collected every hour for each of thousands of servers. We wish to identify servers that are behaving unusually.

We compute a vector of features on each time series, measuring characteristics of the series. The features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and use various bivariate outlier detection methods applied to the first two principal components. This enables the most unusual series, based on their feature vectors, to be identified. The bivariate outlier detection methods used are based on highest density regions and \alpha-hulls.

Associated R package