Finding Breakpoints in a Random Data Stream ✎

Original: "Multiple Change Point Analysis: Fast Implementation And Strong Consistency"
· Jie Ding, Yu Xiang, Lu Shen, Vahid Tarokh
arXiv:1605.00346 · 2016-05-02 · arXiv non-exclusive · independent analysis · 3 min
Mathematicians have devised a fast and reliable way to pinpoint when the behavior of a time series changes.

Context

Detecting structural changes in time series is important for network security, finance, and climatology. Classical approaches either assume independence of observations or require heavy computation for dependent data. This work stands apart from neighboring studies that examine quantum analogs, randomness in evolution, or model selection: here the focus is on a fast algorithm for autoregressive processes and a rigorous proof of consistency for the criterion that selects the number of segments.

Methods

The authors model data as a piecewise autoregressive process: each segment is described by its own linear autoregression. To handle dependence, they split the series into windows, estimate autoregressive coefficients in each window, and obtain a sequence of nearly independent multivariate points. They then apply an algorithm with multiple window sizes that combines results to yield narrow intervals containing the change points. To choose the number of points, a penalized criterion is used: the sum of quadratic losses within segments plus a penalty for each point.

Results

For independent data, it is proven that a penalty of order log-log of the sample size guarantees strong consistency, meaning it almost surely selects the true number of change points. An AIC-type criterion with a constant penalty always gives a nonzero probability of overfitting. For sub-Gaussian data, a penalty of order log of the sample size ensures strong consistency and accurate localization of change points, with error at most of order log. On synthetic data, the method is more accurate and faster than binary segmentation. On eastern U.S. temperatures from 1895 to 2015, change points were found in 1901, 1929, 1944, 2009, and 2012, coinciding with phases of the Atlantic Multidecadal Oscillation. In the El Niño index from 1854 to 2015, a shift around 1979 was detected, linked to the Pacific Decadal Oscillation.

Implications

This work differs from neighboring studies by providing a rigorous proof of strong consistency for the problem of selecting the number of change points in dependent time series. Previously, such guarantees were known only for special cases, but here they are obtained under mild conditions. This provides a theoretical foundation for using penalized criteria in practical time series analysis.

Future development

The method can be extended to other loss functions and more complex data structures, such as multivariate series with correlations between components. Speedup is possible through parallel computing for massive datasets. An open question remains about the minimal penalty that guarantees strong consistency in the general case.

Impact

The results are applicable in climatology, finance, network security, and neuroscience, where it is important to quickly find regime shifts in long time series.

Next steps

Test the method on other real data, such as EEG analysis or network traffic. Investigate adaptive selection of window sizes to improve accuracy.

Key open problems

In time series statistics, the choice of the number of change points for strongly dependent data without parametric assumptions remains an open problem. This work contributes to the theory of penalized criteria but does not fully solve the problem for nonstationary processes with slowly varying parameters.

Eastern U.S. temperatures over the last 120 years have changed their regime only five times, and these shifts coincide with phases of the Atlantic Multidecadal Oscillation, which lasts 20 to 40 years.

\hat{M} = \arg\min_{k=0,\ldots,M} (\hat{e}_k + k f(N))
The number of change points is chosen to minimize the sum of within-segment losses plus a penalty for each point.
\mathrm{Loss}_q(x_{\ell_{j-1}+1},\ldots,x_{\ell_j}) = \sum_{n=\ell_{j-1}+1}^{\ell_j} |x_n - \bar{x}|^2
The loss equals the sum of squared deviations of the segment points from their mean.

Key numbers

  • Number of change points in eastern U.S. temperature: 5
  • Year of regime shift in the El Niño index: 1979
  • Length of eastern U.S. temperature series: 121 years
  • Length of El Niño series: 162 years
  • Penalty order for strong consistency: log-log of sample size
Concepts
information criterion linear regression random forest autoregressive model reanalysis IRI Dynamical Systems stochastic processes principal component analysis seasonal cycle independent component analysis statistical method prediction accuracy benchmark linear inverse model retrospective forecast probabilistic forecast cross validation climate Stochastic modeling