Old weather records hold surprises: the thermometer suddenly jumps by half a degree, not because of the weather, but because the station switched instruments. Such abrupt shifts in data are called changepoints, and separating them from genuine climate signals is a challenge statisticians grapple with. Standard methods either see too many jumps or miss important ones, especially when data have memory.
A restorer of an ancient fresco removes layer after layer, and with each pass, their hand gets better at distinguishing real cracks from random scratches. The new IRFL method works the same way: it iteratively reweights the penalties for each potential jump, so that small, noise-like changes incur increasing penalties while large, genuine ones are preserved. Instead of cutting out everything that looks uneven in one go, the algorithm makes several passes, gradually filtering out false signals.
Data from Mauna Loa volcano is a classic example. It is a time series of carbon dioxide measurements, 768 months long, where seasonality and trend are intertwined with noise. IRFL, after pre-whitening the data to remove dependence between neighboring values, detected five changepoints. They coincided with real events: the Pinatubo eruption and El Niño episodes. The estimated acceleration of CO2 growth from 0.71 to 2.54 ppm per year aligns with NOAA data.
The method is not limited to time series. On images, such as MRIs or plant root scans, IRFL removes noise while preserving sharp boundaries. It is like a restorer cleaning a painting without touching the outlines of figures. In simulations across 14 scenarios, IRFL outperformed competing methods in localization accuracy and false discovery control.
With each pass, the restorer more precisely separates real cracks from random scratches. IRFL does the same with data: by iteratively reweighting penalties, it filters out false jumps and leaves only the changes that truly altered the picture. This framework is already helping climatologists, doctors, and agronomists see real signals in noise.
Instrument changes at U.S. weather stations every 25 years introduce artificial jumps into temperature records, which IRFL can detect and correct.
In Kim Stanley Robinson's novel 'The Ministry for the Future', climate data are analyzed to identify abrupt changes, echoing the task of changepoint detection in CO2 series.