This work stands apart: the nearest neighbors work on machine learning for the large-scale structure of the Universe and gravitational maps, while here the focus is on calibrating probabilistic forecasts. Around us, people already build maps with honest uncertainty and improve prediction accuracy, but the task of recalibration through betting and information theory remains rare. Poor calibration is a long-standing headache in forecasting: a model confidently predicts rain with 80% probability, but rain occurs only half the time. Classical methods require independent and identically distributed data, which is rarely the case in practice.
The authors build a Gaussian process (GP) to estimate the probability density of PIT values for a scalar forecast quantity. PIT is a transformation that maps a forecast to a uniform distribution if the model is perfectly calibrated. The GP density estimate gives a closed-form expression for the entropy of the distribution, and through it, a prediction of the gain in bets against the original system. A key feature is that the method does not require the i.i.d. assumption, making it applicable to time series with dependencies. The scheme is tested on two cases: a laboratory nonlinear circuit and seasonal forecasts of El Niño intensity.
The recalibrated system shows a uniform expected PIT distribution, a hallmark of perfect calibration. In bets against the original system, the new system consistently wins, and the gain is predictable through entropy measures. In the laboratory experiment with a nonlinear circuit, significant calibration improvement over the baseline model was achieved. For seasonal El Niño forecasts, the method also showed superiority despite strong autocorrelation in the data. Numerical results confirm the theoretical derivations.
Unlike neighboring works that focus on prediction accuracy or honest uncertainty maps, this work introduces an explicit betting mechanism as a quality criterion for forecasts. This links calibration to information theory and game theory, opening the way to new methods for evaluating forecasting systems. The practical significance is that the method applies to a wide class of problems, including climate forecasts and engineering systems where data are not i.i.d.
The topic could evolve toward adaptive recalibration schemes that update in real time as new data arrive. Generalization to multivariate forecasts and to cases where PIT values have complex dependence structures is possible. Integration with deep neural network models for density estimation is also of interest.
The method will affect meteorology, climatology, financial forecasting, and any field that uses probabilistic forecasts. It may also change model evaluation practices, making betting a standard tool for verification.
The next step is to apply the method to a broader range of real forecasting systems, including ensemble weather forecasts. It is also worth investigating robustness to the choice of Gaussian process kernel and to outliers in the data.
In the field of probabilistic forecasting, the problem of calibration for dependent data remains open. This work directly addresses this problem by proposing a method that does not require the i.i.d. assumption. This connects it to fundamental questions of uncertainty quantification in dynamical systems.
The idea of using bets to test forecasts dates back to the work of Bruno de Finetti, who believed that probability can be measured through fair bets. This work takes that principle to a practical recalibration scheme.