This paper shows that two things are needed to make data-driven reduced models of chaotic systems work well: randomness inside the model, and training that evaluates whole forecast distributions over finite trajectories. Without both, learned models can look good one step at a time but fail to reproduce real long-term variability.

Coarse-grained models — sometimes called closures — try to represent a complex system with fewer variables. That leaves out many fast or small-scale effects, which creates systematic errors. A common way to learn closures from data is to minimize one-step prediction error, implicitly assuming the missing effects act like memoryless (Markovian) noise. The authors prove that a different common practice — optimizing deterministic pointwise losses such as mean squared error (MSE) over entire chaotic trajectories — causes a mathematical degeneracy: it suppresses the model’s predictive variance and removes physical variability in long simulations.

The cure, the paper argues, is to treat predictions as probability distributions and to train against strictly proper scoring rules. A strictly proper scoring rule rewards a forecast that assigns high probability to the observations and penalizes forecasts that are overconfident or underdispersed. By targeting forecast distributions rather than single realized trajectories, these losses stop penalizing predictive spread and make the long-term optimal model match the system’s invariant measure — the correct long-term statistics.

The authors test these ideas on a canonical chaotic system from fluid dynamics called quasi‑geostrophic turbulence. They find that closures trained one step at a time fail to reproduce stable coarse-grained dynamics. Deterministic closures trained over trajectories do show the predicted loss of variance. In contrast, stochastic closures calibrated over trajectories with a distributional score called the energy score produce skillful ensemble forecasts and realistic long-term statistics.