How to make Gaussian processes work in real time: a signal‑processing tutorial
This paper is a tutorial-style overview of ways to run Gaussian process models in a sequential or streaming setting. Gaussian processes (GPs) are a flexible way to model unknown functions and they give principled uncertainty estimates. But the usual GP methods are slow for long time series or online updates. The authors organize recent methods from a signal‑processing view to show how GPs can be made practical for real‑time tasks.
The article explains the basic problem: classical GP inference with N data points requires inverting an N×N matrix, which costs on the order of N3 operations. That cubic cost and the need for large memory also make simple one-step updates expensive. These computational limits have blocked the straightforward use of GPs in many streaming, adaptive, or resource‑constrained signal‑processing systems.
To address this, the authors survey several families of approximations that restore a sequential, recursive structure. One approach is basis expansions, which approximate the GP prior with a finite set of features. Examples given include random Fourier features and Hilbert‑space constructions. Another approach is sparse and variational approximations that summarize the past via a compact set of inducing variables to get roughly linear or constant memory costs. A third view treats many common GP kernels as solutions of stochastic differential equations (SDEs). That Markovian formulation turns the GP into a state‑space model that can be updated exactly with Kalman filtering and smoothing in continuous time.
Why this matters: these approximations let practitioners keep the interpretability and uncertainty quantification of GPs while getting algorithms that run online, use less memory, and connect directly to classical signal‑processing tools like recursive least squares and Kalman filters. The review points out direct applications in sequential regression and forecasting, anomaly detection in time series, sequential Bayesian optimization, adaptive and active sensing, sequential detection and decision making, and distributed or decentralized estimation.