This paper tackles how to better learn and describe the low-dimensional surface, or manifold, that high-dimensional data lie on. The authors work with the Gaussian Process Latent Variable Model (GP-LVM). In GP-LVM a Gaussian Process (GP) maps points in a low-dimensional latent space to observed data. The GP gives not only a reconstruction but also an estimate of epistemic uncertainty — the model’s uncertainty about the manifold itself.

A common practical approximation in these models is a mean-field variational assumption. Mean-field means the parts of the model that represent the latent variables and the GP “inducing points” are treated as independent in the approximate posterior. Inducing points are a small set of representative inputs used to summarize the GP and make inference tractable. Treating them as independent with the latent variables can limit the quality of the learned manifold and its uncertainty estimates.

To address this, the authors apply Amortized Structured Stochastic Variational Inference. In plain terms, they replace the simple independent approximation with a learned, conditional posterior for the latent variables that depends on the inducing points. “Amortized” refers to using a learned function to produce posterior parameters quickly. “Structured” means the posterior can capture dependencies between the inducing points and the latent variables rather than forcing independence.

The paper reports that this more flexible variational posterior improves several metrics related to reconstructing points on the data manifold. That is, allowing the latent representation to depend on the inducing points led to better reconstructions in the experiments the authors ran. These results suggest the approach can give a more faithful picture of the manifold and its uncertainty than the standard mean-field approximation.