Why message‑passing methods act Gaussian on some deterministic transform matrices
This paper explains when and why a broad class of iterative algorithms behaves the same on many large matrices, including some deterministic transform matrices. The algorithms studied are general first‑order methods (GFOM): they alternate multiplying a state vector by a matrix and applying simple scalar nonlinearities entrywise. A well‑known special case, approximate message passing (AMP), often has a particularly simple limit: the state at each step looks Gaussian in large dimensions. Until now, that Gaussian picture was mainly justified for very random matrices and remained mysterious for more structured, deterministic inputs.
The authors attack the problem with a combinatorial and diagrammatic approach based on the matrix’s limiting “traffic distribution.” Informally, the traffic distribution records the limiting values of all permutation‑invariant polynomials in the matrix entries as the matrix size grows. Using these diagram expansions, they compute the traffic distribution for the first nontrivial deterministic examples, including minor variants of the Walsh–Hadamard transform and the discrete sine and cosine transforms. From that calculation they derive the asymptotic dynamics of GFOM on those matrices, resolving parts of conjectures made by Marinari, Parisi, and Ritort (1994).
They also design a new AMP‑type iteration that unifies several previous AMP variants and extends to new input types. For a large and natural class of traffic distributions — a class that includes both many random ensembles and the deterministic transforms they studied — the algorithm’s limiting dynamics remain Gaussian, but in a conditional sense: the state is asymptotically Gaussian given some latent random variables. The paper gives a simple combinatorial interpretation of the so‑called Onsager correction term that appears in AMP analyses, answering questions posed recently by Wang, Zhong, and Fan (2022). The authors describe this new method as a “treelike AMP” in their development.