A provable Bayesian sampler for nuclear‑norm matrix completion with unknown noise
This paper gives the first Bayesian sampler with an explicit, non‑asymptotic guarantee for a popular matrix completion model that uses the nuclear norm to prefer low‑rank solutions. The main advance is handling the realistic case when the noise level is unknown. The authors show a sampler whose running time is polynomial in the matrix size and in the inverse of the target accuracy. They stress that this is a feasibility result: the sampler is provably polynomial-time, but not presented as a ready‑to‑use tool for large datasets today.
Matrix completion means filling in missing entries of a table from noisy observed entries. A common way to get a single best guess is to minimize a squared error plus a nuclear norm penalty. The nuclear norm is the sum of a matrix’s singular values and encourages low rank. A Bayesian view treats that penalty like a prior and gives a full posterior distribution, which allows uncertainty estimates. If the noise variance were known, the usual nuclear‑norm prior leads to a log‑concave posterior that is easier to sample from. In practice the noise variance is not known, so the authors put a prior on the noise precision (the reciprocal of variance) and work with the joint posterior over precision and the matrix.
A key technical hurdle is that adding a prior on precision makes the target distribution non‑log‑concave in that variable. A naïve approach—estimating the marginal likelihood at each precision by reweighting samples from an untilted distribution—needs exponentially many samples. The authors avoid this by discretizing the precision onto a grid and using thermodynamic integration to build a categorical posterior over the grid. This removes the exponential dependence that the naïve importance‑weighting estimator has.
On the sampling side, the paper extends a mixing‑time analysis for random‑walk Metropolis to targets that combine a smooth strongly convex term with a nonsmooth but Lipschitz term. Concretely, the prior they analyze combines a nuclear‑norm term (nonsmooth) with a Frobenius‑norm squared term (smooth and makes the overall shape strongly log‑concave in parts). Their analysis uses an isoperimetric‑profile framework and gives mixing bounds that are polynomial in dimension and logarithmic in the desired accuracy. They also point out that their bound worsens with the ratio of the nonsmooth term’s Lipschitz constant to the smooth term’s curvature. Other recent work can give better conditioning, but that requires an efficient proximal sampling oracle for the nonsmooth part; random‑walk Metropolis requires only function evaluations and so applies where a proximal oracle is unavailable.