Acc‑Sinkhorn speeds up regularized optimal transport while keeping Sinkhorn’s simple steps
This paper introduces Acc‑Sinkhorn, a small change to the classic Sinkhorn algorithm that often makes it much faster. The authors keep the same per‑step cost and the same kind of row/column scaling that makes Sinkhorn simple. Under a verifiable stability condition they prove a faster mathematical convergence rate and, in experiments on synthetic data, color transfer, and word alignment, report 10×–30× speedups at small regularization strength.
Optimal transport is a way to match two discrete distributions with a cost matrix. Adding an entropy term (entropy‑regularized optimal transport, or EOT) makes the problem smoother and gives a unique, positive solution. The standard numerical method for EOT is Sinkhorn, which alternately rescales the rows and columns of the kernel K = exp(−C/ε). Each Sinkhorn iteration costs O(nm) work and is easy to run on a GPU, but its error typically decays at a 1/k rate in the number of iterations.
The main idea behind Acc‑Sinkhorn is a bilevel view of the dual problem. For a fixed column variable v, the row variable u can be solved exactly by a Sinkhorn row scaling step. That defines a reduced outer objective f(v) for the column variable. The ordinary Sinkhorn step on v is a unit‑step mirror descent on this outer problem. Acc‑Sinkhorn adds a Hessian‑driven Nesterov style acceleration to this outer step while keeping the Sinkhorn scaling form. In practice each Acc‑Sinkhorn iteration still uses essentially one normalized Sinkhorn step plus a few vector operations, so the per‑iteration cost stays at O(nm).
The authors prove that Acc‑Sinkhorn achieves an O(1/k^2) decay of error under a stability condition that can be checked or enforced. As a result, for a target inner accuracy τ their method needs O(τ−1/2) Sinkhorn steps compared with O(τ−1) for plain Sinkhorn. When this is combined with the standard entropic approximation error (about O(ε log n) for replacing unregularized OT by EOT), the paper shows a total complexity of order Õ(n2/ε) for finding an ε‑accurate optimal transport cost. This improves on the Õ(n2/ε2) dependence attributed to Sinkhorn in the same approximation setting.