This paper proves that a standard stochastic algorithm for optimizing several goals at once gets closer to a balanced solution faster than previously shown. The algorithm, called stochastic multi-gradient descent (SMG), uses noisy gradients from mini-batches of data to approximate a direction that tends to reduce all objectives. Under common smoothness and variance assumptions, the author shows the algorithm's measure of suboptimality — the squared Pareto-stationarity (PS) measure, which is the squared norm of the conflict-avoiding multi-gradient direction — falls on the order of 1/T after T iterations (up to logarithmic factors). This improves on an earlier bound of about T^(-1/4) for the same setting.

What the researchers did: they studied SMG under standard hypotheses: each per-sample loss is L-smooth (a mild regularity condition), mini-batch gradients are unbiased, and their variance scales like σ^2 divided by the mini-batch size. The analysis assumes the algorithm computes the minimum-norm convex combination of the mini-batch gradients exactly at each step (the MGDA direction) and uses a randomized output drawn proportional to step sizes. A key practical requirement for the sharper rate is that mini-batch sizes grow linearly over time so that the stochastic variance vanishes as iterations proceed.

How the improvement works, in plain terms: previous work bounded how the MGDA direction changes with data by using a weaker continuity property of the direction itself. This paper instead observes that the PS measure (the length of that direction) is Lipschitz continuous with respect to the matrix of gradients. Lipschitz continuity means small changes in the gradient matrix cause only small changes in the PS measure, with a linear bound. That stronger stability lets the proof control the bias introduced by using stochastic mini-batches more tightly. Combining this with a constant step size and growing batch sizes leads to the improved 1/T rate for the squared PS measure.