New parameter-free first-order methods adapt to unknown smoothness and growth in convex optimization
This paper develops algorithms that minimize convex functions without knowing key problem properties in advance. The authors design methods that adapt automatically to how smooth the objective is and to how fast the objective grows away from its minimizers. The algorithms are “parameter-free” (they do not need user-supplied smoothness or growth constants) and “anytime” (they do not require the user to specify a target accuracy ahead of time). The main claims are provable rates on the number of first-order queries needed to reach a given accuracy.
Concretely, the authors present two main algorithms. The first is a bundle-level W-certificate method called BLW for nonsmooth Lipschitz functions that satisfy a quadratic growth condition. Quadratic growth means the function value exceeds the optimum by at least (µ/2) times the square of the distance to the solution set, for some modulus µ>0. BLW attains the optimal oracle complexity O(M0^2/(µ ε)) for this setting, where M0 measures the Lipschitz size of subgradients and ε is the desired function-error. Crucially, BLW does not need µ or the target accuracy ε as input. The second algorithm, A-BLW, is an accelerated variant that adapts across smoothness regimes. Without knowing the Hölder smoothness exponent ρ (which ranges from 0 for nonsmooth to 1 for fully smooth) or its constant Mρ, or µ, or ε, A-BLW attains the best-known rates in the nonsmooth, weakly smooth (0<ρ<1), and smooth (ρ=1) regimes.
A central technical idea is the affine W-certificate. Informally, the method builds an affine minorant (a linear lower bound) of the objective and measures how slowly that minorant descends. That “descent-slowness” can be turned, under quadratic growth, into a guaranteed bound on how far the current point is from optimality. This certificate lets the bundle-level procedures stop or accelerate in a way that yields the claimed complexity bounds. The paper also includes a stopping-time analysis that shows A-BLW, without changing the algorithm, matches best-known rates for general convex problems and for problems with a broader Hölder-type growth of order α≥2.