This paper is about how researchers should measure and compare techniques that give large language models extra computation during inference — what the authors call “test-time scaling.” These techniques can make models solve harder reasoning problems. But the phrase has come to cover very different methods, and that makes published results hard to compare or reproduce.

The authors formalize test-time scaling as a kind of budgeted search through the partial outputs a model can produce. They describe the model’s possible generations as a “prefix tree”: every partial answer is a branch, and complete answers are leaves. From that view they identify three high-level regimes. Single-trajectory sequential scaling spends more compute extending a single chain of thought. Leaf-level scaling samples many complete answers and then reduces them by voting, reranking, or verification. Prefix-level scaling searches and scores unfinished partial states before finishing them.

They also propose how to evaluate these methods fairly. Instead of treating the base model alone, they evaluate the whole inference system: the checkpoint, the prompt, the decoding rule, the search controller or reducer, the verifier or judge, the stopping rule, and the compute budget. They introduce an “evaluation profile” that reports both end-to-end system performance and diagnostics about the candidate set. The profile and recommended reporting include matched compute accounting and uncertainty estimates so that comparisons do not hide important differences in protocol or cost.

Reproducibility is another focus. The paper distinguishes exact replay (running the same noisy process to get the same traces) from distributional reproducibility (matching the statistical behavior), and lists the artifacts needed to support each. The authors also organize the open-weight reasoning ecosystem by model-side training choices and interface mechanisms (for example verification-based fine-tuning or reward optimization), and they apply their principles to benchmarks in broad knowledge, symbolic reasoning, and competition mathematics.