Researchers introduce LLM-as-a-Verifier, a new way to use large language models (LLMs) to decide whether a proposed solution is correct. Instead of asking a model to output a single score token, the method looks at the model’s full scoring distribution — the raw scores (called logits) it assigns to possible scoring tokens — and turns that distribution into a continuous score. This gives much finer and less tie-prone judgments without any extra training of the model.

At a high level, the team computes an expectation over the model’s scoring-token logits. That means they use the model’s internal probability estimates rather than collapsing them to one discrete choice. They show this view lets verification improve along three axes: score granularity (using more token information), repeated evaluation (running the check multiple times to reduce randomness), and criteria decomposition (breaking a complex check into simpler parts). To make selection among many candidate solutions practical, they also add a cost-efficient ranking algorithm that uses the continuous scores.

The method helps in several practical domains. The paper reports state-of-the-art verification accuracy on multiple benchmarks: Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). They note that many models can produce a correct solution at least once when sampled repeatedly — an “oracle” selector could reach 98.9% on Terminal-Bench V2 — but standard LLM judges often tie and fail to pick the best answer (a 27% tie rate was reported). By producing fine-grained continuous scores, LLM-as-a-Verifier reduces ties and improves discrimination; in robotics it achieved a very high value-order correlation of 0.966.

The authors also show other uses. The verifier’s scores correlate with task progress, so the signals can help monitor multi-step processes. They built extensions for developer tools (Claude Code and Codex) to let agents be monitored and improved. And when used as a dense reward for reinforcement learning, the verifier improved sample efficiency: about 1.8× on a robotics benchmark (LIBERO) when fine-tuning a policy with DSRL-SAC, and about 1.1× on a mathematical-reasoning benchmark when fine-tuning Qwen3-8B with GRPO.