Speech large language models (SpeechLLMs) work directly from audio. They are faster than systems that first convert speech to text and they keep vocal cues that text-only systems lose. But they still fall behind text-only models on hard reasoning tasks, and real-time spoken use limits how long a model can take to think. RetroThinker is a new method that lets a streaming SpeechLLM revise its chain-of-thought — the model’s step-by-step reasoning — on the fly, improving accuracy without adding delay.

The authors built RetroThinker as a multi-stage post-training framework for the Moshi speech model. They trained the model to self-verify and forward-correct its chain-of-thought steps during inference. To do this they used supervised fine-tuning (SFT) on curated “retrospective thinking” examples and a version of direct preference optimization (DPO) that uses length as a signal. Direct preference optimization is a way to nudge a model toward preferred behaviors by comparing alternative outputs; the authors tailor it here to favor better early-stage revisions while the speaker is still speaking.

At a high level, RetroThinker lets the model perform early or concurrent reasoning while audio is still streaming, and then dynamically revise those intermediate reasoning steps as more speech arrives. “Self-verify” means the model checks its own intermediate steps. “Forward-correct” means it updates later steps to fix earlier errors. The approach is designed so these corrections happen during normal streaming inference rather than requiring a separate slow re-think that would add latency.

Why this matters: on a standard benchmark (GSM8K), RetroThinker improved the accuracy–latency trade-off compared to non-retrospective baselines. The paper reports an 11 percentage-point absolute gain in accuracy at a similar latency. That suggests speech agents can become both faster and smarter in step-by-step reasoning, while preserving paralinguistic information that cascaded speech-to-text systems often lose.