This paper presents WordVoice, a new data and model approach that lets large language model (LLM) based text-to-speech (TTS) systems control multiple acoustic attributes at the level of individual words. The authors argue that current end-to-end TTS systems are natural sounding but coarse: they do not let users explicitly set word-level duration, pauses, loudness (energy), pitch, or tone. That lack of fine control is a problem for tasks like audiobook narration and video dubbing where timing and local emphasis matter.

The work has two main parts. First, the team built WordVoice-5A, a 4.7k‑hour bilingual (Chinese and English) dataset with word-level annotations for five acoustic dimensions: duration, boundary (a five-level pause scale), energy, pitch, and tone. They created a linguistically guided annotation pipeline that combines two aligners (Montreal Forced Aligner and Qwen3FA), applies loudness-based optimization of timestamps, and runs consistency checks. The dataset statistics reported are about 2,546 hours for Chinese and 2,138 hours for English, totaling roughly 4,684 hours and 52.26 million words or characters.

Second, they introduce the WordVoice modeling framework. Inside the autoregressive LLM they add a bound-token mechanism that explicitly triggers an “acoustic planning” step: for each word the model predicts the five acoustic attributes before generating the speech tokens. To make those discrete token predictions match smooth waveforms, they add a fine-grained acoustic modulation module in the token-to-waveform stage (integrated with a flow-matching backbone). The paper describes concrete processing choices: word boundaries are discretized into five levels (b0 continuous to b4 >0.4 s); energy is computed from the top 50% of frames to focus on syllable nuclei; pitch uses the central 80% of the F0 contour, resampled to 16 points and modeled with quadratic polynomials for tone/intonation.