Researchers introduce MP-Bench, the first benchmark made to test voice agents as active participants in multiparty conversations. The benchmark focuses on two human skills that matter in group talk: knowing when to take or yield a turn (turn-taking awareness) and producing responses that fit the flow of the conversation (response appropriateness). It also includes comprehension-style question-and-answer checks to see whether agents correctly understand who said what.

To build MP-Bench, the team created scenarios with multiple speakers and designed objective tests that ask the agent whether it should speak next, what a good reply would be, and factual questions about the discussion. They ran the benchmark on 12 voice agents, including systems that operate in real time. The benchmark deliberately used distinct speaker voices so that the test would focus on conversational ability rather than on the technical task of separating similar voices.

The results show clear limits today. Real-time voice agents scored at or below 22% on the multiparty comprehension questions. For turn-taking — deciding when to speak, interrupt, or stay silent — agents performed near chance levels, close to a 50% baseline. In short, current real-time agents often do not understand who is speaking well enough to join a group conversation reliably.

The authors also ran an ablation where speaker voices were made more similar. Abilities that depend on telling voices apart fell dramatically. Comprehension fell to about 4%, speaker-name accuracy dropped to 0%, and discussion appropriateness declined (reported from roughly 75% to 45% in their numbers). Turn-taking remained near or below chance. These tests show that when voice separation gets harder, multiparty understanding collapses.