MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue
Multi-party floor control is the next bottleneck for full-duplex voice agents: systems must decide not just when speech ends, but whether overlapping listeners are backchanneling or genuinely taking the floor. MultiPartyTurnBench introduces 240 synthetically generated, channel-separated conversations spanning two to five speakers and six interaction types, evaluating End-of-Turn (EOT) and Interruption (INT) detection with exact causal annotations. The results reveal a stark capability asymmetry: turn completion is largely tractable, while selective interruption handling remains near chance even for state-of-the-art commercial agents.
The paper presents a meticulously designed benchmark that isolates floor-control decisions in multi-party dialogue, convincingly demonstrating that selective yielding is a distinct and unsolved problem. The discrete-event orchestrator, comprehensive baseline sweep, and full-duplex agent validation combine to make a rigorous case, though the reliance on synthetic speech and limited real-audio validation temper the strength of the findings.
The pre-synthesis floor-state orchestrator is an elegant solution to ground-truth ambiguity, and the evaluation protocol—causal per-speaker predictions with false-positive-constrained recall and signed latency—provides actionable diagnostics. The behavioral breakdown of commercial agents is particularly valuable: opposing failure modes (indiscriminate yielding versus never yielding) are exposed by balanced accuracy and action bias metrics that aggregate scores conceal.
The reliance on synthetic Kokoro-82M speech limits external validity despite strong human validation; annotator consensus drops from 89.8% to 82.1% as speaker count rises, and real-speech validation is limited to just 15 AMI meetings with in-slice threshold tuning. The benchmark is also restricted to English and 24 synthesized voices, leaving demographic and linguistic generalization unaddressed.
Additionally, with only 281 interruption positives in the test split, subgroup analyses by speaker count or interaction type may be underpowered.
The evidence strongly supports the central EOT–INT asymmetry claim: it holds across acoustic VADs, semantic endpointers, turn-projection models, and five commercial agents, with thresholds frozen after development-set selection. Comparisons to prior work in Table 1 are fair, positioning MPTB as the only resource that jointly evaluates multi-party EOT and INT detection with event-specific negatives and causal timestamp scoring.
The generation pipeline is documented with deterministic seeds, explicit timing distributions, and voice inventories, but independent reproduction hinges on the release of audio assets and scorer code that are promised but not explicitly confirmed as available in the provided text. Agent behavior evaluation relies on proprietary closed APIs, which inherently limits exact replication.
Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel without taking it. Existing evaluations study dyadic turn-taking or end-to-end agent behavior, but lack a unified, controlled evaluation of genuine floor transitions against confusable non-events in multi-party settings. We introduce MultiPartyTurnBench (MPTB), a benchmark of 240 synthetically generated, channel-separated conversations spanning two to five speakers and six interaction types. MPTB evaluates two complementary tasks: End-of-Turn (EOT) detection distinguishes genuine turn completion from temporary holds, while Interruption (INT) detection distinguishes successful floor takeovers from non-floor-taking speech. Exact event-conditioned annotations enable causal evaluation using false-positive-constrained recall and signed latency. Across acoustic, semantic, and turn-projection baselines, we find a stark asymmetry: learned systems achieve 82.5-92.6% EOT recall at 4.3-11.8% false-positive rates, while INT detection suffers from missed takeovers or excessive false activations. This asymmetry extends to five full-duplex agents which achieve 86.1-95.8% balanced accuracy on turn entry, yet perform near chance (46.9-54.7%) on selective interruption handling. Gemini-3.8-Live yields to 97.7% of backchannels, whereas MiniCPM-o-4.5 yields to only 1.4% of genuine interruptions. These findings establish selective floor control as a distinct, unresolved challenge for reliable multi-party voice interaction.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.