Dyadic: A Scalable Platform for Human-Human and Human-AI Conversation Research

The platform description reveals genuinely useful features that address practical pain points in conversation research, particularly the ability to inject surveys in situ during conversations rather than relying on post-hoc recall. The slot-based architecture enabling within-room role manipulations is technically sound, and the inclusion of response latency metrics ($t_{first\,keystroke}$, $t_{reply}$) and typing behavior logs demonstrates awareness of granular behavioral measures valued in interaction research. The commitment to browser-based deployment without software installation removes friction for participant recruitment.

The paper commits an important category error: presenting documentation as if it were empirical scholarship. There is no validation that Dyadic actually improves research quality, reduces costs, or enables discoveries that extant tools cannot. Claims of scalability are asserted without performance benchmarks or stress-testing data under high concurrency. The reproducibility concerns regarding API-dependent AI are noted but dismissed rather than addressed—changes to models or discontinuation of services could render studies irreproducible, yet the platform offers no versioning safeguards or local model hosting solutions except vague references to Hugging Face. The tool is commercial-adjacent (requiring paid API keys for full functionality) yet presented as a public good without acknowledging vendor lock-in risks.

Furthermore, claims about mobile compatibility are hedged with explicit warnings that the mobile experience has not been fully optimized, yet the paper simultaneously markets this as a feature. The security discussion focuses on transport encryption and password hashing but omits whether conversation data are encrypted at rest, where servers are physically located, or GDPR/HIPAA compliance status—critical omissions for sensitive conversational data collection.

The evidence provided is descriptive rather than comparative or evaluative. No experiments demonstrate that studies conducted via Dyadic yield different or superior outcomes compared to existing platforms (e.g., oTree, jsPsych, custom Lab.js implementations, or even Qualtrics chat widgets). The table comparing features against unstated alternatives is incomplete without identifying what those alternatives are. The paper cites the need to bridge siloed approaches between AI-mediated communication and psychology of language research (Boyd & Markowitz, 2025, 2026), but offers no empirical demonstration that this bridging actually occurs in practice using Dyadic. The granularity of behavioral measures (typing latencies, mouse clicks) is claimed to facilitate linking verbal behavior to interpersonal outcomes, but no example analyses or case studies demonstrate this linkage.

Reproducibility is compromised by several factors. First, Dyadic is a hosted proprietary platform—not open source—meaning researchers cannot inspect the code, host their own instances, or verify that experimental logic executes as described. The author admits the platform is AI-augmented (coded with Claude Code), introducing opacity about the software's decision-making processes. Second, AI-mediated studies depend on remote API services (OpenAI, Anthropic, Google) where model weights, behaviors, and availability change without notice; as the paper acknowledges, changes to APIs could limit reproducibility (Gundersen & Kjensmo, 2018). Third, while data exports are available in CSV format, the platform lacks preregistration integration, version control for study configurations, or standardized metadata schemas. Hyperparameters for AI behaviors (system prompts, response delays) are configurable but not automatically archived in a machine-readable format, placing the burden on researchers to manually document these settings in methods sections.