Agentic Automation of BT-RADS Scoring: End-to-End Multi-Agent System for Standardized Brain Tumor Follow-up Assessment

The modular multi-agent architecture effectively separates concerns: an extractor agent handles NLP-based clinical variable identification (steroid status, bevacizumab use, radiation date) with evidence-span linking for verification, while a scorer agent applies deterministic BT-RADS logic. This design supports transparency through schema-constrained generation and Pydantic validation. The use of an open-weight LLM (GPT-oss, 20B parameters) with local inference addresses privacy and cost barriers for clinical deployment. The substantial agreement with expert reference (Cohen's κ = 0.708; quadratic weighted κ = 0.803) demonstrates the system captures the ordinality of BT-RADS categories. The root cause analysis provides actionable insights: 44.1% of errors stem from threshold boundary ambiguity—a fundamentally irreducible challenge in categorizing continuous volumetric data.

Several limitations warrant careful scrutiny. First, the comparison to initial clinical assessments may inflate perceived benefit: routine radiology reports often come from non-neuroradiologists with varying training, whereas the reference standard was a single fellowship-trained neuroradiologist. The system achieving 76% vs. ~80% human interrater reliability suggests it matches, but does not surpass, expert-level consistency. Second, the theoretical performance ceiling analysis (Table 4) constructs optimistic scenarios (88.1%, 99.4%) that are not empirically validated; these projections assume perfect extraction and algorithmic logic that may not be achievable. Third, exclusion of 17 cases (9 for baseline issues, 8 for segmentation quality) without sensitivity analysis on their characteristics raises selection bias concerns. Fourth, threshold-dependent categories (BT-3b particularly at 57.1% sensitivity) struggle when FLAIR and enhancement trends diverge—a known limitation of aggregate volumetric approaches that mask lesion-level heterogeneity. The acknowledgment that the BT-3b "enhancement priority rule did not fully align with expert interpretation" signals unaddressed algorithmic refinement needs.

The evidence generally supports the primary claim that multi-agent automation outperforms routine clinical assessment, though the effect size (18.5 percentage points) should be interpreted in context of the comparison baseline. Performance stratification by category type is well-documented and clinically meaningful—high sensitivity for context-dependent categories validates the LLM extraction component, while moderate sensitivity for threshold-dependent categories reflects inherent measurement uncertainty near categorical boundaries. The comparison to Lee et al.'s NLP-only approach (F1 0.68–0.72) is fair but understates the advantage: this work integrates actual imaging volumetrics rather than relying solely on report text. The prior work by Zhang et al. on volumetric analysis alone is appropriately cited as showing "only moderate performance without subcategory differentiation, highlighting the necessity of clinical context integration." However, the paper does not directly compare against the referenced interrater agreement study (Essien et al., Gwet index 0.83) in its results framing—doing so would contextualize that 76% accuracy, while improved over routine practice, remains below optimal human agreement.

Reproducibility is limited by several factors. The source code is "available upon request for academic research" rather than deposited in a public repository with version control. No training data, model weights, or inference pipelines are publicly accessible. Critical implementation details are insufficiently specified: exact prompt templates for the extractor agent, temperature and sampling parameters beyond "temperature 0.0," and specific Pydantic schema constraints are not provided. The CNN segmentation model is described as "previously described and validated" without citation to architecture details or training regimen in the main text. Hyperparameters for volumetric threshold decisions (±20%, >40%) are derived from BT-RADS standards but the rationale for edge-case handling (e.g., exactly 20% change) is not specified. The use of GPT-oss (20B)—a specific open-weight model—means performance may not generalize to other LLM implementations. The single-institution cohort from a high-volume brain tumor center raises questions about generalizability to community practice with different MRI protocols and case distributions.