Evidence-Anchored Overdose Death Prediction from Death Investigation Reports

The systematic comparison of explanation regimes is the strongest element. The authors cleanly show that grounding explanations in exact report spans outperforms templated verbalizations: "Feature-guided rationales outperform feature-verbalized explanations on every split for both models, with the best feature-guided model exceeding the best feature-verbalized model by 2.3% macro-F1." The decoupled pipeline design also proves superior to joint self-rationalization, and the best configuration (Qwen3 with input plus rationales) achieves 0.831 macro-F1 against the SPOT baseline's 0.721, a meaningful gain for public-health decision support.

The silver supervision for rationale extraction is noisy and potentially biased. Three teacher models achieve only moderate aggregate agreement ($\kappa=0.361$), and some features like Prescription Medication(s) and Other Paraphernalia perform poorly in both teacher and student extraction. The paper acknowledges that "the silver-labeled training data for feature-guided rationale extraction is inherently noisy," yet the downstream classifier relies heavily on these spans.

Generalizability is also a major limitation: "Our evaluation is conducted on a single jurisdiction, and it remains an open question how well the learned models transfer to coroner or medical examiner offices with different documentation practices." Additionally, the surprising result that weakly-annotated data yields higher macro-F1 than human-annotated data—despite the latter containing gold feature labels—suggests the feature-annotated split may suffer from extreme class imbalance or label distribution shift that the paper does not fully mitigate.

The evidence broadly supports the claim that grounded rationales help, but the favorable comparison to SPOT should be interpreted carefully. SPOT is a rule-based system requiring extensive expert annotation, so an ML approach beating it is expected; the 15% relative improvement is useful but not benchmarked against modern encoder-only baselines or LLMs operating directly on the report without intermediate rationales beyond the authors' own $I \rightarrow O$ ablation. The logistic regression baseline over binary feature indicators reaches only 0.621 macro-F1, which usefully establishes that "knowing which features are present is not sufficient; the rationale text and report are needed to predict overdose outcomes," though this does not rule out stronger structured-prediction baselines.

Reproducibility is mixed. The authors use only open-weight final models (Llama-3.1-8B, Qwen3-8B) and specify LoRA hyperparameters ($r=32$, $\alpha=32$) and quantization settings, but they do not release code, data, or model checkpoints. The reliance on a teacher ensemble—including MedGemma-27B, Llama-70B, and Qwen3-30B—to produce silver labels means independent reproduction requires substantial compute and proprietary prompt engineering, even though the student model is lightweight. Crucially, the paper notes that "the expert feedback loop we evaluate is retrospective; prospective deployment would require careful interface design," so no prospective validation protocol or inference-time interface is provided.