Does Mechanistic Interpretability Transfer Across Data Modalities? A Cross-Domain Causal Circuit Analysis of Variational Autoencoders

The three methodological refinements are well-justified theoretically: posterior-calibrated CES corrects for distributional mismatch where fixed-range sweeps overestimate influence by 1/σ^eff_d (Proposition 1); path-specific activation patching provides exact telescoping decomposition for sequential architectures (Proposition 2); and FGD generalizes DCI to correlated feature groups while reducing to standard DCI when features are independent (Proposition 3). The experimental scale (75 independent runs with statistical correction) supports the key finding that specificity, not modularity, predicts downstream AUC (r = 0.460, p < .001), offering practical guidance for tabular VAE selection. The discovery that β-VAE capacity collapse is modality-dependent and reconstruction-mediated is mechanistically well-explained via the information bottleneck framework.

The primary limitation is the image benchmark selection: dSprites is a synthetic dataset with independent generative factors and binary pixels, which the authors acknowledge differs from "the complexities present within real world images" (Section 6.6). This idealized structure likely inflates the observed cross-modality gap compared to natural images with correlated features. The strong negative correlation between CES and reconstruction MSE (r = -0.886) creates ambiguity—while the authors interpret this as evidence that the β-VAE bottleneck operates through reconstruction degradation, it is equally plausible that CES simply measures decoder capacity rather than meaningful circuit structure, a confound the authors partially acknowledge in Section 6.2. Additionally, each architecture uses a single hyperparameter setting (e.g., β = 4.0) without ablation studies, making it impossible to determine if the observed β-VAE collapse is continuous or represents a phase transition at that specific value.

The evidence generally supports the core claims, though with noted caveats. The comparison to prior work by Roy and Misra [11] is fair and explicitly acknowledges the limitation that their convolutional encoder study avoided the sequential architecture saturation issue identified here. The statistical analysis appropriately uses non-parametric Wilcoxon tests with Holm–Šídák correction for multiple comparisons, though pooling across datasets assumes homogeneous architectural effects which Table 5 suggests may not hold (β-VAE CES varies dramatically by dataset). The random grouping ablation effectively validates that semantic groupings capture genuine structure (semantic modularity 0.099 vs random 0.080). However, the downstream evaluation uses only logistic regression on latent representations rather than domain-specific tasks (imputation, anomaly detection) that would more directly validate practical utility.

Reproducibility is mixed. The paper specifies training details (Adam optimizer with lr=10⁻³, ReduceLROnPlateau scheduling, early stopping patience=20, batch size 256, latent dimension 10) and seeds (42, 123, 456), and uses public UCI datasets and dSprites. Architecture specifications are detailed in Table 1. However, code and trained model checkpoints are promised "upon publication" rather than being currently available. The deterministic CUDA settings are mentioned but specific hardware (NVIDIA L40S) and software dependencies (PyTorch version, etc.) are not listed. The limited hyperparameter exploration (single β value per architecture) and fixed latent dimensionality without ablation would make it difficult to verify if the dramatic β-VAE collapse generalizes across the hyperparameter space or is specific to the chosen configuration.