I$^2$-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis
Multimodal sentiment analysis fuses linguistic, acoustic, and visual signals, yet standard fusion methods entangle modality-specific and cross-modal contributions in opaque joint representations. This paper proposes I2-KAN, a multi-pathway framework that explicitly maintains intra-modal and pairwise inter-modal information in separate pathways, refines each with a Task-Decoupled Linear-Gated Denoising Bottleneck, and fuses them via a structured Kolmogorov–Arnold Network. The core motivation is that a modality's standalone predictive utility can differ sharply from its interaction-conditioned utility, and the work aims to make this reasoning traceable at pathway, feature, and functional-edge levels.
I2-KAN is a conceptually appealing framework that yields competitive results and delivers genuinely traceable interpretability by preserving explicit pathway associations throughout KAN fusion. Its central empirical observation—that weak standalone pathways can be highly informative through pairwise interactions—is consistently supported by ablations and masking analyses across three benchmarks. That said, the paper tends to foreground favorable metrics while downplaying weaker ones, and its architectural claim of explicit pairwise inter-modal modeling is weaker than presented because cross-modal learning is largely deferred to the final KAN fusion rather than being structurally realized in the pairwise pathways themselves.
The strongest aspect of the work is its systematic demonstration that standalone and interaction-conditioned utility diverge. The pathway ablation in Table III shows that adding all three pairwise pathways improves Acc7 by 5.4 points on CMU-MOSI and by 5.2 points on CMU-MOSEI over an intra-modal-only baseline. Furthermore, the KAN edge visualizations in Fig. 3 empirically validate that the model learns dataset-specific nonlinear routing rather than a fixed fusion topology, supporting the claim that the architecture adapts to distinct multimodal characteristics.
The paper's reporting is selectively optimistic. On CH-SIMS v2 (Table II), I2-KAN records only 72.4 Acc3, trailing KuDA (74.3), CmIR (75.0), and KAN-MCP (75.0), yet the text only highlights that it achieves the best Acc2, F1, MAE, and Corr. Likewise, on CMU-MOSEI (Table I), its Acc7 of 54.4 is below PSA-MF (55.0), MDF (54.8), and ConFEDE (54.9). Methodologically, the inter-modal pathways are built by simple concatenation, and the authors explicitly state that no learnable interaction operator is introduced at that stage. Because each concatenated pair is then processed independently by its own TDL-GDB, true cross-modal interaction is not structurally modeled until the final KAN layer, which weakens the claim of explicit pairwise inter-modal reasoning. Additionally, the compact bottleneck dimension of 3 is extremely aggressive for heterogeneous multimodal pathways and risks discarding discriminative cues.
The ablation evidence supports the utility of the multi-pathway structure, as removing pairwise pathways causes clear degradation across all metrics on CMU-MOSEI. However, baseline comparisons are narrated unevenly: recent methods such as CmIR and KAN-MCP outperform I2-KAN on at least one key metric per dataset, and those gaps are not discussed. The claim that KAN outperforms a comparable MLP is supported only on CMU-MOSEI (Table V), leaving open whether this generalizes. The functional-edge analysis is descriptive and lacks quantitative comparison against post-hoc explanation methods, making it difficult to assess whether the purported interpretability gains translate into reliable actionable insights.
Implementation details specify PyTorch, optimizer settings, and feature extractors, and results are averaged over five random seeds, which is helpful. Nonetheless, no code repository is mentioned, and the KAN configuration is delegated to the default configuration of the adopted implementation, which is insufficient for exact reproduction. The TDL-GDB objective combines multiple auxiliary losses with coefficients that are not individually reported, and without released code, an independent group would need to tune these balances from scratch.
Multimodal sentiment analysis (MSA) aims to infer sentiment by integrating linguistic, acoustic, and visual information. However, many fusion methods integrate modality-specific representations and cross-modal interactions into joint representations, making it difficult to determine whether predictive utility arises from a modality itself or from its interactions with others. More fundamentally, a modality's standalone predictive utility may not faithfully reflect its interaction-conditioned predictive utility. To address this issue, we propose I$^2$-KAN, an interpretable multi-pathway framework that explicitly maintains intra-modal and pairwise inter-modal information in separate structural pathways. Each pathway is refined by a Task-Decoupled Linear-Gated Denoising Bottleneck (TDL-GDB) to learn compact and task-relevant representations, followed by structured Kolmogorov-Arnold Network fusion that retains explicit pathway associations and supports multi-level interpretation. Extensive experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS v2 demonstrate strong overall performance. Ablation and interpretability analyses consistently show across the three benchmarks that low standalone predictive utility does not necessarily imply low interaction-conditioned predictive utility: a modality that appears weak in isolation can remain informative through pairwise interactions. These findings highlight the interaction-dependent nature of multimodal utility and suggest that modality utility should be characterized from both standalone and interaction-conditioned perspectives.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.