MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning
Class-incremental graph learning suffers from catastrophic forgetting because models are repeatedly fine-tuned on new classes, causing severe recency bias where earlier sessions are misclassified as recent ones. MAEM addresses this by freezing a dedicated Mixture-of-Experts expert per session and routing test nodes via the reconstruction error of session-specific masked autoencoders, eliminating the need for task identities or a shared trainable router. To prevent unbounded model growth, it merges least-used experts via Git Re-Basin alignment followed by topology-aware pseudo-data distillation, achieving strong replay-free performance across four benchmarks.
MAEM presents a sound architectural approach to replay-free continual learning on graphs by isolating knowledge in frozen session-specific experts and avoiding shared routers that drift toward recent data. The method achieves competitive or superior results on ogbn-arxiv, Cora-Full, Coauthor-CS, and WikiCS without replay buffers, and the capacity-control mechanism imposes only modest degradation. However, the theoretical guarantees for order invariance and routing reliability rest on strong assumptions---ignored cross-session edges, unique task-wise solutions, and expected-MSE-optimal MAEs---that do not hold exactly in practice, and the practical MAE loss is direction-sensitive rather than MSE, weakening the direct applicability of Theorem 2 as stated.
The strongest part of the paper is the empirical demonstration that frozen experts plus reconstruction-based routing effectively mitigate recency bias without replay. The ablation in Table 2 shows that removing aligned initialization, distillation, or topology-aware loss each causes substantial drops in Micro and AP on Cora-Full, confirming that every fusion component matters. The pseudo-data quality analysis in Appendix D.3 further supports the data-free fusion, indicating that pseudo embeddings recover about 89% of real-embedding accuracy on average.
The reliance on a parameter-free SGC backbone, while stabilizing representations across sessions, severely limits expressiveness and may handicap the method on non-homophilic or structurally complex graphs.
Expert fusion is also sensitive to SGC feature quality; if sessions overlap substantially in representation space---as evidenced by the larger degradation on WikiCS---the pseudo-data synthesis may under-cover difficult classes and amplify errors during forced merges. Theorem 1's claim of order invariance explicitly assumes away cross-session edges, yet the experimental protocol retains inter-session edges in the cumulative graph, making the theorem an idealized statement rather than a practical property.
The evidence largely supports the core claim that MAEM mitigates recency bias and maintains bounded capacity: MAEMN achieves the best Micro accuracy across all four datasets and the best AP on three of them, outperforming replay-based methods like CaT and PDGNNs in several metrics despite being replay-free. However, the comparison does not quantify the inference burden of querying many MAEs and classifiers at test time, and the routing accuracy on Cora-Full is only about 70--71%, meaning nearly 30% of nodes are initially sent to the wrong expert; the paper does not deeply analyze how classification remains robust despite this misrouting.
The paper provides detailed dataset protocols and the practical MAE loss formulation in Appendix F with exact hyperparameters, but critical implementation details for the DeepInversion-style pseudo-data generation---such as the optimization schedule and convergence criteria---are described only at a high level. Reproducing the Git Re-Basin alignment and fusion pipeline requires careful handling of disjoint class spaces and layer-wise Hungarian matching, and the absence of mentioned publicly available source code poses a significant barrier to independent verification. The paper also omits wall-clock time and GPU memory measurements, obscuring the true deployment cost of maintaining multiple frozen expert networks.
Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information unavailable in class-incremental learning. Learnable routers offer an alternative, yet continual fine-tuning the router on new data can bias routing toward recent components. We propose MAEM, a replay-free, capacity-controlled dynamic mixture-of-experts framework for class-incremental graph learning. MAEM builds representations with Simplified Graph Convolution (SGC) and trains a dedicated expert for each new session, then freezes it to protect prior knowledge. For routing without task IDs or a shared trainable router, each expert uses a masked autoencoder (MAE); test nodes are assigned to the expert with the lowest reconstruction error, reducing routing-level recency bias. When the expert pool reaches a fixed capacity, MAEM merges the least-used experts via Git Re-Basin alignment and further distills them into a fused expert using topology-aware pseudo data synthesized from MAE distributions (DeepInversion-style), without storing or revisiting historical graph data. On four class-incremental graph benchmarks, MAEM mitigates recency bias, outperforms strong continual graph learning baselines, and keeps model capacity bounded under strict replay-free learning.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.