GM-GReFEL: Geometry-Guided Spatiotemporal State-Space Learning for Robust Dynamic Facial Expression Recognition
GM-GReFEL attacks two coupled problems in dynamic facial expression recognition (DFER): the quadratic cost of self-attention in long clips and noisy crowdsourced labels. It replaces a spatiotemporal Vision Transformer with a Modulated GroupMamba backbone that scans six channel groups—four spatially and two temporally—at linear cost $\mathcal{O}(N)$, re-mixing them via a Channel Affinity Modulation gate. A geometry-aware reliability module blends the classifier output with trainable facial-landmark anchors, weighted by per-sample normalized entropy, to fall back on anatomy when the prediction is uncertain. The authors report best-fold results within 0.3 points of the top method on DFEW and ahead on MAFW, while openly noting that five-fold means are more mixed.
The paper presents a well-motivated combination of efficient state-space scanning and geometric reliability, but its empirical story is undermined by high fold-to-fold variance, weak ablation support for several components, and a reporting style that foregrounds best-fold peaks over more representative five-fold means. The authors themselves concede that on the five-fold mean GM-GReFEL is below S4D on DFEW and lower than self-supervised ViT methods on MAFW. Although the GroupMamba engine is analytically linear in token count, the full model carries 155.99M parameters and a latency of 47.4 ms, slightly higher than the 147.69M / 46.8 ms ViT-B/16 baseline. A further worry is non-determinism: the authors note that "repeating it on the Fold 5 checkpoint changed the UAR by 0.35 points and the WAR by 0.18 points," which signals uncontrolled variance in evaluation. The geometric reliability module is not shown to improve mean accuracy in Table I, where freezing the anchors actually yields a marginally higher UAR (64.95% versus 64.69%).
The linear-complexity spatiotemporal engine is the paper’s strongest element. Partitioning channels into six groups and scanning four spatially and two temporally is a principled way to keep compute linear in the number of tokens, as the analytical derivation $\mathcal{O}_{\mathrm{GroupMamba}} \approx \mathcal{O}(N \cdot D \cdot N_{state})$ confirms. This aligns with the original Mamba claim that selective state space models achieve linear scaling while matching Transformer long-range capacity. The paired t-test on DFEW also shows a modest but statistically significant temporal benefit (+1.61 UAR, one-sided $p=0.031$). The authors further deserve credit for transparently reporting fold-to-fold variance and evaluation noise rather than hiding it.
The ablation evidence does not support the claimed value of several key components. In Table I, removing trainable anchors, reducing GroupMamba-SSM depth to $L=1$, dropping jittered sampling, or switching to single-stage training all move mean UAR by at most 0.3 points, a range the authors themselves say is within the run-to-run variation of "about two points" in the hyperparameter sweep. Most damningly, freezing the geometric anchors actually raises mean UAR from 64.69% to 64.95%, so the reliability module shows no measurable accuracy gain. The headline results are best-fold peaks (66.57% UAR on DFEW, 45.66% on MAFW) that sit atop wide distributions: on MAFW the remaining folds range from 31.51% to 43.31% UAR, and the five-fold mean is 38.90% with a standard deviation of 5.65. This makes the best fold an outlier rather than a robust achievement, and the authors acknowledge that "the five-fold mean is the like-for-like basis for most baselines" on which GM-GReFEL is not state-of-the-art.
The analytical complexity reduction is well grounded and supported by Eq. (30)–(31), and the comparison to related work is partially fair but inconsistently formatted. The DFEW and MAFW tables headline best-fold results, while the cross-dataset summary in Table VII correctly notes that "the five-fold mean is the like-for-like basis for most baselines, and on it the picture is more balanced." On that basis, GM-GReFEL trails S4D on DFEW and on MAFW is behind T-MEP, DFER-CLIP, and HiCMAE in UAR; on FERV39k, "three pre-trained models (MAE-DFER, VideoMAE, and S4D) report higher UAR (43.1–43.4% versus our 42.3%)." The Grad-CAM and t-SNE visualizations are illustrative but do not quantify the contribution of the geometric reliability module; without an ablation forcing $C_f = 1$ or removing anchors entirely, the reader cannot judge whether the entropy-guided blending in Eq. (23) is doing real work or merely adding parameters.
The authors provide a GitHub repository, specify the optimizer (AdamW, cosine LR $10^{-4}$, dropout 0.5), and detail the two-stage S2D initialization with $\gamma_{\text{freeze}} = 0.01$. However, exact reproduction is threatened by non-deterministic evaluation: the authors state that "repeating it on the Fold 5 checkpoint changed the UAR by 0.35 points and the WAR by 0.18 points," which suggests uncontrolled variance. Batch size is not stated, the number of epochs per fold is relegated to Supplementary Material, and the hyperparameter sweep was limited to "DFEW Fold 1 under an accelerated 50-epoch schedule." Pre-trained static weights and anchor checkpoints are not described as released. These gaps, combined with high cross-fold variance ($\pm$5.65 UAR on MAFW), mean an independent reproduction could easily land anywhere inside the reported spread.
Recognizing facial expressions in unconstrained video suffers from two coupled problems: self-attention grows quadratically in cost with clip length, and crowdsourced labels are noisy. We present GM-GReFEL, a spatiotemporal state-space network with a geometry-aware decision stage. Its Modulated GroupMamba backbone replaces a spatiotemporal Vision Transformer. Six channel groups are scanned separately, four over the frame plane and two over time, so the cost grows linearly with sequence length ($\mathcal{O}(N)$); after each layer, a Channel Affinity Modulation gate mixes the groups again. On top of the backbone, a GReFEL reliability module turns the normalized entropy of each prediction into a mixing weight between the classifier output and a vote from trainable geometric anchors, so that facial geometry counts for more when the network is unsure. The model is initialized by static-to-dynamic weight transfer and trained with a Triple Loss. In five-fold cross-validation, the best fold scores 66.57\%/76.46\% UAR/WAR on DFEW, 0.3 points or less below the strongest compared method, and 45.66\%/62.27\% on MAFW, ahead of all compared methods; on FERV39k the mean over five splits, 42.30\%/52.66\%, is competitive. The five-fold means are 64.07\%/74.54\% on DFEW, on par with recent large pre-trained models, and 38.90\%/54.37\% on MAFW, below the best of them. A paired $t$-test on DFEW indicates a modest benefit of temporal modeling (+1.61 UAR points, one-sided $p=0.031$). Attention stays on plausible facial regions, and latency is close to a ViT-B/16 baseline (47.4 vs.\ 46.8 ms per clip). Code: https://github.com/ardiytama/GroupMamba-DFER.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.