MoG-VLM: Efficient Video Anomaly Detection through Motion-Guided Vision-Language Models
MoG-VLM attacks the prohibitive cost of zero-shot video anomaly detection with Vision-Language Models by treating motion not as a feature but as a lightweight gate that decides when to run expensive VLM inference. It introduces Temporal Gradient Magnitude (TGM) for content-aware frame sampling, anchor-triggered hierarchical lookback windows for temporal refinement, and a tier-aware fusion rule that balances frame-level and video-level语义 signals without any learned parameters. The work reports state-of-the-art zero-shot training-free AUC-ROC on XD-Violence (91.91%), UCF-Crime (84.30%), and MSAD (87.88%), alongside a claimed 1.31× speedup over the prior best method.
The paper offers a conceptually clean, training-free mechanism to reduce VLM inference costs in video anomaly detection, but its headline efficiency-accuracy win is more incremental than transformative. The core idea that motion should act purely as a control signal is well-motivated, yet the gains over the closest competitor are razor-thin (+0.57% on XD-Violence, +0.02% on UCF-Crime), and the speedup entangles adaptive sampling with differences in the underlying VLM backbones. ShanghaiTech performance is the weakest among reported zero-shot methods, which undermines the claim of broad competitiveness.
The training-free motion gating strategy is the strongest contribution. TGM analysis requires no labeled anomalies and avoids the costly learned policies used by prior adaptive-sampling work in action recognition. The tier-aware fusion is equally principled: because $\lambda_t$ inherits from the motion sampling tier rather than being tuned per dataset, the framework generalizes across benchmarks with a single set of scalar weights ($\lambda_{\text{base}}=0.3$, $\lambda_{\text{motion}}=0.4$, $\lambda_{\text{anchor}}=0.7$). The progressive ablation in Table 2 transparently attributes gains to each stage, which lends credibility to the design choices.
Three flaws weaken the empirical claims. First, ShanghaiTech AUC-ROC (78.96%) trails the zero-shot AnyAnomaly baseline (79.70%) by a clear margin and is far below recent supervised results, yet the text calls it "competitive." Second, the 1.31× speedup versus Unified-reasoning likely conflates methodological savings with backbone differences: MoG-VLM uses Qwen3-VL-8B and VideoLLaMA3-7B, whereas the prior method may employ heavier or lighter models, and the paper does not report per-component latencies for the competitor. Third, the TGM gate is intrinsically fragile: "abrupt illumination changes (e.g., lights switching on/off) and camera jitter can produce spurious activations that erode efficiency gains," a limitation that is honestly disclosed but never quantified in the benchmark results.
The benchmark table fairly partitions methods by supervision level, supporting the narrow claim that MoG-VLM advances the zero-shot training-free frontier on three of four datasets. However, the efficiency comparison is thin: the authors correctly note that "Only Unified [12] reports timing information" among prior zero-shot training-free methods, so the throughput baseline consists of a single prior system. The effective FPS metric amortizes cost over entire videos, which hides worst-case latency when sustained motion keeps the pipeline at its densest sampling rate. Moreover, because different VLMs are used across papers, the evidence does not isolate whether the speedup stems from motion-adaptive allocation or simply from cheaper encoders.
The authors release code and disclose all prompts (Appendix C) and core hyperparameters (Section 4.2), which is commendable. They also quantify stochasticity by reporting that "Run-to-run AUC variation across three reruns remains below 0.20% on all benchmarks." Reproduction nevertheless depends on access to very specific model checkpoints—Qwen3-VL-8B-Instruct, VideoLLaMA3-7B, and Qwen3-4B-Instruct-2507—which may not be archived in perpetuity. Timing was measured on a single A100 40 GB GPU, so independent reproduction on different hardware would require recalibration of wall-clock claims.
Vision-Language Models (VLMs) have shown strong potential for video anomaly detection, but dense frame-level inference makes them impractical for real-time deployment. Existing acceleration commonly relies on uniform sampling, which ignores content dynamics and can either waste computation on static segments or miss localized anomalies. We propose MoG-VLM a motion-adaptive anomaly detection framework that leverages motion purely as a lightweight control signal to decide when VLM inference is necessary. To this end, we introduce Temporal Gradient Magnitude (TGM) which performs content-aware sampling to concentrate computation on informative regions while avoiding redundant processing. We further introduce a hierarchical dual-stream design in which dense multi-frame VLM reasoning is activated only around frames with elevated anomaly likelihood, via anchor-triggered context-window expansion with early termination. A fusion module then reconciles these instantaneous semantic cues with temporal action dynamics. Across four benchmarks, our method achieves 91.91%, 84.30%, 87.88%, and 78.96% AUC-ROC on XD-Violence, UCF-Crime, MSAD, and ShanghaiTech, respectively, establishing new state of the art on three benchmarks under the zero-shot, training-free setting. On XD-Violence, we improve over the previous best result by +0.57% while delivering a 1.31x inference speedup. These results demonstrate that content-aware allocation of VLM capacity, processing fewer frames more intelligently rather than all frames uniformly, is a more effective axis of progress than uniformly densifying VLM inference across frames. Code is available on GitHub: https://github.com/kdawoud91/MoG-VLM.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.