Moving Regions, Not Moving Objects: Segmenting What Actually Moves under Camera Motion

The conceptual reframing is rigorous and well-motivated. Conditioning segmentation on ordered frame pairs elegantly separates scene motion from camera-induced displacement, and the target-memory cross-attention decoder is simple yet effective. The MoRSe-5K benchmark is carefully designed: the Static split forces models to prove they can stay silent when only the camera moves, 'a failure mode that sequence-level MOS benchmarks do not express.' Empirically, MoRSe dominates on moving-region metrics ($\mathcal{J}\&\mathcal{F}$ of 77.98 versus 69.64 for the strongest prior method) while producing nearly zero false-positive pixels on Static sequences (PER 0.0009%). The decision to predict camera pose only as an auxiliary task—'its predictions never enter the mask decoder'—prevents the mask path from overfitting to geometric residuals.

Despite the conceptual appeal, several limitations bear emphasis. First, training draws 92.3% of its sampling mass from synthetic 3D point-tracking sources, while real-video labels are curated object masks that only approximate moving regions. Second, the decoder computes each coordinate independently without query–query attention or spatial convolution, which the paper explicitly admits means 'a coordinate cannot recruit its neighbours for spatial context.' This produces known failures on thin structures and uniform surfaces, and critically on small-displacement motion: 'When a region moves by a very small amount between the source and target frames, MoRSe may fail to detect it, yielding missed detections.' Finally, on conventional sequence-level MOS benchmarks the method underperforms GMOS, which leads all three benchmarks because it is architecturally optimized for instance-level prediction.

The reliance on synthetic supervision also raises questions about domain gap. The authors flag that 'benchmarks outline whole instances once any part moves, so region-level masks on real footage must be drawn by hand,' and only 6% of training samples come from real mask sources. Transfer to complex real scenes with thin, deformable, or textureless moving regions is therefore not yet demonstrated.

The evidence largely supports the core claims. All baselines are rerun from released code under a common protocol, and the authors transparently document deviations such as substituting MoVieS's threshold after documenting that 'the released mask_by_motion value of 0.005 is calibrated for rendered splat flow and floods the frame under window-maximum aggregation.' The two-ground-truth evaluation—moving region versus moving object—fairly shows that MoRSe's advantage is largest when partial motion is measured, while GMOS and GMOS-S lead when whole instances are expected. The fetched GMOS paper confirms its use of a 'frozen π3 geometric encoder,' 'frozen SAM2 segmentation encoder,' and 'SAM2-based propagator,' validating the paper's characterization of it as object-level and foundation-model-dependent.

Reproducibility is excellent. The authors commit to releasing code, trained weights, training pair-index files, query banks, the full MoRSe-5K benchmark with Blender scene configurations, and evaluation scripts. Appendix C tabulates every hyperparameter from loss weights ($\lambda_{\mathrm{mask}}=3.0$, $\lambda_{\mathrm{cam}}=0.03$) to architecture constants ('Decoder blocks B 4, Decoder width C 256') and morphological cleanup operators. The only likely blocker is the undisclosed 'Internal synthetic' training source listed in Appendix A, which is not publicly identified and whose absence would alter the exact training mixture.