F4Splat: Feed-Forward Predictive Densification for Feed-Forward 3D Gaussian Splatting
F4Splat tackles inefficient Gaussian allocation in feed-forward 3D Gaussian Splatting (3DGS), where existing methods uniformly assign Gaussians per pixel or voxel, causing redundancy and fixed budgets. The core idea is a learnable densification score that predicts spatial regions needing additional Gaussians based on geometric complexity and multi-view overlap, enabling adaptive allocation and explicit budget control without retraining. This matters because it delivers compact scene representations—using 10–28% of the Gaussians of prior work—while maintaining or improving rendering fidelity.
The paper presents a technically sound solution to a real limitation in feed-forward 3DGS. The densification-score-guided allocation is well-motivated by adaptive density control in optimization-based 3DGS, and the multi-scale prediction framework enables efficient budget control. Quantitative results on RealEstate10K and ACID convincingly demonstrate that spatially adaptive allocation reduces redundancy without sacrificing quality. However, the method relies heavily on recent pretrained backbones (VGGT and DINOv2) and exhibits training instability without careful regularization, which may limit accessibility for reproduction.
The central claim—that learned densification scores enable better allocation than uniform strategies—is strongly supported by ablations. Random allocation (LPIPS 0.194) and frequency-based heuristics (LPIPS 0.160) both underperform the learned score (LPIPS 0.143). The level-wise supervision strategy is validated as necessary for stable optimization. The densification signal, derived from view-space positional gradients $\mathbf{v}_g$ and trained via $\mathcal{L}^{\text{score}}_{\mathcal{G}}$, successfully correlates with reconstruction fidelity, allowing the model to concentrate Gaussians on fine details while avoiding overlap redundancy.
Training stability is a significant issue: removing scene-scale regularization causes catastrophic failure (PSNR drops from 25.47 to 4.82), indicating high sensitivity to the geometric prior. The budget-matching algorithm guarantees a bounded error $0 \leq \bar{N}_{\mathcal{G}} - N_{\mathcal{G}_{\tau}} < 4^{L-1} - 1$, which with $L=3$ allows up to 15 Gaussians of deviation; for tight budgets or larger $L$, this discrete quantization could become problematic. The method also inherits dependencies on very recent external models (VGGT, March 2025), which may not be stable or available long-term. The paper lacks analysis of failure modes under extreme budget constraints (<5% of typical counts) or highly textured scenes where the gradient-based densification signal might mislead.
Comparisons are generally fair: AnySplat was retrained under identical experimental conditions (RE10K multi-view, 27 hours on 8× H200), while other baselines use officially released weights. F4Splat achieves LPIPS 0.128 with 315K Gaussians versus AnySplat’s 0.143 with 1142K Gaussians (24-view setting), demonstrating superior efficiency. The method remains competitive with pose-free approaches (SPFSplat, NoPoSplat) despite operating in the strictly harder uncalibrated setting. Generalization to unseen ACID data is shown, though cross-dataset camera pose estimation accuracy drops relative to training-set performance (RE10K: 0.541 AUC@5° vs ACID: 0.262), suggesting some domain sensitivity.
Reproduction is challenging due to hardware demands (8× NVIDIA H200 GPUs) and dependencies on recent pretrained weights (VGGT and DINOv2). The training pipeline involves complex components: dynamic batch sizing inversely proportional to view count, Sim(3) alignment of target views for novel-view supervision, and a multi-scale decoder with level-specific losses. While the budget-matching algorithm is provided in the supplementary material, the threshold search relies on precomputed lookup tables requiring specific sorting operations. The project page is referenced but code availability is not explicitly confirmed in the provided text, and the reliance on VGGT—published concurrently or recently—poses a versioning risk.
Feed-forward 3D Gaussian Splatting methods enable single-pass reconstruction and real-time rendering. However, they typically adopt rigid pixel-to-Gaussian or voxel-to-Gaussian pipelines that uniformly allocate Gaussians, leading to redundant Gaussians across views. Moreover, they lack an effective mechanism to control the total number of Gaussians while maintaining reconstruction fidelity. To address these limitations, we present F4Splat, which performs Feed-Forward predictive densification for Feed-Forward 3D Gaussian Splatting, introducing a densification-score-guided allocation strategy that adaptively distributes Gaussians according to spatial complexity and multi-view overlap. Our model predicts per-region densification scores to estimate the required Gaussian density and allows explicit control over the final Gaussian budget without retraining. This spatially adaptive allocation reduces redundancy in simple regions and minimizes duplicate Gaussians across overlapping views, producing compact yet high-quality 3D representations. Extensive experiments demonstrate that our model achieves superior novel-view synthesis performance compared to prior uncalibrated feed-forward methods, while using significantly fewer Gaussians.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.