Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
Holi-Spatial 360 targets a key data bottleneck for spatial intelligence: creating large-scale, photorealistic, semantically grounded 3D indoor environments without costly LiDAR or dense video capture. The pipeline converts sparse real-estate panoramas into coherent 3D Gaussian Splatting scenes by anchoring optimization with cross-view correspondences from a large reconstruction model, then densifies trajectories with adapted image and video generative priors. The output is Holi360, a dataset of over 1,000 multi-room scenes with dense RGB-D, 3D instance annotations, and 2 million spatial question-answer pairs, plus a benchmark for VLM spatial reasoning.
Holi-Spatial 360 offers a practical, well-motivated pipeline for reconstructing multi-room homes from sparse panoramas, supported by quantitative gains in rendering, geometry, and 3D grounding over strong baselines. The work is heavily integrative, relying on existing models for reconstruction, restoration, and annotation; its primary novelty is in sequencing and fine-tuning these components rather than in fundamental algorithmic advances. While the dataset and benchmark are valuable contributions, the limited geometric evaluation and significant reproducibility barriers temper the overall strength of the paper.
The correspondence-guided geometric optimization is the paper’s strongest idea. Leveraging DA3 as an offline correspondence teacher yields dense, forward-backward-consistent anchors that stabilize 3DGS in texture-poor regions where COLMAP and i3dgs fail, and the ablation confirms that removing this guidance severely degrades geometry. Training DiFix and Wan-VACE on held-out panorama views within the same scene is also a clever adaptation strategy that preserves real-world artifacts and yields measurable gains in rendering quality.
A primary limitation is the small evaluation scale: geometry is tested on only 10 RealSee3D scenes, rendering on 30 room-random scenes, and grounding ablations on just 5 scenes, raising questions about how the method generalizes across the full diversity of real-world homes.
The pipeline also exhibits compounding model dependence without isolated error analysis: DA3, DiFix, Wan-VACE, Boxer, and Qwen are chained together, and the paper admits that Wan-VACE generalization from rotational clips to translational trajectories is empirically observed and 'not a guarantee for trajectories outside the reconstructed region.' Additionally, the pseudo ground-truth for 3D grounding is built from sparse LiDAR and inconsistent 2D semantic labels that the authors acknowledge 'can introduce inaccuracies during this projection process.'
On the reconstruction side, the evidence is reasonably solid: Table 1 shows consistent NVS improvements, and the fair protocol in Appendix A initializes all methods with DA3 to isolate the contribution of correspondence-guided optimization. The qualitative comparisons in Figure 4 also support the claim that geometry-anchored restoration preserves scene identity better than PanoWorld or DiFix alone. On the VLM side, the comparison is less convincing: the large accuracy gains from fine-tuning Qwen3.5-4B on Holi360 (Table 4) are measured on the paper’s own benchmark, so the improvement partly reflects dataset-specific tuning rather than broad spatial reasoning gains. A stronger evaluation would test fine-tuned models on independent spatial reasoning benchmarks.
Independent reproduction would be difficult. The authors state that any public release will be limited to permissible materials, which given the real-estate source imagery likely excludes both raw panoramas and possibly reconstructed scene data. Although the pipeline architecture is described in Section 3 and initialization details are noted in Appendix A, critical hyperparameters such as the 3DGS loss weights $\lambda_{gen}$ and $\lambda_{mv}$ and the exact training schedules for DiFix and Wan-VACE adaptation are not reported. Without released code, model weights, or the co-visibility graph construction logic, the community will struggle to replicate the results or extend the work.
Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial memory. Adapted single-image and video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360 benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial reasoning capabilities.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.