H-2DGS: Height-Field Gaussian Splatting for Sharp and Deformable Neurosurgical Surface Reconstruction
H-2DGS addresses the need for sharp, photorealistic neurosurgical surface reconstruction that can fuse with MRI and tractography without obscuring internal brain structures. The authors propose a three-stage pipeline that bootstraps appearance with conventional 3D Gaussian Splatting, contracts volumetric density along camera rays using differentiable quantiles, and converts the fused TSDF surface into curved height-field Gaussian patches. By explicitly representing a thin, deformable cortical sheet rather than a diffuse volume, the method targets geometric precision and compactness at the cost of some rendering fidelity.
The paper offers a conceptually clean and clinically motivated extension of Gaussian splatting for a difficult domain—sparse, narrow-baseline surgical microscope video. Its staged conversion from volumetric reconstruction to an explicit height-field surface is well reasoned, and the postoperative deformation strategy is a genuine application contribution. Still, the evidence is limited to five small cases, and the substantial photometric regression relative to baseline methods tempers enthusiasm for the representation as a standalone view synthesizer.
The authors acknowledge that "H-2DGS has lower training-view PSNR and SSIM than the baselines, mainly losing high-frequency highlights." (Section 5.2) They further conclude that "a clear photometric gap remains." (Section 6)
The geometric regularization strategy is the paper's strongest technical element. The ray-wise thickness objective $\mathcal{L}_{\mathrm{thick}}$ uses soft differentiable quantiles to contract density without collapsing the bootstrap appearance, and the subsequent TSDF fusion aggregates multi-view depth into a shared surface before initializing patches. The resulting models are compact (7.97 MB) and render interactively, and qualitative results confirm that competing methods fragment or streak in uncovered views while H-2DGS preserves a localized cortical sheet.
The design rationale is sensible: "We therefore separate appearance acquisition from surface construction. Stage I obtains a robust photometric reconstruction with flexible volumetric Gaussians." (Section 3.1) On uncovered views, "H-2DGS remains a recognizable cortical layer." (Section 5.2)
The most significant limitation is the severe and consistent drop in photometric fidelity. Across all five cases PSNR and SSIM trail the baselines by large margins—for example, Preop 3 falls to 25.20 PSNR versus 40.07 for QGS—indicating that the thin surface representation sacrifices considerable appearance expressiveness despite spatially varying texture bases. Because fine vessels and cortical texture are clinically relevant, this trade-off may limit practical utility. The dataset is also extremely small (five cases, 60–80 images each), and the pipeline failed on at least one additional sequence because "physiological cortical pulsation introduced non-rigid inter-frame motion, causing SfM to confound tissue motion with camera motion and produce inaccurate poses." (Section 6)
The authors correctly admit that "Replacing volumetric support with a thin explicit surface removes geometric ambiguity, but also reduces the rendering expressiveness available for reproducing high-frequency appearance variations." (Section 6)
Quantitative results in Table 1 support the geometric claims: H-2DGS attains the lowest multi-view depth consistency (MVDC) values ($1\times10^{-5}$ to $4\times10^{-5}$) and the smallest memory footprint. However, the comparison is asymmetrical because the authors "do not match primitive counts to the baselines, since volumetric primitives lacking local texture generally need more of them to fit the same images." (Section 6) While this argument is reasonable, the lack of an ablation isolating primitive count from representation type makes it impossible to attribute the photometric gap solely to the compact budget. Qualitative fusion figures convincingly show tractography beneath a sharp boundary, yet without perceptual or clinical user studies the claim that this improves surgical cognition remains speculative.
The justification for the height-field primitive is well stated: "This separation lets appearance vary rapidly without introducing corresponding high-frequency oscillations in geometry." (Section 3.4)
Independently reproducing this work will be difficult until code and data are released. The authors state that "Open source code will be provided via Github upon publication," (Introduction) but it is not yet available. Hyperparameters and hardware (NVIDIA RTX 3050, 4 GB) are disclosed, yet critical algorithmic details—such as the exact TSDF voxel size, view-consistency filtering thresholds, and Newton iteration counts for ray-patch intersection—are only described qualitatively. The proprietary surgical dataset is governed by institutional agreements and cannot be easily replicated. Furthermore, because the pipeline depends on COLMAP-style SfM poses, its known failure under cortical pulsation means that reproduction requires either exceptionally stable tissue or externally tracked camera geometry.
Hardware and budget are noted: "All experiments run on an NVIDIA RTX 3050 GPU with 4 GB memory, 32 GB system memory, and an Intel i7-12700H CPU. H-2DGS uses 2,400 primitives per case, with 50 color and 20 opacity bases per primitive." (Section 5.1)
Neurosurgical navigation commonly presents magnetic resonance imaging (MRI) as 2D slices, making it difficult to relate the anatomy in those slices to the cortical surface observed through the surgical microscope. We seek a common 3D visualization that combines a photorealistic reconstruction of the exposed cortex during tumor surgery with whole-brain MRI context and diffusion tractography pathways. This requires the reconstructed cortex to form a clear boundary around the brain anatomy rather than a diffuse volume that obscures internal structures. Conventional 3D Gaussian Splatting (3DGS) can reproduce observed views but, under limited surgical view coverage, may distribute opacity through a thick and geometrically ambiguous band. We introduce height-field 2D Gaussian Splatting (H-2DGS), a three-stage pipeline that concentrates a robust 3DGS reconstruction, fuses its depth into a shared truncated signed distance field (TSDF) surface, and represents that surface with curved, surface-oriented Gaussian primitives. Separate height, color, and opacity bases provide local geometry and appearance, while a neighbor-transition objective coordinates overlapping patches. The explicit surface further supports postoperative alignment: MRI-derived depth deforms the reconstruction, and tracked-color supervision with alternating optimization preserves its appearance. Across five real neurosurgical cases, H-2DGS gives low multi-view depth inconsistency, a compact 7.97MB representation, and interactive rendering at 29--42ms per frame. The resulting sharp, deformable surface integrates clearly with tractography and MRI anatomy, including a postoperative resection cavity.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.