GSHeadRelight: Fast Relightability for 3D Gaussian Head Synthesis
GSHeadRelight tackles real-time, 3D-consistent portrait relighting from in-the-wild images by combining a generative 3D Gaussian Splatting backbone with a lightweight Unified Neural Radiance Transfer model. It assigns per-Gaussian spherical-harmonics coefficients that are view-conditioned to model low-frequency specular effects, trains on FFHQ under a white-light assumption without costly light-stage data, and achieves roughly 240 FPS by making relighting a simple per-Gaussian linear operation after a single forward pass.
The paper offers an elegant engineering solution: radiance transfer coefficients $\mathbf{t}_p(\hat{\mathbf{v}})$ are stored as Gaussian attributes, so changing illumination requires only a dot product with SH light coefficients. The efficiency gain over NeRF-based relighting is real, and the ablations validate the white-light assumption and view-conditioning. However, the speedup claims are complicated by ambiguous GPU reporting (H800 in the text versus RTX A6000 in Table 1), the quantitative lighting metrics rely on an imperfect off-the-shelf estimator, and the method is fundamentally limited to low-frequency lighting by its SH basis.
The ablation studies convincingly justify the design. Training with RGB lighting labels instead of white light causes the model to bake illumination into albedo and collapse into style transfer, whereas the proposed unified radiance transfer yields stable metrics. The view-conditioned transfer vector also successfully removes specular residuals from albedo, with a Lighting Error of 0.6520 against 0.7019 when the view condition is removed.
First, the speed comparison is potentially misleading. The introduction claims ~240 FPS on an NVIDIA H800 and a 12\times speedup, yet Table 1 states prior measurements were taken on a single RTX A6000 GPU without clarifying whether the same hardware was used for the proposed method. Second, the quantitative lighting metrics are indirect and noisy: Lighting Error and Lighting Stability rely solely on an off-the-shelf estimator, and spatial-consistency evaluation with NeUS2 is circular because EG3D scores highly by sharing the same implicit-rendering biases. Third, the method cannot claim superiority over all Gaussian-based competitors because concurrent work such as PRTGaussian is cited but never compared, and high-frequency reflections are explicitly outside the model's reach.
The evidence supports the core efficiency and 3D-consistency claims: the explicit Gaussian representation achieves the highest identity-similarity score in Table 3 (0.7538) and avoids the 2D super-resolution artifacts that plague EG3D baselines. On the other hand, evidence for lighting quality is weaker. The proposed method is merely comparable to NeRFFaceLighting in Lighting Error (0.6213 versus 0.6377), not dominant. Comparisons are also skewed by rendering resolution differences and by the absence of a public code release, which prevents independent verification of the speed and metric numbers.
Implementation details are adequate for architecture but insufficient for full reproduction. The paper specifies 15M images on FFHQ, 3rd-order SH labels from DPR, MODNet foreground extraction, and a one-hidden-layer decoder with 64 units. However, no code repository is mentioned, and the project page does not guarantee model release. Without access to the exact GSGAN backbone configuration, discriminator conditioning, and training hyperparameters, independent reproduction of the reported FID 5.71 and 243 FPS values would require substantial trial and error.
Relighting and novel view synthesis of human portraits are essential in applications such as portrait photography, virtual reality (VR), and augmented reality (AR). Despite recent progress, 3D-aware portrait relighting remains challenging due to the demands for photorealistic rendering, real-time performance, and generalization to unseen subjects. Existing works either rely on supervision from limited and expensive light stage captured data or produce suboptimal results. Moreover, many works are based on generative NeRFs, which suffer from poor 3D consistency and low real-time performance. We resort to recent progress on generative 3D Gaussians and design a lighting model based on a unified neural radiance transfer representation, which responds linearly to incident light. Using only in-the-wild images, our method achieves state-of-the-art relighting results and a significantly faster rendering speed ($\times12$) compared to previous 3D-aware portrait relighting research. Project page: http://www.geometrylearning.com/GSHeadRelight/.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.