RDCompGS: Rate-Distortion Optimization for Compression of 3D Gaussian Splatting with Image Codecs
3D Gaussian Splatting enables high-quality real-time rendering but its millions of primitives incur heavy storage costs. RDCompGS targets compression pipelines that map Gaussian attributes into 2D grids for mature image codecs like H.265/HEVC and JPEG XL. The core idea is to close the loop between Gaussian optimization and subsequent coding by adding a differentiable coding-rate estimate, derived from DCT coefficient statistics, to the standard rendering loss, alongside an adaptive constraint that balances rate and distortion across quantization steps. By jointly optimizing for both reconstruction fidelity and compressibility during training, the method delivers substantial bit-rate savings without sacrificing rendering quality.
RDCompGS presents a clean and well-motivated formulation for end-to-end rate-distortion optimization within the image-codec 3DGS compression paradigm. By integrating a differentiable rate term $\mathcal{L}_{\mathrm{rate}} = k\sum_{i=1}^{N}\log_{2}\left(1+\frac{2e\sqrt{x_{i}^{2}+\epsilon}}{\Delta}\right)$ with an adaptive weight $\lambda_{\mathrm{rate}}(\Delta) = \lambda_{\mathrm{ref}}\left(\frac{\Delta}{\Delta_{\mathrm{ref}}}\right)^{1+\beta}$, the method effectively couples splatting optimization to the quantization step of subsequent coding. The reported BD-Rate improvements over SOG are large, yet the evaluation scope is narrow, the rate adjustment adds only marginal gain, and the optimization explicitly ignores compression-induced distortion. As the authors state, "The distortion considered in RDCompGS refers to the rendering distortion defined above, while the distortion introduced by compression is not explicitly modeled in the optimization process." (Section III-C)
The strongest part of the paper is the theoretically grounded differentiable rate estimator. Modeling DCT coefficients with a zero-mean Laplace distribution yields a compact, differentiable rate expression that can be backpropagated during Gaussian optimization. The empirical validation is convincing: "At QP values of 4, 10, 16, and 22, the Pearson correlation coefficients are 0.997, 0.996, 0.990, and 0.973, respectively." (Section IV-C). The ablation confirms that this component alone drives the majority of the gain: "incorporating rate estimation consistently improves the R-D performance across all three datasets, with BD-Rate reductions exceeding 50%." (Section IV-F1). The fact that gains hold across both H.265/HEVC and JPEG XL supports the generality of the rate model within the 2D-representation setting.
The rate constraint adjustment model offers surprisingly little benefit relative to its analytical complexity. The ablation shows that "Averaged over the three datasets, the adjustment model achieves a BD-Rate reduction of 1.76%." (Section IV-F2), and on Tanks & Temples the BD-Rate actually worsens by 0.77%. More importantly, the paper compares almost exclusively against SOG [31], which shares the same PLAS sorting and image-codec pipeline; it does not benchmark against native 3D compression methods (e.g., HAC++, ContextGS, LightGaussian outside of a pruning-combination test) or against CodecGS [30], which also optimizes Gaussians jointly with 2D feature planes. Position attributes are excluded from the rate term and coded losslessly, leaving a major portion of the bitstream unoptimized. Finally, key hyperparameters $\Delta_{\mathrm{ref}}$, $\lambda_{\mathrm{ref}}$, and $\beta$ are tuned on the Truck scene, which is then excluded from R-D evaluation: "Since the Truck scene is used to determine the reference parameters, it is excluded from all subsequent R-D evaluations to avoid potential bias." (Section IV-A2). This does not eliminate the risk that the reported gains are partially specific to that tuning.
The evidence strongly supports superiority over SOG within the PLAS-plus-codec paradigm: RDCompGS reports consistent BD-Rate improvements on Tanks & Temples, Deep Blending, and Mip-NeRF 360 under both H.265/HEVC and JPEG XL. The abstract claims that "RDCompGS achieves average BD-Rate reductions of 37.84% and 49.31% with H.265/HEVC and JPEG XL, respectively." (Abstract). However, the absence of comparisons to native 3D compression schemes—which typically achieve >100x compression—and to CodecGS, which also restructures the 3DGS representation during optimization, makes it impossible to judge whether the 2D-codec approach itself is competitive in absolute terms. Moreover, the R-D curves are built from only four operating points per codec (QP 4, 10, 16, 22 for HEVC), which can make BD-Rate estimates sensitive to interpolation behavior. Visual comparisons in Fig. 7 are illustrative but limited to three cherry-picked scenes.
The authors state that "The proposed method is implemented in PyTorch. Experiments are conducted on an NVIDIA GeForce RTX 5070 Ti GPU." (Section IV-A2), and they specify HEVC QP values and matching JPEG XL distances. However, no code, checkpoint, or complete training log has been released. Several practical details remain underspecified: for instance, "The coefficient k=1/40 is derived from the encoder frame rate of 25 frames per second." (Section IV-A2) lacks derivation, and the exact DCT block assembly, normalization constants, and PLAS re-schedule during densification would need to be guessed from Algorithm 1. Because the rate term must be evaluated on sorted 2D grids after every densification step, the additional per-iteration compute overhead is nontrivial, yet training time and memory usage are not reported. Independent reproduction is therefore possible in principle but blocked in practice by missing implementation artifacts.
3D Gaussian Splatting (3DGS) enables high-quality, real-time novel view synthesis but requires significant storage and transmission resources. Mapping Gaussian attributes to 2D representations enables efficient compression using mature image codecs. However, Gaussian representations that achieve comparable rendering quality are generally treated similarly during optimization, despite substantial differences in coding rate, leading to limited rate-distortion (R-D) performance after compression. To address this limitation, we propose RDCompGS, a 3DGS compression method that enables overall R-D optimization of Gaussian representations with respect to subsequent coding. A differentiable estimate of the coding rate is incorporated into the R-D optimization with rendering distortion. The rate constraint adjustment model further regulates the R-D trade-off across different compression conditions. Extensive experiments on three benchmark datasets demonstrate that RDCompGS achieves average Bj{\o}ntegaard Delta Rate reductions of 37.84% and 49.31% with H.265/HEVC and JPEG XL, respectively. The estimated rate also shows a strong correlation with the actual bit-rate under different compression conditions.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.