In inverse rendering, accurately modeling visibility and indirect radiance for incident light is essential for capturing secondary effects. Due to the absence of a powerful Gaussian ray tracer, previous 3DGS-based methods have either adopted a simplified rendering equation or used learnable parameters to approximate incident light, resulting in inaccurate material and lighting estimations. To this end, we introduce inter-reflective Gaussian splatting (IRGS) for inverse rendering. To capture inter-reflection, we apply the full rendering equation without simplification and compute incident radiance on the fly using the proposed differentiable 2D Gaussian ray tracing. Additionally, we present an efficient optimization scheme to handle the computational demands of Monte Carlo sampling for rendering equation evaluation. Furthermore, we introduce a novel strategy for querying the indirect radiance of incident light when relighting the optimized scenes. Extensive experiments on multiple standard benchmarks validate the effectiveness of IRGS, demonstrating its capability to accurately model complex inter-reflection effects.
Schematic illustration of the proposed IRGS. Starting from a set of 2D Gaussians equipped with material properties, we apply rasterization to generate albedo, roughness, position, and normal maps. We then evaluate the rendering equation using stratified sampling at the corresponding position, drawing geometry and material values from these feature maps. The radiance of incident light is decomposed into direct radiance from the environment map, and indirect radiance with visibility, obtained via 2D Gaussian ray tracing.
Performance of directly applying Gaussian ray tracing on a pretrained Gaussian splatting checkpoint in 3D Gaussian and 2D Gaussian cases, respectively.
Applying ray tracing directly on a 3DGS checkpoint results in a noticeable decline in rendering quality compared to splatting. To address this issue, we propose 2D Gaussian ray tracing (2DGRT), which performs ray tracing on 2D Gaussian primitives. Since 2D Gaussian disks have a well-defined ray-splat intersection, this approach eliminates inconsistencies in intersection points present in 3D Gaussians. Our proposed 2DGRT achieves significantly less quality degradation than 3DGRT in both quantitative metrics and visual results.
Qualitative comparison of NVS, material and lighting estimation, and relighting results on the Synthetic4Relight dataset.
Qualitative comparison of relighting results on TensoIR dataset.
Acknowledgements: The website template was borrowed from Lior Yariv. Image sliders are based on dics.
