BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting

  • 1Tencent PCG, USA
  • 2George Mason University, USA
  • 3ETH Zurich, Switzerland
  • 4ISTA, Austria
Arxiv (to appear at 3D Vision 2025)

Abstract

We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique designed to represent and render 3D objects with surface and volumetric materials under dynamic illumination. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects. To unify surface and volumetric material within a cohesive appearance model, we adopt a light- and view-dependent scattering representation via bidirectional spherical harmonics. Our model does not use a specific surface normal-related reflectance function, making it more compatible with volumetric representations like Gaussian splatting, where the normals are undefined. We demonstrate our method by reconstructing and rendering objects with complex materials. Using One-Light-At-a-Time (OLAT) data as input, we can reproduce photorealistic appearances under novel lighting conditions in real time.

BiGS can relight objects with both near-field lights and distant light, and applies to objects of both surface-based and volumetric materials including subsurface scattering, iridescent reflection, and fuzzy surface.


Intrinsic Decomposition

The radiance is intrinsically decomposed into four components, from left to right, diffuse scattering, directional scattering, direct light transport, and indirect light transport. OLAT datasets are used to optimize the components for physical plausible decomposition results.


Point Light Relighting

Relight with a point light with varying distance, and view with a moving camera. When the light gets nearer, the object gets more lit.


Environment Map Relighting

BiGS also supports relighting using an environment map.

We demonstrate our method on capture data as well.


Acknowledgments

We thank the authors and contributors of gsplat and nerfstudio for developing these open-source projects; Jeremy Chew and Rafael Bischof for their feedback on the manuscript; Sergej Majboroda, Andreas Mischok, Oliksiy Yakovlyev, Jarod Guest and Dimitrios Savva for releasing their HDRI images into public domain which are used as environment maps in this project.

Citation

				
					@misc{zhenyuan2024bigs,
     title={BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting}, 
     author={Liu Zhenyuan and Yu Guo and Xinyuan Li and Bernd Bickel and Ran Zhang},
     year={2024},
     eprint={2408.13370},
     url={https://arxiv.org/abs/2408.13370}
 }
Paper

Links