Differentiable Lighting

Kaolin Library implements low-level Spherical Harmonic (sh) and Spherical Gaussian (sg) lighting, including diffuse and specular lighting effects. These lighting models are different from light sources like point light and spot light, common in computer graphics, and instead approximate a wide range of lighting conditions with a fixed number of optimizable parameters. For example, Spherical Gaussians are used in “DIB-R++: learning to predict lighting and material with a hybrid differentiable renderer.” Chen, Wenzheng, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, and Sanja Fidler. NeurIPS 2021.

See low-level lighting functions in kaolin.render.lighting, as well as Easy PBR Shader, which incorporates SphericalGaussian lighting as an argument into an easy-to-render function for meshes.

Tutorials: Lighting Optimization

Optimize Diffuse and Specular Lighting with Spherical Gaussians

The examples/tutorial/sg_specular_lighting.ipynb tutorial shows a synthetic example of optimizing for lighting effects represented as Spherical Gaussians using standard PyTorch optimizers and photometric loss.

Optimizing Diffuse Lighting

The examples/tutorial/diffuse_lighting.ipynb tutorial shows how to optimize lighting parameters with spherical gaussians and spherical harmonics.

Tutorials: Visualization

Other tutorials include Spherical Gaussian lighting with kaolin.render.easy_render.render_mesh function, and show its usage for visualization and synthetic data rendering. For example:

• GLTF Visualizer showing how to render GLTF / glb meshes, such as those from the Objaverse datset, with PBR materials and lighting.

• Easy Mesh Render Tutorial showing how to render meshes from different formats with PBR materials and lighting.