Tutorial Index

Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links point to the main branch and are not tied to the version. Here is a brief introduction to our tutorials and repository structure:

Detailed Tutorials

  • Representation-Agnostic Physics Simulation with Simplicits:
  • Differentiable Rendering Docs Tutorials:
  • Camera and Rasterization: Rasterize ShapeNet mesh with nvdiffrast and camera:

    • Load ShapeNet mesh

    • Preprocess mesh and materials

    • Create a camera with from_args() general constructor

    • Render a mesh with multiple materials with nvdiffrast

    • Move camera and see the resulting rendering

  • Optimizing Diffuse Lighting: Optimize lighting parameters with spherical gaussians and spherical harmonics:

    • Load an obj mesh with normals and materials

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse lighting: * Spherical harmonics * Spherical gaussian with inner product implementation * Spherical gaussian with fitted approximation

  • Optimize Diffuse and Specular Lighting with Spherical Gaussians:

    • Load an obj mesh with normals and materials

    • Generate view rays from camera

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse and specular lighting with spherical gaussians

  • Working with Surface Meshes:

    • loading and constructing kaolin.rep.SurfaceMesh objects

    • batching of meshes

    • auto-computing common attributes (like face_normals)

  • Working with 3D Gaussian Splat Models:

    • reading and writing kaolin.rep.GaussianSplatModel from PLY and USD formats

    • inspecting model contents and per-tensor statistics

    • moving tensors between devices and dtypes

    • applying affine transforms with as_transformed()

    • concatenating multiple Gaussian models into a single scene

    • filtering Gaussians by opacity mask and saving to disk

  • Understanding Structured Point Clouds (SPCs): walks through SPC features, covering:

    • under-the-hood explanation of SPC, why it’s useful and key ops

    • loading a mesh

    • sampling a point cloud

    • converting a point cloud to SPC

    • setting up camera

    • rendering SPC with ray tracing

    • storing features in an SPC

  • Differentiable Rendering: optimizes a triangular mesh from images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • computing mesh laplacian

    • DIB-R rasterization

    • differentiable texture mapping

    • computing mask intersection-over-union loss (IOU)

    • using Timelapse API for 3D checkpoints

    • visualizing 3D results of training

  • Fitting a 3D Bounding Box: fits a 3D bounding box around an object in images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • DIB-R rasterization

    • computing mask intersection-over-union loss (IOU)

  • 3D Checkpoint Visualization: explains saving 3D checkpoints and visualizing them, covering:

    • using Timelapse API for writing 3D checkpoints

    • understanding output file format

    • visualizing 3D checkpoints using Omniverse Kaolin App

    • visualizing 3D checkpoints using bundled kaolin-dash3d commandline utility

  • Reconstructing Point Cloud with DMTet: Trains an SDF estimator to reconstruct a mesh from a point cloud covering:

    • using point clouds data generated with Omniverse Kaolin App

    • loading point clouds from an USD file.

    • defining losses and regularizer for a mesh with point cloud ground truth

    • applying marching tetrahedra

    • using Timelapse API for 3D checkpoints

    • visualizing 3D checkpoints using kaolin-dash3d

Simple Recipes