Tutorial Index

Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links point to master.

Detailed Tutorials

  • Deep Marching Tetrahedra: reconstructs a tetrahedral mesh from point clouds with DMTet, covering:

    • generating data with Omniverse Kaolin App

    • loading point clouds from a .usd file

    • chamfer distance as a loss function

    • differentiable marching tetrahedra

    • using Timelapse API for 3D checkpoints

    • visualizing 3D results of training

  • Understanding Structured Point Clouds (SPCs): walks through SPC features, covering:

    • under-the-hood explanation of SPC, why it’s useful and key ops

    • loading a mesh

    • sampling a point cloud

    • converting a point cloud to SPC

    • setting up camera

    • rendering SPC with ray tracing

    • storing features in an SPC

  • Differentiable Rendering: optimizes a triangular mesh from images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • computing mesh laplacian

    • DIB-R rasterization

    • differentiable texture mapping

    • computing mask intersection-over-union loss (IOU)

    • using Timelapse API for 3D checkpoints

    • visualizing 3D results of training

  • Fitting a 3D Bounding Box: fits a 3D bounding box around an object in images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • DIB-R rasterization

    • computing mask intersection-over-union loss (IOU)

  • 3D Checkpoint Visualization: explains saving 3D checkpoints and visualizing them, covering:

    • using Timelapse API for writing 3D checkpoints

    • understanding output file format

    • visualizing 3D checkpoints using Omniverse Kaolin App

    • visualizing 3D checkpoints using bundled kaolin-dash3d commandline utility

  • Reconstructing Point Cloud with DMTet: Trains an SDF estimator to reconstruct a mesh from a point cloud covering:

    • using point clouds data generated with Omniverse Kaolin App

    • loading point clouds from an USD file.

    • defining losses and regularizer for a mesh with point cloud ground truth

    • applying marching tetrahedra

    • using Timelapse API for 3D checkpoints

    • visualizing 3D checkpoints using kaolin-dash3d

  • Camera and Rasterization: Rasterize ShapeNet mesh with nvdiffrast and camera:

    • Load ShapeNet mesh

    • Preprocess mesh and materials

    • Create a camera with from_args() general constructor

    • Render a mesh with multiple materials with nvdiffrast

    • Move camera and see the resulting rendering

  • Optimizing Diffuse Lighting: Optimize lighting parameters with spherical gaussians and spherical harmonics:

    • Load an obj mesh with normals and materials

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse lighting: * Spherical harmonics * Spherical gaussian with inner product implementation * Spherical gaussian with fitted approximation

  • Optimize Diffuse and Specular Lighting with Spherical Gaussians:

    • Load an obj mesh with normals and materials

    • Generate view rays from camera

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse and specular lighting with spherical gaussians

Simple Recipes