Tutorial Index¶
Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links point to master.
Detailed Tutorials¶
- Camera and Rasterization: Rasterize ShapeNet mesh with nvdiffrast and camera:
Load ShapeNet mesh
Preprocess mesh and materials
Create a camera with
from_args()
general constructorRender a mesh with multiple materials with nvdiffrast
Move camera and see the resulting rendering
- Optimizing Diffuse Lighting: Optimize lighting parameters with spherical gaussians and spherical harmonics:
Load an obj mesh with normals and materials
Rasterize the diffuse and specular albedo
Render and optimize diffuse lighting: * Spherical harmonics * Spherical gaussian with inner product implementation * Spherical gaussian with fitted approximation
- Optimize Diffuse and Specular Lighting with Spherical Gaussians:
Load an obj mesh with normals and materials
Generate view rays from camera
Rasterize the diffuse and specular albedo
Render and optimize diffuse and specular lighting with spherical gaussians
- Working with Surface Meshes:
loading and constructing
kaolin.rep.SurfaceMesh
objectsbatching of meshes
auto-computing common attributes (like
face_normals
)
- Deep Marching Tetrahedra: reconstructs a tetrahedral mesh from point clouds with DMTet, covering:
generating data with Omniverse Kaolin App
loading point clouds from a
.usd
filechamfer distance as a loss function
differentiable marching tetrahedra
using Timelapse API for 3D checkpoints
visualizing 3D results of training
- Understanding Structured Point Clouds (SPCs): walks through SPC features, covering:
under-the-hood explanation of SPC, why it’s useful and key ops
loading a mesh
sampling a point cloud
converting a point cloud to SPC
setting up camera
rendering SPC with ray tracing
storing features in an SPC
- Differentiable Rendering: optimizes a triangular mesh from images using DIB-R renderer, covering:
generating data with Omniverse Kaolin App, and loading this synthetic data
loading a mesh
computing mesh laplacian
DIB-R rasterization
differentiable texture mapping
computing mask intersection-over-union loss (IOU)
using Timelapse API for 3D checkpoints
visualizing 3D results of training
- Fitting a 3D Bounding Box: fits a 3D bounding box around an object in images using DIB-R renderer, covering:
generating data with Omniverse Kaolin App, and loading this synthetic data
loading a mesh
DIB-R rasterization
computing mask intersection-over-union loss (IOU)
- 3D Checkpoint Visualization: explains saving 3D checkpoints and visualizing them, covering:
using Timelapse API for writing 3D checkpoints
understanding output file format
visualizing 3D checkpoints using Omniverse Kaolin App
visualizing 3D checkpoints using bundled
kaolin-dash3d
commandline utility
- Reconstructing Point Cloud with DMTet: Trains an SDF estimator to reconstruct a mesh from a point cloud covering:
using point clouds data generated with Omniverse Kaolin App
loading point clouds from an USD file.
defining losses and regularizer for a mesh with point cloud ground truth
applying marching tetrahedra
using Timelapse API for 3D checkpoints
visualizing 3D checkpoints using
kaolin-dash3d
Simple Recipes¶
- I/O and Data Processing:
usd_kitchenset.py: loading multiple meshes from a
.usd
file and savingspc_from_pointcloud.py: converting a point cloud to SPC object
occupancy_sampling.py: computing occupancy function of points in a mesh using
check_sign
spc_basics.py: showing attributes of an SPC object
spc_dual_octree.py: computing and explaining the dual of an SPC octree
spc_trilinear_interp.py: computing trilinear interpolation of a point cloud on an SPC
- Visualization:
visualize_main.py: using Timelapse API to write mock 3D checkpoints
fast_mesh_sampling.py: Using CachedDataset to preprocess a ShapeNet dataset we can sample point clouds efficiently at runtime
- Camera:
cameras_differentiable.py: optimize a camera position
camera_transforms.py: using
Camera.transform()
functioncamera_ray_tracing.py: how to design a ray generating function using
Camera
objectscamera_properties.py: exposing some the camera attributes and properties
camera_opengl_shaders.py: Using the camera with glumpy
camera_movement.py: Manipulating a camera position and zoom
camera_init_simple.py: Making Camera objects with the flexible
Camera.from_args()
constructorcamera_init_explicit.py: Making
CameraIntrinsics
andCameraExtrinsics
with all the different constructors availablecamera_coordinate_systems.py: Changing coordinate system in a
Camera
object