SIGGRAPH 2026¶
Kaolin at SIGGRAPH 2026 — Los Angeles
Join us in Los Angeles for two SIGGRAPH 2026 sessions showcasing new Kaolin capabilities and latest research from NVIDIA Creative and Applied AI Tools group (CAAT) — a talk on rapid prototyping of interactive Web tools over emerging AI and 3D research, and a hands-on lab on an in-the-wild capture-to-simulation pipeline for 3D Gaussian Splat scenes.
Get notified: Most of the features highlighted here will ship in an upcoming Kaolin release. Watch our GitHub repository and select Custom → Releases to be notified when it drops.
From 3D Captures to Simulated Digital Environments¶
Bring your laptop! This hands-on course walks through a capture-to-simulation pipeline for in-the-wild 3D Gaussian Splat scenes. Attendees will segment objects, predict volumetric mechanical properties, run physics simulation directly on Gaussian splats with a new version of Simplicits, and export shareable USD files with Kaolin’s custom physics schema — powered by recent Kaolin features and research:
ArtisanGS — interactive Gaussian splat selection and segmentation (AI + human in the loop)
VoMP — feed-forward prediction of volumetric mechanical property fields (Young’s modulus, Poisson’s ratio, density)
FreeForm / RKPM — mesh-free reduced-order simulation integrated in Kaolin Simplicits (CVPR 2026)
Kaolin USD I/O — export simulation-ready assets with Kaolin’s custom physics material schema
Content¶
Time |
Topic |
Supporting Materials |
|---|---|---|
10:15–10:30 am |
Introduction and environment setup, DLI, Lab overview. |
|
10:30–10:50 am |
ArtisanGS overview. Load an in-the-wild 3D Gaussian Splat capture, and interactively segment objects in a 3DGS scene; save in USD (hands on). |
|
10:50–11:05 am |
VoMP overview. Load splat segments, predict volumetric mechanical properties; save in USD (hands on). |
VoMP. |
11:05–11:35 am |
Simplicits/FreeForm overview. Multi-object Gaussian splat physics simulation using predicted properties; save in USD (hands on). Intro to newton coupling. |
FreeForm / RKPM, Simplicits, Simplicits-newton coupling, newton. |
11:35–11:45 am |
Concluding remarks and Q&A. |
— |
The pipeline emphasizes emerging research and new Kaolin capabilities, including custom USD schema — mechanical properties, skinned physics, and mixed representations are saved in easy-to-share USD files for easy collaboration.
Run It Yourself¶
Full lab resources will be available after the conference.
For similar examples, see:
To try, install Kaolin from source, following Installation.
Companion research code:
ArtisanGS, code coming soon and currently on web_framework_prerelease branch
VoMP, code and model available.
FreeForm / RKPM, already on
master.
Accelerating Interactive Prototypes Over Cutting Edge AI & 3D Research¶
This talk introduces Kaolin’s new web client-server framework (kaolin/visualize/dash) for
rapid prototyping of interactive browser interfaces over a large class of emerging technologies in
3D representations and AI. The design encapsulates patterns distilled from building many interactive
tool prototypes presented at SIGGRAPH technical papers, Real-Time Live, and Labs — supporting
flexible research workflows that combine user interaction with cutting-edge AI and 3D tooling.
Note
🌱 This toolkit is young and evolving, so use it with caution. Our target is rapid research prototype development on a secure internal network, and not production deployment.
The framework is not yet on master; it lives on the public
web_framework_prerelease branch.
See documentation (👈 start with kaolin.visualize for the
high-level Python API, examples, and tutorials; the JavaScript API covers the browser-side
window.kaolin utilities):
Content¶
Time |
Topic |
Supporting Materials |
|---|---|---|
3:00–3:08 pm |
Motivation: interactive prototypes over AI & 3D research |
Findings from prior CAAT interactive tools at SIGGRAPH (technical papers, RTL, Labs) |
3:08–3:15 pm |
Web client-server architecture in Kaolin |
|
3:15–3:18 pm |
Example applications on the branch |
See sample apps below |
3:18–3:20 pm |
Q & A |
— |
Sample applications (web_framework_prerelease branch)¶
Example client-server apps built with the framework:
Gaussian Splat Segmentation — interactive 2D/3D selection and SAM2-based segmentation of Gaussian splats (segment app README)
Toy Gaussian Splat Inpainter — server-side splat rendering with 2D diffusion inpainting baked back into the 3D scene (inpaint app README)
Run It Yourself¶
Check out the web_framework_prerelease branch.
Build Kaolin from source following Installation (Python >= 3.11, Node.js required for the web components).
Explore the sample apps under
kaolin/app/(see sample apps).TBD: step-by-step setup notebook and launch instructions.
Organizers¶
Clement Fuji Tsang
Clement is a Senior Research Scientist at NVIDIA, leading Kaolin Library development and working on Deep Learning applied to 3D and computer vision. Previously Clement was working on operators fusion and TensorRT integration in MXNet, as well as large scale training of Deep Learning models. His current focus is to develop and share Deep Learning solutions that are efficient and scalable on GPUs for 3D, computer vision and NLP tasks. He has been presenting Kaolin at SIGGRAPH 2022, 2024, and multiple GTCs.
Vismay Modi
Vismay is a Research Scientist at NVIDIA, working on Kaolin's representation-agnostic physics simulator. His focus is to enable interactive simulation of 3D objects in various representations, empowering artists, researchers and engineers to easily prototype, animate and simulate their generated or reconstructed 3D assets. His research goal is to ensure that simulation tools support a diverse set of interactive physics-based phenomena, including elasto-dynamics, muscle activation, joints, cloth, collisions with frictional contact, on any 3D representation, including NeRFs, 3D Gaussian splats, CT scans and more.
Masha (Maria) Shugrina
Masha is a Senior Research Scientist at NVIDIA and leader of the Creative and Applied AI Tools group (CAAT), which focuses on interactive applications of AI and on efforts to accelerate research, including the NVIDIA Kaolin Library. Her core research interest is advancing techniques that integrate AI and latest technologies into the interactive loop. She defended her PhD at the University of Toronto, and Master’s at MIT. She has also worked as a Research Engineer at Adobe and Senior Software Engineer and Tech Lead at Google.