kaolin.visualize¶
API¶
- class kaolin.visualize.BaseIpyVisualizer(height, width, camera, render, fast_render=None, watched_events=None, max_fps=None)¶
Bases:
object
Base class for ipython visualizer.
To create a visualizer one must define the class attribute _WATCHED_EVENTS and the method
_handle_event()
.the method
_handle_event()
must use the methodsself.render()
orself.fast_render()
to update the canvasYou can overload the constructor (make sure to reuse the base class one so that ipycanvas and ipyevents are properly used)
- Parameters
height (int) – height of the canvas.
width (int) – width of the canvas.
camera (kal.render.camera.Camera) – Camera used for the visualization.
render (Callable) – render function that take a
kal.render.camera.Camera
as input. Must return a torch.uint8 tensor as output, of shape \((\text{output_height}, \text{output_width})\), height and width don’t have to match canvas dimension.fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than
render
.watched_events (list of str) – Events to be watched by the visualizer (see ipyevents main documentation).
max_fps (float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with
render()
close to real-time, and lower to 10 fps with slower rendering or network latency.
- fast_render_update()¶
Update the Canvas with
fast_render()
- property max_fps¶
maximum fps for handling consecutive events
- render_update()¶
Update the Canvas with
render()
- show()¶
display the Canvas with interactive features
- class kaolin.visualize.IpyFirstPersonVisualizer(height, width, camera, render, fast_render=None, world_up=None, zoom_sensitivity=0.001, rotation_sensitivity=0.4, translation_sensitivity=1.0, key_move_sensitivity=0.05, max_fps=24.0, up_key='i', down_key='k', left_key='j', right_key='l', forward_key='o', backward_key='u', update_only_on_release=False, additional_watched_events=None, additional_event_handler=None)¶
Bases:
BaseIpyVisualizer
An interactive first person visualizer that can display on jupyter notebook.
You can move the orientation with the left button of the mouse, move the position of the camera with the right button of the mouse or the associated key, and zoom with the wheel.
- Parameters
height (int) – height of the canvas.
width (int) – width of the canvas.
camera (kal.render.camera.Camera) – Camera used for the visualization.
render (Callable) – render function that take a
kal.render.camera.Camera
as input. Must return a torch.uint8 tensor as output, of shape \((\text{output_height}, \text{output_width})\), height and width don’t have to match canvas dimension.fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than
render
.world_up (optional, torch.Tensor) – World up axis, of shape \((3,)\). If provided the camera will be reoriented to avoid roll. Default:
camera.cam_up()
.zoom_sensitivity (float) – Sensitivity of the wheel on zoom. Default: 1e-3.
rotation_sensitivity (float) – Sensitivity of the mouse on rotations. Default: 0.4.
translation_sensitivity (float) – Sensitivity of the mouse on camera translation. Default: 1.
key_move_sensitivity (float) – Amount of camera movement on key press. Default 0.05.
max_fps (optional, float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with
render()
close to real-time. And you lower to 10 fps with slower rendering or network latency. Default: 24 fps.up_key (str) – key associated to moving up. Default ‘i’.
down_key (str) – key associated to moving up. Default ‘k’.
left_key (str) – key associated to moving up. Default ‘j’.
right_key (str) – key associated to moving up. Default ‘l’.
forward_key (str) – key associated to moving up. Default ‘o’.
backward_key (str) – key associated to moving up. Default ‘u’.
update_only_on_release (bool) – If true, the canvas won’t be updated while the mouse button is pressed and only when it’s released. To avoid freezes with very slow rendering functions. Default: False.
additional_watched_events (optional, list of str) – Additional events to be watched by the visualizer (see ipyevents main documentation). To be used for customed events such as enabling / disabling a feature on a key press. [‘wheel’, ‘mousedown’, ‘mouseup’, ‘mousemove’, ‘mouseleave’] are already watched. Default: None.
additional_event_handler (optional, Callable) – Additional event handler to be used for customed events such as enabling / disabling a feature on a key press. The Callable must take as input a tuple of (this visualizer object, the event). (see ipyevents main documentation).
- camera¶
The camera used for rendering.
- Type
kal.render.camera.Camera
- canvas¶
The canvas on which the rendering is copied to.
- Type
ipycanvas.Canvas
- out¶
An output where error and prints are displayed.
- Type
ipywidgets.Output
- render¶
The rendering function.
- Type
Callable
- fast_render¶
- Type
Callable
- world_up¶
- Type
- show()¶
display the Canvas with interactive features.
- fast_render_update()¶
Update the Canvas with
fast_render()
.
- class kaolin.visualize.IpyTurntableVisualizer(height, width, camera, render, fast_render=None, focus_at=None, world_up_axis=1, zoom_sensitivity=0.001, forward_sensitivity=0.001, mouse_sensitivity=1.5, max_fps=24.0, update_only_on_release=False, additional_watched_events=None, additional_event_handler=None)¶
Bases:
BaseIpyVisualizer
An interactive turntable visualizer that can display on jupyter notebook.
You can move around with the mouse (using the left button), zoom with the wheel and get closer to the center with the wheel + control key.
- Parameters
height (int) – height of the canvas.
width (int) – width of the canvas.
camera (kal.render.camera.Camera) – Camera used for the visualization. Note: The camera will be reoriented to look at
focus_at
and with respect toworld_up
.render (Callable) – render function that take a
kal.render.camera.Camera
as input. Must return a torch.uint8 tensor as output of shape \((\text{output_height}, \text{output_width})\), height and width don’t have to match canvas dimension.fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than
render
.focus_at (optional, torch.Tensor) – The center of the turntable on which the camera is focusing on. Default: (0, 0, 0).
world_up_axis (optional, int) – The up axis of the world, in the coordinate system. Default: 1.
zoom_sensitivity (float) – Sensitivity of the wheel on zoom. Default: 1e-3.
forward_sensitivity (float) – Sensitivity of the wheel on forward. Default: 1e-3.
mouse_sensitivity (float) – Sensitivity of the mouse on movements. Default: 1.5.
max_fps (optional, float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with
render()
close to real-time. And you lower to 10 fps with slower rendering or network latency. Default: 24 fps.update_only_on_release (bool) – If true, the canvas won’t be updated while the mouse button is pressed and only when it’s released. To avoid freezes with very slow rendering functions. Default: False.
additional_watched_events (optional, list of str) – Additional events to be watched by the visualizer (see ipyevents main documentation). To be used for customed events such as enabling / disabling a feature on a key press. [‘wheel’, ‘mousedown’, ‘mouseup’, ‘mousemove’, ‘mouseleave’] are already watched. Default: None.
additional_event_handler (optional, Callable) – Additional event handler to be used for customed events such as enabling / disabling a feature on a key press. The Callable must take as input a tuple of (this visualizer object, the event). (see ipyevents main documentation).
- camera¶
The camera used for rendering.
- Type
kal.render.camera.Camera
- canvas¶
The canvas on which the rendering is copied to.
- Type
ipycanvas.Canvas
- out¶
An output where error and prints are displayed.
- Type
ipywidgets.Output
- render¶
The rendering function.
- Type
Callable
- fast_render¶
- Type
Callable
- focus_at¶
- Type
- show()¶
display the Canvas with interactive features.
- fast_render_update()¶
Update the Canvas with
fast_render()
.
- class kaolin.visualize.Timelapse(log_dir, up_axis='Y')¶
Bases:
object
- add_mesh_batch(iteration=0, category='', vertices_list=None, faces_list=None, uvs_list=None, face_uvs_idx_list=None, face_normals_list=None, materials_list=None)¶
Add meshes to visualizer output.
- Parameters
iteration (int) – Positive integer identifying the iteration the supplied meshes belong to.
category (str, optional) – Batch name.
vertices_list (list of tensors, optional) – Vertices for N meshes of shape (num_vertices, 3).
faces_list (list of tensors, optional) – Faces for N meshes of shape (num_faces, face_size).
uvs_list (list of tensors, optional) – UV coordinates for N meshes of shape (num_uvs, 2).
face_uvs_idx_list (list of tensors, optional) – Index of UV coordinates for N meshes of shape (num_faces, face_size).
face_normals_list (list of tensors, optional) – Face normals for N meshes of shape (num_faces, face_size, 3).
materials_list (list, optional) – List of materials for N meshes. For each mesh, if a list of io.Materials is supplied, each material is applied to the mesh as a ShadingVariant. A name for each material can be defined by supplying a dictionary in the form of {‘material_name’: material}.
- add_pointcloud_batch(iteration=0, category='', pointcloud_list=None, colors=None, points_type='point_instancer')¶
Add pointclouds to visualizer output.
- Parameters
iteration (int) – Positive integer identifying the iteration the supplied pointclouds belong to.
category (str, optional) – Batch name.
pointcloud_list (list of tensors, optional) – Batch of point clouds as (B x N x 3) tensor or list of variable length point cloud tensors, each (N_i x 3).
colors (list of tensors, optional) – Batch of RGB colors of length N.
points_type (str) – String that indicates whether to save pointcloud as UsdGeomPoints or PointInstancer. “usd_geom_points” indicates UsdGeomPoints and “point_instancer” indicates PointInstancer. Please refer here for UsdGeomPoints: https://graphics.pixar.com/usd/docs/api/class_usd_geom_points.html and here for PointInstancer https://graphics.pixar.com/usd/docs/api/class_usd_geom_point_instancer.html. Default: “point_instancer”.
- add_voxelgrid_batch(iteration=0, category='', voxelgrid_list=None, colors=None, semantic_ids=None)¶
Add voxelgrids to visualizer output.
- Parameters
iteration (int) – Positive integer identifying the iteration the supplied voxelgrids belong to.
category (str, optional) – Batch name.
voxelgrid_list (list of tensors, optional) – Batch of points of length N defining N pointclouds.
colors (list of tensors, optional) – Batch of RGB colors of length N.
semantic_ids (list of int, optional) – Batch of semantic IDs.
- class kaolin.visualize.TimelapseParser(log_dir)¶
Bases:
object
Utility class for working with log directories created using the Timelapse interface. For example, this class can be used to extract raw data from the written checkpoints as follows:
Example
# During training timelapse = Timelapse(log_dir) timelapse.add_pointcloud_batch(iteration=idx, category=”prediction”, pointcloud_list=[predictions])
# Examining training run parser = TimelapseParser(log_dir) path = parser.get_file_path(“pointcloud”, “prediction”, 0) cloud = kaolin.io.usd.import_pointclouds(path, time=iteration_number) # time should be iteration number # Visualize, save or analyze as desired
- class CategoryInfo(category, ids=None, end_time=0)¶
Bases:
object
Corresponds to a “category” specified in Timelapse for a specific type like “mesh”. The ids corresponds to the number of objects saved in calls like add_mesh_batch, and end_time is the largest end time in the group.
- add_instance(new_id, end_timecode)¶
- serializable()¶
- check_for_updates()¶
Updates parse information if it has changed in the logdir.
- Returns
(bool) - True if updates exist, False if not
- get_category_info(type, category)¶
- get_category_names_by_type(type)¶
- get_file_path(type, category, id)¶
Gets file path by keys. :param type: one of “mesh”, “pointcloud”, “voxelgrid” :type type: str :param category: category passed to Timelapse when writing checkpoints :type category: str :param id: id of the item within its batch :type id: int
- Returns
(str) or None
- static get_filepaths(logdir)¶
Get all USD file paths within a directory that match naming conventions imposed by Timelapse.
- num_mesh_categories()¶
- num_mesh_items()¶
- num_pointcloud_categories()¶
- num_pointcloud_items()¶
- num_voxelgrid_categories()¶
- num_voxelgrid_items()¶
- static parse_filepath_info(filepaths)¶
Parses a directory of checkpoints written by Timelapse module into a summary format.
- Parameters
filepaths – dictionary output by get_filepaths
- Returns
- dictionary keyed by checkpoint type (“mesh”, “pointcloud”, “voxelgrid”)
with each value a list of (serializable) CategoryInfo
- kaolin.visualize.update_canvas(canvas, image)¶