kaolin.visualize.ipython

This module contains utilities for interactive visualization in IPython / Jupyter notebooks.

For relevant tutorials, see 👉 Tutorial Page.

Misc Utilities

kaolin.visualize.ipython.quick_viz(imgs, nrow=None, inches=15)

Display a batch of images in a grid using matplotlib.

Parameters

  • imgs (torch.Tensor) – Images of shape (B, C, H, W) or (C, H, W), C in [1, 3, 4], with values in [0, 1].

  • nrow (int, optional) – Images per row. Defaults to batch size.

  • inches (float) – Image width multiplier, default: 15.

Returns

The axes containing the displayed image.

Return type

matplotlib.axes.Axes

kaolin.visualize.ipython.update_canvas(canvas, image, format='PNG', quality=100)

Update an ipycanvas Canvas with an image tensor.

Parameters

  • canvas (ipycanvas.Canvas) – Target canvas to update.

  • image (torch.Tensor) – Image of shape (H, W, C) with dtype uint8.

  • format (str) – Image encoding format (‘PNG’ or ‘JPEG’). Default: ‘PNG’.

  • quality (int) – Compression quality (0-100). Default: 100.

Interactive 3D Visualizers

class kaolin.visualize.BaseIpyVisualizer(height, width, camera, render, fast_render=None, watched_events=None, max_fps=None, canvas=None, event_canvas=None, img_format='PNG', img_quality=100)

Bases: object

Base class for ipython visualizer.

To create a visualizer one must define the class attribute _WATCHED_EVENTS and the method _handle_event().

the method _handle_event() must use the methods self.render() or self.fast_render() to update the canvas

You can overload the constructor (make sure to reuse the base class one so that ipycanvas and ipyevents are properly used)

Parameters

  • height (int) – Height of the canvas.

  • width (int) – Width of the canvas.

  • camera (kaolin.render.camera.Camera) – Camera used for the visualization.

  • render (Callable) – render function that take a kal.render.camera.Camera as input. Must return a torch.ByteTensor as output, or a dictionary where the element ‘img’ is a torch.ByteTensor to be displayed, of shape \((\text{output_height}, \text{output_width}, 3)\), height and width don’t have to match canvas dimension.

  • fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than render.

  • watched_events (list of str) – Events to be watched by the visualizer (see ipyevents main documentation).

  • max_fps (optional, float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with render() close to real-time, and lower to 10 fps with slower rendering or network latency.

  • canvas (optional, ipycanvas.Canvas) – If you don’t want the visualizer to create a canvas automatically, pass the canvas object to be drawn on to this function. By default, this canvas will also be used for processing events. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events, so if you are drawing of a sub-canvas of a MultiCanvas, pass the MultiCanvas object as event_canvas=. Dimensions must be matching height and width.

  • event_canvas (optional, ipywidgets.DOMWidget) – If you want visualizer to receive events from a different canvas object from the one that the rendering shows on, pass this object here. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events. By default, the same canvas is used for events and drawing.

  • img_format (optional, str) – Format of the image provided to the canvas, change to ‘JPEG’ reduce latency. Default: ‘PNG’

  • img_quality (optional, int) – Quality of the image provided to the canvas, to reduce latency. Default: 100 (percent)

camera

The camera used for rendering.

Type

(kaolin.render.camera.Camera)

canvas

The canvas on which the rendering is displayed.

Type

(ipycanvas.Canvas)

current_output

The current output of the rendering function.

Type

(torch.Tensor)

event

the event handler.

Type

(ipyevents.Event)

event_canvas

The widget used to handle the events.

Type

(ipywidgets.DOMWidget)

fast_render

The fast rendering function.

Type

(Callable)

fast_render_update()

Update the Canvas with fast_render()

get_values_under_cursor(event)

returns output values under cursor provided by current event

height

The Canvas height.

Type

(int)

property max_fps

maximum fps for handling consecutive events

out

An output where error and prints are displayed.

Type

(ipywidgets.Output)

render

The rendering function.

Type

render (Callable)

render_update()

Update the Canvas with render()

show()

display the Canvas with interactive features

width

The Canvas width.

Type

(int)

class kaolin.visualize.IpyFirstPersonVisualizer(height, width, camera, render, fast_render=None, world_up=None, zoom_sensitivity=0.001, rotation_sensitivity=0.4, translation_sensitivity=1.0, key_move_sensitivity=0.05, max_fps=24.0, up_key='i', down_key='k', left_key='j', right_key='l', forward_key='o', backward_key='u', update_only_on_release=False, additional_watched_events=None, additional_event_handler=None, canvas=None, event_canvas=None, img_format='PNG', img_quality=100)

Bases: BaseIpyVisualizer

An interactive first person visualizer that can display on jupyter notebook.

You can move the orientation with the left button of the mouse, move the position of the camera with the right button of the mouse or the associated key, and zoom with the wheel.

Parameters

  • height (int) – Height of the canvas.

  • width (int) – Width of the canvas.

  • camera (kal.render.camera.Camera) – Camera used for the visualization.

  • render (Callable) – render function that take a kal.render.camera.Camera as input. Must return a torch.ByteTensor as output, or a dictionary where the element ‘img’ is a torch.ByteTensor to be displayed, of shape \((\text{output_height}, \text{output_width})\), height and width don’t have to match canvas dimension.

  • fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than render.

  • world_up (optional, torch.Tensor) – World up axis, of shape \((3,)\). If provided the camera will be reoriented to avoid roll. Default: camera.cam_up().

  • zoom_sensitivity (float) – Sensitivity of the wheel on zoom. Default: 1e-3.

  • rotation_sensitivity (float) – Sensitivity of the mouse on rotations. Default: 0.4.

  • translation_sensitivity (float) – Sensitivity of the mouse on camera translation. Default: 1.

  • key_move_sensitivity (float) – Amount of camera movement on key press. Default 0.05.

  • max_fps (optional, float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with render() close to real-time. And you lower to 10 fps with slower rendering or network latency. Default: 24 fps.

  • up_key (str) – key associated to moving up. Default ‘i’.

  • down_key (str) – key associated to moving up. Default ‘k’.

  • left_key (str) – key associated to moving up. Default ‘j’.

  • right_key (str) – key associated to moving up. Default ‘l’.

  • forward_key (str) – key associated to moving up. Default ‘o’.

  • backward_key (str) – key associated to moving up. Default ‘u’.

  • update_only_on_release (bool) – If true, the canvas won’t be updated while the mouse button is pressed and only when it’s released. To avoid freezes with very slow rendering functions. Default: False.

  • additional_watched_events (optional, list of str) – Additional events to be watched by the visualizer (see ipyevents main documentation). To be used for customed events such as enabling / disabling a feature on a key press. [‘wheel’, ‘mousedown’, ‘mouseup’, ‘mousemove’, ‘mouseleave’] are already watched. Default: None.

  • additional_event_handler (optional, Callable) – Additional event handler to be used for customed events such as enabling / disabling a feature on a key press. The Callable must take as input a tuple of (this visualizer object, the event). (see ipyevents main documentation).

  • res (optional, tuple of ints) – height and width of the canvas, if not defined will get inferred from canvas if defined, otherwise from the output of render.

  • canvas (optional, ipycanvas.Canvas) – If you don’t want the visualizer to create a canvas automatically, pass the canvas object to be drawn on to this function. By default, this canvas will also be used for processing events. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events, so if you are drawing of a sub-canvas of a MultiCanvas, pass the MultiCanvas object as event_canvas=. Dimensions must be matching height and width.

  • event_canvas (optional, ipywidgets.DOMWidget) – If you want visualizer to receive events from a different canvas object from the one that the rendering shows on, pass this object here. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events. By default, the same canvas is used for events and drawing.

  • img_format (optional, str) – Format of the image provided to the canvas, change to ‘JPEG’ reduce latency. Default: ‘PNG’

  • img_quality (optional, int) – Quality of the image provided to the canvas, to reduce latency. Default: 100 (percent)

world_up
Type

torch.Tensor

zoom_sensitivity
Type

float

rotation_sensitivity
Type

float

translation_sensitivity
Type

float

key_move_sensitivity
Type

float

update_only_on_release
Type

bool

class kaolin.visualize.IpyTurntableVisualizer(height, width, camera, render, fast_render=None, focus_at=None, world_up_axis=1, zoom_sensitivity=0.001, forward_sensitivity=0.001, rotation_sensitivity=1.5, translation_sensitivity=1.0, max_fps=24.0, update_only_on_release=False, additional_watched_events=None, additional_event_handler=None, canvas=None, event_canvas=None, img_format='PNG', img_quality=100)

Bases: BaseIpyVisualizer

An interactive turntable visualizer that can display on jupyter notebook.

You can move around with the mouse (using the left button), zoom with the wheel and get closer to the center with the wheel + control key.

Parameters

  • height (int) – Height of the canvas.

  • width (int) – Width of the canvas.

  • camera (kal.render.camera.Camera) – Camera used for the visualization. Note: The camera will be reoriented to look at focus_at and with respect to world_up.

  • render (Callable) – render function that take a kal.render.camera.Camera as input. Must return a torch.ByteTensor as output, or a dictionary where the element ‘img’ is a torch.ByteTensor to be displayed, of shape \((\text{output_height}, \text{output_width})\), height and width don’t have to match canvas dimension.

  • fast_render (optional, Callable) – A faster rendering function that may be used when doing high frequency manipulation such as moving the camera with a mouse. Default: same than render.

  • focus_at (optional, torch.Tensor) – The center of the turntable on which the camera is focusing on. Default: (0, 0, 0).

  • world_up_axis (optional, int) – The up axis of the world, in the coordinate system. Default: 1.

  • zoom_sensitivity (float) – Sensitivity of the wheel on zoom. Default: 1e-3.

  • forward_sensitivity (float) – Sensitivity of the wheel on forward. Default: 1e-3.

  • rotation_sensitivity (float) – Sensitivity of the mouse on left click movements. Default: 1.5.

  • translation_sensitivity (float) – Sensitivity of the mouse on right click movements. Default: 1.

  • max_fps (optional, float) – maximum framerate for handling consecutive events, this is useful when the rendering is slow to avoid freezes. Typically 24 fps is great when working on a local machine with render() close to real-time. And you lower to 10 fps with slower rendering or network latency. Default: 24 fps.

  • update_only_on_release (bool) – If true, the canvas won’t be updated while the mouse button is pressed and only when it’s released. To avoid freezes with very slow rendering functions. Default: False.

  • additional_watched_events (optional, list of str) – Additional events to be watched by the visualizer (see ipyevents main documentation). To be used for customed events such as enabling / disabling a feature on a key press. [‘wheel’, ‘mousedown’, ‘mouseup’, ‘mousemove’, ‘mouseleave’] are already watched. Default: None.

  • additional_event_handler (optional, Callable) – Additional event handler to be used for customed events such as enabling / disabling a feature on a key press. The Callable must take as input a tuple of (this visualizer object, the event). (see ipyevents main documentation).

  • canvas (optional, ipycanvas.Canvas) – If you don’t want the visualizer to create a canvas automatically, pass the canvas object to be drawn on to this function. By default, this canvas will also be used for processing events. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events, so if you are drawing of a sub-canvas of a MultiCanvas, pass the MultiCanvas object as event_canvas=. Dimensions must be matching height and width.

  • event_canvas (optional, ipywidgets.DOMWidget) – If you want visualizer to receive events from a different canvas object from the one that the rendering shows on, pass this object here. Note that in case of ipycanvas.MultiCanvas, only the parent MultiCanvas object can process events. By default, the same canvas is used for events and drawing.

  • img_format (optional, str) – Format of the image provided to the canvas, change to ‘JPEG’ reduce latency. Default: ‘PNG’

  • img_quality (optional, int) – Quality of the image provided to the canvas, to reduce latency. Default: 100 (percent)

focus_at
Type

torch.Tensor

world_up_axis
Type

int

zoom_sensitivity
Type

float

rotation_sensitivity
Type

float

mouse_sensitivity
Type

float

update_only_on_release
Type

bool