kaolin.visualize.web.sockets

This module contains utlities for handling communication through Web Sockets for client-server applications. The base communication is handled by the kaolin.visualize.web.sockets.WebSocketHandlerManager, which can be configured with any number of available or custom message handlers implementing kaolin.visualize.web.sockets.SyncMessageHandlerProtocol or kaolin.visualize.web.sockets.AsyncMessageHandlerProtocol.

Handler Protocols

The library user only needs to understand and implement the following protocols.

class kaolin.visualize.web.sockets.MessageHandlerProtocol(*args, **kwargs)

Bases: Protocol

Protocol for WebSocket message handlers.

accepted_message_tags() → list[str]

Return list of message tags this handler responds to.

on_connection_open(write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Called when WebSocket connection opens.

class kaolin.visualize.web.sockets.SyncMessageHandlerProtocol(*args, **kwargs)

Bases: MessageHandlerProtocol, Protocol

Requires synchronous message handling.

on_message(message_tag: str, message_content: Dict, write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Handle incoming message.

Parameters

• message_tag –

• message_content – The message content to handle (bytes or string)

• write_message_fn –

class kaolin.visualize.web.sockets.AsyncMessageHandlerProtocol(*args, **kwargs)

Bases: MessageHandlerProtocol, Protocol

Requires asynchronous message handling.

async on_message(message_tag: str, message_content: Dict, write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Available Handlers and Other API

class kaolin.visualize.web.sockets.AnyRendererMessageHandler(render_func, device='cuda', tag_suffix='', rendering_options: Optional[RemoteRenderingOptions] = None, low_rendering_options: Optional[RemoteRenderingOptions] = None, high_rendering_options: Optional[RemoteRenderingOptions] = None)

Bases: AsyncMessageHandlerProtocol

Async handler that renders images on camera updates and render requests.

Handles ‘set_camera’ and ‘render’ messages (with optional tag_suffix). Runs rendering in a separate thread to avoid blocking the event loop.

Bookkeeping: tracks the rate of incoming render requests (request_fps) and the throughput implied by render+encode time (rendering_fps); see stats(). These are recorded but not yet used to adapt quality.

When high_rendering_options is configured, a final high-resolution frame is sent once render requests stop (a gap longer than _DEFAULT_IDLE_THRESHOLD_S); if it is None no such frame is sent.

Parameters

• render_func – Callable(camera, **kwargs) -> RGBA tensor (H, W, 4) or dict of outputs.

• device – Torch device the camera is moved to before rendering.

• tag_suffix – Optional suffix for message tags (e.g., ‘_secondary’).

• rendering_options – Default RemoteRenderingOptions used for requests. Defaults to a fresh RemoteRenderingOptions.

• low_rendering_options – If configured, can interpolate from rendering_options toward these faster options to keep up response speed (not yet used).

• high_rendering_options – If configured, the final frame after a gap in requests is rendered with these options.

accepted_message_tags() → list[str]

Return list of message tags this handler responds to.

encode_rendering(render_res, options: Optional[RemoteRenderingOptions] = None)

Encode render result (tensor or dict) as binary message.

on_connection_open(write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Called when WebSocket connection opens.

async on_message(message_tag: str, message_content: Dict, write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Handle incoming message.

render_and_send(write_message_fn, cam=None, options: Optional[RemoteRenderingOptions] = None)

Render with given camera/options, recording render+encode time, and send.

send_render_message(render_res, write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any], options: Optional[RemoteRenderingOptions] = None)

Encode and send render result as binary message.

stats() → Dict[str, Optional[float]]

Snapshot of bookkeeping metrics.

Returns a dict with request_fps (rate of incoming render requests), rendering_fps (throughput implied by render+encode time) and last_render_s (most recent render+encode duration). Values may be None before enough samples are collected.

class kaolin.visualize.web.sockets.GlobalWebSocketConnectionManager

Bases: object

Process-wide registry of active WebSocket connections.

Tracks live WebSocketHandlerManager instances (one per connected client) so server-side code can iterate over them, broadcast updates, or look up a specific connection independent of which client initiated an action.

This is a thread-safe singleton; access it via instance(). Connections register themselves automatically from WebSocketHandlerManager.open() and unregister from WebSocketHandlerManager.on_close(), so application code typically only needs to read this registry.

broadcast(raw_message)

connection_ids() → list[str]

Snapshot list of all currently-active connection ids.

connections() → list[kaolin.visualize.web.sockets.WebSocketHandlerManager]

Snapshot list of all currently-active connection handlers.

get(conn_id: str) → Optional[WebSocketHandlerManager]

Return the handler for conn_id if it is still connected, else None.

get_handlers_by_tab(tab_uuid: str) → list[kaolin.visualize.web.sockets.WebSocketHandlerManager]

Snapshot list of all live handlers that belong to the given browser tab.

The list may be empty if the tab has no open WS connection (e.g. mid-reload). Handlers register themselves under their tab_uuid automatically when WebSocketHandlerManager.open() parses the tab query argument.

classmethod instance() → GlobalWebSocketConnectionManager

Return the process-wide singleton, creating it on first call.

items() → list[tuple[str, kaolin.visualize.web.sockets.WebSocketHandlerManager]]

Snapshot list of (conn_id, handler) pairs for active connections.

register(handler: WebSocketHandlerManager) → str

Record a newly-opened connection and return its assigned id.

The id is allocated via UniqueIdGenerator with the websocket prefix so it is unique across all kaolin-issued ids in the process.

If handler.tab_uuid is set (typically populated by WebSocketHandlerManager.open() from the tab query argument), the handler is also indexed under that uuid so get_handlers_by_tab() can fan out updates to every WS handler belonging to a single browser tab.

unregister(conn_id: str) → None

Remove a closed connection. Idempotent: unknown ids are silently ignored.

class kaolin.visualize.web.sockets.MeshGeometryStreamer(mesh: SurfaceMesh)

Bases: SyncMessageHandlerProtocol

Streams mesh geometry to connected WebSocket clients.

Sends mesh data (vertices, normals, UVs, materials) on connection open and on ‘get_mesh’ requests.

Parameters

mesh – The SurfaceMesh to stream.

accepted_message_tags() → list[str]

Return list of message tags this handler responds to.

convert_material(m)

Convert PBRMaterial to dict suitable for WebSocket transmission.

encode_mesh()

Encode mesh as binary message for transmission.

on_connection_open(write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Called when WebSocket connection opens.

on_message(message_tag: str, message_content: Dict, write_message_fn: Callable[[Union[bytes, str, Dict[str, Any]], bool], Any])

Handle incoming message.

Parameters

• message_tag –

• message_content – The message content to handle (bytes or string)

• write_message_fn –

class kaolin.visualize.web.sockets.RemoteRenderingOptions(encode_format: str = 'png', jpeg_quality: int = 90, max_resolution: Optional[int] = None, kwargs: Optional[Dict[str, Any]] = None)

Bases: object

Options controlling how a remote frame is rendered and encoded.

A single set of these is applied per render; AnyRendererMessageHandler holds a default set plus optional low/high sets so quality can be traded off against responsiveness.

Parameters

• encode_format – How the rendered image is encoded on the wire: one of 'raw', 'png' or 'jpeg' (see kaolin.visualize.web.io.ImageFormat). Defaults to 'png'.

• jpeg_quality – 0..100 quality used only when encode_format is 'jpeg'. Defaults to _DEFAULT_JPEG_QUALITY.

• max_resolution – If set (and > 0), the camera is scaled so its largest image dimension equals this many pixels before rendering (aspect ratio preserved); None renders at the camera’s native resolution.

• kwargs – Extra keyword arguments forwarded to the render function, or None to call it with the camera only.

encode_format: str = 'png'

classmethod interpolate(a: RemoteRenderingOptions, b: RemoteRenderingOptions, alpha: float = 0.5) → RemoteRenderingOptions

Blend two option sets. alpha=0 returns a, alpha=1 returns b.

Numeric fields (jpeg_quality, max_resolution) are linearly interpolated and rounded when both endpoints define them; otherwise (and for the non-numeric encode_format / kwargs) the nearer endpoint is used (a for alpha < 0.5, else b).

jpeg_quality: int = 90

kwargs: Optional[Dict[str, Any]] = None

max_resolution: Optional[int] = None

class kaolin.visualize.web.sockets.RenderTimeFps(window_s: float = 1.0, now: ~typing.Callable[[], float] = <built-in function monotonic>)

Bases: object

Tracks render+encode durations and reports the implied throughput.

Unlike SlidingWindowFps (which measures how often requests arrive), this measures how fast frames could be produced back-to-back: fps = n / sum(durations) over the most recent window_s (i.e. the reciprocal of the mean per-frame render+encode time). This is the response-time bookkeeping used to decide whether rendering can keep up. Thread-safe (durations are recorded from the render worker thread).

fps() → Optional[float]

Implied renders/second from recent durations, or None if no samples.

last_duration() → Optional[float]

Most recent render+encode duration in seconds, or None.

record(duration_s: float, timestamp: Optional[float] = None) → None

Record the time taken to render and encode one frame (seconds).

reset() → None

class kaolin.visualize.web.sockets.SlidingWindowFps(window_s: float = 1.0, idle_threshold_s: float = 0.5, now: ~typing.Callable[[], float] = <built-in function monotonic>)

Bases: object

Sliding-window event-rate tracker (renders-per-second of arrivals).

Python port of the client-side InteractiveFps (kaolin/visualize/dash/components/src/ts/interact/fps.tsx). Interactive viewers render on demand, so frame/request arrivals come in bursts separated by arbitrarily long idle gaps; a naive moving average would drag the reported rate toward zero during those gaps. This measures the rate only over the most recent window_s of events and treats any gap longer than idle_threshold_s as the end of a burst (earlier samples are discarded). fps() returns None when no meaningful rate is available (fewer than two samples, or currently idle). Thread-safe.

event(timestamp: Optional[float] = None) → None

Record that an event (e.g. a render request) just occurred.

fps() → Optional[float]

Current rate (events/second) over the window, or None if idle.

is_idle() → bool

True iff no event has arrived within idle_threshold_s.

reset() → None

class kaolin.visualize.web.sockets.TuidCallable(func)

Bases: object

Wrap arguments to handlers in this to return a fresh per-client instance (handled by WebAppBuilder).

WebSocket Handler

The following websocket handler can be used with tornado-based applications for handling custom messages.

class kaolin.visualize.web.sockets.WebSocketHandlerManager(application: Application, request: HTTPServerRequest, **kwargs: Any)

Bases: WebSocketHandler

Tornado WebSocket handler that dispatches messages to registered handlers.

Routes incoming messages by tag to appropriate sync/async handlers. Provides thread-safe message writing via write_message_safe().

Handlers are constructed per-connection from the handler specs passed at application setup time; each WebSocket connection gets its own handler instances so that per-connection state (e.g. cameras, render queues) is not shared across clients.

Each live connection is also registered in GlobalWebSocketConnectionManager for the duration of its open socket so other server-side code can enumerate or broadcast to clients.

check_origin(origin)

Allow connections from any origin

property connection_id: Optional[str]

Id assigned by GlobalWebSocketConnectionManager while open.

initialize(handler_specs: list[tuple])

Construct per-connection message handlers from specs.

Parameters

handler_specs – list of (handler_class, args) or (handler_class, args, kwargs) tuples, where args is a tuple/list of positional arguments and kwargs is a dict of keyword arguments to pass to the handler constructor. A fresh handler instance is built for each WebSocket connection.

on_close()

Invoked when the WebSocket is closed.

If the connection was closed cleanly and a status code or reason phrase was supplied, these values will be available as the attributes self.close_code and self.close_reason.

Changed in version 4.0: Added close_code and close_reason attributes.

async on_message(raw_message)

Handles new messages on the socket.

open()

Open socket connection and send test message.

register_message_handler(handler: kaolin.visualize.web.sockets.SyncMessageHandlerProtocol | kaolin.visualize.web.sockets.AsyncMessageHandlerProtocol)

Register a handler for its accepted message tags.

send_sample_json_message()

set_camera(camera: Camera)

Programmatically set camera (triggers handlers).

write_message_safe(message, binary=False)

Thread-safe message write via IOLoop callback.