SurfaceMesh

Tutorial

For a walk-through of kaolin.rep.SurfaceMesh features, see working_with_meshes.ipynb.

API

class kaolin.rep.SurfaceMesh(vertices, faces, normals=None, uvs=None, face_uvs_idx=None, face_normals_idx=None, material_assignments=None, materials=None, vertex_normals=None, vertex_tangents=None, face_normals=None, face_uvs=None, face_vertices=None, strict_checks=True, unset_attributes_return_none=True, allow_auto_compute=True)

This container class manages data attributes (pytorch tensors) of a homogeneous surface mesh (i.e. with all faces of an equal number of vertices, such as triangle mesh), or a batch of meshes following three Batching strategies. SurfaceMesh allows converting between these batching strategies, and automatically computes some attributes (such as face normals) on access (see supported attributes). This data type does not extend to volumetric tetrahedral meshes at this time and has limited support for materials.

Overview

To construct a SurfaceMesh object, pass vertices and faces (can be 0-length) tensors and any other supported attributes; batching strategy will be automatically determined from the inputs:

vertices = torch.rand((B, V, 3), dtype=torch.float32, device=device)
faces = torch.randint(0, V - 1, (F, 3), dtype=torch.long)
mesh = SurfaceMesh(faces, vertices)

To load a SurfaceMesh object:

from kaolin.io import obj, usd
mesh = obj.load_mesh(path)
mesh2 = usd.load_mesh(path2)

Examine mesh properties:

>>> print(mesh)  # Note auto-computable attributes
SurfaceMesh object with batching strategy NONE
            vertices: [42, 3] (torch.float32)[cpu]
               faces: [80, 3] (torch.int64)[cpu]
       face_vertices: if possible, computed on access from: (faces, vertices)
        face_normals: if possible, computed on access from: (normals, face_normals_idx) or (vertices, faces)
            face_uvs: if possible, computed on access from: (uvs, face_uvs_idx)
      vertex_normals: if possible, computed on access from: (faces, face_normals)

>>> mesh.face_normals  # Causes attribute to be computed
>>> print(mesh.describe_attribute("face_normals"))
        face_normals: [80, 3, 3] (torch.float32)[cpu]

To get a sense for what tensors the mesh can contain for different batching strategies see table below, or run:

>>> print(SurfaceMesh.attribute_info_string(SurfaceMesh.Batching.FIXED))
Expected SurfaceMesh contents for batching strategy FIXED
            vertices: (torch.FloatTensor) of shape ['B', 'V', 3]
               faces: (torch.IntTensor)   of shape ['F', 'FSz']
       face_vertices: (torch.FloatTensor) of shape ['B', 'F', 'FSz', 3]
             normals: (torch.FloatTensor) of shape ['B', 'VN', 3]
    face_normals_idx: (torch.IntTensor)   of shape ['B', 'F', 'FSz']
        face_normals: (torch.FloatTensor) of shape ['B', 'F', 'FSz', 3]
                 uvs: (torch.FloatTensor) of shape ['B', 'U', 2]
        face_uvs_idx: (torch.IntTensor)   of shape ['B', 'F', 'FSz']
            face_uvs: (torch.FloatTensor) of shape ['B', 'F', 'FSz', 2]
      vertex_normals: (torch.FloatTensor) of shape ['B', 'V', 3]
     vertex_tangents: (torch.FloatTensor) of shape ['B', 'V', 3]
material_assignments: (torch.IntTensor)   of shape ['B', 'F']
           materials: non-tensor attribute

Note

This class is using python logging, so set up logging to get diagnostics:

import logging
import sys
logging.basicConfig(level=logging.INFO, stream=sys.stdout)

Supported Attributes:

SurfaceMesh supports the following attributes, which can be provided to the constructor or set on the object. See supported batching strategies.

Attribute

Batching.NONE

Batching.FIXED

Batching.LIST

Computable?

vertices

V x 3

B x V x 3

[V_i x 3]

N

faces

F x FSz

F x FSz

[F_i x FSize]

N

face_vertices

F x FSz x 3

B x F x FSz x 3

[F_i x FSz_i x 3]

Y

normals

VN x 3

B x VN x 3

[VN_i x 3]

N

face_normals_idx

F x FSz

B x F x FSz

[F_i x FSz]

N

face_normals

F x FSz x 3

B x F x FSz x 3

[F_i x FSz_i x 3]

Y

uvs

U x 2

B x U x 2

[U_i x 2]

N

face_uvs_idx

F x FSz

B x F x FSz

[F_i x FSz]

N

face_uvs

F x FSz x 2

B x F x FSz x 2

[F_i x FSz_i x 2]

Y

vertex_normals

V x 3

B x V x 3

[V_i x 3]

Y

vertex_tangents

V x 3

B x V x 3

[V_i x 3]

Y

material_assignments

F

B x F

[F_i]

N

materials (non-tensor)

list

list of lists

list of lists

N

Legend: B - batch size, V - number of vertices, VN - number of vertex normals, U - number of UV coordinates, F - number of faces, FSz - number of vertices per face, {?}_i - count for the ith mesh, […] - list of tensors of shapes.

Note

SurfaceMesh will not sanity check consistency of manually set attributes.

__init__(vertices, faces, normals=None, uvs=None, face_uvs_idx=None, face_normals_idx=None, material_assignments=None, materials=None, vertex_normals=None, face_normals=None, face_uvs=None, face_vertices=None, strict_checks=True, unset_attributes_return_none=True, allow_auto_compute=True)

Initializes the object, while automatically detecting the batching strategy (see above for expected tensor dimensions). The vertices and faces tensors are required, but the number of faces/vertices can be 0. Any or all of the other attributes can be also provided or set later. See table above for expected tensor dimensions for different batching strategies.

Parameters

  • vertices (Union[torch.FloatTensor, list]) – xyz locations of vertices.

  • faces (Union[torch.LongTensor, list]) – indices into vertices array for each vertex of each face; this is the only fixed topology item for Batching.FIXED.

  • face_vertices (Optional[Union[torch.FloatTensor, list]]) – xyz locations for each vertex of each face; can be set directly or is auto-computable by indexing vertices with faces.

  • normals (Optional[Union[torch.FloatTensor, list]]) – xyz normal values, indexed by face_normals_idx.

  • face_normals_idx (Optional[Union[torch.LongTensor, list]]) – indices into normals for each vertex in each face.

  • face_normals (Optional[Union[torch.FloatTensor, list]]) – xyz normal values for each face; can be set directly or is auto-computable by 1) indexing normals with face_normals_idx, or (if either is missing and mesh is triangular) by 2) using vertex locations.

  • uvs (Optional[Union[torch.FloatTensor, list]]) – uv texture coordinates, indexed by face_uvs_idx.

  • face_uvs_idx (Optional[Union[torch.LongTensor, list]]) – indices into uvs for each vertex of each face.

  • face_uvs (Optional[Union[torch.FloatTensor, list]]) – uv coordinate values for each vertex of each face; can be set directly or is auto-computable by indexing uvs with face_uvs_idx.

  • vertex_normals (Optional[Union[torch.FloatTensor, list]]) – xyz normal values, corresponding to vertices; can be set directly or is auto-computable by averaging face_normals of faces incident to a vertex.

  • vertex_tangents (Optional[Union[torch.FloatTensor, list]]) – tangents values used to compute the orientation of normal perturbation, corresponding to vertices; can be set directly of is auto-computable, from vertices and face_uvs.

  • material_assignments (Optional[Union[torch.Tensor, list]]) – indices into materials list for each face.

  • materials (Optional[list]) – raw materials as output by the io reader.

  • strict_checks (bool) – if True, will raise exception if any tensors passed to the construcor have unexpected shapes (see shapes matrix above); note that checks are less strict for Batching.LIST batching (default: True).

  • unset_attributes_return_none (bool) – if set to False exception will be raised when accessing attributes that are not set (or cannot be computed), if True will simply return None (default: True).

  • allow_auto_compute (bool) – whether to allow auto-computation of attributes on mesh attribute access; see supported attributes (default: True).

Supported Batching Strategies

SurfaceMesh can be instantiated with any of the following batching strategies, and supports conversions between batching strategies. Current batching strategy of a mesh object can be read from mesh.batching or by running print(mesh).

For example:

mesh = kaolin.io.obj.load_mesh(path)
print(mesh)
mesh.to_batched()
print(mesh)
class Batching(value)

Batching strategies supported by the SurfaceMesh.

Batching.FIXED = 'FIXED'

a batch of meshes with fixed topology (i.e. same faces array)

Batching.LIST = 'LIST'

a list of meshes of any topologies

Batching.NONE = 'NONE'

a single unbatched mesh

classmethod attribute_info_string(batching)

Outputs information about expected mesh contents and tensor sizes, given a batching strategy. Only includes tensor and material attributes.

Parameters

batching (SurfaceMesh.Batching) – batching strategy

Returns

multi-line string of attributes and their shapes

Return type

(str)

check_sanity()

Checks that tensor attribute sizes are consistent for the current batching strategy. Will log any inconsistencies.

Returns

true if sane, false if not

Return type

(bool)

set_batching(batching, skip_errors=False)

Converts a mesh to a different batching strategy. Modifies the mesh in place and returns self.

All conversions are supported, except the following:

  • to NONE from FIXED or LIST batch with more than one mesh

  • to FIXED from LIST batch where fixed topology items are different

Parameters

  • batching (SurfaceMesh.Batching) – desirable batching strategy.

  • skip_errors – if true, will simply unset attributes that cannot be converted (useful if e.g. vertices are of fixed topology, but meshes have variable number of normals that cannot be stacked)

Returns

(self)

to_batched()

Convenience shorthand to convert unbatched mesh to FIXED topology batched mesh. Modifies the mesh in place and returns self.

Returns

(self)

getattr_batched(attr, batching)

Same as getattr or mesh.attr, but converts the attribute value to desired batching strategy before returning.

All conversions are supported, except the following:

  • to NONE from FIXED or LIST batch with more than one mesh

  • to FIXED from LIST batch where fixed topology items are different

Parameters
Returns

attribute value

classmethod cat(meshes, fixed_topology=True, skip_errors=False)

Concatenates meshes or batches of meshes to create a FIXED (if fixed_topology) or LIST batched mesh. Only attributes present in all the meshes will be preserved.

Parameters

  • meshes (Sequence[SurfaceMesh]) – meshes to concatenate; any batching is supported

  • fixed_topology (bool) – if to create a FIXED batched mesh (input must comply to assumptions)

  • skip_errors (bool) – if True, will not fail if some of the attributes fail to convert to target batching

Returns

new mesh

Return type

(SurfaceMesh)

static convert_attribute_batching(val, from_batching, to_batching, is_tensor=True, fixed_topology=False, batch_size=None)

Converts tensors between different SurfaceMesh.Batching strategies. The input value is expected to respect the provided from_batching. Will fail if conversion cannot be done

Approximate summary of conversions for tensor values:

  • NONE -> LIST: return [val]

  • NONE -> FIXED: return val.unsqueeze(0) unless fixed_topology

  • LIST -> NONE: return val[0], fails if list longer than 1

  • LIST -> FIXED: return torch.stack(val) (or val[0] if fixed_topology)

  • FIXED -> NONE: return val.squeeze(0) unless fixed_topology, fails if list longer than 1

  • FIXED -> LIST: return [val[i, ...] for i ...] (or [val for i ...] if fixed_topology)

Non-tensor values are stored as lists for FIXED and LIST batching.

Note

This method is only useful for converting batching of custom attributes and is not needed if only working with attributes natively supported by the SurfaceMesh.

Parameters

  • val (Union[Tensor, list]) – value to convert, must be consistent with from_batching

  • from_batching (Batching) – batching type to convert from

  • to_batching (Batching) – batching type to convert to

  • is_tensor (bool) – if the converted value is a tensor attribute (and not e.g. unstructured value to store in lists)

  • fixed_topology (bool) – if the attribute should be the same across items in a FIXED batching

  • batch_size (Optional[int]) – desirable batch size; must be consistent with val (will be guessed in most cases, but when converting fixed_topology items to e.g. LIST batching, this value is needed)

Attribute Access

By default, SurfaceMesh will attempt to auto-compute missing attributes on access. These attributes will be cached, unless their ancestors have requires_grad == True. This behavior of the mesh object can be changed at construction time (allow_auto_compute=False) or by setting mesh.allow_auto_compute later. In addition to this convenience API, explicit methods for attribute access are also supported.

For example, using convenience API:

# Caching is enabled by default
mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
print(mesh)
print(mesh.has_attribute('face_normals'))  # False
fnorm = mesh.face_normals  # Auto-computed
print(mesh.has_attribute('face_normals'))  # True (cached)

# Caching is disabled when gradients need to flow
mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
mesh.vertices.requires_grad = True   # causes caching to be off
print(mesh.has_attribute('face_normals'))  # False
fnorm = mesh.face_normals  # Auto-computed
print(mesh.has_attribute('face_normals'))  # False (caching disabled)

For example, using explicit API:

mesh = kaolin.io.obj.load_mesh(path, with_normals=False)
print(mesh.has_attribute('face_normals'))  # False
fnorm = mesh.get_or_compute_attribute('face_normals', should_cache=False)
print(mesh.has_attribute('face_normals'))  # False
get_attributes(only_tensors=False)

Returns names of all attributes that are currently set.

Parameters

only_tensors – if true, will only include tensor attributes

Returns

list of string names

Return type

(list)

has_attribute(attr)

Checks if a given attribute is present without trying to compute it, if not.

Parameters

attr (str) – attribute name

Returns

True if attribute is set and is not None

Return type

(bool)

has_or_can_compute_attribute(attr)

Returns true if this attribute is set or has all the requirements to be computed. Note that actual computation may still fail at run time.

Parameters

attr (str) – attribute name to check

Returns

True if exists or likely to be computable.

Return type

(bool)

probably_can_compute_attribute(attr)

Checks that the attributes required for computing attribute exist and returns true if the attribute is likely to be computable (not that it is not possible to determine this for sure without actually computing the attribute, as there could be runtime errors that occur during computation).

Parameters

attr (str) – attribute name to check

Returns

(bool) True if likely to be computable

get_attribute(attr)

Gets attribute without any auto-computation magic. If attribute is not set will either return None if mesh.unset_attributes_return_none or raise an exception.

Parameters

attr (str) – attribute name, see attributes

Returns

attribute value

Raises

AttributeError – if attribute nate is not supported, or if attribute is not set and not mesh.unset_attributes_return_none

get_or_compute_attribute(attr, should_cache=None)

Gets or computes an attribute, while allowing explicit control of caching of the computed value. If attribute is not set and cannot be computed will either return None if mesh.unset_attributes_return_none or raise an exception.

Parameters

  • attr (str) – attribute name, see attributes

  • should_cache (Optional[bool]) – if True, will cache attribute if it was computed; if False, will not cache; by default will decide if to cache based on requires_grad of variables used in computation (will not cache if any has requires_grad is True).

Returns

attribute value

Inspecting and Copying Meshes

To make it easier to work with, SurfaceMesh supports detailed print statements, as well as len(), copy(), deepcopy() and can be converted to a dictionary.

Supported operations:

import copy
mesh_copy = copy.copy(mesh)
mesh_copy = copy.deepcopy(mesh)
batch_size = len(mesh)

# Print default attributes
print(mesh)

# Print more detailed attributes
print(mesh.to_string(detailed=True, print_stats=True))

# Print specific attribute
print(mesh.describe_attribute('vertices'))
to_string(print_stats=False, detailed=False)

Returns information about tensor attributes currently contained in the mesh as a multi-line string.

Parameters

  • print_stats (bool) – if to print statistics about values in each tensor

  • detailed (bool) – if to include additional information about each tensor

Returns

multi-line string with attribute information

Return type

(str)

describe_attribute(attr, print_stats=False, detailed=False)

Outputs an informative string about an attribute; the same method used for all attributes in to_string.

Args:

print_stats (bool): if to print statistics about values in each tensor detailed (bool): if to include additional information about each tensor

Returns

multi-line string with attribute information

Return type

(str)

as_dict(only_tensors=False)

Returns currently set items as a dictionary. Does not auto-compute any items, but returns raw values.

Parameters

only_tensors (bool) – if true, will only include tensor attributes

Returns

currently set attributes as a dictionary

Return type

(dict)

Tensor Operations

Convenience operations for device and type conversions of some or all member tensors.

cuda(device=None, attributes=None)

Calls cuda on all or only on select tensor attributes, returns a copy of self.

Parameters

  • device – device to set

  • attributes (list of str) – if set, will only call cuda() on select attributes

Returns

(SurfaceMesh) shallow copy, with the exception of attributes that were converted

cpu(attributes=None)

Calls cpu() on all or only on select tensor attributes, returns a copy of self.

Parameters

attributes (list of str) – if set, will only call cpu() on select attributes

Returns

(SurfaceMesh) shallow copy, with the exception of attributes that were converted

to(device, attributes=None)

Converts all or select tensor attributes to provided device; returns copy of self.

Parameters

  • device (str, torch.device) – device to call torch tensors’ to method with

  • attributes (list of str) – if set, will only convert select attributes

Returns

(SurfaceMesh) shallow copy, with the exception of attributes that were converted

float_tensors_to(float_dtype)

Converts all floating point tensors to the provided type; returns shallow copy.

Parameters

float_dtype – torch dtype such as torch.float16, torch.float32

Returns

(SurfaceMesh) shallow copy, with the exception of attributes that were converted

detach(attributes=None)

Detaches all or select attributes in a shallow copy of self.

Parameters

attributes (list of str) – if set, will only call cuda on select attributes

Returns

(SurfaceMesh) shallow copy, with the exception of attributes that were converted

Other

static assert_supported(attr)
vertex_tangents