kaolin.ops.mesh

A mesh is a 3D object representation consisting of a collection of vertices and polygons.

Triangular meshes

Triangular meshes comprise of a set of triangles that are connected by their common edges or corners. In Kaolin, they are usually represented as a set of two tensors:

  • vertices: A torch.Tensor, of shape \((\text{batch_size}, \text{num_vertices}, 3)\), contains the vertices coordinates.

  • faces: A torch.LongTensor, of shape \((\text{batch_size}, \text{num_faces}, 3)\), contains the mesh topology, by listing the vertices index for each face.

Both tensors can be combined using kaolin.ops.mesh.index_vertices_by_faces(), to form face_vertices, of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\), listing the vertices coordinate for each face.

Tetrahedral meshes

A tetrahedron or triangular pyramid is a polyhedron composed of four triangular faces, six straight edges, and four vertex corners. Tetrahedral meshes inside Kaolin are composed of two tensors:

  • vertices: A torch.Tensor, of shape \((\text{batch_size}, \text{num_vertices}, 3)\), contains the vertices coordinates.

  • tet: A torch.LongTensor, of shape \((\text{batch_size}, \text{num_tet}, 4)\), contains the tetrahedral mesh topology, by listing the vertices index for each tetrahedron.

Both tensors can be combined, to form tet_vertices, of shape \((\text{batch_size}, \text{num_tet}, 4, 3)\), listing the tetrahedrons vertices coordinates for each face.

API

kaolin.ops.mesh.adjacency_matrix(num_vertices, faces, sparse=True)

Calculates a adjacency matrix of a mesh.

Parameters
  • num_vertices (int) – Number of vertices of the mesh.

  • faces (torch.LongTensor) – Faces of shape \((\text{num_faces}, \text{face_size})\) of the mesh.

  • sparse (bool) – Whether to return a sparse tensor or not. Default: True.

Returns

adjacency matrix

Return type

(torch.FloatTensor or torch.sparse.FloatTensor)

Example

>>> faces = torch.tensor([[0, 1, 2]])
>>> adjacency_matrix(3, faces)
tensor(indices=tensor([[0, 0, 1, 1, 2, 2],
                       [1, 2, 0, 2, 0, 1]]),
       values=tensor([1., 1., 1., 1., 1., 1.]),
       size=(3, 3), nnz=6, layout=torch.sparse_coo)
kaolin.ops.mesh.check_sign(verts, faces, points, hash_resolution=512)

Checks if a set of points is contained inside a watertight triangle mesh.

Shoots a ray from each point to be checked and calculates the number of intersections between the ray and triangles in the mesh. Uses the parity of the number of intersections to determine if the point is inside the mesh.

Parameters
  • verts (torch.Tensor) – Vertices, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).

  • faces (torch.Tensor) – Faces, of shape \((\text{num_faces}, 3)\).

  • points (torch.Tensor) – Points to check, of shape \((\text{batch_size}, \text{num_points}, 3)\).

  • hash_resolution (int) – Resolution used to check the points sign. Only used with CPU. Default: 512.

Returns

Tensor indicating whether each point is inside the mesh, of shape \((\text{batch_size}, \text{num_points})\).

Return type

(torch.BoolTensor)

Example

>>> device = 'cuda' if torch.cuda.is_available() else 'cpu'
>>> verts = torch.tensor([[[0., 0., 0.],
...                       [1., 0.5, 1.],
...                       [0.5, 1., 1.],
...                       [1., 1., 0.5]]], device = device)
>>> faces = torch.tensor([[0, 3, 1],
...                       [0, 1, 2],
...                       [0, 2, 3],
...                       [3, 2, 1]], device = device)
>>> axis = torch.linspace(0.1, 0.9, 3, device = device)
>>> p_x, p_y, p_z = torch.meshgrid(axis + 0.01, axis + 0.02, axis + 0.03)
>>> points = torch.cat((p_x.unsqueeze(-1), p_y.unsqueeze(-1), p_z.unsqueeze(-1)), dim=3)
>>> points = points.view(1, -1, 3)
>>> check_sign(verts, faces, points)
tensor([[ True, False, False, False, False, False, False, False, False, False,
         False, False, False,  True, False, False, False,  True, False, False,
         False, False, False,  True, False,  True, False]], device='cuda:0')
kaolin.ops.mesh.compute_vertex_normals(faces, face_normals, num_vertices=None)

Computes normals for every vertex by averaging face normals assigned to that vertex for every face that has this vertex.

Parameters
  • faces (torch.LongTensor) – vertex indices of faces of a fixed-topology mesh batch with shape \((\text{num_faces}, \text{face_size})\).

  • face_normals (torch.FloatTensor) – pre-normalized xyz normal values for every vertex of every face with shape \((\text{batch_size}, \text{num_faces}, \text{face_size}, 3)\).

  • num_vertices (int, optional) – number of vertices V (set to max index in faces, if not set)

Returns

of shape (B, V, 3)

Return type

(torch.FloatTensor)

kaolin.ops.mesh.face_areas(vertices, faces)

Compute the areas of each face of triangle meshes.

Parameters
  • vertices (torch.Tensor) – The vertices of the meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).

  • faces (torch.LongTensor) – the faces of the meshes, of shape \((\text{num_faces}, 3)\).

Returns

the face areas of same type as vertices and of shape \((\text{batch_size}, \text{num_faces})\).

Return type

(torch.Tensor)

kaolin.ops.mesh.face_normals(face_vertices, unit=False)

Calculate normals of triangle meshes. Left-hand rule convention is used for picking normal direction.

Parameters
  • face_vertices (torch.Tensor) – of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\).

  • unit (bool) – if true, return normals as unit vectors. Default: False.

Returns

face normals, of shape \((\text{batch_size}, \text{num_faces}, 3)\)

Return type

(torch.FloatTensor)

kaolin.ops.mesh.index_vertices_by_faces(vertices_features, faces)

Index vertex features to convert per vertex tensor to per vertex per face tensor.

Parameters
  • vertices_features (torch.FloatTensor) – vertices features, of shape \((\text{batch_size}, \text{num_points}, \text{knum})\), knum is feature dimension, the features could be xyz position, rgb color, or even neural network features.

  • faces (torch.LongTensor) – face index, of shape \((\text{num_faces}, \text{num_vertices})\).

Returns

the face features, of shape \((\text{batch_size}, \text{num_faces}, \text{num_vertices}, \text{knum})\).

Return type

(torch.FloatTensor)

kaolin.ops.mesh.inverse_vertices_offset(tet_vertices)

Given tetrahedrons with 4 vertices A, B, C, D. Compute the inverse of the offset matrix w.r.t. vertex A for each tetrahedron. The offset matrix is obtained by the concatenation of \(B - A\), \(C - A\) and \(D - A\). The resulting shape of the offset matrix is \((\text{batch_size}, \text{num_tetrahedrons}, 3, 3)\). The inverse of the offset matrix is computed by this function.

Parameters

tet_vertices (torch.Tensor) – Batched tetrahedrons, of shape \((\text{batch_size}, \text{num_tetrahedrons}, 4, 3)\).

Returns

Batched inverse offset matrix, of shape \((\text{batch_size}, \text{num_tetrahedrons}, 3, 3)\). Each offset matrix is of shape \((3, 3)\), hence its inverse is also of shape \((3, 3)\).

Return type

(torch.Tensor)

Example

>>> tet_vertices = torch.tensor([[[[-0.0500,  0.0000,  0.0500],
...                                [-0.0250, -0.0500,  0.0000],
...                                [ 0.0000,  0.0000,  0.0500],
...                                [0.5000, 0.5000, 0.4500]]]])
>>> inverse_vertices_offset(tet_vertices)
tensor([[[[   0.0000,   20.0000,    0.0000],
          [  79.9999, -149.9999,   10.0000],
          [ -99.9999,  159.9998,  -10.0000]]]])
kaolin.ops.mesh.packed_face_areas(vertices, first_idx_vertices, faces, num_faces_per_mesh)

Compute the areas of each face of triangle meshes.

Parameters
  • vertices (torch.Tensor) – The packed vertices of the meshes, of shape \((\text{num_vertices}, 3)\).

  • first_idx_vertices (torch.Tensor) – The first_idx associated to vertices, of shape \((\text{batch_size})\).

  • faces (torch.LongTensor) – The packed faces of the meshes, of shape \((\text{num_faces}, 3)\).

  • num_faces_per_mesh – The number of faces per mesh, of shape \((\text{batch_size})\).

Returns

The face areas of same type as vertices and of shape \((\text{num_faces})\).

Return type

(torch.Tensor)

kaolin.ops.mesh.packed_sample_points(vertices, first_idx_vertices, faces, num_faces_per_mesh, num_samples, areas=None)

Uniformly sample points over the surface of triangle meshes.

First face on which the point is sampled is randomly selected, with the probability of selection being proportional to the area of the face. then the coordinate on the face is uniformly sampled.

The return pointclouds are with fixed batching.

Parameters
  • vertices (torch.Tensor) – The packed vertices of the meshes, of shape \((\text{num_vertices}, 3)\).

  • first_idx_vertices (torch.Tensor) – The first_idx associated to vertices, of shape \((\text{batch_size})\).

  • faces (torch.LongTensor) – The packed faces of the meshes, of shape \((\text{num_faces}, 3)\).

  • num_faces_per_mesh – The number of faces per mesh, of shape \((\text{batch_size})\).

  • num_samples (int) – The number of point sampled per mesh.

  • areas (torch.Tensor, optional) – The areas of each face, of shape \((\text{num_faces})\), can be preprocessed, for fast on-the-fly sampling, will be computed if None (default).

Returns

  • The pointclouds, of shape \((\text{batch_size}, \text{num_points}, 3)\).

  • The indexes of the faces selected (as merged faces), of shape \((\text{batch_size}, \text{num_points}).\)

Return type

(torch.Tensor, torch.LongTensor)

kaolin.ops.mesh.sample_points(vertices, faces, num_samples, areas=None, face_features=None)

Uniformly sample points over the surface of triangle meshes.

First face on which the point is sampled is randomly selected, with the probability of selection being proportional to the area of the face. then the coordinate on the face is uniformly sampled.

If face_features is defined for the mesh faces, the sampled points will be returned with interpolated features as well, otherwise, no feature interpolation will occur.

Parameters
  • vertices (torch.Tensor) – The vertices of the meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).

  • faces (torch.LongTensor) – The faces of the mesh, of shape \((\text{num_faces}, 3)\).

  • num_samples (int) – The number of point sampled per mesh.

  • areas (torch.Tensor, optional) – The areas of each face, of shape \((\text{batch_size}, \text{num_faces})\), can be preprocessed, for fast on-the-fly sampling, will be computed if None (default).

  • face_features (torch.Tensor, optional) –

    Per-vertex-per-face features, matching faces order, of shape \((\text{batch_size}, \text{num_faces}, 3, \text{feature_dim})\). For example:

    1. Texture uv coordinates would be of shape \((\text{batch_size}, \text{num_faces}, 3, 2)\).

    2. RGB color values would be of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\).

    When specified, it is used to interpolate the features for new sampled points.

See also

index_vertices_by_faces() for conversion of features defined per vertex and need to be converted to per-vertex-per-face shape of \((\text{num_faces}, 3)\).

Returns

the pointclouds of shape \((\text{batch_size}, \text{num_samples}, 3)\), and the indexes of the faces selected, of shape \((\text{batch_size}, \text{num_samples})\).

If face_features arg is specified, then the interpolated features of sampled points of shape \((\text{batch_size}, \text{num_samples}, \text{feature_dim})\) are also returned.

Return type

(torch.Tensor, torch.LongTensor, (optional) torch.Tensor)

kaolin.ops.mesh.subdivide_tetmesh(vertices, tetrahedrons, features=None)

Subdivide each tetrahedron in tetmesh into 8 smaller tetrahedrons by adding midpoints. If per-vertex features (e.g. SDF value) are given, the features of the new vertices are computed by averaging the features of vertices on the edge. For more details and example usage in learning, see Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis NeurIPS 2021.

Parameters
  • vertices (torch.Tensor) – batched vertices of tetrahedral meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).

  • tetrahedrons (torch.LongTensor) – unbatched tetrahedral mesh topology, of shape \((\text{num_tetrahedrons}, 4)\).

  • features (optional, torch.Tensor) – batched per-vertex feature vectors, of shape \((\text{batch_size}, \text{num_vertices}, \text{feature_dim})\).

Returns

  • batched vertices of subdivided tetrahedral meshes, of shape \((\text{batch_size}, \text{new_num_vertices}, 3)\)

  • unbatched tetrahedral mesh topology, of shape \((\text{num_tetrahedrons} * 8, 4)\).

  • batched per-vertex feature vectors of subdivided tetrahedral meshes, of shape \((\text{batch_size}, \text{new_num_vertices}, \text{feature_dim})\).

Return type

(torch.Tensor, torch.LongTensor, (optional) torch.Tensor)

Example

>>> vertices = torch.tensor([[[0, 0, 0],
...               [1, 0, 0],
...               [0, 1, 0],
...               [0, 0, 1]]], dtype=torch.float)
>>> tetrahedrons = torch.tensor([[0, 1, 2, 3]], dtype=torch.long)
>>> sdf = torch.tensor([[[-1.], [-1.], [0.5], [0.5]]], dtype=torch.float)
>>> new_vertices, new_tetrahedrons, new_sdf = subdivide_tetmesh(vertices, tetrahedrons, sdf)
>>> new_vertices
tensor([[[0.0000, 0.0000, 0.0000],
         [1.0000, 0.0000, 0.0000],
         [0.0000, 1.0000, 0.0000],
         [0.0000, 0.0000, 1.0000],
         [0.5000, 0.0000, 0.0000],
         [0.0000, 0.5000, 0.0000],
         [0.0000, 0.0000, 0.5000],
         [0.5000, 0.5000, 0.0000],
         [0.5000, 0.0000, 0.5000],
         [0.0000, 0.5000, 0.5000]]])
>>> new_tetrahedrons
tensor([[0, 4, 5, 6],
        [1, 7, 4, 8],
        [2, 5, 7, 9],
        [3, 6, 9, 8],
        [4, 5, 6, 8],
        [4, 5, 8, 7],
        [9, 5, 8, 6],
        [9, 5, 7, 8]])
>>> new_sdf
tensor([[[-1.0000],
         [-1.0000],
         [ 0.5000],
         [ 0.5000],
         [-1.0000],
         [-0.2500],
         [-0.2500],
         [-0.2500],
         [-0.2500],
         [ 0.5000]]])
kaolin.ops.mesh.subdivide_trianglemesh(vertices, faces, iterations, alpha=None)

Subdivide triangular meshes following the scheme of Loop subdivision proposed in Smooth Subdivision Surfaces Based on Triangles. If the smoothing factor alpha is not given, this function performs exactly as Loop subdivision. Elsewise the vertex position is updated using the given per-vertex alpha value, which is differentiable and the alpha carries over to subsequent subdivision iterations. Higher alpha leads to smoother surfaces, and a vertex with alpha = 0 will not change from its initial position during the subdivision. Thus, alpha can be learnable to preserve sharp geometric features in contrast to the original Loop subdivision. For more details and example usage in learning, see Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis NeurIPS 2021.

Parameters
  • vertices (torch.Tensor) – batched vertices of triangle meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).

  • faces (torch.LongTensor) – unbatched triangle mesh faces, of shape \((\text{num_faces}, 3)\).

  • iterations (int) – number of subdivision iterations.

  • alpha (optional, torch.Tensor) – batched per-vertex smoothing factor, alpha, of shape \((\text{batch_size}, \text{num_vertices})\).

Returns

  • batched vertices of triangle meshes, of shape

    \((\text{batch_size}, \text{new_num_vertices}, 3)\).

  • unbatched triangle mesh faces, of shape

    \((\text{num_faces} \cdot 4^\text{iterations}, 3)\).

Return type

(torch.Tensor, torch.LongTensor)

Example

>>> vertices = torch.tensor([[[0, 0, 0],
...                           [1, 0, 0],
...                           [0, 1, 0],
...                           [0, 0, 1]]], dtype=torch.float)
>>> faces = torch.tensor([[0, 1, 2],[0, 1, 3],[0, 2, 3],[1, 2, 3]], dtype=torch.long)
>>> alpha = torch.tensor([[0, 0, 0, 0]], dtype=torch.float)
>>> new_vertices, new_faces = subdivide_trianglemesh(vertices, faces, 1, alpha)
>>> new_vertices
tensor([[[0.0000, 0.0000, 0.0000],
         [1.0000, 0.0000, 0.0000],
         [0.0000, 1.0000, 0.0000],
         [0.0000, 0.0000, 1.0000],
         [0.3750, 0.1250, 0.1250],
         [0.1250, 0.3750, 0.1250],
         [0.1250, 0.1250, 0.3750],
         [0.3750, 0.3750, 0.1250],
         [0.3750, 0.1250, 0.3750],
         [0.1250, 0.3750, 0.3750]]])
>>> new_faces
tensor([[1, 7, 4],
        [0, 4, 5],
        [2, 5, 7],
        [5, 4, 7],
        [1, 8, 4],
        [0, 4, 6],
        [3, 6, 8],
        [6, 4, 8],
        [2, 9, 5],
        [0, 5, 6],
        [3, 6, 9],
        [6, 5, 9],
        [2, 9, 7],
        [1, 7, 8],
        [3, 8, 9],
        [8, 7, 9]])
kaolin.ops.mesh.uniform_laplacian(num_vertices, faces)

Calculates the uniform laplacian of a mesh. \(L[i, j] = \frac{1}{num\_neighbours(i)}\) if i, j are neighbours. \(L[i, j] = -1\) if i == j. \(L[i, j] = 0\) otherwise.

Parameters
  • num_vertices (int) – Number of vertices for the mesh.

  • faces (torch.LongTensor) – Faces of shape \((\text{num_faces}, \text{face_size})\) of the mesh.

Returns

Uniform laplacian of the mesh of size \((\text{num_vertices}, \text{num_vertices})\)

Return type

(torch.Tensor)

Example

>>> faces = torch.tensor([[0, 1, 2]])
>>> uniform_laplacian(3, faces)
tensor([[-1.0000,  0.5000,  0.5000],
        [ 0.5000, -1.0000,  0.5000],
        [ 0.5000,  0.5000, -1.0000]])
kaolin.ops.mesh.vertex_tangents(faces, face_vertices, face_uvs, vertex_normals)

Compute vertex tangents.

The vertex tangents are useful to apply normal maps during rendering.

Parameters
  • faces (torch.LongTensor) – unbatched triangle mesh faces, of shape \((\text{num_faces}, 3)\).

  • face_vertices (torch.Tensor) – unbatched triangle face vertices, of shape \((\text{num_faces}, 3, 3)\).

  • face_uvs (torch.Tensor) – unbatched triangle UVs, of shape \((\text{num_faces}, 3, 2)\).

  • vertex_normals (torch.Tensor) – unbatched vertex normals, of shape \((\text{num_vertices}, 3)\).

Returns

The vertex tangents, of shape \((\text{num_vertices, 3})\)

Return type

(torch.Tensor)