kaolin.ops.mesh¶
A mesh is a 3D object representation consisting of a collection of vertices and polygons.
Triangular meshes¶
Triangular meshes comprise of a set of triangles that are connected by their common edges or corners. In Kaolin, they are usually represented as a set of two tensors:
vertices
: Atorch.Tensor
, of shape \((\text{batch_size}, \text{num_vertices}, 3)\), contains the vertices coordinates.faces
: Atorch.LongTensor
, of shape \((\text{batch_size}, \text{num_faces}, 3)\), contains the mesh topology, by listing the vertices index for each face.
Both tensors can be combined using kaolin.ops.mesh.index_vertices_by_faces()
, to form face_vertices
, of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\), listing the vertices coordinate for each face.
Tetrahedral meshes¶
A tetrahedron or triangular pyramid is a polyhedron composed of four triangular faces, six straight edges, and four vertex corners. Tetrahedral meshes inside Kaolin are composed of two tensors:
vertices
: Atorch.Tensor
, of shape \((\text{batch_size}, \text{num_vertices}, 3)\), contains the vertices coordinates.tet
: Atorch.LongTensor
, of shape \((\text{batch_size}, \text{num_tet}, 4)\), contains the tetrahedral mesh topology, by listing the vertices index for each tetrahedron.
Both tensors can be combined, to form tet_vertices
, of shape \((\text{batch_size}, \text{num_tet}, 4, 3)\), listing the tetrahedrons vertices coordinates for each face.
API¶
- kaolin.ops.mesh.adjacency_matrix(num_vertices, faces, sparse=True)¶
Calculates a adjacency matrix of a mesh.
- Parameters
- Returns
adjacency matrix
- Return type
(torch.FloatTensor or torch.sparse.FloatTensor)
Example
>>> faces = torch.tensor([[0, 1, 2]]) >>> adjacency_matrix(3, faces) tensor(indices=tensor([[0, 0, 1, 1, 2, 2], [1, 2, 0, 2, 0, 1]]), values=tensor([1., 1., 1., 1., 1., 1.]), size=(3, 3), nnz=6, layout=torch.sparse_coo)
- kaolin.ops.mesh.check_sign(verts, faces, points, hash_resolution=512)¶
Checks if a set of points is contained inside a watertight triangle mesh.
Shoots a ray from each point to be checked and calculates the number of intersections between the ray and triangles in the mesh. Uses the parity of the number of intersections to determine if the point is inside the mesh.
- Parameters
verts (torch.Tensor) – Vertices, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).
faces (torch.Tensor) – Faces, of shape \((\text{num_faces}, 3)\).
points (torch.Tensor) – Points to check, of shape \((\text{batch_size}, \text{num_points}, 3)\).
hash_resolution (int) – Resolution used to check the points sign. Only used with CPU. Default: 512.
- Returns
Tensor indicating whether each point is inside the mesh, of shape \((\text{batch_size}, \text{num_points})\).
- Return type
(torch.BoolTensor)
Example
>>> device = 'cuda' if torch.cuda.is_available() else 'cpu' >>> verts = torch.tensor([[[0., 0., 0.], ... [1., 0.5, 1.], ... [0.5, 1., 1.], ... [1., 1., 0.5]]], device = device) >>> faces = torch.tensor([[0, 3, 1], ... [0, 1, 2], ... [0, 2, 3], ... [3, 2, 1]], device = device) >>> axis = torch.linspace(0.1, 0.9, 3, device = device) >>> p_x, p_y, p_z = torch.meshgrid(axis + 0.01, axis + 0.02, axis + 0.03) >>> points = torch.cat((p_x.unsqueeze(-1), p_y.unsqueeze(-1), p_z.unsqueeze(-1)), dim=3) >>> points = points.view(1, -1, 3) >>> check_sign(verts, faces, points) tensor([[ True, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, True, False, False, False, False, False, True, False, True, False]], device='cuda:0')
- kaolin.ops.mesh.compute_vertex_normals(faces, face_normals, num_vertices=None)¶
Computes normals for every vertex by averaging face normals assigned to that vertex for every face that has this vertex.
- Parameters
faces (torch.LongTensor) – vertex indices of faces of a fixed-topology mesh batch with shape \((\text{num_faces}, \text{face_size})\).
face_normals (torch.FloatTensor) – pre-normalized xyz normal values for every vertex of every face with shape \((\text{batch_size}, \text{num_faces}, \text{face_size}, 3)\).
num_vertices (int, optional) – number of vertices V (set to max index in faces, if not set)
- Returns
of shape (B, V, 3)
- Return type
(torch.FloatTensor)
- kaolin.ops.mesh.face_areas(vertices, faces)¶
Compute the areas of each face of triangle meshes.
- Parameters
vertices (torch.Tensor) – The vertices of the meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).
faces (torch.LongTensor) – the faces of the meshes, of shape \((\text{num_faces}, 3)\).
- Returns
the face areas of same type as vertices and of shape \((\text{batch_size}, \text{num_faces})\).
- Return type
- kaolin.ops.mesh.face_normals(face_vertices, unit=False)¶
Calculate normals of triangle meshes. Left-hand rule convention is used for picking normal direction.
- Parameters
face_vertices (torch.Tensor) – of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\).
unit (bool) – if true, return normals as unit vectors. Default: False.
- Returns
face normals, of shape \((\text{batch_size}, \text{num_faces}, 3)\)
- Return type
(torch.FloatTensor)
- kaolin.ops.mesh.index_vertices_by_faces(vertices_features, faces)¶
Index vertex features to convert per vertex tensor to per vertex per face tensor.
- Parameters
vertices_features (torch.FloatTensor) – vertices features, of shape \((\text{batch_size}, \text{num_points}, \text{knum})\),
knum
is feature dimension, the features could be xyz position, rgb color, or even neural network features.faces (torch.LongTensor) – face index, of shape \((\text{num_faces}, \text{num_vertices})\).
- Returns
the face features, of shape \((\text{batch_size}, \text{num_faces}, \text{num_vertices}, \text{knum})\).
- Return type
(torch.FloatTensor)
- kaolin.ops.mesh.inverse_vertices_offset(tet_vertices)¶
Given tetrahedrons with 4 vertices A, B, C, D. Compute the inverse of the offset matrix w.r.t. vertex A for each tetrahedron. The offset matrix is obtained by the concatenation of \(B - A\), \(C - A\) and \(D - A\). The resulting shape of the offset matrix is \((\text{batch_size}, \text{num_tetrahedrons}, 3, 3)\). The inverse of the offset matrix is computed by this function.
- Parameters
tet_vertices (torch.Tensor) – Batched tetrahedrons, of shape \((\text{batch_size}, \text{num_tetrahedrons}, 4, 3)\).
- Returns
Batched inverse offset matrix, of shape \((\text{batch_size}, \text{num_tetrahedrons}, 3, 3)\). Each offset matrix is of shape \((3, 3)\), hence its inverse is also of shape \((3, 3)\).
- Return type
Example
>>> tet_vertices = torch.tensor([[[[-0.0500, 0.0000, 0.0500], ... [-0.0250, -0.0500, 0.0000], ... [ 0.0000, 0.0000, 0.0500], ... [0.5000, 0.5000, 0.4500]]]]) >>> inverse_vertices_offset(tet_vertices) tensor([[[[ 0.0000, 20.0000, 0.0000], [ 79.9999, -149.9999, 10.0000], [ -99.9999, 159.9998, -10.0000]]]])
- kaolin.ops.mesh.packed_face_areas(vertices, first_idx_vertices, faces, num_faces_per_mesh)¶
Compute the areas of each face of triangle meshes.
- Parameters
vertices (torch.Tensor) – The packed vertices of the meshes, of shape \((\text{num_vertices}, 3)\).
first_idx_vertices (torch.Tensor) – The first_idx associated to vertices, of shape \((\text{batch_size})\).
faces (torch.LongTensor) – The packed faces of the meshes, of shape \((\text{num_faces}, 3)\).
num_faces_per_mesh – The number of faces per mesh, of shape \((\text{batch_size})\).
- Returns
The face areas of same type as vertices and of shape \((\text{num_faces})\).
- Return type
- kaolin.ops.mesh.packed_sample_points(vertices, first_idx_vertices, faces, num_faces_per_mesh, num_samples, areas=None)¶
Uniformly sample points over the surface of triangle meshes.
First face on which the point is sampled is randomly selected, with the probability of selection being proportional to the area of the face. then the coordinate on the face is uniformly sampled.
The return pointclouds are with fixed batching.
- Parameters
vertices (torch.Tensor) – The packed vertices of the meshes, of shape \((\text{num_vertices}, 3)\).
first_idx_vertices (torch.Tensor) – The first_idx associated to vertices, of shape \((\text{batch_size})\).
faces (torch.LongTensor) – The packed faces of the meshes, of shape \((\text{num_faces}, 3)\).
num_faces_per_mesh – The number of faces per mesh, of shape \((\text{batch_size})\).
num_samples (int) – The number of point sampled per mesh.
areas (torch.Tensor, optional) – The areas of each face, of shape \((\text{num_faces})\), can be preprocessed, for fast on-the-fly sampling, will be computed if None (default).
- Returns
The pointclouds, of shape \((\text{batch_size}, \text{num_points}, 3)\).
The indexes of the faces selected (as merged faces), of shape \((\text{batch_size}, \text{num_points}).\)
- Return type
(torch.Tensor, torch.LongTensor)
- kaolin.ops.mesh.sample_points(vertices, faces, num_samples, areas=None, face_features=None)¶
Uniformly sample points over the surface of triangle meshes.
First face on which the point is sampled is randomly selected, with the probability of selection being proportional to the area of the face. then the coordinate on the face is uniformly sampled.
If
face_features
is defined for the mesh faces, the sampled points will be returned with interpolated features as well, otherwise, no feature interpolation will occur.- Parameters
vertices (torch.Tensor) – The vertices of the meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).
faces (torch.LongTensor) – The faces of the mesh, of shape \((\text{num_faces}, 3)\).
num_samples (int) – The number of point sampled per mesh.
areas (torch.Tensor, optional) – The areas of each face, of shape \((\text{batch_size}, \text{num_faces})\), can be preprocessed, for fast on-the-fly sampling, will be computed if None (default).
face_features (torch.Tensor, optional) –
Per-vertex-per-face features, matching
faces
order, of shape \((\text{batch_size}, \text{num_faces}, 3, \text{feature_dim})\). For example:Texture uv coordinates would be of shape \((\text{batch_size}, \text{num_faces}, 3, 2)\).
RGB color values would be of shape \((\text{batch_size}, \text{num_faces}, 3, 3)\).
When specified, it is used to interpolate the features for new sampled points.
See also
index_vertices_by_faces()
for conversion of features defined per vertex and need to be converted to per-vertex-per-face shape of \((\text{num_faces}, 3)\).- Returns
the pointclouds of shape \((\text{batch_size}, \text{num_samples}, 3)\), and the indexes of the faces selected, of shape \((\text{batch_size}, \text{num_samples})\).
If
face_features
arg is specified, then the interpolated features of sampled points of shape \((\text{batch_size}, \text{num_samples}, \text{feature_dim})\) are also returned.- Return type
(torch.Tensor, torch.LongTensor, (optional) torch.Tensor)
- kaolin.ops.mesh.subdivide_tetmesh(vertices, tetrahedrons, features=None)¶
Subdivide each tetrahedron in tetmesh into 8 smaller tetrahedrons by adding midpoints. If per-vertex features (e.g. SDF value) are given, the features of the new vertices are computed by averaging the features of vertices on the edge. For more details and example usage in learning, see Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis NeurIPS 2021.
- Parameters
vertices (torch.Tensor) – batched vertices of tetrahedral meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).
tetrahedrons (torch.LongTensor) – unbatched tetrahedral mesh topology, of shape \((\text{num_tetrahedrons}, 4)\).
features (optional, torch.Tensor) – batched per-vertex feature vectors, of shape \((\text{batch_size}, \text{num_vertices}, \text{feature_dim})\).
- Returns
batched vertices of subdivided tetrahedral meshes, of shape \((\text{batch_size}, \text{new_num_vertices}, 3)\)
unbatched tetrahedral mesh topology, of shape \((\text{num_tetrahedrons} * 8, 4)\).
batched per-vertex feature vectors of subdivided tetrahedral meshes, of shape \((\text{batch_size}, \text{new_num_vertices}, \text{feature_dim})\).
- Return type
(torch.Tensor, torch.LongTensor, (optional) torch.Tensor)
Example
>>> vertices = torch.tensor([[[0, 0, 0], ... [1, 0, 0], ... [0, 1, 0], ... [0, 0, 1]]], dtype=torch.float) >>> tetrahedrons = torch.tensor([[0, 1, 2, 3]], dtype=torch.long) >>> sdf = torch.tensor([[[-1.], [-1.], [0.5], [0.5]]], dtype=torch.float) >>> new_vertices, new_tetrahedrons, new_sdf = subdivide_tetmesh(vertices, tetrahedrons, sdf) >>> new_vertices tensor([[[0.0000, 0.0000, 0.0000], [1.0000, 0.0000, 0.0000], [0.0000, 1.0000, 0.0000], [0.0000, 0.0000, 1.0000], [0.5000, 0.0000, 0.0000], [0.0000, 0.5000, 0.0000], [0.0000, 0.0000, 0.5000], [0.5000, 0.5000, 0.0000], [0.5000, 0.0000, 0.5000], [0.0000, 0.5000, 0.5000]]]) >>> new_tetrahedrons tensor([[0, 4, 5, 6], [1, 7, 4, 8], [2, 5, 7, 9], [3, 6, 9, 8], [4, 5, 6, 8], [4, 5, 8, 7], [9, 5, 8, 6], [9, 5, 7, 8]]) >>> new_sdf tensor([[[-1.0000], [-1.0000], [ 0.5000], [ 0.5000], [-1.0000], [-0.2500], [-0.2500], [-0.2500], [-0.2500], [ 0.5000]]])
- kaolin.ops.mesh.subdivide_trianglemesh(vertices, faces, iterations, alpha=None)¶
Subdivide triangular meshes following the scheme of Loop subdivision proposed in Smooth Subdivision Surfaces Based on Triangles. If the smoothing factor alpha is not given, this function performs exactly as Loop subdivision. Elsewise the vertex position is updated using the given per-vertex alpha value, which is differentiable and the alpha carries over to subsequent subdivision iterations. Higher alpha leads to smoother surfaces, and a vertex with alpha = 0 will not change from its initial position during the subdivision. Thus, alpha can be learnable to preserve sharp geometric features in contrast to the original Loop subdivision. For more details and example usage in learning, see Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis NeurIPS 2021.
- Parameters
vertices (torch.Tensor) – batched vertices of triangle meshes, of shape \((\text{batch_size}, \text{num_vertices}, 3)\).
faces (torch.LongTensor) – unbatched triangle mesh faces, of shape \((\text{num_faces}, 3)\).
iterations (int) – number of subdivision iterations.
alpha (optional, torch.Tensor) – batched per-vertex smoothing factor, alpha, of shape \((\text{batch_size}, \text{num_vertices})\).
- Returns
- batched vertices of triangle meshes, of shape
\((\text{batch_size}, \text{new_num_vertices}, 3)\).
- unbatched triangle mesh faces, of shape
\((\text{num_faces} \cdot 4^\text{iterations}, 3)\).
- Return type
(torch.Tensor, torch.LongTensor)
Example
>>> vertices = torch.tensor([[[0, 0, 0], ... [1, 0, 0], ... [0, 1, 0], ... [0, 0, 1]]], dtype=torch.float) >>> faces = torch.tensor([[0, 1, 2],[0, 1, 3],[0, 2, 3],[1, 2, 3]], dtype=torch.long) >>> alpha = torch.tensor([[0, 0, 0, 0]], dtype=torch.float) >>> new_vertices, new_faces = subdivide_trianglemesh(vertices, faces, 1, alpha) >>> new_vertices tensor([[[0.0000, 0.0000, 0.0000], [1.0000, 0.0000, 0.0000], [0.0000, 1.0000, 0.0000], [0.0000, 0.0000, 1.0000], [0.3750, 0.1250, 0.1250], [0.1250, 0.3750, 0.1250], [0.1250, 0.1250, 0.3750], [0.3750, 0.3750, 0.1250], [0.3750, 0.1250, 0.3750], [0.1250, 0.3750, 0.3750]]]) >>> new_faces tensor([[1, 7, 4], [0, 4, 5], [2, 5, 7], [5, 4, 7], [1, 8, 4], [0, 4, 6], [3, 6, 8], [6, 4, 8], [2, 9, 5], [0, 5, 6], [3, 6, 9], [6, 5, 9], [2, 9, 7], [1, 7, 8], [3, 8, 9], [8, 7, 9]])
- kaolin.ops.mesh.uniform_laplacian(num_vertices, faces)¶
Calculates the uniform laplacian of a mesh. \(L[i, j] = \frac{1}{num\_neighbours(i)}\) if i, j are neighbours. \(L[i, j] = -1\) if i == j. \(L[i, j] = 0\) otherwise.
- Parameters
num_vertices (int) – Number of vertices for the mesh.
faces (torch.LongTensor) – Faces of shape \((\text{num_faces}, \text{face_size})\) of the mesh.
- Returns
Uniform laplacian of the mesh of size \((\text{num_vertices}, \text{num_vertices})\)
- Return type
Example
>>> faces = torch.tensor([[0, 1, 2]]) >>> uniform_laplacian(3, faces) tensor([[-1.0000, 0.5000, 0.5000], [ 0.5000, -1.0000, 0.5000], [ 0.5000, 0.5000, -1.0000]])
- kaolin.ops.mesh.vertex_tangents(faces, face_vertices, face_uvs, vertex_normals)¶
Compute vertex tangents.
The vertex tangents are useful to apply normal maps during rendering.
- Parameters
faces (torch.LongTensor) – unbatched triangle mesh faces, of shape \((\text{num_faces}, 3)\).
face_vertices (torch.Tensor) – unbatched triangle face vertices, of shape \((\text{num_faces}, 3, 3)\).
face_uvs (torch.Tensor) – unbatched triangle UVs, of shape \((\text{num_faces}, 3, 2)\).
vertex_normals (torch.Tensor) – unbatched vertex normals, of shape \((\text{num_vertices}, 3)\).
- Returns
The vertex tangents, of shape \((\text{num_vertices, 3})\)
- Return type