Clustered Meshlet Optimizer

Overview

The Clustered Meshlet Optimizer is a specialized Simplygon tool that processes input geometry into a set of Meshlets — small, self-contained sub‑meshes. The tool takes an input PackedGeometryData object, splits it into meshlets of a set size, and iteratively merges, reduces, and splits these initial meshlets into a set of many levels of detail. The reduction is always run to its natural end point, which can be either that only a single meshlet remains, or more commonly that the reduction can not be taken further because of irreducible features or the used settings.

All generated meshlets are sorted into a directed acyclic graph (DAG) structure which can be efficiently queried at runtime to display continuous LOD based on the camera parameters (for example utilizing mesh shaders).

The meshlets in a processed geometry are not simply a subdivision of the original mesh, but a potentially large set of spatially overlapping meshlets of various reduction levels. The additional meshlet metadata in the DAG generated by the tool supports a renderer to efficiently select a suitable subset of meshlets to render at runtime. This is covered in the Selecting meshlets for rendering section.

Meshlets in the geometry are also divided into groups of streaming Levels — sets of meshlets that together approximate the geometry to some level of detail. With level information, a renderer can implement data streaming to avoid large up-front load times. This is covered in the Meshlet streaming section.

Algorithm details

The optimizer first creates a base set of meshlets that collectively represent the original mesh exactly. The subdivision can be influenced by user parameters, but generally the algorithm produces contiguous triangle groups to improve packing and locality.

Next, the tool iteratively merges, reduces, and splits meshlets and produces a directed acyclic graph hierarchy (resembling hierarchical LODs) of additional meshlets that spatially overlap their predecessors. The data structure is progressively filled with coarser and coarser meshlets, where each meshlet has several predecessors (referred to as children) of higher detail level. The reduction is always run to its natural end point, which can be either that only a single meshlet remains, or more commonly that the reduction can not be taken further because of irreducible features or the used settings.

Lastly, the optimizer associates each meshlet with its bounding sphere and the reduction deviation error it has been reduced to, together with the bounds and error of its parent in the datastructure - that is, the bounds and error of the cluster of meshlets that represent a coarser version of this part of the original mesh.

Left: The yellow meshlet (part of some meshlet cluster in the left figure) is associated with its own bounds (blue circle), as well as the bounds of the coarser parent cluster, visualized to the right

Selecting meshlets for rendering

Combined with user provided per‑frame data (camera position, projection, and an error tolerance), the meshlet metadata generated by the Clustered Meshlet Optimizer is enough for a renderer to be able to at runtime select a subset of meshlets across levels of reduction that satisfies a target visual error.

A very important property of the data structure is that the selection decision for a meshlet can be done completely independent from data in other meshlets. That independence enables highly parallel evaluation and is well suited for GPU‑based culling and selection.

Meshlet data fields

The ClusteredMeshletOptimizer overwrites the input geometry data (triangle, corner, and vertex data) and generates a set of additional custom fields on the geometry that contains meshlet-specific metadata:

Custom field name	Item type	Tuple size	Tuple count	Description
cmo:MeshletTriangles	UnsignedInt	1	nMeshlets	Index of the first triangle for each meshlet
cmo:MeshletBounds	Real	4	nMeshlets	Bounds for each meshlet *
cmo:MeshletErrors	Real	1	nMeshlets	Error for each meshlet
cmo:MeshletParentBounds	Real	4	nMeshlets	Bounds for each meshlet parent *
cmo:MeshletParentErrors	Real	1	nMeshlets	Error for each meshlet parent
cmo:MeshletChildLevels	UnsignedInt	1	nMeshlets	level that needs to be loaded for this meshlet to be replacable by higher detail counterpart **

nMeshlets is the number of generated meshlets.
* cmo:MeshletBounds and cmo:MeshletParentBounds represent bounds as 4-tuples (center.x, center.y, center.z, radius)
** cmo:MeshletChildLevels relates to meshlet streaming levels, covered in Meshlet streaming

Extracting meshlet metadata

The meshlet data of the processed geometry is stored in a format efficient for serialization, but somewhat cumbersome for further processing and handling by user code. Consider this code for extraction of meshlet metadata to convenient structures, they will be referred to throughout this documentation:

struct Bounds
{
    glm::vec3 center;
    float radius;
};

struct Meshlet
{
    unsigned int startTriangle; // index of first triangle of this meshlet
    unsigned int triangleCount; // number of triangles in this meshlet

    Bounds bounds;       // the bounds of this meshlet and its sibling meshlets 
    float error;         // the error of this meshlet and its sibling meshlets 
    Bounds parentBounds; // the bounds of the meshlets that make up the coarser level to this
    float parentError;   // the error of the meshlets that make up the coarser level to this

    unsigned int childLevel; // streaming level requirement (see Meshlet streaming)
};

struct ViewParams
{
    glm::vec3 pos;  // Camera position
    float proj;     // Camera projection factor: 1.0 / tan(0.5 * FOV)
    float zNear     // Camera near clipping plane
}

std::vector<Meshlet> ExtractMeshletData(const spPackedGeometryData& geometry)
{
    // Get output fields from geometry
    auto meshletTriangles = spUnsignedIntArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletTriangles"));
    auto meshletBounds = spRealArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletBounds"));
    auto meshletErrors = spRealArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletErrors"));
    auto meshletParentBounds = spRealArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletParentBounds"));
    auto meshletParentErrors = spRealArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletParentErrors"));
    auto meshletChildLevels = spUnsignedIntArray::SafeCast(geometry->GetCustomFields()->FindValueArray("cmo:MeshletChildLevels"));


    std::vector<Meshlet> meshlets(meshletTriangles->GetItemCount());

    // Collect all meshlets
    for (unsigned int meshletInx = 0; meshletInx < meshletTriangles->GetItemCount(); ++meshletInx)
    {
        meshlets[meshletInx].startTriangle = meshletTriangles->GetItem(meshletInx);
        if (meshletInx < meshletTriangles->GetItemCount() - 1)
            meshlets[meshletInx].triangleCount = meshletTriangles->GetItem(meshletInx + 1) - meshlets[meshletInx].startTriangle;
        else
            meshlets[meshletInx].triangleCount = geometry->GetTriangleCount() - meshlets[meshletInx].startTriangle;

        meshlets[meshletInx].bounds.center[0] = meshletBounds->GetItem(meshletInx * 4 + 0);
        meshlets[meshletInx].bounds.center[1] = meshletBounds->GetItem(meshletInx * 4 + 1);
        meshlets[meshletInx].bounds.center[2] = meshletBounds->GetItem(meshletInx * 4 + 2);
        meshlets[meshletInx].bounds.radius = meshletBounds->GetItem(meshletInx * 4 + 3);
        meshlets[meshletInx].error = meshletErrors->GetItem(meshletInx);

        meshlets[meshletInx].parentBounds.center[0] = meshletParentBounds->GetItem(meshletInx * 4 + 0);
        meshlets[meshletInx].parentBounds.center[1] = meshletParentBounds->GetItem(meshletInx * 4 + 1);
        meshlets[meshletInx].parentBounds.center[2] = meshletParentBounds->GetItem(meshletInx * 4 + 2);
        meshlets[meshletInx].parentBounds.radius = meshletParentBounds->GetItem(meshletInx * 4 + 3);
        meshlets[meshletInx].parentError = meshletParentErrors->GetItem(meshletInx);

        meshlets[meshletInx].childLevel = meshletChildLevels->GetItem(meshletInx);   
    }

    return meshlets;
}

Runtime meshlet selection

Clustered meshlet geometry needs to be treated differently in runtime compared to traditional geometry - the renderer will be tasked with selecting which meshlets to render based on the current camera view and an error threshold. Thanks to the monotonically increasing error and expanding bounds of the meshlet clusters, the selection code is quite simple and easily parallelizable.

Consider the following code:

float CalcBoundsError( 
        const Bounds& bounds, 
        const float error, 
        const ViewParams& viewParams )
{
    auto delta = bounds.center - viewParams.pos;
    float d = glm::length( delta ) - bounds.radius;
    return error / ( d > viewParams.zNear ? d : viewParams.zNear ) * ( viewParams.proj * 0.5f );
}

bool ShouldRenderMeshlet( 
        float threshold, 
        const Meshlet& meshlet, 
        const ViewParams& viewParams )
{
    float currentError = CalcBoundsError( 
                meshlet.bounds, 
                meshlet.error, 
                viewParams );

    float parentError = CalcBoundsError( 
                meshlet.parentBounds, 
                meshlet.parentError, 
                viewParams );

    return currentError <= threshold && parentError > threshold;
}

With these utilities, meshlet selection becomes a simple loop which is run on the currently loaded meshlets. In pseudocode:

// consts for this frame
ViewParams viewParams;
viewParams.pos = {{current view position}}
viewParams.proj = {{projection, 1.f / tanf(half_fov) }}
viewParams.zNear = {{near clipping plane distance}}
const float threshold = {{current threshold, like pixel error}}

// List of source meshlets
const std::vector<const Meshlet*> meshlets = {{input meshlet metadata, generated offline}};

// list of meshlets to render
std::vector<const Meshlet*> renderMeshlets;

// filter out meshlets to render from full list of possible meshlets
for( const auto meshlet : meshlets )
{
    if( ShouldRenderMeshlet(threshold, meshlet, viewParams ) )
        renderMeshlets.push_back( meshlet );
}

Notice how the decision whether to render a meshlet or not can be made in complete isolation, there are no interdependencies between the meshlets. This fact makes the selection code fully parallelizable and easily transferrable to a GPU process.

Meshlet streaming

Meshlets in a geometry are serialized with streamability in mind. Each meshlet belongs to a streaming Level — a set of meshlets that together will allow some meshlets from previous level to be swapped out with ones of finer detail. A single level is not guaranteed to contain enough meshlets to represent the entire mesh.

Aided by level-related metadata for meshlets in the geometry, a renderer can chose to stream in the geometry layer-by layer to reduce load-time stalls.

Important note: A meshlet level is not suitable as LOD itself. Levels are designed as steps at which a streaming data loader may yield to the renderer, but meshlet selection code will require that for a given layer, all previous layers are also loaded. Using levels at all is optional and only relevant for users that want to iteratively stream in the geometry.

Meshlets are sorted in the data arrays per-level and in increasing level order. This means that the meshlets that make the coarsest and cheapest approximation of the geometry (meshlets of the lowest Level) are placed at the beginning of the data, followed by the meshlets in the next level, etc.

Note: Levels as complete as this above example can only be expected in the trivial case where the geometry is 2-manifold and without borders. In the general case, levels do not necessarily contain meshlets from the entire mesh.

Level data fields

Similar to the basic meshlet data, level data is stored in custom fields on the geometry:

Field name	Item type	Tuple size	Tuple count	Description
cmo:LevelMeshlets	UnsignedInt	1	nLevels	First meshlet for each level. Indexes into meshlet data arrays (see above)
cmo:LevelRequiredVertices	UnsignedInt	1	nLevels	Number of vertices required for each layer

nLevels is the number of generated meshlet levels.

Extracting level metadata

Level metadata can, similar to the meshlet metadata, be extracted into more usable structures. Consider the following struct and code for extraction:

struct Level
{
    unsigned int startMeshlet;
    unsigned int meshletCount;
    unsigned int requiredVertices;
};

std::vector<Level> ExtractLevelData(const spPackedGeometryData& geometry)
{
    // Get output fields from geometry
    auto levelMeshlets = spUnsignedIntArray::SafeCast(packedGeom->GetCustomFields()->FindValueArray("cmo:LevelMeshlets"));
    auto levelRequiredVertices = spUnsignedIntArray::SafeCast(packedGeom->GetCustomFields()->FindValueArray("cmo:LevelRequiredVertices"));

    std::vector<Level> levels(levelMeshlets->GetItemCount());

    // collect all levels
    for (unsigned int level = 0; level < levelMeshlets->GetItemCount(); ++level)
    {
        levels[level].startMeshlet = levelMeshlets->GetItem(level);
        levels[level].requiredVertices = levelRequiredVertices->GetItem(level);
        if (level < levelMeshlets->GetItemCount() - 1)
            levels[level].meshletCount = levelMeshlets->GetItem(level + 1) - levels[level].startMeshlet;
        else
            levels[level].meshletCount = meshletTriangles->GetItemCount() - levels[level].startMeshlet;
    }

    return levels;
}

With this data, a streaming renderer can easily identify the amount of geometry data that needs to be streamed in to be able to render at any given layer.

Runtime meshlet selection - streamed

With level information available, the meshlet selection code changes slightly compared to previously. To decide whether a meshlet should render, the number of levels loaded needs to be taken into account:

bool ShouldRenderMeshletStreamed( float threshold,
                          const Meshlet& meshlet,
                          const ViewParams& viewParams,
                          const unsigned int loadedLevel )
{
    float currentError = meshlet.childLevel > loadedLevel ?
            0.0f :
            CalcBoundsError( 
                meshlet.bounds, 
                meshlet.error, 
                viewParams );

    float parentError = CalcBoundsError( 
                meshlet.parentBounds, 
                meshlet.parentError, 
                viewParams );

    return currentError <= threshold && parentError > threshold;
}

Accordingly, the pseudocode for the renderer is slightly different.

// consts for this frame
ViewParams viewParams;
viewParams.pos = {{current view position}}
viewParams.proj = {{projection, 1.f / tanf(half_fov) }}
viewParams.zNear = {{near clipping plane distance}}
const float threshold = {{current threshold, like pixel error}}
const float loadedLevel = {{the last Level in the meshlet structure which has been loaded in its entirety}}

// list of source meshlets
const std::vector<const Meshlet*> meshlets = {{input meshlet metadata, generated offline}};

// list of meshlets to render
std::vector<const Meshlet*> renderMeshlets;

// filter out meshlets to render from full list of possible meshlets,
// taking the loaded level into account.
for( const auto meshlet : meshlets )
{
    if( ShouldRenderMeshletStreamed(threshold, meshlet, viewParams, loadedLevel ) )
        renderMeshlets.push_back( meshlet );
}