Magic Tools
Developer ToolsBy CooconAugust 24, 20266 views5 min read

A 36GB scene that OOMs a 192GB machine now processes in under 4 minutes. Here's what meshoptimizer changed.

A 36GB scene that OOMs a 192GB machine now processes in under 4 minutes. Here's what meshoptimizer changed.

Let me set the scale first. One glTF file, 36GB, geometry only, 1.64 billion triangles (18.9 billion with instancing).

Feed it to Blender: ~10 minutes to import, then out of memory. Unreal Engine does it faster — crashes in under 5 minutes. NVIDIA's own sample code with 16 threads? OOM. Drop to 8 threads and it takes 30 minutes while chewing through 180+GB of RAM. A 192GB workstation barely survives this file.

This is the Zorah scene from NVIDIA's RTX Mega Geometry demo — the one that shows off clustered raytracing. It looks stunning on screen. Its geometry pipeline is a nightmare.

zeux, the author of meshoptimizer, decided to fix that. He used the file as a benchmark to take the library's clustered LOD support from "technically runs" to "processes 1.6 billion triangles in minutes." The write-up of that process is a genuine case study for anyone working on game engines, 3D toolchains, or large-scale mesh processing.

What this tech actually does: a mesh, chopped into Lego bricks

Zorah uses a Nanite-style hierarchical LOD scheme that meshoptimizer has supported since 2024.

The idea: split a mesh into small clusters (max 128 triangles each), merge neighboring clusters into groups, simplify each group independently, then split, merge, and simplify again until nothing can be simplified further. The result is a DAG of clusters. At runtime, the renderer streams in the right level of detail for the camera position — a coarser cluster is only swapped in when the visual error stays under 1 pixel.

Think of it as Lego: from "every brick visible" up close to "one block for the whole wall" from far away. You only assemble what the camera can see.

Start to finish, the pipeline has three phases: generating the hierarchy, compressing it for streaming, and rendering it in real time. This article covers only the first one — generation. Which is also the part people skip, because it's the most compute-hungry.

Baseline: not slow. Unusable.

meshoptimizer provides the three ingredients — clusterization (chopping), partitioning (grouping), simplification (reducing). But the original example code was written to experiment with algorithms, not to handle real scenes.

zeux did the groundwork first: refactored the code into a reusable API, added memory mapping for the 36GB file (including a PR to cgltf to make that work), and reindexed the meshes. The last one matters — one mesh in the file has 90M vertices for 30M triangles, an indexing ratio so bad that vertex count was 3x the triangle count.

After all that, 16 threads: 9m20s with 54.6GB RAM for the raster-optimized path; 7m10s with 57.6GB for the new raytracing-optimized clusterizer.

Better than 30 minutes. zeux's verdict: still too slow. "Getting a cup of coffee doesn't take that long." Then the real work started.

Three optimizations: from 9 minutes to 3.5

Trap #1: doing pointless work for 30 million vertices

Profiling with Superluminal, the loudest hotspot was... memset.

The clusterizer keeps an array indexed by vertex to track whether a vertex is already assigned to the current meshlet. Every time it processes a sub-mesh, it resets the whole thing:

memset(used, -1, vertex_count * sizeof(short));

On a small mesh, that line is nothing. Here, the code repeatedly clusterizes subsets of a 30M-triangle mesh — vertex counts in the millions, full wipe every time. Pure waste. The simplifier had the same disease: a bit-array initialization (memset(filter, 0, (vertex_count + 7) / 8)) that also scaled with the full mesh.

The fix is brutally simple: when sparse access is detected (index_count < vertex_count), only initialize the entries the index buffer actually touches.

Result: raster path from 9m20s to 3m31s. Raytrace path from 7m10s to 3m57s. A memset fix more than doubled throughput.

Trap #2: 16 threads, 12 cores of actual work

/usr/bin/time -v showed CPU usage of 1240–1260%. Sounds busy? The theoretical ceiling for 16 threads is 1600%. Four threads' worth of compute was idling.

Blame load imbalance: the scene is hundreds of meshes with wildly different sizes — uneven task granularity, so threads finish their chunk and wait. You can't tune your way out of this; the scheduling logic has to change. The deeper lesson zeux drives home: profiler-reported time distributions lie. Wall clock time is set by the slowest thread.

Trap #3: raster and raytracing clusterizers are different animals

meshoptimizer now ships two clusterization algorithms: one tuned for rasterization and mesh shaders (eight years of incremental improvements), and a new one built specifically for clustered raytracing — written after NVIDIA released RTX Mega Geometry.

Why two? Raytracing is extremely sensitive to where cluster boundaries fall. Cut them well and each cluster can build its own micro-BVH, then one BVH over all clusters — traversal becomes fast. Cut them badly and your hit rates tank. For raytracing, the clusterization IS the performance.

Hold the applause: the measurement has caveats

Fair warning before you quote those numbers in a meeting.

First, the test code doesn't write results to disk — it measures in-memory processing only. A real pipeline serializes output, which costs extra time and memory. Second, 36GB is just the geometry; the full Zorah scene includes a 62GB render cache. Third, meshoptimizer is a C++ library — integrating it into an existing pipeline is real work, not a drop-in.

But the direction is unambiguous: the mesh scale a single CPU can handle just moved up an order of magnitude. A few years ago this class of asset meant "rendering farm, don't bother." Now a dev machine handles it.

Three things you can do today

  1. Profile before you optimize. Trust nothing. The biggest win here came from a memset — nobody would have guessed that hotspot without a profiler. Measure first, then change.
  2. Hunt your own sparse loops. If you're fully initializing large arrays in a hot loop, try touching only the entries your index buffer uses. This pattern shows up in indexing, texture, and cache-clearing code everywhere.
  3. Re-run meshoptimizer's clustered LOD example on your data. If your build pipeline skipped batch LOD generation because "it's too slow," the cost just changed. It's time to redo that math.

As for 1.6-billion-triangle assets like Zorah — that used to be "boss, we need the render farm." Now it's "give me a minute, I'll make coffee."


Based on zeux's blog post Billions of triangles in minutes and the daily-intel headline of the day. All figures quoted from the original article.

✨ Drafted by DeepSeek, reviewed and polished by Claude.

Published by Magic Tools