Best way to setup bvh raycasting for meshes?

[tueboesen]

In the release of v.1.11.0 an example of bvh raycasting for box volumes is shown:

# Build a BVH in Python containing multiple worlds
lowers = wp.array(...)  # Shape bounds for all worlds
uppers = wp.array(...)
world_ids = wp.array([0, 0, 1, 1, 2, 2, ...], dtype=int)

bvh = wp.Bvh(lowers, uppers, groups=world_ids)

@wp.kernel
def raycast_world(
    bvh_id: wp.uint64,
    world_id: int,
    ray_origin: wp.vec3,
    ray_dir: wp.vec3
):
    # Get the root node for this world's sub-tree
    root = wp.bvh_get_group_root(bvh_id, world_id)
    
    # Query only intersects geometry from this world
    query = wp.bvh_query_ray(bvh_id, ray_origin, ray_dir, root)
    
    # Process hits
    shape_idx = int(0)
    while wp.bvh_query_next(query, shape_idx):
        # Handle intersection with shape_idx
        pass

# Launch kernel to query world 2
wp.launch(raycast_world, dim=1, inputs=[bvh.id, 2, origin, direction])

However it is not clear to me whether it is done in exactly the same way when working with meshes?
Suppose I have some meshes and I want to similarly use bvh raycast, do I then manually determine the bounding volume for each mesh, and from these bounding volumes create the bvh object or is there a more direct way of doing this?

        lower_bounds = np.empty((n,3),dtype=np.float32)
        upper_bounds = np.empty_like(lower_bounds)
        mesh_ids = np.empty(n, dtype=np.uint64)
        meshes = []
        for i, mesh in enumerate(self.meshes):
            vertices = mesh.vertices
            lower_bounds[i] = np.min(vertices,axis=0)
            upper_bounds[i] = np.max(vertices,axis=0)
            mesh2 = wp.Mesh(points=wp.array(vertices, dtype=wp.vec3), velocities=None, indices=wp.array(mesh.indices, dtype=int))
            meshes.append(mesh2)
            mesh_ids[i] = mesh2.id

        device_lowers = wp.array(lower_bounds, dtype=wp.vec3, device="cuda:0")
        device_uppers = wp.array(upper_bounds, dtype=wp.vec3, device="cuda:0")
        bvh = wp.Bvh(device_lowers, device_uppers)

The above seems to work but it also feels like I'm doing something stupid since I am manually creating the lower and upper bounds rather than use the meshes directly for the bvh calculation.
However I do not see any examples on how to use meshes and nor can I find anything in the constructors or documentation about how to do it differently.

[shi-eric]

@StafaH can you please weigh in as you have most recently worked on this code?

[StafaH]

Hi @tueboesen,

There are two examples you can follow for how to set-up the BVH for rendering. There is an example being built out in mujoco-warp that will be released soon, it can be found in the render branch of mujoco-warp: https://github.com/google-deepmind/mujoco_warp/tree/render

The second example is currently being built out in netwon, it is called warp-raytrace: https://github.com/newton-physics/newton/blob/main/newton/_src/sensors/warp_raytrace/render.py

The example you have shown is correct in concept. Rendering usually takes the form of Top Level Acceleration Structures (TLAS) and Bottom Level Acceleration Structures (BLAS). At the top level you have your bounding volumes which are your primitives in world space. And at the bottom level you have meshes and other geometric types.

You only need to build your lowers/uppers array once for the TLAS. Then every frame you should be updating that array with a kernel that adjusts the box based on movement of the geometry. The BLAS usually stays static unless you have deformable geometry.

Those two repos have clean code that should be easy to follow to get an idea of how to build a rendering pipeline using this BVH (or you can use those libraries directly to avoid re-writing your own renderer).

Hope this helps!

[tueboesen]

Your explanation of TLAS and BLAS was quite helpful to get an idea about how these things are conceptually used.

However, the example shown in https://github.com/newton-physics/newton/blob/main/newton/_src/sensors/warp_raytrace/render.py does not really answer my question about how one is supposed to actually build the bvh for meshes, since it just assumes that you already have a mesh and a bvh.
What I'm missing is a minimal example illustrating how I am supposed to do bvh raycasting if I have a set of mesh objects.
My own solution to the issue is to build the bounding boxes myself and based on those construct the bvh object. However, this feels clunky and I'm guessing there should be a smarter way of doing this I just cannot find any information on it.

[StafaH]

Mesh BVHs are built like this: https://github.com/google-deepmind/mujoco_warp/blob/render/mujoco_warp/_src/bvh.py#L300

Static meshes need to only be built once. When you build the mesh you should calculate the canonical size of the mesh.

Then when you update your lowers/uppers (which you would do every frame since geometry is moving), you can use the original size calculated, update it based on the new position/rotation, and update lowers/uppers for that mesh: https://github.com/google-deepmind/mujoco_warp/blob/render/mujoco_warp/_src/bvh.py#L210

As you mentioned, there is no smarter way to do this, other than streamlining your example to make sure you create the meshes in advance and store their sizes, and then have a build and refit method that uses the original size + the current pose to update the bounding box location in lowers/uppers