Newton USD import discards authored physics:approximation; convexDecomposition assets become a single convex hull

[lgulich]

Summary

When importing USD through the Newton backend, Isaac Lab discards each asset's authored physics:approximation and applies a convex hull to every collision mesh in the scene. Assets that declare convexDecomposition — the standard way to give a concave collider usable physics — silently get a single convex hull instead. Concave geometry becomes solid: containers cannot be entered, gaps and slots fill in.

There is no configuration path around it. The relevant argument defaults to True and is never passed by any caller.

Affected code

source/isaaclab_newton/isaaclab_newton/cloner/newton_replicate.py#L86-L95 (_build_newton_builder_from_mapping):

p.add_usd(
    stage,
    root_path=src_path,
    load_visual_shapes=True,
    skip_mesh_approximation=True,        # (1)
    ...
)
if simplify_meshes:                      # (2) defaults True, never passed by any caller
    p.approximate_meshes("convex_hull", keep_visual_shapes=True)

These are two separate problems, and fixing either alone does not help.

(1) skip_mesh_approximation=True makes Newton's importer skip reading the attribute at all. From newton/_src/utils/import_usd.py:

if not skip_mesh_approximation:
    approximation = usd.get_attribute(prim, "physics:approximation", None)
    if approximation is not None:
        remeshing_method = approximation_to_remeshing_method.get(approximation.lower(), None)
        ...
        remeshing_queue[remeshing_method].append(shape_id)

With it set, nothing is ever queued for remeshing — the authored intent is gone before Isaac Lab's own line runs.

(2) The unconditional approximate_meshes("convex_hull", ...) then hulls everything. simplify_meshes is a parameter of a private function; NewtonCfg does not expose it, and no call site in the repository passes it, so downstream code cannot opt out without monkey-patching.

If only (2) were fixed, every mesh would be imported raw and concave, because (1) already discarded the attribute. If only (1) were fixed, the blanket hull on the next line would immediately overwrite the decomposition. Both must change together.

Newton itself is not at fault and already supports this: it maps convexdecomposition to CoACD (approximation_to_remeshing_method), exposes approximate_meshes(method, shape_indices=...) for a subset of shapes, and ships coacd as a dependency. The capability is present and is being switched off.

Reproduction

Self-contained — needs only newton, pxr, numpy, scipy. No Isaac Sim app, no external assets. It builds an L-shaped (concave) collider in a temporary stage, authors convexDecomposition on it, and imports it both ways. The probe point sits in the L's notch: empty space that a convex hull would fill.

import tempfile
from pathlib import Path

import numpy as np
from pxr import Usd, UsdGeom, UsdPhysics, Gf

from newton import ModelBuilder, ShapeFlags

# L-shaped prism. Concave, so its convex hull is strictly larger than the shape.
#
#   y
#   2 +---+
#     |   |
#   1 |   +---+
#     |       |
#   0 +-------+
#     0   1   2  x
#
_L = [(0, 0), (2, 0), (2, 1), (1, 1), (1, 2), (0, 2)]
_NOTCH = (1.5, 1.5, 0.5)   # outside the L, inside its convex hull


def author_stage(path):
    stage = Usd.Stage.CreateNew(path)
    UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z)
    UsdGeom.Xform.Define(stage, "/World")
    body = UsdGeom.Xform.Define(stage, "/World/Body")
    UsdPhysics.RigidBodyAPI.Apply(body.GetPrim())
    UsdPhysics.MassAPI.Apply(body.GetPrim()).CreateMassAttr(1.0)

    mesh = UsdGeom.Mesh.Define(stage, "/World/Body/Collider")
    points, counts, indices = [], [], []
    for z in (0.0, 1.0):
        for x, y in _L:
            points.append(Gf.Vec3f(float(x), float(y), z))
    n = len(_L)
    counts.append(n); indices.extend(range(n - 1, -1, -1))     # bottom
    counts.append(n); indices.extend(range(n, 2 * n))          # top
    for i in range(n):                                         # sides
        j = (i + 1) % n
        counts.append(4); indices.extend([i, j, j + n, i + n])
    mesh.CreatePointsAttr(points)
    mesh.CreateFaceVertexCountsAttr(counts)
    mesh.CreateFaceVertexIndicesAttr(indices)

    UsdPhysics.CollisionAPI.Apply(mesh.GetPrim())
    UsdPhysics.MeshCollisionAPI.Apply(mesh.GetPrim()) \
        .CreateApproximationAttr().Set(UsdPhysics.Tokens.convexDecomposition)
    stage.Save()
    return stage


def collision_meshes(builder):
    """Meshes that collide, excluding visual-only shapes kept by keep_visual_shapes."""
    out = []
    for i, source in enumerate(getattr(builder, "shape_source", [])):
        verts = getattr(source, "vertices", None)
        if verts is None:
            continue
        if not (builder.shape_flags[i] & int(ShapeFlags.COLLIDE_SHAPES)):
            continue
        out.append(np.asarray(verts, dtype=float))
    return out


def solid_at(meshes, point):
    from scipy.spatial import ConvexHull, Delaunay
    target = np.asarray(point, dtype=float)
    for verts in meshes:
        if len(verts) < 4:
            continue
        try:
            hull = ConvexHull(verts)
            if Delaunay(verts[hull.vertices]).find_simplex(target) >= 0:
                return True
        except Exception:
            continue
    return False


with tempfile.TemporaryDirectory() as tmp:
    stage = author_stage(str(Path(tmp) / "concave.usda"))

    a = ModelBuilder()
    a.add_usd(stage, collapse_fixed_joints=False)                       # Newton's default
    a_meshes = collision_meshes(a)
    print(f"A. honouring physics:approximation -> {len(a_meshes)} collision mesh(es), "
          f"notch solid: {solid_at(a_meshes, _NOTCH)}")

    b = ModelBuilder()
    b.add_usd(stage, collapse_fixed_joints=False, skip_mesh_approximation=True)
    b.approximate_meshes("convex_hull", keep_visual_shapes=True)        # what Isaac Lab does
    b_meshes = collision_meshes(b)
    print(f"B. Isaac Lab's call sequence       -> {len(b_meshes)} collision mesh(es), "
          f"notch solid: {solid_at(b_meshes, _NOTCH)}")

Actual output

A. honouring physics:approximation -> 2 collision mesh(es), notch solid: False
B. Isaac Lab's call sequence       -> 1 collision mesh(es), notch solid: True

convexDecomposition was authored. Path A honours it: CoACD returns 2 convex pieces and the notch stays empty. Path B — the sequence Isaac Lab executes — produces one hull and the notch becomes solid.

Expected

Path B should match path A: the authored convexDecomposition should produce a decomposition, and empty space should stay empty.

Why this is easy to miss

keep_visual_shapes=True preserves the visual mesh, so the object still renders with its concavity. Only the collider is wrong. A container looks open and behaves closed — objects lowered into it stop in mid-air on an invisible surface, which reads as a solver or contact problem rather than as wrong geometry. Nothing is logged.

Proposed fix

Honour the authored attribute, and keep the convex hull as a fallback only for shapes that authored nothing — which is presumably why the blanket hull is there. That fallback matters: with skip_mesh_approximation=False, a mesh with no authored attribute is imported as a raw concave triangle mesh, verified with the script above:

none authored     -> 1 collision mesh, 12 verts   (raw concave mesh)
convexHull        -> 1 collision mesh, 10 verts   (hulled)
convexDecomposition -> 2 collision meshes         (decomposed)

So simply dropping skip_mesh_approximation=True would leave unauthored assets with concave colliders. The suggestion:

 p.add_usd(
     stage,
     root_path=src_path,
     load_visual_shapes=True,
-    skip_mesh_approximation=True,
+    skip_mesh_approximation=False,      # honour authored physics:approximation
     schema_resolvers=schema_resolvers,
     ignore_paths=_deformable_ignore_paths if _deformable_ignore_paths else None,
 )
-if simplify_meshes:
-    p.approximate_meshes("convex_hull", keep_visual_shapes=True)
+if simplify_meshes:
+    # Fall back to a convex hull only where the asset expressed no preference.
+    unauthored = [i for i in range(len(p.shape_label)) if not _authored_approximation(p, stage, i)]
+    if unauthored:
+        p.approximate_meshes("convex_hull", shape_indices=unauthored, keep_visual_shapes=True)

Whatever the exact shape of the implementation, the two properties that matter are:

1. an asset that authors convexDecomposition gets a decomposition;
2. an asset that authors nothing still gets a convex hull, so this is not a performance regression.

Separately, it would help downstream users if simplify_meshes were reachable through NewtonCfg. Today the only way to influence this behaviour is to monkey-patch a private function.

Environment

• isaaclab_newton from Isaac Lab release/3.0.0-beta2 (af1bab4); same code present in the isaaclab 3.0.0b2 wheel
• newton 1.4.0, warp-lang 1.15.0, coacd installed
• Linux x86_64, Python 3.12

[kellyguo11]

was this already addressed by #6596?

[kellyguo11]

Closing as fixed by #6596 on develop. The report was based on release/3.0.0-beta2; the merged fix makes the Newton cloner honor authored physics:approximation, applies the convex-hull fallback only to meshes without an authored approximation, and adds the CoACD dependency and regression coverage for convexDecomposition.