Joint Dependent Mesh/BBox for Embodiment

[zhx06]

Summary

Joint-dependent mesh/bbox for embodiments

Detailed description

• EmbodimentBase.get_bounding_box / get_collision_mesh now pose the USD at init_state.joint_pos
• Support for loading pre-computed mesh from Omniverse
• Add robot_library_folder per embodiment and export_ready_pose_collision_meshes.py to generate them

[greptile-apps]

Initial joint pose is ignored

When a Franka or Droid embodiment receives a non-default initial_joint_pose, the reset event applies that pose while placement geometry still reads robot.init_state.joint_pos, causing relation placement to use a bounding box and collision mesh for the wrong arm configuration.

Knowledge Base Used:

• Scene Composition, Affordances, and Relation Solving
• Tasks and MDP Terms

[qianl-nv]

This is better handle in droid/franka. set_initial_joint_pose currently only updates the reset event's default_pose.
can expand set_initial_joint_pose to also all self.set_joint_initial_pos(...) with a name→value map matching set_default_joint_pose order.

[greptile-apps]

That's the right fix. Looking at both constructors, set_initial_joint_pose only writes to the event's positional default_pose list but leaves init_state.joint_pos (what get_placement_geometry_source reads) unchanged.

One ordering issue to watch in FrankaEmbodimentBase.__init__: set_initial_joint_pose is called before self.scene_config is assigned:

self.event_config = FrankaEventCfg()
if initial_joint_pose is not None:
    self.set_initial_joint_pose(initial_joint_pose)  # scene_config is still None here
...
self.scene_config = FrankaSceneCfg()  # assigned after

So if you expand set_initial_joint_pose to also call self.set_joint_initial_pos(...), it'll hit the assert self.scene_config is not None in the base class. Droid doesn't have this problem since self.scene_config = DroidSceneCfg() is assigned before the call.

The simplest fix for Franka is to move scene_config assignment before the joint pose call:

self.scene_config = FrankaSceneCfg()
self.event_config = FrankaEventCfg()
if initial_joint_pose is not None:
    self.set_initial_joint_pose(initial_joint_pose)

Then the expanded set_initial_joint_pose in both Franka and Droid can safely call self.set_joint_initial_pos(joint_name_map) alongside the existing default_pose update, and placement geometry will reflect the correct configuration.

[greptile-apps]

Source changes retain stale meshes

When a robot USD is updated without changing its stem or configured pose, artifact validation still accepts the previous mesh because it checks only the stem and pose key. Developers must manually clear or republish artifacts before placement reflects the updated geometry.

Knowledge Base Used: Scene Composition, Affordances, and Relation Solving

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Greptile Summary

This PR introduces joint-dependent placement geometry and reusable collision-mesh artifacts.

• Poses embodiment bounding boxes and collision meshes from configured articulation joints.
• Adds persistent local and published mesh loading, validation, eviction, and export tooling.
• Adds per-embodiment robot-library folders, USD articulation kinematics, visualization helpers, and extensive simulation tests.

Confidence Score: 3/5

The PR should not merge until Franka and Droid placement geometry honors constructor-supplied initial joint poses; source-aware cache invalidation should also be strengthened.

Franka and Droid can reset into a constructor-selected arm pose while their newly introduced bounding box and collision mesh are computed from a different joint mapping, causing incorrect placement geometry on a supported path.

Files Needing Attention: isaaclab_arena/embodiments/embodiment_base.py, isaaclab_arena/embodiments/franka/franka.py, isaaclab_arena/embodiments/droid/droid.py, isaaclab_arena/utils/collision_mesh_store.py

Important Files Changed

Filename	Overview
isaaclab_arena/embodiments/embodiment_base.py	Routes embodiment geometry through joint-aware helpers, but reads a different joint-pose source than Franka and Droid constructor overrides.
isaaclab_arena/utils/usd_articulation.py	Adds offline USD articulation forward kinematics for revolute, prismatic, fixed, instanced, and closed-loop geometry.
isaaclab_arena/utils/usd_helpers.py	Adds cached posed-mesh extraction and posed Gprim bounding-box computation.
isaaclab_arena/utils/collision_mesh_store.py	Adds persistent and published mesh artifacts, though source identity does not invalidate artifacts after in-place USD updates.
isaaclab_arena/scripts/export_ready_pose_collision_meshes.py	Adds a simulation-backed exporter that deduplicates embodiment variants and reports partial failures.
isaaclab_arena/tests/test_usd_articulation.py	Adds broad articulation-kinematics and real-robot geometry coverage.
isaaclab_arena/tests/test_collision_mesh_store.py	Covers pose keys, artifact validation, scaling, atomic storage, publication, and cache trimming.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Config[Embodiment scene config] --> Source[PlacementGeometrySource]
  Source --> Pose[USD articulation posing]
  Pose --> BBox[Posed bounding box]
  Pose --> Extract[Mesh extraction]
  Source --> Store{Stored artifact valid?}
  Store -->|yes| Mesh[Scaled collision mesh]
  Store -->|no| Extract
  Extract --> Cache[Local/published mesh store]
  Cache --> Mesh
  BBox --> Placement[Relation placement]
  Mesh --> Placement

Loading

_{Reviews (1): Last reviewed commit: "add joint support for robots" | Re-trigger Greptile}

[arena-review-bot]

🟡 Does the published robot library need to ship now?

The in-process lru_cache on the posed-geometry helpers already covers the relation-solver hot path (repeated bbox queries within a run), so this disk + Nucleus layer's marginal benefit is the one-time 0.1–2.5 s extraction per fresh process. Weighed against that, it adds a publish pipeline that must be re-run on every USD/joint change, a Nucleus dependency in the placement path, and a 1 GiB ~/.cache cache that is now on by default for every user (a default-path change, not opt-in). Could we ship the in-process cache first — plus a plain local disk cache if the cross-process cost actually bites — and defer the exported-library machinery until it's shown to be a bottleneck?

[arena-review-bot]

🤖 Isaac Lab-Arena Review Bot

Summary

This PR makes an embodiment's placement bounding box and collision mesh reflect the robot as actually spawned — posed at its configured init_state.joint_pos via offline USD forward kinematics — instead of the arbitrary joint configuration the asset was authored in. That is a real correctness improvement for relation-based placement, and it is backed by an unusually strong test suite (PhysX ground-truth link-pose comparison, closed-loop articulations, instanced geometry, prismatic/revolute cases, LRU eviction, and stale/foreign-artifact rejection). The FK and geometry code is careful and well-documented.

Design, Boundaries & Scope

My one real question is scope, raised inline on collision_mesh_store.py: the change ships a two-tier persistence layer — a 1 GiB on-disk LRU cache plus a Nucleus-published robot library with an export/upload pipeline. The in-process lru_cache on the posed-geometry helpers already covers the relation-solver hot path within a run, so the disk + Nucleus layer only saves the one-time 0.1–2.5 s extraction per fresh process. Against that it adds ongoing maintenance (re-export on every USD/joint change), a Nucleus dependency in the placement path, and a cache that grows in every user's ~/.cache on the default path. Worth confirming that cross-process cost actually bites before taking on the exported-library machinery; the in-process cache (plus perhaps a plain local disk cache) may deliver most of the value for far less surface.

Boundaries otherwise hold: the new FK/store code is generic USD/IO utility, no robot-specific logic leaks into core, and the embodiment geometry methods stay pure (no live env).

Findings

🟡 collision_mesh_store.py — question whether the published-library + disk-LRU persistence needs to ship now, or could be deferred behind the in-process cache (inline).

Test Coverage

Excellent. New tests follow the inner/outer run_simulation_app_function pattern with deferred sim imports and land in Phase 1 (in-process persistent app, no cameras/subprocess), matching the existing sibling test. Coverage spans unit FK, real-Droid geometry, PhysX agreement, and the full store lifecycle including negative cases. No gaps worth calling out.

Verdict

Minor fixes needed — essentially ship-ready; please just weigh in on the persistence-layer scope question before merge.

[qianl-nv]

I have some general questions on why we need the "local/Omniverse cache for pre-computed mesh" part.

• collecting the mesh for droid is only taking about 1s, it's hardly the bottlenet in the overall pipeline atm. we don't think we need to go done for the perf there using cache. Finding ways the speed up the mesh mode for Background (where we absolutely need it) is more important imo.
• for embodiment, what's blocking is actually the joint-angle-based bounding box collection. unless we are confident of shipping v0.3 with both background and embodiment using mesh mode (so far it has always take forever for solver), we need a working version of background in mesh mode + embodiment in bbox mode.

[qianl-nv]

is there any risk of having an empty directory? if not let's just create the dir here following get_arena_asset_cache_dir

[qianl-nv]

This is better handle in droid/franka. set_initial_joint_pose currently only updates the reset event's default_pose.
can expand set_initial_joint_pose to also all self.set_joint_initial_pos(...) with a name→value map matching set_default_joint_pose order.