Skip to content

Enable lightwheel kitchen mesh placement - #972

Open
qianl-nv wants to merge 3 commits into
mainfrom
qianl/feature/lw-kitchen-mesh
Open

Enable lightwheel kitchen mesh placement#972
qianl-nv wants to merge 3 commits into
mainfrom
qianl/feature/lw-kitchen-mesh

Conversation

@qianl-nv

@qianl-nv qianl-nv commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Set lightwheel kitchen placement to mesh and fix crashes from placement pool

Detailed description

  • Pass mesh collision mode through PlaceableAsset kwargs; set it to mesh for the droid pick and place yaml to avoid foreground objects on counter top intersecting with existing background scene objects. Keep the default as bbox for other existing environments.
  • For the droid pick and place yaml with kitchen in mesh mode, env bringup crashes with two issues: EventTermCfg deepcopying placement pool with Warp ctypes caches ("ctypes objects containing pointers cannot be pickled"); configclass validation on the placement pool recurse into cyclic asset graphs throwing RecurssionError. Fix both by introducing a PlacementPoolHandle the wraps the pool to deepcopy and limits validation depth.

Known issue: With Fabric enabled, live Warp caches on the pool can hide the Droid stand_instanceable in Kit viz while physics stays correct; use --disable_fabric --device cpu for viewport runs (4 env / 2000-step benchmark: 87.0 vs 86.5 ms/step rollout, no meaningful difference). Alternative to fix this problem requires releasing the warp cache before passing into EventTermCfg. This slows down the first refill solve significantly (x2).

  • With background bbox mode (collision with backgroudn object)
image
  • With background mesh mode + fix + fabric (stand visual disappear)
image
  • With background mesh mode + fix + disable fabric
image

@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Enables mesh-based Lightwheel kitchen placement while preventing placement resources from interfering with configuration validation and simulator startup.

  • Configures the Lightwheel kitchen to use its raw mesh for collision-aware relation placement.
  • Releases solver and validator mesh caches after construction; later solves rebuild them lazily.
  • Introduces an opaque, deepcopy-stable handle for reset-event placement pools and derives runtime assets from the pool.
  • Updates placement-event and relation-solver tests for the new lifecycle and parameter contract.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code defect identified.

Mesh resources cleared after construction are recreated by later solver and validator paths, while reset events consistently unwrap the shared pool handle and use the pool's own object set.

Important Files Changed

Filename Overview
isaaclab_arena/assets/background_library.py Configures the Lightwheel kitchen background for unrepaired mesh collision so placement can use its detailed geometry.
isaaclab_arena/environments/relation_solver_interface.py Releases construction-time mesh resources before storing the placement pool in a reset event through an opaque handle.
isaaclab_arena/relations/placement_events.py Adds the opaque pool handle, unwraps it at runtime, and sources placement assets directly from the pool.
isaaclab_arena/relations/relation_solver.py Adds explicit cleanup for solver mesh managers, pair caches, collision state, and cached orientations.
isaaclab_arena/relations/placement_validators.py Adds a validator cleanup hook and releases the no-overlap validator's CPU mesh cache.
isaaclab_arena/relations/pooled_object_placer.py Exposes mesh-resource cleanup through the pooled placer's underlying ObjectPlacer.
isaaclab_arena/tests/test_placement_events.py Updates reset-event tests for pool-owned assets and verifies opaque-handle resolution and config validation.
isaaclab_arena/tests/test_relation_solver_interface.py Verifies resource release, handle-backed event parameters, and deepcopy/config validation after a mesh solve.

Sequence Diagram

sequenceDiagram
  participant Builder as Environment Builder
  participant Pool as PooledObjectPlacer
  participant Solver as Relation Solver
  participant Event as Reset Event
  Builder->>Pool: Construct placement pool
  Pool->>Solver: Solve initial mesh-aware layouts
  Solver-->>Pool: Ranked layouts
  Builder->>Pool: Release mesh collision resources
  Builder->>Event: Store opaque PlacementPoolHandle
  Event->>Pool: Sample layouts on reset
  alt Pool needs refill
    Pool->>Solver: Solve additional layouts
    Solver->>Solver: Lazily rebuild mesh resources
    Solver-->>Pool: Additional ranked layouts
  end
  Pool-->>Event: Per-environment placements
Loading

Reviews (1): Last reviewed commit: "Add placement pool handle." | Re-trigger Greptile

PLACEMENT_RESET_EVENT_NAME = "placement_reset"


class PlacementPoolHandle:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Live runtime pool stored in config params

PlacementPoolHandle exists only so a live, mutating PooledObjectPlacer can survive configclass validation/deep-copy while sitting in EventTermCfg.params — an opaque token with __slots__ plus a __deepcopy__ that returns self. That's a fair bit of machinery to make runtime state pass as config.

Since the pool is runtime state (it mutates as layouts are consumed and holds warp caches), would it be cleaner long-term to hold it on the env/runtime and have the reset event look it up, rather than store it in event params? Not a blocker for this fix — mostly asking whether the handle is papering over the pool living in the config layer. The docstring here is great context either way.

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

This PR enables MESH collision on the Lightwheel Robocasa kitchen, drops warp mesh caches after the construction solve so CUDA BVHs are not pinned through Kit startup/reset, and wraps the live placement pool in an opaque PlacementPoolHandle so configclass validation does not recurse into cyclic asset graphs. The change is small, well-documented, and comes with solid regression tests. Nice cleanup dropping the redundant assets event param in favor of placement_pool.objects.

Design, Boundaries & Scope

One non-blocking question (also inline): PlacementPoolHandle is a fair bit of machinery — an opaque __slots__ token plus a __deepcopy__-returns-self — whose sole job is to let live, mutating runtime state survive configclass validation/deep-copy inside EventTermCfg.params. It fixes a real crash and is clearly deliberate, but it hints that the runtime pool may belong on the env/runtime layer (looked up by the reset event) rather than in config params. Storing the pool there is pre-existing (get_placement_pool already reads it from the event term), so this is a follow-up thought, not a blocker for this fix.

Findings

🟡 relations/placement_events.py:29PlacementPoolHandle makes runtime state pass as config to dodge validation recursion; worth considering whether the pool should live on the runtime layer instead. Non-blocking.

The release_mesh_collision_resources chain (solver → validators → placer → pool) is consistent and safe: every released attribute is recomputed lazily on the next solve/validate, so refills rebuild caches correctly.

Test Coverage

Good. test_dynamic_spawn_pose_event_cfg_deepcopy_after_mesh_solve is a real regression test — it deep-copies and _validates the event cfg after a MESH solve, which is exactly the crash path. Handle unwrapping (get_placement_pool, resolve_placement_pool) and the mesh-release call are asserted. Tests follow the file's existing deferred-import convention and run in Phase 1 (no sim spin-up).

Verdict

Ship it — clean, well-tested fix; the one design question is a follow-up, not a blocker.

Base automatically changed from qianl/feature/lw-kitchen-yaml to main July 29, 2026 15:02
@qianl-nv
qianl-nv force-pushed the qianl/feature/lw-kitchen-mesh branch 2 times, most recently from 949ab01 to cc40fa2 Compare July 30, 2026 10:31
@qianl-nv qianl-nv changed the title Enable lightweel kitchen mesh placement Enable lightwheel kitchen mesh placement Jul 30, 2026
class PlacementPoolHandle:
"""Opaque EventTermCfg param holding a runtime placement pool.

Isaac Lab ``configclass._validate`` recursively walks any object with ``__dict__`` and has no

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion to rewrite this docstring in human :)

It exists because Isaac Lab deep-copies every termination parameter and would then hold a potential outdated copy?

@qianl-nv qianl-nv Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not for potential outdated copy.
on main, if we switch any asset to mesh mode instead of bbox, a warp GPU cache (wp.Mesh for BVHs) is created, which causes deepcopy to crash (ctype no pickled). so we need to either release that cache before sim starts (which requires recreating it on the first placement layout refill with extra runtime), or just avoid deepcopying all together.
After that fix there's also another RecurssionError issue with config validation.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the docstring to make that more clear.

Comment thread isaaclab_arena/relations/placement_events.py Outdated
Comment thread isaaclab_arena/relations/placement_events.py Outdated
Comment thread isaaclab_arena/relations/placement_events.py Outdated
Comment thread isaaclab_arena/relations/placement_asset.py Outdated
PLACEMENT_RESET_EVENT_NAME = "placement_reset"


class PlacementPoolHandle:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PlacementPoolHandle doesn’t make its purpose very clear. Could we use a more descriptive name, such as PlacementResetState?

@qianl-nv qianl-nv Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle is a commonly used program object to holds reference to other resources. It's used for passing references through APIs that cannot accept the underlying object directly, and avoid accidental copying/serialization. This is exactly the purpose we introduce this wrapper here.
I think the name is fitting for what it is & its purpose. Prefer to keep.

@qianl-nv
qianl-nv force-pushed the qianl/feature/lw-kitchen-mesh branch from cc40fa2 to 7d3c7dc Compare July 31, 2026 17:56
Store the live placement pool behind an opaque handle in
EventTermCfg params instead of the pool object. Mesh-mode
placement builds Warp BVHs on the pool's solver; configclass
deepcopy then fails with "ctypes objects containing pointers
cannot be pickled". The handle shares one pool across
deep-copies and keeps validation from recursing into cyclic
asset graphs.

Leaving those Warp mesh caches on the pool through Fabric
startup also hides the Droid stand_instanceable in Kit viz
(physics and placement stay correct). Use --disable_fabric for
viewport runs until that is resolved; a 4-env / 2000-step
benchmark showed no rollout cost from the flag (87.0 vs 86.5
ms/step).

Drop the redundant assets event kwarg; layouts come from
placement_pool.objects.

Signed-off-by: Qian Lin <qianl@nvidia.com>
@qianl-nv
qianl-nv force-pushed the qianl/feature/lw-kitchen-mesh branch from 7d3c7dc to 69ddc99 Compare August 3, 2026 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants