Skip to content

Enable lightwheel kitchen mesh placement - #981

Closed
qianl-nv wants to merge 2 commits into
mainfrom
qianl/feature/lw-kitchen-mesh2
Closed

Enable lightwheel kitchen mesh placement#981
qianl-nv wants to merge 2 commits into
mainfrom
qianl/feature/lw-kitchen-mesh2

Conversation

@qianl-nv

Copy link
Copy Markdown
Collaborator

Summary

Mesh kitchen placement; bind pool on env runtime

Detailed description

  • Lightwheel kitchen bench uses mesh collision mode through PlaceableAsset kwargs; set it to mesh for the droid pick and place yaml; keep the default as bbox for other existing environments.
  • Pass the live placement pool through env_kwargs instead of EventTermCfg params so mesh placement Warp caches do not hit configclass deepcopy/pickle ("ctypes objects containing pointers cannot be pickled")
  • Reset placement_reset event uses empty params and reads the pool from IsaacLabArenaManagerBasedRLEnv.placement_pool

Note: With Fabric enabled, Warp caches on the pool can hide the Droid stand_instanceable in Kit viz while physics stays correct; use --disable_fabric for viewport runs (4 env / 2000-step benchmark: 87.0 vs 86.5 ms/step rollout, no meaningful difference)

  • With background bbox mode (collision with backgroudn object)
image
  • With background mesh mode + fix + fabric (stand visual disappear)
image
  • With background mesh mode + fix + disable fabric
image

qianl-nv added 2 commits July 30, 2026 16:16
Pass the live PooledObjectPlacer through env_kwargs (like
variation_recorder) instead of EventTermCfg params. Mesh-mode
placement builds Warp BVHs on the pool; storing it in config
deepcopy fails with "ctypes objects containing pointers cannot
be pickled" and configclass validation recurses into cyclic
asset graphs. The reset event now has empty params and reads the
pool from the env.

Leaving Warp mesh caches on the pool through Fabric startup
also hides the Droid stand_instanceable in Kit viz (physics and
placement stay correct). Use --disable_fabric for viewport runs
until that is resolved; a 4-env / 2000-step benchmark showed no
rollout cost from the flag (87.0 vs 86.5 ms/step).

Signed-off-by: Qian Lin <qianl@nvidia.com>
Comment on lines 406 to +408
env_kwargs: dict[str, Any] = {"variation_recorder": variation_recorder}
if self._placement_pool is not None:
env_kwargs["placement_pool"] = self._placement_pool

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Placement pool lost in callback

When a relation-based environment with reset-time placement is launched through the IsaacLab external callback, the callback discards the env_kwargs containing placement_pool. The environment is then constructed with a null pool, causing its initial placement reset to fail with AssertionError: Env has no placement pool; build through ArenaEnvBuilder.

Knowledge Base Used: Environment builder flow

@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR enables mesh-based Lightwheel kitchen collision placement and moves the reset placement pool out of event configuration.

  • Adds configurable collision mode and non-watertight mesh handling to placeable assets and the Lightwheel kitchen background.
  • Returns the live placement pool from relation solving and passes it to the runtime environment.
  • Makes reset placement retrieve assets and layouts from the environment-bound pool.
  • Updates placement and background-collision tests for the new runtime ownership model.

Confidence Score: 4/5

The external IsaacLab callback path must preserve the live placement pool before this PR is safe to merge.

Reset-time relation placement now depends on an environment constructor keyword that the direct builder path forwards, but the supported external callback discards before IsaacLab constructs and initially resets the environment.

Files Needing Attention: isaaclab_arena/environments/arena_env_builder.py and isaaclab_arena/environments/isaaclab_interop.py

Important Files Changed

Filename Overview
isaaclab_arena/environments/arena_env_builder.py Returns the live placement pool through environment kwargs, but that transport does not survive the external-callback registration flow.
isaaclab_arena/environments/relation_solver_interface.py Separates the reset event configuration from the live placement pool and returns both to the builder.
isaaclab_arena/environments/isaaclab_arena_manager_based_env.py Stores an optionally supplied placement pool before base-environment initialization and exposes it through a property.
isaaclab_arena/relations/placement_events.py Retrieves reset state from the runtime environment and fails fast when the pool was not forwarded.
isaaclab_arena/relations/placement_asset.py Accepts per-asset collision mode and mesh-repair settings while retaining existing defaults.
isaaclab_arena/assets/background_library.py Forwards Lightwheel kitchen parameters through the asset constructor hierarchy.
isaaclab_arena_environments/kitchen_bench/droid_pick_and_place_lightwheel_kitchen.yaml Enables unrepaired mesh collision for the Lightwheel kitchen background.

Sequence Diagram

sequenceDiagram
  participant Builder as ArenaEnvBuilder
  participant Callback as IsaacLab external callback
  participant Gym as gym.make
  participant Env as Arena runtime env
  participant Reset as placement_reset
  Builder->>Builder: Solve relations and create pool
  Builder->>Callback: build_registered() returns cfg + env_kwargs
  Callback--xGym: placement_pool kwargs discarded
  Gym->>Env: "Construct with placement_pool=None"
  Env->>Reset: Initial reset
  Reset-->>Env: Assertion: placement pool missing
Loading

Reviews (1): Last reviewed commit: "Bind placement pool on env at runtime" | Re-trigger Greptile

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

Enables mesh-mode collision placement for the Lightwheel kitchen background and moves the live PooledObjectPlacer off EventTermCfg.params onto the env at runtime (via env_kwargs), so mesh-mode Warp caches no longer trip configclass deepcopy/pickle. The refactor is clean and mirrors the existing variation_recorder plumbing: the reset event now reads the pool from env.unwrapped.placement_pool and derives its assets from pool.objects (the same list previously passed as params).

Findings

No blocking issues. The collision_mode: mesh YAML string threads correctly through **kwargs to PlaceableAsset, where CollisionMode("mesh") normalizes it; storing the pool on the runtime env is the right layer; and get_placement_pool's getattr(..., "placement_pool", None) reads a declared property with a graceful None for non-pooled envs (a contract callers like placement_pool_validation already rely on), so it is not the ad-hoc-storage smell.

Test Coverage

Good. Existing non-sim unit tests are updated to the new signatures, and two new regression tests cover the exact deepcopy/_validate path that previously failed under mesh mode — test_dynamic_spawn_pose_event_cfg_deepcopy_after_mesh_solve and test_solve_and_apply_relation_placement_drops_embodiment_from_runtime_pool. These stay sim-free (MagicMock envs), so the inner/outer pattern is not needed.

Verdict

Ship it.

@qianl-nv

Copy link
Copy Markdown
Collaborator Author

Won't go with this approach. willl use #972 instead.

@qianl-nv qianl-nv closed this Jul 30, 2026
@qianl-nv
qianl-nv deleted the qianl/feature/lw-kitchen-mesh2 branch July 30, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant