Add Flexiv command shaping to DisplayPort env - #8
Open
ashwinvkNV wants to merge 64 commits into
Open
Conversation
The Rizon 4s cable-insertion task was failing every episode at reset: the IK + open-loop close sequence put the gripper next to the plug instead of around it, the fingers closed on empty space, and the plug fell to the floor (or twisted away when a finger clipped its edge). Root cause was a geometry mismatch between the un-flattened plug USD and the IK target. The un-flattened plug has its visible/collision ``Mesh`` offset ~3 cm from the rigid-body origin (``bbox_center_local = (0.0312, 0.0009, -0.0107)`` in body frame). ``set_robot_to_object_grasp_pose`` snapped the plug body to the gripper-achieved pose, but the achieved pose was a function of the *body* origin, not the visible centre, so the fingertips landed ~3 cm from the actual plug. Switch to the flattened plug + socket USDs (``flattened/`` directory) where ``bbox_center == (0, 0, 0)`` and the visible mesh coincides with the rigid-body origin. With that, the grasp parameters reduce to the gripper tool length only: * ``grasp_offset = (0, 0, -0.13)`` -- only backs ``link7`` off by the Grav tool length so the fingertips land at the plug body origin (which is now also the visible centre). * ``grasp_rot_offset = (-0.707, 0.707, 0, 0)`` (unchanged) keeps ``link7`` pointing straight down with the gripper finger axis perpendicular to the plug's long body Y axis. * Plug + socket use identity ``init_state.rot``; the flattened USDs already align the long axis with body Y. * Plug spawns 5 cm above the socket so the policy has descent room. * ``hand_close_width = 0.0`` matches the gear-assembly small-gear close target. ``-0.155`` (the joint hard limit) was tried and ejected the plug because the drive over-commanded the fingers. Also extend ``set_robot_to_object_grasp_pose``: * Snap the held object to the achieved hand pose after IK so any IK residual error or geometry offset is washed out before the gripper closes. * One-shot ``[GRASP-DBG]`` log dumps the initial pose, the IK target, the achieved hand pose and the snapped object pose -- invaluable for diagnosing future grasp regressions.
The Flexiv Rizon 4s in the NVIDIA Hubble Lab is wall-mounted with the arm rotated about the negative X-axis. Update the ROS inference env config so the simulator's reset state matches the physical robot: * Set the home joint pose to the lab's joint convention. * Set the robot base orientation to the wall-mount quaternion so the arm hangs the same way in sim and in the real cell. This keeps the policy roll-out and the live deployment in the same robot frame, which is required for the policy obs (computed relative to ``base_link``) to be valid at inference time.
Three small scripts under ``scripts/tools/`` to diagnose and repair USD assets whose visible/collision geometry is offset from the rigid-body origin -- the failure mode that broke the Rizon 4s cable-insertion grasp. * ``inspect_cable_usd.py``: standalone Pixar-USD walker. Lists xform ops, ``RigidBodyAPI`` / ``MassAPI`` / ``CollisionAPI`` markers and per-mesh ``bbox_center_local`` / ``bbox_size``. Use this from any shell that already has ``pxr`` on PYTHONPATH (e.g. a Kit Python). * ``inspect_cable_usd_kit.py``: same walker run inside an IsaacLab ``AppLauncher`` context so ``pxr`` is available without extra setup. This is what we actually used during debugging from the IsaacLab conda env. * ``flatten_cable_usd.py``: bake the chain of intermediate xforms down into the leaf mesh's points, then reset all xform ops on the rigid-body and its children to identity. The result is a USD whose visible mesh is centered at the rigid-body origin -- exactly the invariant the IK-based grasp event assumes. Supports an optional ``--rotate-z-deg`` knob for assets whose body-local pose needs an additional rotation baked in. Both inspectors print the bounding-box centre in body-local frame so ``grasp_offset`` can be set deterministically from the asset rather than tuned by trial and error.
The ``-Play-v0`` variants for Rizon 4s and UR10e gear assembly omitted ``rsl_rl_cfg_entry_point`` in their gym registration, so ``play.py`` could not resolve the agent config and would error out with ``KeyError`` before ever loading a checkpoint. Add the entry-point so ``./isaaclab.sh -p .../play.py --task Isaac-Deploy-GearAssembly-...-Play-v0`` works end-to-end. Also pick up a pre-commit whitespace cleanup in ``play_hardcoded.py``.
Required for training: env cfg references these files but they were excluded by the *.usd gitignore rule.
…eam rebase. Upstream moved deploy tasks from manager_based/manipulation/deploy to contrib/deploy; update module paths and scripts so DP env registration and imports match the new layout while keeping existing Isaac-Deploy task IDs.
Remove debug scripts, asset tooling, passive drop-test env, legacy GB300 configs, unused assets, and disabled sysid action plumbing. Keep only the displayport_rizon_4s joint/task/ROS env variants and the two SDF USD assets used by the training envs.
shauryadNv
force-pushed
the
shauryad/dp_cable_ship
branch
from
July 30, 2026 21:11
1fdfcfa to
1516ffc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds Flexiv Rizon4s command-response shaping to the DisplayPort cable insertion ROS-inference environment.
The change builds on
shauryad/dp_cable_shipand updates the DisplayPort Rizon4s ROS-inference configs to use a shaped/delayed relative joint-position action term:2.0 rad/s3.0 rad/s^220 msThe DisplayPort environment timing is intentionally left unchanged:
240 Hz830 HzThis PR does not change the actuator/PD gains and does not add the tuned SysID actuator YAML. The robot continues to use the existing DisplayPort actuator configuration.
The DisplayPort env already logs insertion success metrics:
Metrics/success_rateMetrics/plug_socket_pos_error_mMetrics/plug_socket_keypoint_dist_mMetrics/terminal_success_rateThe success threshold is currently
3 mmplug/socket mate-point distance.No new external dependencies are required.
Fixes # N/A
Type of change
Screenshots
N/A
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there