Skip to content

omni.tools.array crashing in Isaac Sim 6 #727

Description

@vitusli

Description

Using the Array Tool on physics-enabled prims and then pressing Play hard-crashes Isaac Sim. The tool internally creates and deletes copies of prims while the user adjusts parameters (preview mode). These deleted prims are gone from the scene, but Python holds onto them in memory a little longer than expected. When Play is pressed, PhysX kicks off a parallel background scan of all physics objects — and if it happens to touch one of those "not quite dead yet" prims at the same moment Python finally lets go of it, the two collide at the C++ level and the process aborts immediately with no error message in the UI.

I let my agent run through my Isaac Sim installation and fix it. It works for me now. By opening this issue, the behavior might be fixed for other users too. Because my agent fixed this, it also wrote this issue. I read through it and corrected it where necessary.

Isaac Sim version

6.0.0

Operating System (OS)

Windows 11 Pro, Build 26200, 64-bit

GPU Name

RTX 3090

GPU Driver and CUDA versions

610.62 (WDDM: 32.0.16.1062)

Logs

kit_20260715_103232_stripped.log
kit_20260723_165120_stripped.log
kit_20260709_151757_stripped.log

Additional information

Bug 1 — Hard Crash (C++ Fatal) on Play after using the Array Tool

Frequency

Reproducible. Crash confirmed in 3 independent sessions across multiple days.
All Python threads were idle at crash time — this is a pure C++ crash, not a Python exception.

Steps to Reproduce

  1. Open a USD stage containing prims with PhysX schemas
    (PhysicsRigidBodyAPI, PhysicsCollisionAPI, or similar).
  2. Select one or more of those prims.
  3. Open the Array Tool (omni.tools.array) and create an array
    (e.g. 1D, count = 5). Leave Preview mode enabled (default).
  4. Adjust any parameter so the Array Tool runs its create/delete cycle at least once.
  5. Click Apply (or leave the tool open).
  6. Press the Play button in the toolbar.

→ Isaac Sim crashes immediately with a C++ Fatal error.

Expected Behavior

Isaac Sim starts the simulation normally.

Actual Behavior

The process aborts with a [Fatal] crash report from carb.crashreporter-breakpad.plugin.


Crash Log Evidence

Here are exeptions from the crash logs, all ending with the identical [Fatal] call stack:

lastCommands field from crash reporter (Log 1 — verbatim from log)

lastCommand  = 'ToolbarPlayButtonClicked'
lastCommands = 'TransformMultiPrimsSRTCpp(...),
                Group, Group, Group, Group, Group,
                ChangeProperty(...xformOp:orient...),
                MovePrim(...),
                SelectPrimsCommand(...),
                ToolbarPlayButtonClicked'

The Group × 5 sequence is the Array Tool creating a 5-element array.
The crash happens on the very next action: pressing Play.

Full crash call stack (Log 1, thread 29368, bottom → top)

[Fatal] Crash detected in pid 13024 thread 29368

000: usd_usd.dll!pxrInternal_v0_25_11__pxrReserved__::UsdObject::_GetDefiningSpecType+0xdbace ***
001: usd_usd.dll!pxrInternal_v0_25_11__pxrReserved__::UsdObject::_GetDefiningSpecType+0x61e96a ***
002: usd_usd.dll!pxrInternal_v0_25_11__pxrReserved__::UsdObject::_GetDefiningSpecType+0x624bb4 ***
003: tbb12.dll!tbb::detail::r1::isolate_within_arena+0xdd ***          <-- TBB worker thread
004: usd_usd.dll!...+0x5de778 ***
005: usd_usd.dll!...+0x3617c8 ***
...
020: usd_sdf.dll!pxrInternal_v0_25_11__pxrReserved__::Sdf_Identity::_UnregisterOrDelete+0x96268 ***
021: omni.physx.plugin.dll!carbOnPluginStartup+0xadf10 ***             <-- PhysX scene init
022: omni.physx.plugin.dll!carbOnPluginStartup+0x40f83 ***
023: omni.physx.plugin.dll!carbOnPluginStartup+0x38d0f ***
024: physicsumbrella.dll!carbOnPluginStartup+0x4c46 ***
025: omni.stageupdate.plugin.dll!+0xfcfe ***
...
041: omni.usd.dll!omni::usd::UsdContext::unregisterViewOverrideToHydraEngines+0x15ee ***
042: carb.eventdispatcher.plugin.dll!+0x11ade ***
044: omni.timeline.plugin.dll!+0x62cb ***                              <-- timeline.Play() dispatched
...
048: omni.kit.loop-isaac.plugin.dll!+0x19fa1 ***                       <-- main loop tick
055: ntdll.dll!RtlUserThreadStart+0x2c ***

NOTE: *** indicates a low-confidence frame (symbols not found)

Root Cause Analysis

1. Stale Usd.Prim C++ objects from preview mode

In preview mode, ArrayCore repeatedly:

  1. Copies source prims via CopyPrimCommand — copies inherit all USD schemas, including PhysX schemas.
  2. Groups them via GroupPrimsCommand.
  3. On each parameter change, calls _clear_resulting_prims():
    • Calls DeletePrimsCommand to remove them from the USD stage.
    • Calls .clear() on the internal Python lists.

Problem: Python's garbage collector does not immediately release the underlying Usd.Prim
C++ objects. They remain in memory holding live references to SdfPath identities in the SDF
layer — even though the prims have already been deleted from the stage.

Additionally, ArrayCore._array_core_instance = self (the singleton) is never set to None
in clean(). All Usd.Prim references therefore survive until extension shutdown.

2. PhysX uses TBB parallel traversal — not safe against concurrent Python GC

When Play is pressed, omni.physx.plugin.dll initializes the physics scene by traversing
all physics-schema prims on Intel TBB worker threads (confirmed by tbb12.dll frame [003]).

Race condition:

  • Python GC finalizes the stale Usd.Prim objects from prior preview cycles.
  • Their ref count drops to zero → Sdf_Identity::_UnregisterOrDelete is called (frame [020]).
  • Simultaneously, PhysX TBB threads are calling UsdObject::_GetDefiningSpecType on the
    same SDF path identities (frames [000]–[002]).
  • _UnregisterOrDelete mutates the SDF identity registry while TBB threads read it → crash.

Proposed Fix

array_core.pyclean(): release singleton to allow immediate GC

# BEFORE:
def clean(self):
    self._clear_resulting_prims()

# AFTER:
def clean(self):
    self._clear_resulting_prims()
    self._target_prims = []
    self._valid_prims = []
    self._valid_prim_transforms = {}
    ArrayCore._array_core_instance = None  # allow GC to collect all Usd.Prim refs

array_core.py_clear_resulting_prims(): force GC after prim deletion

def _clear_resulting_prims(self):
    # ... (existing logic to collect valid_prim_paths) ...

    self._one_d_result_prims.clear()
    self._two_d_result_prims.clear()
    self._three_d_result_prims.clear()
    self.delete_prims(valid_prim_paths)

    # Force GC immediately so Usd.Prim C++ objects are released before
    # any subsequent PhysX scene init (Play) triggers TBB traversal.
    import gc
    gc.collect()

Bug 2 — Deprecated get_stage_event_stream() warning in Kit 110

Location

action_window.py, line 49:

self._stage_event_sub = self._usd_context.get_stage_event_stream().create_subscription_to_pop(
    self._on_stage_event
)

Problem

Kit 110.x uses Events 2.0. The above call goes through a deprecated carb wrapper that
emits a warning on every startup. It may also mis-sequence stage events relative to the
omni.physics.stageupdate plugin ordering, contributing to the race condition above.

Proposed Fix

Replace with the direct omni.usd API (no deprecated wrapper):

import omni.usd
self._stage_event_sub = (
    omni.usd.get_context()
    .get_stage_event_stream()
    .create_subscription_to_pop(self._on_stage_event)
)

Bug 3 — Window drifts to the left screen edge every time it is opened

Steps to Reproduce

  1. Open Isaac Sim.
  2. Open the Array Tool window.
  3. Observe: the window moves steadily to the left until it hits the screen edge.

Root Cause

action_window.py sets auto_resize=True on the ui.Window:

self._window = ui.Window(ui_c.NAME_TOOL, visible=False, width=430, auto_resize=True)

In Kit 110, auto_resize computes the window's bounding box from the full visual extent of
all child widgets, including ui.Placer elements with negative offsets.

Inside _create_multi_float_drag_with_labels(), multiple ui.Placer elements use:

with ui.Placer(offset_x=-20 - i, width=0):
    ...

Kit 110 includes these negative offset_x values in the bounding box calculation, shifting the
computed left edge further left on every frame. This did not occur in Kit 108 (the extension's
original target), where auto_resize only affected height.

Proposed Fix

# BEFORE:
self._window = ui.Window(ui_c.NAME_TOOL, visible=False, width=430, auto_resize=True)

# AFTER:
self._window = ui.Window(ui_c.NAME_TOOL, visible=False, width=430)
# auto_resize removed: Kit 110 includes ui.Placer negative offsets in the bounding box,
# causing the window to drift left every frame until it hits the screen edge.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions