Fix isaaclab.* INFO logs silenced on kitless backends - #6813
Fix isaaclab.* INFO logs silenced on kitless backends#6813mataylor-nvidia wants to merge 1 commit into
Conversation
The isaaclab_info_stream log handler was installed only inside AppLauncher, which is skipped entirely for kitless backends (Newton, OvPhysX). As a result, logger.info() calls such as "Created new renderer for simulation" and "Using renderer" in RenderContext and Camera were silenced even though the handler was designed to surface them. Extract ensure_isaaclab_info_stream_handler() from AppLauncher into logging_utils so it can be shared. The kitless path in sim_launcher now calls it and raises the root logger to INFO (matching what AppLauncher already does for Kit-based backends), making isaaclab.* INFO records visible on the console for all backends.
|
Too many files changed for review. ( |
There was a problem hiding this comment.
Isaac Lab Review Bot
The shared isaaclab_info_stream handler extraction preserves the Kit path and enables INFO output for kitless Newton/OvPhysX runs. However, the kitless root-level adjustment unintentionally overrides explicitly stricter logging levels such as ERROR or CRITICAL.
- Design and architecture: Centralizing the idempotent handler installer in
logging_utilsand retainingAppLauncher._ensure_isaaclab_info_stream_handleras a delegator is a sound way to share logging behavior across Kit and kitless launch paths. - API: The existing
AppLauncherstatic method remains available, while the new helper is documented consistently with the neighboring logging utilities. The patch-tier changelog fragment follows the required format. - Implementation: The extracted handler retains the existing filter, formatter, and name-based idempotence. In the kitless path, however,
level >= logging.WARNINGalso matches ERROR and CRITICAL, so the subsequent root logger change to INFO defeats a caller's stricter configured level. Limit this adjustment to the intended default WARNING case or scope the INFO level to theisaaclablogger.
Minor fixes needed. Posted 1 actionable finding inline.
Automated review; human maintainers own approval decisions.
| level = resolve_python_logging_level(launcher_args) | ||
| apply_python_logging_level(level) | ||
| ensure_isaaclab_info_stream_handler() | ||
| if level >= logging.WARNING: |
There was a problem hiding this comment.
🟡 Warning · Implementation — Root level override also matches ERROR/CRITICAL
level >= logging.WARNING matches ERROR and CRITICAL too. A caller that set the root logger to ERROR gets that level applied by apply_python_logging_level(level) and then silently reset to INFO on the next line, re-enabling isaaclab.* INFO output through the scoped handler and record creation for all loggers. The comment states the intent is only the default WARNING case; use level == logging.WARNING, or raise only the isaaclab logger instead of root.
|
This shows how logging has changed: Run 1 — generic
|
| Log file | Command | Created new renderer |
Using renderer |
|---|---|---|---|
| cmd1_rtx_newton_mjwarp.log | renderer=rtx physics=newton_mjwarp |
✅ OVRTXRenderer |
✅ OVRTXRenderer |
| cmd2_rtx_ovphysx.log | renderer=rtx physics=ovphysx |
✅ OVRTXRenderer |
✅ OVRTXRenderer |
| cmd3_rtx_viz_kit.log | renderer=rtx --viz=kit |
✅ IsaacRtxRenderer |
✅ IsaacRtxRenderer |
Key log lines
cmd1 (renderer=rtx physics=newton_mjwarp — OVRTX kitless + Newton MJWarp):
[INFO]: Created new renderer for simulation: OVRTXRenderer
[INFO]: Using renderer: OVRTXRenderer
cmd2 (renderer=rtx physics=ovphysx — OVRTX kitless + OVPhysX):
[INFO]: Created new renderer for simulation: OVRTXRenderer
[INFO]: Using renderer: OVRTXRenderer
cmd3 (renderer=rtx --viz=kit — Isaac Sim Kit path):
[INFO]: Created new renderer for simulation: IsaacRtxRenderer
[INFO]: Using renderer: IsaacRtxRenderer
|
@ndahile-nvidia for review |
| if level >= logging.WARNING: | ||
| # Root logger is at WARNING; raise it to INFO so that isaaclab.* INFO | ||
| # records can propagate to the scoped stream handler above. | ||
| logging.getLogger().setLevel(logging.INFO) |
There was a problem hiding this comment.
would this make all logging level as info for the process? i.e. we'd end up seeing a lot of prints in the terminal?
There was a problem hiding this comment.
AI-generated follow-up: Yes. The scoped handler filters what it prints, but setting the root logger to INFO changes effective levels process-wide. I reproduced a stronger two-launch case: the first default call left root at INFO; the second resolved that internal state as the requested level, lowered an existing root handler to INFO, and emitted an unrelated third-party INFO record. The bot is also correct that the current comparison overrides ERROR and CRITICAL. Please elevate only the isaaclab namespace and leave the root level unchanged.
There was a problem hiding this comment.
AI-generated suggestion: Centralize the policy in one private logging helper called by both AppLauncher and sim_launcher. It should: (1) apply the resolved level to the root logger and leave that root level unchanged; (2) set logging.getLogger("isaaclab") to INFO only when the resolved level is the default WARNING, otherwise use the resolved level; and (3) install or reconfigure one INFO handler filtered to record.name == "isaaclab" or record.name.startswith("isaaclab."). Remove the AppLauncher delegating wrapper. This surfaces Isaac Lab INFO records while Kit and third-party Python INFO records remain hidden by the WARNING root handlers. Carb or native stdout messages are outside Python logging and are unaffected. A focused standard-library test should cover two consecutive default configurations, ERROR/CRITICAL preservation, no duplicate handler, and rejection of a name such as isaaclab_plugin; no simulator integration test is needed.
AntoineRichard
left a comment
There was a problem hiding this comment.
AI-generated review
Requesting changes.
The review bot's >= logging.WARNING finding is correct, and Kelly's process-wide logging concern is reproducible. With the sequence in this PR, a first default kitless launch resolves WARNING but leaves the root logger at INFO; a second launch then resolves that internally modified INFO level, lowers pre-existing root handlers to INFO, and allows unrelated third-party INFO records through. Explicit ERROR and CRITICAL levels are also reset to INFO. Please keep the INFO enablement scoped to the isaaclab logger namespace and preserve the caller's root level.
The inline comments cover the minimal regression coverage, exact namespace matching, unnecessary API/indirection, and duplicated changelog rationale.
Verification: source/isaaclab_tasks/test/core/test_sim_launcher_visualizer_intent.py passed (3 tests), but its current logging test does not exercise this behavior. ./isaaclab.sh -f passed all hooks. The PR currently conflicts with develop and will also need a rebase.
| apply_python_logging_level(resolve_python_logging_level(launcher_args)) | ||
| level = resolve_python_logging_level(launcher_args) | ||
| apply_python_logging_level(level) | ||
| ensure_isaaclab_info_stream_handler() |
There was a problem hiding this comment.
AI-generated review — Important: The existing test_launch_simulation_kitless_applies_python_logging_level mocks only the resolver and applier. This new real call, together with the root-level branch below, mutates global logging state; the test's fake level 42 enters that branch without asserting or restoring the state. Mock this helper in the plumbing-only test, then add one focused standard-library logging test that covers two default launches, ERROR/CRITICAL, and an unrelated logger. No simulator integration test is needed.
| handler.setLevel(level) | ||
|
|
||
|
|
||
| def ensure_isaaclab_info_stream_handler() -> None: |
There was a problem hiding this comment.
AI-generated review — Moderate: This helper has only internal launcher callers, but its public name and changelog cross-reference turn an implementation detail into API that would later require deprecation. Please make it private, call it directly from both launcher modules, and keep its docstring to the functional contract. The current regardless of what Kit does claim also contradicts the root-level prerequisite documented immediately below it.
| handler.addFilter(_IsaacLabInfoFilter()) | ||
| handler.setFormatter(logging.Formatter("[INFO]: %(message)s")) | ||
| root_logger.addHandler(handler) | ||
| ensure_isaaclab_info_stream_handler() |
There was a problem hiding this comment.
AI-generated review — Moderate: This private one-line delegator no longer owns any behavior and duplicates the shared helper's name. Remove it and invoke the module helper directly at the two _load_extensions call sites; that leaves one implementation and one functional contract to maintain.
|
|
||
| class _IsaacLabInfoFilter(logging.Filter): | ||
| def filter(self, record: logging.LogRecord) -> bool: | ||
| return record.levelno == logging.INFO and record.name.startswith("isaaclab") |
There was a problem hiding this comment.
AI-generated review — Moderate: startswith("isaaclab") also accepts unrelated logger names such as isaaclab_plugin, although the contract says the isaaclab.* namespace. Match record.name == "isaaclab" or record.name.startswith("isaaclab."), and cover that boundary in the focused logging test.
| * Fixed ``logger.info`` calls in :class:`~isaaclab.renderers.RenderContext` and | ||
| :class:`~isaaclab.sensors.camera.Camera` (e.g. "Created new renderer for simulation" and | ||
| "Using renderer") being silenced on kitless backends (Newton, OvPhysX). | ||
| The ``isaaclab_info_stream`` log handler was only installed inside |
There was a problem hiding this comment.
AI-generated review — Moderate: Keep the changelog user-facing. These implementation details duplicate the PR description and helper docstring, and they advertise a helper that should remain internal. Reduce the entry to the observable outcome, for example: Fixed renderer and camera INFO messages being hidden when using kitless backends.
|
AI-generated implementation suggestion I recommend centralizing the complete logging policy in one private helper. The root logger should remain at the resolved level, while the In _ISAACLAB_INFO_HANDLER_NAME = "isaaclab_info_stream"
def _ensure_isaaclab_info_stream_handler() -> None:
"""Install the scoped Isaac Lab INFO handler if needed."""
root_logger = logging.getLogger()
for handler in root_logger.handlers:
if handler.name == _ISAACLAB_INFO_HANDLER_NAME:
# apply_python_logging_level() may have changed this.
handler.setLevel(logging.INFO)
return
class _IsaacLabInfoFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool:
is_isaaclab_logger = record.name == "isaaclab" or record.name.startswith("isaaclab.")
return record.levelno == logging.INFO and is_isaaclab_logger
handler = logging.StreamHandler(sys.stdout)
handler.name = _ISAACLAB_INFO_HANDLER_NAME
handler.setLevel(logging.INFO)
handler.addFilter(_IsaacLabInfoFilter())
handler.setFormatter(logging.Formatter("[INFO]: %(message)s"))
root_logger.addHandler(handler)
def _configure_python_logging(level: int) -> None:
"""Configure Python logging with default INFO output scoped to Isaac Lab."""
apply_python_logging_level(level)
isaaclab_logger = logging.getLogger("isaaclab")
isaaclab_logger.setLevel(logging.INFO if level == logging.WARNING else level)
if level <= logging.WARNING:
_ensure_isaaclab_info_stream_handler()Then both launch paths can use the same policy. In _configure_python_logging(self._python_logging_level)In if not needs_kit:
level = resolve_python_logging_level(launcher_args)
_configure_python_logging(level)This removes the process-wide A focused standard-library test should verify:
No simulator integration test is needed. Per the regression-test policy, temporarily restoring the old root-level override should make the new test fail. |
Summary
isaaclab_info_streamlog handler was only installed insideAppLauncher, which is skipped entirely for kitless backends (Newton, OvPhysX)logger.info()calls inRenderContext("Created new renderer for simulation") andCamera("Using renderer") were silenced on the Newton path even though they should appear at default log levelensure_isaaclab_info_stream_handler()fromAppLauncherintologging_utilsso it can be shared; the kitless path insim_launchernow calls it and raises the root logger to INFO, matching whatAppLauncheralready does for Kit-based backendsTest plan
./isaaclab.sh train --rl_library skrl --task Isaac-Cartpole-Camera-Direct renderer=newton_renderer physics=newton_mjwarpand confirm[INFO]: Created new renderer for simulation: NewtonWarpRendererand[INFO]: Using renderer: NewtonWarpRendererappear in stdout[INFO]:output--verbose/--infoflags on both paths and confirm no duplicate or missing log lines