Skip to content

Fix isaaclab.* INFO logs silenced on kitless backends - #6813

Open
mataylor-nvidia wants to merge 1 commit into
isaac-sim:developfrom
mataylor-nvidia:mataylor/fix-kitless-renderer-logging
Open

Fix isaaclab.* INFO logs silenced on kitless backends#6813
mataylor-nvidia wants to merge 1 commit into
isaac-sim:developfrom
mataylor-nvidia:mataylor/fix-kitless-renderer-logging

Conversation

@mataylor-nvidia

Copy link
Copy Markdown

Summary

  • The isaaclab_info_stream log handler was only installed inside AppLauncher, which is skipped entirely for kitless backends (Newton, OvPhysX)
  • logger.info() calls in RenderContext ("Created new renderer for simulation") and Camera ("Using renderer") were silenced on the Newton path even though they should appear at default log level
  • Extracted ensure_isaaclab_info_stream_handler() from AppLauncher into logging_utils so it can be shared; the kitless path in sim_launcher now calls it and raises the root logger to INFO, matching what AppLauncher already does for Kit-based backends

Test plan

  • Run ./isaaclab.sh train --rl_library skrl --task Isaac-Cartpole-Camera-Direct renderer=newton_renderer physics=newton_mjwarp and confirm [INFO]: Created new renderer for simulation: NewtonWarpRenderer and [INFO]: Using renderer: NewtonWarpRenderer appear in stdout
  • Run with Kit-based renderer (default) and confirm no regression in existing [INFO]: output
  • Run with --verbose / --info flags on both paths and confirm no duplicate or missing log lines

The isaaclab_info_stream log handler was installed only inside
AppLauncher, which is skipped entirely for kitless backends (Newton,
OvPhysX). As a result, logger.info() calls such as "Created new
renderer for simulation" and "Using renderer" in RenderContext and
Camera were silenced even though the handler was designed to surface
them.

Extract ensure_isaaclab_info_stream_handler() from AppLauncher into
logging_utils so it can be shared. The kitless path in sim_launcher
now calls it and raises the root logger to INFO (matching what
AppLauncher already does for Kit-based backends), making isaaclab.*
INFO records visible on the console for all backends.
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review. (3000 files found, 100 file limit)

@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Jul 30, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The shared isaaclab_info_stream handler extraction preserves the Kit path and enables INFO output for kitless Newton/OvPhysX runs. However, the kitless root-level adjustment unintentionally overrides explicitly stricter logging levels such as ERROR or CRITICAL.

  • Design and architecture: Centralizing the idempotent handler installer in logging_utils and retaining AppLauncher._ensure_isaaclab_info_stream_handler as a delegator is a sound way to share logging behavior across Kit and kitless launch paths.
  • API: The existing AppLauncher static method remains available, while the new helper is documented consistently with the neighboring logging utilities. The patch-tier changelog fragment follows the required format.
  • Implementation: The extracted handler retains the existing filter, formatter, and name-based idempotence. In the kitless path, however, level >= logging.WARNING also matches ERROR and CRITICAL, so the subsequent root logger change to INFO defeats a caller's stricter configured level. Limit this adjustment to the intended default WARNING case or scope the INFO level to the isaaclab logger.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

level = resolve_python_logging_level(launcher_args)
apply_python_logging_level(level)
ensure_isaaclab_info_stream_handler()
if level >= logging.WARNING:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Implementation — Root level override also matches ERROR/CRITICAL

level >= logging.WARNING matches ERROR and CRITICAL too. A caller that set the root logger to ERROR gets that level applied by apply_python_logging_level(level) and then silently reset to INFO on the next line, re-enabling isaaclab.* INFO output through the scoped handler and record creation for all loggers. The comment states the intent is only the default WARNING case; use level == logging.WARNING, or raise only the isaaclab logger instead of root.

@mataylor-nvidia

Copy link
Copy Markdown
Author

This shows how logging has changed:

Run 1 — generic renderer=rtx preset

Logs: rtx_preset/

./isaaclab.sh train --rl_library skrl --task Isaac-Cartpole-Camera-Direct renderer=rtx physics=newton_mjwarp
./isaaclab.sh train --rl_library skrl --task Isaac-Cartpole-Camera-Direct renderer=rtx physics=ovphysx
./isaaclab.sh train --rl_library skrl --task Isaac-Cartpole-Camera-Direct renderer=rtx --viz=kit
Log file Command Created new renderer Using renderer
cmd1_rtx_newton_mjwarp.log renderer=rtx physics=newton_mjwarp OVRTXRenderer OVRTXRenderer
cmd2_rtx_ovphysx.log renderer=rtx physics=ovphysx OVRTXRenderer OVRTXRenderer
cmd3_rtx_viz_kit.log renderer=rtx --viz=kit IsaacRtxRenderer IsaacRtxRenderer

Key log lines

cmd1 (renderer=rtx physics=newton_mjwarp — OVRTX kitless + Newton MJWarp):

[INFO]: Created new renderer for simulation: OVRTXRenderer
[INFO]: Using renderer: OVRTXRenderer

cmd2 (renderer=rtx physics=ovphysx — OVRTX kitless + OVPhysX):

[INFO]: Created new renderer for simulation: OVRTXRenderer
[INFO]: Using renderer: OVRTXRenderer

cmd3 (renderer=rtx --viz=kit — Isaac Sim Kit path):

[INFO]: Created new renderer for simulation: IsaacRtxRenderer
[INFO]: Using renderer: IsaacRtxRenderer

rtx_preset.zip

@mataylor-nvidia

Copy link
Copy Markdown
Author

@ndahile-nvidia for review

if level >= logging.WARNING:
# Root logger is at WARNING; raise it to INFO so that isaaclab.* INFO
# records can propagate to the scoped stream handler above.
logging.getLogger().setLevel(logging.INFO)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this make all logging level as info for the process? i.e. we'd end up seeing a lot of prints in the terminal?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated follow-up: Yes. The scoped handler filters what it prints, but setting the root logger to INFO changes effective levels process-wide. I reproduced a stronger two-launch case: the first default call left root at INFO; the second resolved that internal state as the requested level, lowered an existing root handler to INFO, and emitted an unrelated third-party INFO record. The bot is also correct that the current comparison overrides ERROR and CRITICAL. Please elevate only the isaaclab namespace and leave the root level unchanged.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated suggestion: Centralize the policy in one private logging helper called by both AppLauncher and sim_launcher. It should: (1) apply the resolved level to the root logger and leave that root level unchanged; (2) set logging.getLogger("isaaclab") to INFO only when the resolved level is the default WARNING, otherwise use the resolved level; and (3) install or reconfigure one INFO handler filtered to record.name == "isaaclab" or record.name.startswith("isaaclab."). Remove the AppLauncher delegating wrapper. This surfaces Isaac Lab INFO records while Kit and third-party Python INFO records remain hidden by the WARNING root handlers. Carb or native stdout messages are outside Python logging and are unaffected. A focused standard-library test should cover two consecutive default configurations, ERROR/CRITICAL preservation, no duplicate handler, and rejection of a name such as isaaclab_plugin; no simulator integration test is needed.

@AntoineRichard AntoineRichard left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review

Requesting changes.

The review bot's >= logging.WARNING finding is correct, and Kelly's process-wide logging concern is reproducible. With the sequence in this PR, a first default kitless launch resolves WARNING but leaves the root logger at INFO; a second launch then resolves that internally modified INFO level, lowers pre-existing root handlers to INFO, and allows unrelated third-party INFO records through. Explicit ERROR and CRITICAL levels are also reset to INFO. Please keep the INFO enablement scoped to the isaaclab logger namespace and preserve the caller's root level.

The inline comments cover the minimal regression coverage, exact namespace matching, unnecessary API/indirection, and duplicated changelog rationale.

Verification: source/isaaclab_tasks/test/core/test_sim_launcher_visualizer_intent.py passed (3 tests), but its current logging test does not exercise this behavior. ./isaaclab.sh -f passed all hooks. The PR currently conflicts with develop and will also need a rebase.

apply_python_logging_level(resolve_python_logging_level(launcher_args))
level = resolve_python_logging_level(launcher_args)
apply_python_logging_level(level)
ensure_isaaclab_info_stream_handler()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review — Important: The existing test_launch_simulation_kitless_applies_python_logging_level mocks only the resolver and applier. This new real call, together with the root-level branch below, mutates global logging state; the test's fake level 42 enters that branch without asserting or restoring the state. Mock this helper in the plumbing-only test, then add one focused standard-library logging test that covers two default launches, ERROR/CRITICAL, and an unrelated logger. No simulator integration test is needed.

handler.setLevel(level)


def ensure_isaaclab_info_stream_handler() -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review — Moderate: This helper has only internal launcher callers, but its public name and changelog cross-reference turn an implementation detail into API that would later require deprecation. Please make it private, call it directly from both launcher modules, and keep its docstring to the functional contract. The current regardless of what Kit does claim also contradicts the root-level prerequisite documented immediately below it.

handler.addFilter(_IsaacLabInfoFilter())
handler.setFormatter(logging.Formatter("[INFO]: %(message)s"))
root_logger.addHandler(handler)
ensure_isaaclab_info_stream_handler()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review — Moderate: This private one-line delegator no longer owns any behavior and duplicates the shared helper's name. Remove it and invoke the module helper directly at the two _load_extensions call sites; that leaves one implementation and one functional contract to maintain.


class _IsaacLabInfoFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool:
return record.levelno == logging.INFO and record.name.startswith("isaaclab")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review — Moderate: startswith("isaaclab") also accepts unrelated logger names such as isaaclab_plugin, although the contract says the isaaclab.* namespace. Match record.name == "isaaclab" or record.name.startswith("isaaclab."), and cover that boundary in the focused logging test.

* Fixed ``logger.info`` calls in :class:`~isaaclab.renderers.RenderContext` and
:class:`~isaaclab.sensors.camera.Camera` (e.g. "Created new renderer for simulation" and
"Using renderer") being silenced on kitless backends (Newton, OvPhysX).
The ``isaaclab_info_stream`` log handler was only installed inside

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review — Moderate: Keep the changelog user-facing. These implementation details duplicate the PR description and helper docstring, and they advertise a helper that should remain internal. Reduce the entry to the observable outcome, for example: Fixed renderer and camera INFO messages being hidden when using kitless backends.

@AntoineRichard

Copy link
Copy Markdown
Collaborator

AI-generated implementation suggestion

I recommend centralizing the complete logging policy in one private helper. The root logger should remain at the resolved level, while the isaaclab namespace is raised to INFO only for the default WARNING case.

In logging_utils.py:

_ISAACLAB_INFO_HANDLER_NAME = "isaaclab_info_stream"


def _ensure_isaaclab_info_stream_handler() -> None:
    """Install the scoped Isaac Lab INFO handler if needed."""
    root_logger = logging.getLogger()

    for handler in root_logger.handlers:
        if handler.name == _ISAACLAB_INFO_HANDLER_NAME:
            # apply_python_logging_level() may have changed this.
            handler.setLevel(logging.INFO)
            return

    class _IsaacLabInfoFilter(logging.Filter):
        def filter(self, record: logging.LogRecord) -> bool:
            is_isaaclab_logger = record.name == "isaaclab" or record.name.startswith("isaaclab.")
            return record.levelno == logging.INFO and is_isaaclab_logger

    handler = logging.StreamHandler(sys.stdout)
    handler.name = _ISAACLAB_INFO_HANDLER_NAME
    handler.setLevel(logging.INFO)
    handler.addFilter(_IsaacLabInfoFilter())
    handler.setFormatter(logging.Formatter("[INFO]: %(message)s"))
    root_logger.addHandler(handler)


def _configure_python_logging(level: int) -> None:
    """Configure Python logging with default INFO output scoped to Isaac Lab."""
    apply_python_logging_level(level)

    isaaclab_logger = logging.getLogger("isaaclab")
    isaaclab_logger.setLevel(logging.INFO if level == logging.WARNING else level)

    if level <= logging.WARNING:
        _ensure_isaaclab_info_stream_handler()

Then both launch paths can use the same policy.

In AppLauncher._load_extensions():

_configure_python_logging(self._python_logging_level)

In launch_simulation():

if not needs_kit:
    level = resolve_python_logging_level(launcher_args)
    _configure_python_logging(level)

This removes the process-wide logging.getLogger().setLevel(logging.INFO) override, the level >= logging.WARNING branch, the AppLauncher delegating wrapper, and the duplicated configuration sequence.

A focused standard-library test should verify:

  • Default configuration emits isaaclab.* INFO records but not Kit or third-party INFO records.
  • Two consecutive configurations install only one handler and behave identically.
  • ERROR and CRITICAL remain unchanged.
  • A logger such as isaaclab_plugin does not pass the namespace filter.

No simulator integration test is needed. Per the regression-test policy, temporarily restoring the old root-level override should make the new test fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants