Skip to content

refactor(deviceio): make FrameMetadataTrackerOak single-stream - #875

Merged
jiwenc-nv merged 1 commit into
mainfrom
jiwenc-nv/IT868
Aug 4, 2026
Merged

refactor(deviceio): make FrameMetadataTrackerOak single-stream#875
jiwenc-nv merged 1 commit into
mainfrom
jiwenc-nv/IT868

Conversation

@jiwenc-nv

@jiwenc-nv jiwenc-nv commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Description

FrameMetadataTrackerOak held N SchemaTrackers, one per OAK stream, while every other tracker is 1:1 with a tensor collection — the one thing blocking it from #853's generated template. Narrowed to one stream per instance; create one tracker per stream. Supersedes #874.

The ctor takes a plain collection_id ("oak_camera/Color") rather than the (prefix, stream) in @shaosu-nvidia's comment: that pair is the generated template's exact signature, and every sibling tracker already takes a bare collection_id.

Breaking. get_stream_data/stream_countget_data. Each tracker records under its own MCAP base name to oak/oak_tracked rather than one tracker writing a channel per stream, so McapRecordingConfig takes N entries and readers keyed on <base>/Color need updating. Two channels because the generated live template hardcodes reader indices 0, 1.

Out of scope. No replay impl — OAK never had one and codegen emits it. Manifest migration (last task of #868) is blocked on #853.

Refs #868

Type of change

  • Breaking change (fix or feature that would cause existing functionality to change)

Testing

Built clean with cmake --preset py3.12 on Linux aarch64 (Orin); ctest -LE "gpu|window" passes 233/233 (the 4 failures in a full run are gpu/window-labelled viz tests, untouched here). Smoke-tested the wheel's new API surface.

No new automated tests: the existing OAK tests cover the schema, not the tracker, which needs a live OpenXR runtime plus hardware. examples/oxr/python/test_oak_camera.py --mode schema-pusher is updated but not yet run on an OAK-D.

Checklist

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

OAK metadata tracking now creates one FrameMetadataTrackerOak per camera stream. The tracker accepts a collection ID and exposes get_data() without a stream index. The new oak_collection_id() helper validates stream types and builds collection identifiers for C++ and Python. Live tracking uses one schema reader and tracked-data object. Plugins, examples, recording configuration, and documentation now register and consume per-stream trackers.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • NVIDIA/IsaacTeleop#870: Updates device metadata-tracker architecture and recording integration for Orbbec trackers.
  • NVIDIA/IsaacTeleop#874: Implements the same one-stream-per-FrameMetadataTrackerOak refactor across APIs, bindings, examples, and documentation.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refactoring FrameMetadataTrackerOak to support a single stream.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jiwenc-nv/IT868

Comment @coderabbitai help to get the list of available commands.

FrameMetadataTrackerOak was the last schema-based tracker that did not fit the
generated passthrough template from #853: it held N SchemaTrackers, one per OAK
camera stream, while every other tracker is a 1:1 map between tracker and tensor
collection. Narrow it to one stream per instance and create one tracker per
stream instead, so its facade, base interface and live impl now match the
generated in/ shape exactly, and its constructor matches every sibling tracker's
plain collection_id.

Breaking: the ctor takes a collection id ("oak_camera/Color") instead of a
prefix plus stream list, and get_stream_data/stream_count are gone in favour of
get_data. MCAP layout changes with it -- each tracker records under its own base
name to "oak" and "oak_tracked" rather than one tracker writing a channel per
stream name, so McapRecordingConfig now takes N entries and readers keyed on
"<base>/Color" need updating.

The two recording channels are deliberate: the generated live template writes
every sample to channel 0 and the last sample per update to channel 1, and the
plugin pushes from the camera callback independently of the host update rate, so
samples do accumulate between updates.

Refs #868.

Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
@shaosu-nvidia

Copy link
Copy Markdown
Contributor

Tested with test_oak_camera.py with device and confirmed the mcap is able to be written. Each tracker represents one stream and has both oak and oak_tracked channels written.

@jiwenc-nv
jiwenc-nv merged commit 01f6bc9 into main Aug 4, 2026
42 checks passed
@jiwenc-nv
jiwenc-nv deleted the jiwenc-nv/IT868 branch August 4, 2026 04:35
@jiwenc-nv

Copy link
Copy Markdown
Collaborator Author

@nv-jakob I hope this will unblock you to run the template refactor with the OAK-D stuff as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants