Bug fixes 01302026

.gitignore

@@ -55,3 +55,10 @@ huggingface/
 
 # Logs
 *.log
+
+nul
+
+
+# Claude AI
+.claude/
+CLAUDE.md

CHANGELOG.md

@@ -0,0 +1,120 @@
+# Changelog
+
+## [Unreleased] - 2026-01-31
+
+### TensorRT 10.3 Validation - Phase 1 Complete
+
+- **TensorRT 10.3 Performance Validated**:
+  - Platform: Jetson Orin NX 16GB
+  - Model: DA3-SMALL at 518x518 FP16
+  - Throughput: 35.3 FPS
+  - GPU Latency: 26.4ms median (25.5ms min)
+  - Engine Size: 58MB
+  - Speedup: 6.8x over PyTorch baseline (~5.2 FPS)
+  - Test Date: 2026-01-31
+  - Validation: Host script `scripts/test_trt10.3_host.sh`
+
+### Critical Findings (Resolved)
+
+- **TensorRT 8.6 Fundamentally Incompatible with DA3**:
+  - Root cause: DINOv2 backbone exports Einsum operations unsupported by TRT 8.6
+  - Error: "caskConvolutionV2Forward could not find any supported formats"
+  - NVIDIA GitHub Issue #4537 confirms DINOv2 failures persist until TRT 10.8+
+  - Workarounds (opset 17 re-export, graph surgery) not viable
+  - **Solution Validated:** Docker base image L4T r36.4.0 (TensorRT 10.3)
+  - Full analysis: `docs/TENSORRT_DA3_PLAN.md`
+
+### Docker Base Image Update
+
+- **Updated Jetson base image to L4T r36.4.0**:
+  - Previous: `dustynv/ros:humble-ros-base-l4t-r36.2.0` (TensorRT 8.6.2)
+  - Current: `dustynv/ros:humble-pytorch-l4t-r36.4.0` (TensorRT 10.3)
+  - Benefits: Full DINOv2/ViT support, validated 6.8x speedup
+  - TRT 10.x syntax: `--memPoolSize=workspace:2048MiB`
+  - ONNX 5D input: `pixel_values:1x1x3x518x518`
+
+### Performance Baseline
+
+- **Measured on Jetson Orin NX 16GB** (JetPack 6.0, L4T r36.2.0, CUDA 12.2):
+  - Model: DA3-SMALL (PyTorch, FP32)
+  - Resolution: 518x518
+  - Inference Time: ~193ms per frame
+  - FPS: ~5.2 FPS
+
+### Added
+
+- **Jetson Hardware Detection** (`depth_anything_3_ros2/jetson_detector.py`):
+  - Platform detection for Jetson modules (Orin AGX, Orin NX, Orin Nano, Xavier AGX, Xavier NX)
+  - GPU memory and VRAM detection
+  - JetPack and L4T version detection
+  - CUDA availability checking
+
+- **Model Setup System**:
+  - `config/model_catalog.yaml`: Model catalog with VRAM requirements and platform recommendations
+  - `scripts/setup_models.py`: Interactive model selection and download script
+  - Hardware-aware model recommendations based on detected platform
+
+- **Dockerfile - Jetson Support**:
+  - `L4T_VERSION=r36.2.0` build argument
+  - New `jetson-base` stage using `dustynv/ros:humble-ros-base-l4t-${L4T_VERSION}`
+  - OpenCV 4.8.1 version verification check
+  - cv_bridge and image_geometry built from source (resolves OpenCV 4.8.1 vs 4.5.4 conflict)
+  - `-DBUILD_TESTING=OFF` flag for cv_bridge build (avoids ament_lint_auto dependency)
+  - PyTorch 2.3.0 via direct NVIDIA wheel download (CUDA 12.2 support for L4T r36.2.0)
+  - torchvision 0.18.0 compatible with PyTorch 2.3.0
+  - OpenMPI and OpenBLAS runtime dependencies for PyTorch on ARM64
+  - Depth Anything 3 installation with `--no-deps` flag (avoids pycolmap/open3d ARM64 build failures)
+  - Windows line ending fix (`sed -i 's/\r$//'`) for ros_entrypoint.sh
+  - Model download at build time via `DOWNLOAD_MODELS_AT_BUILD` and `INSTALL_MODELS` args
+
+- **docker-compose.yml**:
+  - New `depth-anything-3-jetson` service with `BUILD_TYPE: jetson-base`
+  - Enhanced GPU access configuration with nvidia-container-runtime
+
+- **Tests**:
+  - `test/__init__.py`: Package marker for test directory
+  - `test/conftest.py`: Shared fixtures and ROS2 availability detection
+  - `test/test_jetson_detector.py`: Unit tests for Jetson hardware detection
+  - Updated `test/test_node.py` with conditional imports and skipif decorators
+  - Updated `test/test_generic_camera.py` with graceful ROS2 module handling
+
+- **.gitignore**:
+  - Added `CLAUDE.md`, `nul`, `.claude/settings.local.json`
+  - Line ending rules (`eol=lf`) for shell scripts
+
+### Fixed
+
+- **PyTorch CUDA Support on Jetson**:
+  - Changed from cu126 index to direct NVIDIA wheel (cu122 for L4T r36.2.0)
+  - Added libopenmpi3 and libopenblas0 to both builder and runtime stages
+  - `torch.cuda.is_available()` now returns `True` in container
+
+- **cv_bridge OpenCV Conflict**:
+  - ROS Humble apt packages expect OpenCV 4.5.4
+  - dustynv base image ships with OpenCV 4.8.1 (CUDA-enabled)
+  - Solution: Build cv_bridge from source against existing OpenCV 4.8.1
+
+- **ARM64 Python Package Dependencies**:
+  - pycolmap and open3d lack ARM64 wheels
+  - Solution: Install Depth Anything 3 with `--no-deps`, manually install required inference dependencies
+  - Runtime patch: api.py patched at container startup to handle missing pycolmap/evo imports
+
+- **torchvision Source Build Required**:
+  - NVIDIA PyTorch wheel has ABI mismatch with pip torchvision
+  - NMS operator crashes at runtime with pip-installed torchvision
+  - Solution: Build torchvision 0.18.0 from source against NVIDIA PyTorch wheel
+
+- **Windows CRLF Line Endings**:
+  - ros_entrypoint.sh fails with CRLF line endings on Windows-cloned repos
+  - Solution: Added `sed -i 's/\r$//'` in Dockerfile for entrypoint scripts
+
+- **Test Collection Failures (Issue #21)**:
+  - Tests failed when ROS2 environment not sourced
+  - Solution: Added ROS2 availability detection and pytest skipif decorators
+
+### Changed
+
+- **Dockerfile**:
+  - Base image for Jetson changed from `nvcr.io/nvidia/l4t-ros` to `dustynv/ros` (no NGC auth required)
+  - PyTorch installation method changed from pip index to direct wheel download
+  - cv_bridge installation changed from apt to source build

CLAUDE.md

@@ -0,0 +1,144 @@
+# CLAUDE.md
+
+## Always offer to pull down issues with Github CLI, address issues before beginning
+
+## DO NOT MAKE ANY COMMITS, USER WILL MAKE COMMITS
+
+## Always see if there are specialized agents to helps with tasks and troubleshooting, orchestrate agents to work together
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Build & Development Commands
+
+```bash
+# Build the package
+colcon build --packages-select depth_anything_3_ros2
+
+# Run tests
+colcon test --packages-select depth_anything_3_ros2
+colcon test-result --verbose
+
+# Run a single test file
+python3 -m pytest test/test_inference.py -v
+
+# Lint and format
+flake8 depth_anything_3_ros2/
+black --check depth_anything_3_ros2/
+black depth_anything_3_ros2/  # auto-format
+
+# Launch with USB camera
+ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py image_topic:=/camera/image_raw
+
+# Docker (GPU)
+docker-compose up -d depth-anything-3-gpu
+
+# Deploy to Jetson (via SCP)
+scp -r . gerdsenai@10.69.7.112:~/depth_anything_3_ros2/
+```
+
+## Architecture
+
+This is a ROS2 Humble wrapper for ByteDance's Depth Anything 3 monocular depth estimation, targeting >30 FPS on NVIDIA Jetson Orin AGX.
+
+### 3-Layer Design
+- **Node Layer** (`depth_anything_3_node.py`, `*_optimized.py`): ROS2 interface, parameter handling, topic management
+- **Inference Layer** (`da3_inference.py`, `*_optimized.py`): Model loading via HuggingFace, CUDA/CPU inference
+- **Utility Layer** (`utils.py`, `gpu_utils.py`): Depth processing, colorization, GPU acceleration
+
+### Dual Implementation Pattern
+- Standard nodes: Baseline functionality
+- Optimized nodes (`*_optimized.py`): TensorRT, async processing, >30 FPS target
+- Both expose identical ROS2 interfaces - changes to one should be reflected in the other
+
+### Inference Wrapper Return Format
+```python
+{'depth': np.ndarray,  # (H, W) float32
+ 'confidence': np.ndarray,  # (H, W) float32, optional
+ 'camera_params': dict}  # optional
+```
+
+## Critical Design Principles
+
+### Camera-Agnostic Design (Non-Negotiable)
+- NEVER add camera-specific logic to core modules
+- Camera integration ONLY via topic remapping and example launch files in `launch/examples/`
+- All cameras work through standard `sensor_msgs/Image` interface
+
+### ROS2 Patterns
+- Use relative topic names with `~` prefix (e.g., `~/depth`, `~/image_raw`)
+- BEST_EFFORT QoS for image subscribers (allows frame drops)
+- Declare all parameters in node constructor
+
+## Coding Standards
+
+- **No emojis** - Forbidden in code, comments, docstrings, logs, and commits
+- **Line length**: 88 characters (Black formatter)
+- **Docstrings**: Google-style with type hints on all functions
+- **Naming**: `PascalCase` classes, `snake_case` functions, `_private_methods`, `UPPER_SNAKE_CASE` constants
+
+## Testing
+
+Tests use mocked DA3 model (doesn't require GPU):
+- `test/test_inference.py` - Unit tests for inference wrapper
+- `test/test_node.py` - Integration tests for ROS2 node
+- `test/test_generic_camera.py` - Camera-agnostic functionality
+
+## Key Files
+
+- `package.xml`, `setup.py` - ROS2 ament_python package config
+- `launch/depth_anything_3.launch.py` - Main launch file with 13 configurable arguments
+- `config/params.yaml` - Default parameters
+- `.github/copilot-instructions.md` - Extended AI coding guidelines
+
+## Specialized Agents
+
+This repository includes specialized agents in `.claude/agents/`. Use them proactively for domain-specific tasks.
+
+### Available Agents
+
+| Agent | Domain | Use When |
+|-------|--------|----------|
+| `jetson-expert` | Hardware | Module selection, flashing, BSP, carrier boards, GPIO/CSI, thermal, boot issues |
+| `nvidia-expert` | Software | CUDA, TensorRT, DeepStream, Isaac ROS, containers, profiling, PyTorch/TensorFlow |
+
+### Agent Selection Guide
+
+**Hardware questions** -> `jetson-expert`:
+- "Which Jetson module should I use?"
+- "How do I flash JetPack 6.x?"
+- "Camera not detected on CSI port"
+- "Thermal throttling issues"
+- "Carrier board GPIO configuration"
+- "Boot hangs after flashing"
+- "Device tree or pinmux setup"
+
+**Software questions** -> `nvidia-expert`:
+- "How do I convert ONNX to TensorRT?"
+- "Optimize inference performance"
+- "DeepStream pipeline design"
+- "Isaac ROS node optimization"
+- "CUDA memory management"
+- "Container can't access GPU"
+- "INT8 calibration for TensorRT"
+
+### Multi-Agent Scenarios
+
+Some issues require both agents working together:
+
+| Scenario | Primary Agent | Secondary Agent | Reason |
+|----------|---------------|-----------------|--------|
+| Slow inference on Orin NX | `nvidia-expert` | `jetson-expert` | Software first, then check thermal/power |
+| Container can't access GPU | `nvidia-expert` | `jetson-expert` | Runtime config first, then driver/L4T check |
+| CSI camera not detected | `jetson-expert` | - | Hardware/device tree issue |
+| TensorRT build fails | `nvidia-expert` | - | Software/model issue |
+| JetPack 6.x upgrade | `jetson-expert` | `nvidia-expert` | Flash first, then container compatibility |
+| Performance varies wildly | `nvidia-expert` | `jetson-expert` | Profile first, then check thermal throttling |
+
+### Proactive Agent Usage
+
+ALWAYS consider using specialized agents when:
+1. User mentions Jetson hardware or deployment -> Consider `jetson-expert`
+2. User asks about AI/ML optimization -> Consider `nvidia-expert`
+3. Troubleshooting involves both HW and SW -> Use both agents sequentially
+4. Task is outside ROS2/Python expertise -> Use appropriate agent
+5. Performance issues arise -> Start with `nvidia-expert`, escalate to `jetson-expert` if thermal