Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[flake8]
max-line-length = 88
extend-ignore = E203, W503, Q000, I100, I101, I201, D200, D401
exclude = .venv, build, install, log, .git, __pycache__
112 changes: 112 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Depth Anything 3 ROS2 Wrapper - AI Coding Instructions

## Project Overview

Camera-agnostic ROS2 (Humble) wrapper for ByteDance's Depth Anything 3 monocular depth estimation. Targets real-time performance (>30 FPS) on NVIDIA Jetson Orin AGX.

## Architecture

```
depth_anything_3_ros2/
├── depth_anything_3_node.py # Standard ROS2 node (simpler, flexible)
├── depth_anything_3_node_optimized.py # High-performance node (TensorRT, CUDA streams)
├── da3_inference.py # HuggingFace model wrapper (DepthAnything3.from_pretrained)
├── da3_inference_optimized.py # TensorRT/INT8 optimized inference
├── gpu_utils.py # CUDA-accelerated upsampling (GPUDepthUpsampler)
└── utils.py # Depth normalization, colorization, metrics
```

**Data Flow**: Camera `sensor_msgs/Image` → Node subscribes on `~/image_raw` → `DA3InferenceWrapper.inference()` → Publishes depth on `~/depth`, colored on `~/depth_colored`, confidence on `~/confidence`

## Critical Patterns

### Camera-Agnostic Design (Mandatory)
- **Never** add camera-specific logic to core modules
- Use ROS2 topic remapping for camera integration: `image_topic:=/camera/image_raw`
- Camera configs exist only in `config/camera_configs/*.yaml` and example launch files

### ROS2 Node Structure
```python
# Standard patterns used throughout:
self.declare_parameter('param_name', default_value) # Always declare with defaults
qos = QoSProfile(reliability=ReliabilityPolicy.BEST_EFFORT, ...) # Use BEST_EFFORT for images
self.create_subscription(Image, '~/image_raw', self.callback, qos) # Relative topics with ~
self.create_publisher(Image, '~/depth', 10)
```

### Inference Wrapper Pattern
```python
# da3_inference.py loads model from HuggingFace:
from depth_anything_3.api import DepthAnything3
self._model = DepthAnything3.from_pretrained(model_name)

# Returns dict: {'depth': np.ndarray, 'confidence': np.ndarray}
result = wrapper.inference(rgb_image, return_confidence=True)
```

## Build & Run Commands

```bash
# Build (source ROS2 first)
source /opt/ros/jazzy/setup.bash # or humble
colcon build --packages-select depth_anything_3_ros2 --symlink-install
source install/setup.bash

# Run standard node
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
image_topic:=/camera/image_raw model_name:=depth-anything/DA3-BASE

# Run optimized node (>30 FPS)
ros2 launch depth_anything_3_ros2 depth_anything_3_optimized.launch.py \
backend:=tensorrt_int8 model_input_height:=384

# Run tests
colcon test --packages-select depth_anything_3_ros2
colcon test-result --verbose
```

## Code Style Requirements

- **PEP 8** with 88 char line length (Black)
- **Google-style docstrings** with type hints on all functions
- **No emojis** in code, docs, or commit messages
- Naming: `PascalCase` classes, `snake_case` functions, `_private_methods`

Example:
```python
def process_image(self, image: np.ndarray, normalize: bool = True) -> Dict[str, np.ndarray]:
"""
Process input image and return depth estimation.

Args:
image: Input RGB image as numpy array (H, W, 3)
normalize: Whether to normalize depth output

Returns:
Dictionary containing depth map and confidence
"""
```

## Key Configuration

- **Models**: `depth-anything/DA3-SMALL`, `DA3-BASE`, `DA3-LARGE`, `DA3-GIANT`
- **Parameters** in `config/params.yaml`: `model_name`, `device` (cuda/cpu), `inference_height/width`, `colormap`
- **Performance**: Use 384x384 input with DA3-SMALL + TensorRT INT8 for >30 FPS

## Testing Patterns

Tests use `unittest` with mocked model loading:
```python
@patch('depth_anything_3_ros2.da3_inference.DepthAnything3')
def test_inference(self, mock_da3):
mock_model = MagicMock()
mock_da3.from_pretrained.return_value = mock_model
# Test logic...
```

## Important Files

- `launch/depth_anything_3.launch.py` - Primary launch with all configurable args
- `launch/multi_camera.launch.py` - Multi-camera namespace isolation pattern
- `OPTIMIZATION_GUIDE.md` - TensorRT conversion and performance tuning
- `docker/README.md` - Docker deployment (CPU/GPU/dev modes)
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ models/

# Virtual environments
venv/
.venv/
ENV/
env/

Expand Down
12 changes: 6 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,12 @@ RUN if [ "$BUILD_TYPE" = "cuda-base" ]; then \

# Install other Python dependencies
RUN pip3 install --no-cache-dir \
transformers>=4.35.0 \
huggingface-hub>=0.19.0 \
opencv-python>=4.8.0 \
pillow>=10.0.0 \
numpy>=1.24.0,<2.0 \
timm>=0.9.0
"transformers>=4.35.0" \
"huggingface-hub>=0.19.0" \
"opencv-python>=4.8.0" \
"pillow>=10.0.0" \
"numpy>=1.24.0,<2.0" \
"timm>=0.9.0"

# Install Depth Anything 3
RUN pip3 install --no-cache-dir \
Expand Down
25 changes: 24 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,30 @@ ros2 run depth_anything_3_ros2 depth_anything_3_node --ros-args \

## Docker Deployment

Docker images are provided for easy deployment on both CPU and GPU systems.
Docker configuration files are provided for building and deploying on both CPU and GPU systems.

> **Important**: No pre-built Docker images are published to Docker Hub or any container registry. You must build the images locally using `docker-compose build` or `docker-compose up` (which auto-builds).

### Complete Docker Installation (3 Steps)

```bash
# Step 1: Clone the repository
git clone https://github.com/GerdsenAI/GerdsenAI-Depth-Anything-3-ROS2-Wrapper.git
cd GerdsenAI-Depth-Anything-3-ROS2-Wrapper

# Step 2: Build and run (choose GPU or CPU)
docker-compose up -d depth-anything-3-gpu # For GPU (requires nvidia-docker)
# OR
docker-compose up -d depth-anything-3-cpu # For CPU-only

# Step 3: Enter container and run the node
docker exec -it da3_ros2_gpu bash # For GPU container
# OR
docker exec -it da3_ros2_cpu bash # For CPU container

# Inside the container:
ros2 run depth_anything_3_ros2 depth_anything_3_node --ros-args -p device:=cuda
```

### Quick Start with Docker Compose

Expand Down
2 changes: 1 addition & 1 deletion depth_anything_3_ros2/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
A camera-agnostic ROS2 wrapper for Depth Anything 3 monocular depth estimation.
"""

__version__ = '1.0.0'
__version__ = "1.0.0"
44 changes: 19 additions & 25 deletions depth_anything_3_ros2/da3_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def __init__(
self,
model_name: str = "depth-anything/DA3-BASE",
device: str = "cuda",
cache_dir: Optional[str] = None
cache_dir: Optional[str] = None,
):
"""
Initialize the DA3 inference wrapper.
Expand Down Expand Up @@ -67,26 +67,25 @@ def _setup_device(self, requested_device: str) -> str:
Raises:
ValueError: If requested device is invalid
"""
if requested_device not in ['cuda', 'cpu']:
if requested_device not in ["cuda", "cpu"]:
raise ValueError(
f"Invalid device '{requested_device}'. "
f"Must be 'cuda' or 'cpu'"
f"Invalid device '{requested_device}'. " f"Must be 'cuda' or 'cpu'"
)

# Check CUDA availability
if requested_device == 'cuda':
if requested_device == "cuda":
if not torch.cuda.is_available():
logger.warning(
"CUDA requested but not available. Falling back to CPU. "
"Performance may be degraded."
)
return 'cpu'
return "cpu"
else:
cuda_device = torch.cuda.get_device_name(0)
logger.info(f"Using CUDA device: {cuda_device}")
return 'cuda'
return "cuda"

return 'cpu'
return "cpu"

def _load_model(self) -> None:
"""
Expand All @@ -104,8 +103,7 @@ def _load_model(self) -> None:
# Load model with optional cache directory
if self.cache_dir:
self._model = DepthAnything3.from_pretrained(
self.model_name,
cache_dir=self.cache_dir
self.model_name, cache_dir=self.cache_dir
)
else:
self._model = DepthAnything3.from_pretrained(self.model_name)
Expand All @@ -130,7 +128,7 @@ def inference(
self,
image: np.ndarray,
return_confidence: bool = True,
return_camera_params: bool = False
return_camera_params: bool = False,
) -> Dict[str, np.ndarray]:
"""
Run depth inference on an input image.
Expand Down Expand Up @@ -161,9 +159,7 @@ def inference(
)

if image.dtype != np.uint8:
logger.warning(
f"Expected uint8 image, got {image.dtype}. Converting..."
)
logger.warning(f"Expected uint8 image, got {image.dtype}. Converting...")
image = image.astype(np.uint8)

try:
Expand All @@ -175,22 +171,20 @@ def inference(
prediction = self._model.inference([pil_image])

# Extract results
result = {
'depth': prediction.depth[0].astype(np.float32)
}
result = {"depth": prediction.depth[0].astype(np.float32)}

if return_confidence:
result['confidence'] = prediction.conf[0].astype(np.float32)
result["confidence"] = prediction.conf[0].astype(np.float32)

if return_camera_params:
result['extrinsics'] = prediction.extrinsics[0].astype(np.float32)
result['intrinsics'] = prediction.intrinsics[0].astype(np.float32)
result["extrinsics"] = prediction.extrinsics[0].astype(np.float32)
result["intrinsics"] = prediction.intrinsics[0].astype(np.float32)

return result

except torch.cuda.OutOfMemoryError as e:
# Clear CUDA cache on OOM
if self.device == 'cuda':
if self.device == "cuda":
torch.cuda.empty_cache()
raise RuntimeError(
f"CUDA out of memory during inference. Try reducing image size or "
Expand All @@ -207,16 +201,16 @@ def get_gpu_memory_usage(self) -> Optional[Dict[str, float]]:
Dictionary with 'allocated_mb' and 'reserved_mb' if CUDA is available,
None otherwise
"""
if self.device == 'cuda' and torch.cuda.is_available():
if self.device == "cuda" and torch.cuda.is_available():
return {
'allocated_mb': torch.cuda.memory_allocated() / (1024 ** 2),
'reserved_mb': torch.cuda.memory_reserved() / (1024 ** 2)
"allocated_mb": torch.cuda.memory_allocated() / (1024**2),
"reserved_mb": torch.cuda.memory_reserved() / (1024**2),
}
return None

def clear_cache(self) -> None:
"""Clear CUDA cache to free up memory."""
if self.device == 'cuda' and torch.cuda.is_available():
if self.device == "cuda" and torch.cuda.is_available():
torch.cuda.empty_cache()
logger.info("CUDA cache cleared")

Expand Down
Loading
Loading