Tensor rt testing#24
Merged
Merged
Conversation
Update TODO.md to mark Phase 2 as COMPLETE: replace the planned files list with a completed implementation checklist (host TRT service, SharedMemoryInference, and host-aware deploy script), revise the IPC protocol filenames/status format, add a Jetson deployment snippet, and bump the last-updated date. Also add .DS_Store files in the repository root and examples/ (macOS metadata).
Add auto-install for huggingface_hub and onnx dependencies in deploy script
Fix PATH for huggingface-cli - add ~/.local/bin
- Replace huggingface-cli with huggingface_hub.snapshot_download() - More reliable: CLI binary not always in PATH after pip install - Auto-install huggingface_hub and onnx if missing - Tested on fresh Jetson deployment
- Add JETSON_BENCHMARKS.md with full performance documentation - Add benchmark_models.sh script for model size testing - Update TODO.md with Phase 3 complete: - Resolution benchmarks: 40-110 FPS (518-256px) - Model benchmarks: Small (40 FPS), Base (19 FPS), Large (7.5 FPS) - Recommend DA3-Small @ 308-400px for real-time robotics
- 10-minute sustained load test: PASSED - Throughput: 40.79 FPS (stable throughout) - Latency: 24.73ms mean, 25.19ms p99 - No thermal throttling detected - Add thermal_stability_test.sh script - Update benchmark documentation
Refresh Jetson/TensorRT documentation and deployment scripts to reflect new validation results and tooling changes. Key changes: - README: update TensorRT performance claims (≈40 FPS @ 518x518, 93 FPS @ 308x308), speeds (7.7x / up to 17.8x), latency and benchmark references. - docs/BASELINES.md: major rewrite with validated TensorRT FP16 tables, model/size throughput, thermal stability results and recommended configurations; update JetPack/CUDA versions and PyTorch baseline. - docs/JETSON_DEPLOYMENT_GUIDE.md: adjust JetPack/CUDA versions, engine size (58 → 64 MB), performance numbers, host-container diagram formatting, shared-memory protocol (request/status), add model/benchmark/thermal script references, update validation date. - Remove obsolete planning docs: docs/TENSORRT_DA3_PLAN.md and docs/TENSORRT_OPTIMIZATION_PLAN.md (deleted). - scripts/demo.sh & scripts/deploy_jetson.sh: add auto-install and dependency checks (huggingface_hub, onnx, numpy, pycuda), improved ONNX download/embedding flow, better fallbacks to PyTorch when TRT unavailable, update displayed backend performance text and expected FPS. These changes align docs and scripts with recently validated Jetson results, improve robustness of automated deployment (dependency handling and clearer fallbacks), and remove outdated planning artifacts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request updates all documentation and benchmarking results to reflect the latest, significantly improved TensorRT performance for Depth Anything 3 (DA3) on Jetson Orin NX 16GB. The new results show much higher throughput, lower latency, and improved speedup over PyTorch, with comprehensive new benchmarks and validation. The PR also marks the completion of several project phases and adds a detailed benchmarking document.
Key changes:
Major Performance Updates
Documentation and Benchmarking
docs/JETSON_BENCHMARKS.mdfile with comprehensive performance data, deployment recommendations, methodology, and thermal/stability validation.README.md,TODO.md, anddocs/BASELINES.mdto reflect new results, phase completions, and best-practice recommendations for real-time robotics and quality-focused applications. [1] [2] [3]Technical and Platform Details
Stability and Validation
Miscellaneous
These updates provide clear, accurate, and actionable performance data for users, and establish robust documentation for future development and deployment.## Description
Please include a summary of the changes and the related issue. Include relevant motivation and context.
Fixes # (issue)
Type of Change
Please delete options that are not relevant.
Testing
Please describe the tests you ran to verify your changes. Provide instructions so we can reproduce.
Test Configuration:
Checklist
Camera-Agnostic Design
Performance Impact
Screenshots (if applicable)
Add screenshots to help explain your changes.
Additional Notes
Add any other context about the pull request here.