ros_deep_learning
Deep learning inference nodes for ROS/ROS2 with support for NVIDIA Jetson devices and TensorRT.
What is ros_deep_learning?
ros_deep_learning is a ROS/ROS2 package that provides deep learning inference nodes for computer vision tasks on NVIDIA Jetson embedded platforms. It solves the problem of integrating real-time AI perception—like object detection and image classification—into robotic systems by offering optimized nodes that leverage TensorRT for performance. The package includes nodes for multiple DNN tasks and supports various camera and streaming inputs.
Robotics engineers and researchers developing perception systems for autonomous robots, drones, or embedded AI applications using ROS/ROS2 on NVIDIA Jetson hardware.
Developers choose ros_deep_learning for its tight integration with the ROS ecosystem, support for both ROS1 and ROS2, and optimized performance on Jetson devices via TensorRT. It provides a ready-to-use solution with pretrained models while allowing customization with user-trained models, reducing the development time for AI-powered robotics.
Overview
Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT
Use Cases
Best For
- Adding real-time object detection to a ROS-based mobile robot
- Implementing semantic segmentation for autonomous drone navigation
- Building a vision system for industrial inspection robots on Jetson
- Streaming and processing video from network cameras (RTSP/RTP) in ROS
- Developing embedded AI applications with custom trained models on Jetson
- Creating a unified perception pipeline supporting multiple DNN tasks in ROS2
Not Ideal For
- Projects targeting non-NVIDIA embedded platforms like Raspberry Pi or Intel NUC
- Teams using robotic frameworks other than ROS/ROS2, such as Autoware or custom middleware
- Applications requiring deep learning models in formats other than Caffe or ONNX, like TensorFlow SavedModel or PyTorch directly
- Developers needing server-side inference on cloud GPUs without Jetson hardware
Pros & Cons
Pros
Leverages TensorRT and the jetson-inference library for high-speed, real-time inference on Jetson devices, as highlighted in the integration with NVIDIA's optimized stack.
Same codebase supports multiple ROS distributions including Melodic, Noetic, Foxy, Galactic, Humble, and Iron, simplifying development across ROS versions as stated in the compatibility section.
Handles diverse inputs like MIPI CSI cameras, V4L2, RTP/RTSP streams, and outputs to displays or files, detailed in the video_source and video_output node parameters.
Containerized setup via Docker scripts automates dependency installation and model mounting, making deployment straightforward per the installation instructions.
Cons
Tightly coupled with Jetson hardware and the jetson-inference library, creating vendor dependency and limiting portability to other platforms like x86 or ARM without NVIDIA GPUs.
Legacy installation requires manual building of jetson-inference and ROS setup, which is error-prone compared to the Docker method, as noted in the detailed but cumbersome legacy instructions.
Only supports Caffe and ONNX models, excluding popular formats like TensorFlow or PyTorch directly, which may necessitate conversion steps for broader model compatibility.
Frequently Asked Questions
Related Projects
detectron2Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
EasyOCRReady-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
imgaugImage augmentation for machine learning experiments.
meshroomNode-based Visual Programming Toolbox
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.