A neural network for object detection using multi-level fusion of camera and radar data, built on Keras RetinaNet.
CRF-Net (Camera and Radar Fusion Network) is a deep learning framework for object detection that combines camera images and radar data through multi-level neural network fusion. It solves the problem of unreliable single-sensor perception in autonomous systems by leveraging complementary sensor modalities to improve detection accuracy in challenging environmental conditions. The system is built on the Keras RetinaNet architecture and is specifically designed for autonomous driving applications.
Autonomous vehicle researchers and engineers working on perception systems, particularly those focusing on multimodal sensor fusion for robust object detection. Also suitable for computer vision practitioners interested in extending RetinaNet architectures with additional sensor modalities.
Developers choose CRF-Net because it provides a proven, configurable framework for camera-radar fusion that outperforms single-sensor approaches, with ready integration for the nuScenes dataset and flexible architecture supporting both feature-level and decision-level fusion strategies.
CRF-Net (Camera and Radar Fusion Network) is a deep learning architecture designed for robust object detection by fusing camera images and radar point clouds. It addresses the limitations of single-sensor systems by combining complementary data sources, improving detection accuracy in challenging conditions like poor visibility or lighting. The network is specifically tested on the nuScenes autonomous driving dataset but can be extended to other multimodal sensor datasets.
CRF-Net follows a sensor-agnostic fusion philosophy that treats camera and radar data as complementary information streams, leveraging the strengths of each modality to overcome individual sensor limitations in autonomous perception systems.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Integrates camera and radar data at multiple neural network stages, enhancing detection accuracy in challenging conditions like poor visibility, as demonstrated in the nuScenes benchmark.
Builds on the proven Keras RetinaNet framework, leveraging focal loss for class imbalance and providing a reliable object detection backbone.
Offers a containerized environment with Docker support, ensuring consistent training and evaluation across different systems, as detailed in the installation guide.
Includes pre-configured compatibility with the nuScenes autonomous driving dataset, complete with 3D ground truth annotations and pretrained weights for quick start.
Requires specific, older versions like CUDA 10.0 and Python 3.5, which may conflict with modern deep learning libraries and hardware, limiting upgrade paths.
Primarily optimized for nuScenes; extending to other camera-radar datasets requires significant manual configuration and code adaptation, as noted in the README.
Involves error-prone steps like creating custom config files and managing environment variables, with no automated tools or simplified workflows for beginners.