CameraRadarFusionNet
A neural network for object detection using multi-level fusion of camera and radar data, built on Keras RetinaNet.
What is CameraRadarFusionNet?
CRF-Net (Camera and Radar Fusion Network) is a deep learning framework for object detection that combines camera images and radar data through multi-level neural network fusion. It solves the problem of unreliable single-sensor perception in autonomous systems by leveraging complementary sensor modalities to improve detection accuracy in challenging environmental conditions. The system is built on the Keras RetinaNet architecture and is specifically designed for autonomous driving applications.
Autonomous vehicle researchers and engineers working on perception systems, particularly those focusing on multimodal sensor fusion for robust object detection. Also suitable for computer vision practitioners interested in extending RetinaNet architectures with additional sensor modalities.
Developers choose CRF-Net because it provides a proven, configurable framework for camera-radar fusion that outperforms single-sensor approaches, with ready integration for the nuScenes dataset and flexible architecture supporting both feature-level and decision-level fusion strategies.
Overview
CRF-Net (Camera and Radar Fusion Network) is a deep learning architecture designed for robust object detection by fusing camera images and radar point clouds. It addresses the limitations of single-sensor systems by combining complementary data sources, improving detection accuracy in challenging conditions like poor visibility or lighting. The network is specifically tested on the nuScenes autonomous driving dataset but can be extended to other multimodal sensor datasets.
Key Features
- Multi-Level Sensor Fusion — Integrates radar and camera data at multiple stages within the neural network architecture for enhanced feature representation.
- RetinaNet Foundation — Builds upon the proven Keras RetinaNet framework for efficient object detection with focal loss handling class imbalance.
- Radar Augmented Images (RAI) — Generates fused visual representations combining camera frames with radar point cloud data for improved input preprocessing.
- Configurable Training Pipeline — Flexible configuration system allows customization of training parameters, model architectures, and dataset specifications.
- Docker Support — Provides containerized environment for reproducible training and evaluation across different systems.
- nuScenes Compatibility — Ready-to-use implementation for the popular autonomous driving dataset with 3D ground truth annotations.
Philosophy
CRF-Net follows a sensor-agnostic fusion philosophy that treats camera and radar data as complementary information streams, leveraging the strengths of each modality to overcome individual sensor limitations in autonomous perception systems.
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
