pyroSAR

CameraRadarFusionNet

CRF-Net (Camera and Radar Fusion Network) is a deep learning architecture designed for robust object detection by fusing camera images and radar point clouds. It addresses the limitations of single-sensor systems by combining complementary data sources, improving detection accuracy in challenging conditions like poor visibility or lighting. The network is specifically tested on the nuScenes autonomous driving dataset but can be extended to other multimodal sensor datasets. ## Key Features - **Multi-Level Sensor Fusion** — Integrates radar and camera data at multiple stages within the neural network architecture for enhanced feature representation. - **RetinaNet Foundation** — Builds upon the proven Keras RetinaNet framework for efficient object detection with focal loss handling class imbalance. - **Radar Augmented Images (RAI)** — Generates fused visual representations combining camera frames with radar point cloud data for improved input preprocessing. - **Configurable Training Pipeline** — Flexible configuration system allows customization of training parameters, model architectures, and dataset specifications. - **Docker Support** — Provides containerized environment for reproducible training and evaluation across different systems. - **nuScenes Compatibility** — Ready-to-use implementation for the popular autonomous driving dataset with 3D ground truth annotations. ## Philosophy CRF-Net follows a sensor-agnostic fusion philosophy that treats camera and radar data as complementary information streams, leveraging the strengths of each modality to overcome individual sensor limitations in autonomous perception systems.