Enhancing Autonomous Driving By Exploiting Thermal Object Detection Through Feature Fusion

Autonomous driving does not imply self-driving vehicles; self-driving vehicles are the final evolution level of autonomous driving. Autonomous driving aims to add more benefits and features to the vehicle, including driver assistance technologies and automated driving systems. Object detection is a fundamental pillar in the autonomous vehicle industry. Recently, thermal imaging is increasingly gaining an essential role in autonomous driving. The use of a thermal camera in the sensor suite of an automotive vehicle has become more and more popular to increase reliability for robust and safe operation. Thermal imaging can detect the infrared radiation heat emitted by objects. Therefore, a thermal camera can outperform other sensors, such as LiDAR, radar, and RGB, in adverse lighting conditions. In this work, we study object detection in thermal imaging. We propose the Multi-Domain Feature Fusion Thermal Detector (MDFFTDet) framework that fuses the thermal weights with the RGB weights using only thermal images from the FLIR ADAS dataset. The proposed framework inherits the architecture of EfficientDet and further proposes a technique for the feature fusion of two different modalities: thermal and RGB. We use UNIT GAN for the image-to-image translation task to generate pseudo-RGB images from the input thermal images. The proposed framework could tackle the problem of the lack of large datasets in the thermal domain and improve detection accuracy. The experimental results on the FLIR ADAS dataset demonstrate that the proposed framework outperforms the detection accuracy of other state-of-the-art object detectors in thermal imagery, achieving this using much fewer parameters and FLOPS. Two variants of the framework, MDFFTDet-D0 and MDFFTDet-D2, are proposed, achieving mAP of 63.15% and 77.81%, respectively. On the other hand, EfficientDet-D0 and EfficientDet-D2, which are used as a baseline for the proposed frameworks, achieved mAP of 59.4% and 69.12%, respectively.