Kinect-Based Outdoor Navigation for the Visually Challenged Using Deep Learning

In this paper, we propose an outdoor navigation system, intended for people with visual impairments. Our system makes use of a Microsoft Kinect which is reconfigured for mobile use with a portable power supply. An object detection model was trained to detect commonly found obstacles on roads, namely cars, pedestrians, bicycles and motorcycles, based on the inputs from the Kinect. In the process, we select an optimal object detection model for an embedded environment by carrying out extensive training, benchmarking and experimentation on three single shot detection models (SSD) with different feature extractors and a RetinaNet model, while also applying quantization techniques to obtain real-time performance with relatively minor losses in performance. The detections from the network are leveraged to calculate the distance between the person and the object detected, using the depth map from the Kinect, and the information is relayed to the user using a text-to-speech system, through Bluetooth earphones paired to the system. The entire setup is constructed on a white cane, where a Raspberry Pi 3B is connected to the Kinect for reading the input frames and performing onboard processing. The results of testing the model in outdoor footage indicate its viability as a tool for outdoor navigation.