Neural Modeling and Real-Time Environment Training of Human Binocular Stereo Visual Tracking

Simulating the human natural visual system is beneficial for understanding brain intelligence and exploiting new aspects of computer vision. Previous studies have proposed many progressive models and experiments for visual tracking; however, only a few consider all factors involved in visual tracking. Improvements in cross-modal sensory fusion, online physical environment training, and leveraging machine learning are required. In this paper, we present a balanced visual tracking study between neuroscience models and deep-learning methods. In our visual tracking framework, we modify the original region proposal network and interconnect binocular R-CNNs with a new region of interest (RoI) model. Ground frame prediction can be implemented by localization fusion from binocular R-CNNs, as well as external sensory information, such as a dense disparity map. In the behavior stage, visual-motor transformation is implemented through the online training of saccades, pursuit, and vergence networks in the real environment. As demonstrated on a robot, our framework can learn tracking skills through online parameter updates using physical data collected from the robot. The framework achieves performance highly similar to human behaviors and better accuracy than recent models. Moreover, using prediction from our ground vision model to guide binocular, RoI pooling can improve the efficiency of object recognition and localization and reduce visual tracking errors by 27% compared with the original network. In conclusion, this study proposed an effective binocular tracking framework that draws inspiration from brain structures. The performance showed improved accuracy and robustness in tracking random moving targets.