A novel classification framework using multiple bandwidth method with optimized CNN for brain–computer interfaces with EEG-fNIRS signals

The most effective way to communicate between the brain and electronic devices in the outside world is the brain–computer interface (BCI) systems. BCI systems use signals of being through neural activity in the brain to fulfill this function. Traditional BCI systems use electroencephalography (E.E.G.) signals due to their characteristics, such as temporal resolution, cost, and noninvasive nature. However, the inherent complex features make the analysis process very difficult. In addition, its sensitivity to internal and external noise affects performance negatively. Near-infrared spectroscopy (NIRS), which describes brain hemodynamics, is a noninvasive method and robust against the problems that E.E.G. suffers. We present an effective study examining the effects of E.E.G. and NIRS signals for BCI and investigating the contribution of their combination to performance. Also, a novel classification framework using multiple bandwidth method with optimized convolution neural network (CNN) is proposed. The proposed method classifies the recorded E.E.G. and NIRS signals according to the imagination of opening and closing the subjects' right and left hands. A CNN architecture including fully connected layer optimization using E.E.G. and NIRS signals is trained in an end-to-end manner. Instead of using a single bandwidth as in the literature, multiple bandwidths are used in the training process. In this way, information loss in band filtering tasks is prevented. Performance indicators obtained from experiments performed using the proposed framework are superior to current state-of-the-art methods in the literature in the most significant performance metrics: accuracy and stability. The proposed approach has a higher classification performance than current state-of-the-art methods, with an accuracy performance of 99.85%. On the other hand, in order to test the performance of the proposed CNN method, a detailed ablation study section on single-band experiments and including analysis of each component is presented.