Brain Informatics

Speech is an easy and useful way to detect speakers’ mental and psychological health, and automatic emotion recognition in speech has been investigated widely in the fields of human-machine interaction, psychology, psychiatry, etc. In this paper, we extract prosodic and spectral features including pitch, MFCC, intensity, ZCR and LSP to establish the emotion recognition model with SVM classifier. In particular, we find different frame duration and overlap have different influences on final results. So, Depth-First-Search method is applied to find the best parameters. Experimental results on two known databases, EMODB and RAVDESS, show that this model works well, and our speech features are enough effectively in characterizing and recognizing emotions.

Inspired by elements from neuroscience and psychological literature, a computational model of forming and changing of behaviours is presented which can be used as the basis of a human-aware assistance system. The presented computational model simulates the dynamics of mental states of a human during formation and change of behaviour. The application domain focuses on sustainable behavior.

Desire regulation can make use of different regulation strategies; this implies an underlying decision making process, which makes use of some form of self-monitoring. The aim of this work is to develop a neurologically inspired computational cognitive model of desire regulation and these underlying self-monitoring and decision making processes. In this model four desire regulation strategies have been incorporated. Simulation experiments have been performed based for the domain of food choice.

In this paper, we present a novel idea where we analyzed EEG signals to classify what type of video a person is watching which we believe is the first step of a BCI based video recommender system. For this, we setup an experiment where 13 subjects were shown three different types of videos. To be able to classify each of these videos from the EEG data of the subjects with a very good classification accuracy, we carried out experiments with several state-of-the-art algorithms for each of the submodules (pre-processing, feature extraction, feature selection and classification) of the Signal Processing module of a BCI system in order to find out what combination of algorithms best predicts what type of video a person is watching. We found, the best results (80.0% with 32.32 ms average total execution time per subject) are obtained when data of channel AF8 are used (i.e. data recorded from the electrode located at the right frontal lobe of the brain). The combination of algorithms that achieved this highest average accuracy of 80.0% are FIR Least Squares, Welch Spectrum, Principal Component Analysis and Adaboost for the submodules pre-processing, feature extraction, feature selection and classification respectively.

Music can convey and evoke powerful emotions. But it is very challenging to recognize the music emotions accurately by computational models. The difficulty of the problem can exponentially increase when the music segments delivery multiple and complex emotions. This paper proposes a novel quantum convolutional neural network (QCNN) to learn music emotions. Inheriting the distinguished abstraction ability from deep learning, QCNN automatically extracts the music features that benefit emotion classification. The main contribution of this paper is that we utilize measurement postulate to simulate the human emotion awareness in music appreciation. Statistical experiments on the standard dataset shows that QCNN outperforms the classical algorithms as well as the state-of-the-art in the task of music emotion classification. Moreover, we provide demonstration experiment to explain the good performance of the proposed technique from the perspective of physics and psychology.

It is of great significance to infer activation extents under different cognitive tasks in neuroscience research as well as clinical applications. However, the EEG electrodes measure electrical potentials on the scalp instead of directly measuring activities of brain sources. To infer the activated cortex sources given the EEG data, many approaches were proposed with different neurophysiological assumptions. Traditionally, the EEG inverse problem was solved in an unsupervised way without any utilization of the brain status label information. We propose that by leveraging label information, the task related discriminative extended source patches can be much better retrieved from strong spontaneous background signals. In particular, to find task related source extents, a novel supervised EEG source imaging model called Graph regularized Variation-Based Sparse Cortical Current Density (GVB-SCCD) was proposed to explicitly extract the discriminative source extents by embedding the label information into the graph regularization term. The graph regularization was derived from the constraint that requires consistency for all the solutions on different time points within the same class. An optimization algorithm based on the alternating direction method of multipliers (ADMM) is derived to solve the GVB-SCCD model. Numerical results show the effectiveness of our proposed framework.

The paper presents a novel artificial bee colony clustering (ABCC) algorithm with a self-adaptive multidimensional search mechanism based on difference bias for insula functional parcellation, called as DABCC. In the new algorithm, the preprocessed functional magnetic resonance imaging (fMRI) data was mapped into a low-dimension space by spectral mapping to reduce its dimension in the initialization. Then, clustering centers in the space were searched by the search procedure composed of employed bee search, onlooker bee search and scout bee search, where a self-adaptive multidimensional search mechanism based on difference bias for employed bee search was developed to improve search capability of ABCC. Finally, the experiments on fMRI data demonstrate that DABCC not only has stronger search ability, but can produce better parcellation structures in terms of functional consistency and regional continuity.

Electroencephalograph (EEG) signals reveal much of our brain states and have been widely used in emotion recognition. However, the recognition accuracy is hardly ideal mainly due to the following reasons: (i) the features extracted from EEG signals may not solely reflect one’s emotional patterns and their quality is easily affected by noise; and (ii) increasing feature dimension may enhance the recognition accuracy, but it often requires extra computation time. In this paper, we propose a feature smoothing method to alleviate the aforementioned problems. Specifically, we extract six statistical features from raw EEG signals and apply a simple yet cost-effective feature smoothing method to improve the recognition accuracy. The experimental results on the well-known DEAP dataset demonstrate the effectiveness of our approach. Comparing to other studies on the same dataset, ours achieves the shortest feature processing time and the highest classification accuracy on emotion recognition in the valence-arousal quadrant space.

Even though how bilinguals process the second language (L2) still remain disputable, it is agreed that L2 processing involve more brain areas and activate common regions more strongly. It interested us to probe why heavier manipulation of cortical regions did not guarantee a high language proficiency. Since the responses of individual brain regions were inadequate to explain how the brain enabled behavior, we sought to explore this question at the neural network prospect via the Psychophysiological interaction (PPI) analysis. We found that Chinese English bilinguals adopted the assimilation/accommodation strategy to read L2, and English activated common brain areas more strongly. However, the whole brain voxel-wise analysis of effective connectivity showed that these brain areas formed a less synchronized network, which may indicate an ineffective neural network of L2. Our findings provided a possible explanation why the proficiency level of L2 was always lower than L1, and suggested that future fMRI studies may better explore language issues by depicting functional connectivity efficacy.

Brain imaging studies in humans have reported each object category was associated with different neural response pattern reflecting visual, structure or semantic attributes of visual appearance, and the representation of an object is distributed across a broader expanse of cortex rather than a specific region. These findings suggest the feature-map model of object representation. The present object categorization study provided another evidence for feature-map representation of objects. Linear Support Vector Machine (SVM) was used to analyze the functional magnetic resonance imaging (fMRI) data when subjects viewed four representative categories of objects (house, face, car and cat) to investigate the representation of different categories of objects in human brain. We designed 6 linear SVM classifiers to discriminate one category from the other one (1 vs. 1), 12 linear SVM classifiers to discriminate one category from other two categories (1 vs. 2), 3 linear SVM classifiers to discriminate two categories of objects from the other two categories (2 vs. 2). Results showed that objects with visually similar features have lower classification accuracy under all conditions, which may provide new evidences for the feature-map representation of different categories of objects in human brain.

Previous studies have extensively reported an advantage of females over males in facial expression recognition. However, few studies have concerned the gender role differences. In this study, gender role differences on facial recognition were investigated by reaction time and the early event-related potentials (ERPs), N170 and Vertex Positive Potential (VPP). A total of 466 female college students were investigated by gender role inventory, and 34 of them were chosen as subjects, with equal numbers in masculinity and femininity. Subjects were asked to discriminate fearful, happy and neutral expressions explicitly in two emotional states: neutral and fearful. First, N170 and VPP showed greater activity in femininities than in masculinities. Second, subjects showed a predominance of negative face processing, as VPP was more positive in response to fearful expressions than neutral and happy expressions, but no gender role difference was found. Third, in fearful state, the reaction time was shorter, especially for fear expression, and N170 showed enhanced negativity, suggesting that fearful state could promote individuals to recognize expressions, and there was no gender role difference. In conclusion, gender role differences exist in the early stage of facial expressions recognition and femininities are more sensitive than masculinities. Our ERP results provide neuroscience evidence for differences in the early components of facial expression cognition process between the two gender roles of females.

Electroencephalogram (EEG) signals are unique neurons’ electrical activity representation, which can support biometric recognition. This paper investigates the potential to identify an individual using brain signals and highlight the challenges of using EEG as a biometric modality in a recognition system. The understanding of designing an effective acquisition protocol is essential to the performance of the EEG-based biometric system. Different acquisition protocols of EEG based recognition i.e. relaxation, motor and non-motor imaginary, and evoked potentials were presented and discussed. Universality, permanence, uniqueness, and collectability are suggested as key requirements for constructing a viable biometric recognition system. Lastly, a summary of recent EEG biometrics studies was depicted before concluding on the findings. It is observed that both motor and non-motor imagery and event-related potential (ERP) outperformed the method of relaxation in acquisition protocol.

Sleep episodes are generally classified according to EEG, EMG, ECG, EOG and other signals. Many experts at home and abroad put forward many automatic sleep staging classification methods, however the accuracy of most methods still remain to be improved. This paper firstly improves the initial center of clustering by combining the correlation coefficient and the correlation distance and uses the idea of piecewise function to update the clustering center. Based on the improvement of K-means clustering algorithm, an automatic sleep stage classification algorithm is proposed and is adopted after the wavelet denoising, EEG data feature extraction and spectrum analysis. The experimental results show that the classification accuracy is improved and the sleep automatic staging algorithm is effective by comparison between the experimental results with the artificial markers and the original algorithms.

In order to improve speaker verification accuracy in the complex environment, a two-layer Gaussian mixture model-universal background model (GMM-UBM) model based on speaker verification method is proposed. For different layer, a GMM-UBM model was trained by different combination of speech features. The voice data of 3 days (36 h) were recorded from the complex environment, and the collected data was manually segmented into four classes: quiet, noise, target speaker and other speaker. Not only the segment data can be used to train GMM-UBM model, but also it can provide a criterion to assess the effectiveness of the model. The results show that the highest recall for the second and third day were 0.75 and 0.74 respectively, and the corresponding specificity were 0.29 and 0.19, which indicates the proposed GMM-UBM model is viable to verify the target speaker in the complex environment.

Recently there has attracted wide attention in EEG-based emotion recognition (ER), which is one of the utilization of Brain Computer Interface (BCI). However, due to the ambiguity of human emotions and the complexity of EEG signals, the EEG-ER system which can recognize emotions with high accuracy is not easy to achieve. In this paper, by combining discrete wavelet transform, correlation analysis, and neural network methods, we propose an Emotional Recognition model based on rhythm synchronization patterns to distinguish the emotional stimulus responses to different emotional audio and video. In this model, the entire scalp conductance signal is analyzed from a joint time-frequency-space correlation, which is beneficial to the depth learning and expression of affective pattern, and then improve the accuracy of recognition. The accuracy of the proposed multi-layer EEG-ER system is compared with various feature extraction methods. For analysis results, average and maximum classification rates of 64% and 67.0% were obtained for arousal and 66.6% and 76.0% for valence.

Patients with major depressive disorder (MDD) show an impaired ability to modulate emotional states and deficits in processing emotional information. Many studies in healthy individuals showed a major involvement of the medial prefrontal cortex (MPFC) in the modulation of emotional processing. Recently, we used emotional expected pictures with high or low salience in a functional MRI (fMRI) paradigm to study altered modulation of MPFC in MDD patients and to explore the neural correlates of pathological cognitive bias, and investigated the effect of symptom severity on this functional impairment. Data were obtained from 18 healthy subjects and 18 MDD patients, diagnosed according to the ICD-10 criteria. Subjects lay in a 3 T Siemens Trio scanner while viewing emotional pictures, which were randomly preceded by an expectancy cue in 50% of the cases. Our study showed a lower effect of salience on dorsal MPFC (DMPFC) activation during anticipatory preparation in depressed subjects. Differential effects for high salient versus low salient pictures viewing were also found in DMPFC in depressed and healthy subjects, and this effect was significantly higher for depressed subjects and correlated positively with Hamilton rating scale for depression (HAMD) scores. Comparing the anticipation effect on subsequent picture viewing between high and low salient pictures, differential effects were higher for depressed subjects during high salient picture viewing and lower during low salient picture viewing. Therefore, we could shed light on DMPFC functioning in depressive disorder by separating cognitive and consumatory effects in this region.

The prevalence rate of ADHD varies from age to age. To better understand the development of ADHD from childhood to adolescence, different age groups of ADHD from large dataset are needed to explore the development pattern of brain activities. In this study, amplitude of low frequency fluctuation (ALFF), fractional amplitude of low frequency fluctuation (fALFF) and regional homogeneity (ReHo) were extracted from resting-state functional magnetic resonance imaging (rs-fMRI) of both ADHD subjects and typical developing (TD) subjects from 7 to 16 years old. The result showed that the different areas mainly appear at the bilateral superior frontal cortex, anterior cingulate cortex (ACC), precentral gyrus, right superior occipital lobe, cerebellum and parts of basal ganglia between all ADHD subjects and all TD subjects. Besides, compared with TD, there were different brain activity patterns at different ages in ADHD, which appear at the left ACC and left occipital lobe. The result can inspire more studies on comparisons between functional connectivity methods.

It has been reported that depression can be detected by electrophysiological signals. However, few studies investigate how to daily monitor patient’s electrophysiological signals through a more convenient way for a doctor, especially on the monitoring of electroencephalogram (EEG) signals for depression diagnosis. Since a person’s mental state and physiological state are changing over time, the most insured diagnosis of depression requires doctors to collect and analyze subject’s EEG signals every day until two weeks for the clinical practice. In this work, we designed a real-time depression monitoring system to capture the user’s EEG data by a wearable device and to perform real-time signal filtering, artifacts removal and power spectrum visualization, which could be combined with psychological test scales as an auxiliary diagnosis. In addition to collecting the resting EEG signals for real-time analysis or diagnosis of depression, we also introduced an external audio stimulus paradigm to further make a detection of depression. Through the machine learning method, system can give a credible probability of depression under each stimulus as a user’s self-rating score from continuous EEG data. EEG signals collected from 81 early-onset patients and 89 normal controls are used to build the final classification model and to verify the practical performance.

Alzheimer’s disease (AD), the most common form of dementia, causes progressive impairment of cognitive functions of patients. There is thus an urgent need to (1) accurately predict the cognitive performance of the disease, and (2) identify potential MRI (Magnetic Resonance Imaging)-related biomarkers most predictive of the estimation of cognitive outcomes. The main objective of this work is to build a multi-task learning based on MRI in the presence of structure in the features. In this paper, we simultaneously exploit the interrelated structures within the MRI features and among the tasks and present a novel Group guided Sparse group lasso (GSGL) regularized multi-task learning approach, to effectively incorporate both the relatedness among multiple cognitive score prediction tasks and useful inherent group structure in features. An Alternating Direction Method of Multipliers (ADMM) based optimization is developed to efficiently solve the non-smooth formulation. We demonstrate the performance of the proposed method using the Alzheimer’s Disease Neuroimaging Initiative (ADNI) datasets and show that our proposed methods achieve not only clearly improved prediction performance for cognitive measurements, but also finds a compact set of highly suggestive biomarkers relevant to AD.

Alzheimer’s Disease is a severe neurological brain disorder. It destroys brain cells causing people to lose their memory, mental functions and ability to continue daily activities. Alzheimer’s Disease is not curable, but earlier detection can help improve symptoms in a great deal. Machine learning techniques can vastly improve the process for accurate diagnosis of Alzheimer’s Disease. In recent days deep learning techniques have achieved major success in medical image analysis. But relatively little investigation has been done to applying deep learning techniques for Alzheimer’s Disease detection and classification. This paper presents a novel deep learning model for multi-Class Alzheimer’s Disease detection and classification using Brain MRI Data. We design a very deep convolutional network and demonstrate the performance on the Open Access Series of Imaging Studies (OASIS) database.

In order to recognize the major depressive mood status of inpatients and achieve its daily change information, a POMS-BCN scale was used to rate the mood status. Meanwhile, a personalized quantified model based on portable EEG was built, which aimed at objectively assessing the major depressive mood status for each patient. 6 inpatients were recruited to join the experiment. The Principal Component Analysis method is used to extract first principal component curve from the POMS-BCN data. The feature extraction method is used to extract linear and nonlinear features from portable EEG data. The regression analysis based on Random Forest is adopted to build the personalized quantified model. The principal component analysis result shows that the first principal component curve is able to recognize the major emotional factor and depict its daily change information. Additionally, the expected quantitative value outputted from the personalized quantified model is highly correlated (the absolute value of correlation coefficient 0.7, P-value 0.05) with the actual first principal component data, which implies that the personalized quantified model can give an accurate objective assessment for the major depressive mood status.

Along with the popularization and rapid development of Internet, there is a growing interest in the research to identify the trend of social events on social media. Currently news could quickly spread on various social media (e.g. Sina Weibo) with a limited time, which may trigger the severity of the events that requires timely attention and responses from government. This paper proposes to predict the trend of social events on Sina Weibo, which is the most popular social media in China now. In this study, combining social psychology and communication sciences, we extracted comprehensive and effective features which may relate to the trend of social events on social media, and constructed the trend prediction models using three classical regression algorithms. The real social events data was used to verify the performance of our model, and the outstanding performance with precision of 0.56 and an f-measure of 0.71 demonstrate the efficiency of our features and models.

Depression has become a disease, which may threaten millions of families’ well-being. The current method of screening depression is subjective, labor-consuming and costly. Study on Electroencephalogram (EEG) has become a new direction to explore an objective, low-cost and accurate method to detect depression. In this paper, three-electrode EEG data of 158 subjects (90 depressed and 68 normal control) in resting state, and under audio stimulation (positive and negative) were collected and processed. After feature selection using Sequential Floating Forward Selection (SFFS), four popular classification methods were applied and classification accuracies were verified using 10-fold cross validation. Results have shown the accuracy of classification will be improved when male and female are classified separately. The highest accuracy of male and female classification are 91.98%, 79.76%, respectively, compare to 77.43% when the classification is processed as gender-free. The effective depressive features of male and female are also different, which may be caused by the differences of brain structure. This research suggests a possible pervasive method of depression classification for future clinical application.

Microstructural changes of corticospinal tract (CST) correlate with motor performance in ischemic stroke patients. However, the findings about CST structural alteration after stroke varied due to different lesion sites, recovery degree and different disrupted pathways. Basal ganglia (BG) plays an important role in motor control and execution. Despite the intimate anatomical relation between BG and CST, the impact of BG stroke lesion on CST integrity and its association with motor performance remains unclear. In this study, we recruited 10 stroke patients with lesion specifically in BG area and investigate the CST structural alteration 1–3 months post stroke using diffusion tensor imaging (DTI) methodology. The bilateral cerebral peduncle (CP), posterior limb of internal capsule (PLIC) and superior cornal radiation (sCR) areas were investigated and the regional DTI parameters were calculated. Our results showed a significant decline of ipsileional FA in CP, PLIC and sCR, which is in correlation with patient’s concurrent Fugl-Meyer index (FMI) score. Moreover, the lateralization of FA in CP and PLIC negatively correlated with FMI. Our work showed that the CST structural alteration associated with motor function of BG stroke patients within subacute stage. The FA value and its lateralization served as informative markers for motor performance evaluation.

Detecting depression in speech is a hot topic in recent years. Some inconsistent results in previous researches imply a few important influence factors are ignored. In this paper, we investigated a sample of 184 subjects (108 females, 76 males) to examine the influence of speaking style and emotional valence on depression detection. First, classification accuracy was used to measure the influence of these two factors. Then, two-way analysis of variance was employed to determine interactive acoustical features. Finally, normalized features by subtracting got higher classification accuracies. Results show that both speaking style and emotional valence are important factors. Spontaneous speech is better than automatic speech and neutral is the best choice among three emotional valences in depression detection. Normalized features improve the detection performance.

The article summarizes scientific advances on consciousness up to the present (the year 2017). The remarkable milestones of experimental research on consciousness, in particular those in response to some philosophic meanings, are selected. These chosen achievements are within more than half centuries and narrowed on five fields: (1) modeling consciousness, (2) analysis on consciousness quantum indeterminacy, (3) finding core-consciousness-function cells, (4) brain-machine interface, (5) brain research plans on brain information access and analysis with large-scale. The main conclusions cover that (1) Piaget consciousness model (PCM), which asserts that consciousness is the homomorphism between functional cells and their mapped objects in respective laws of motion, is a universal frame defining the consciousness in philosophic, scientific ways; (2) Receptive Field, Place Cell and Grid Cell, and some functional brain cells which specially make decisions are PCM instances; (3) consciousness has not been confirmed to be related to quantum states, but some tentative plans to be confirmed have been suggested; (4) brain-machine interface shows PCM too by physical or artificial ways. Meanwhile, some important data, analog or relevant technologies about the above achievements are described, and philosophic explanations are tried to be given.

Visualization plays an important role in helping neuroscientist understanding human brain data. Most publicly available software focuses on visualizing a specific brain imaging modality. Here we present an extensible visualization platform, BECA, which employ a plugin architecture to facilitate rapid development and deployment of visualization for human brain data. This paper will introduce the architecture and discuss some important design decisions in implementing the BECA platform and its visualization plugins.

The incidence of depression has increased dramatically in recent years and the quality use of antidepressants has drawn extensive concerns worldwide. In this paper, firstly, we analyze current barriers regarding the use of antidepressants. Secondly, a new informative system which is developed using knowledge techniques is proposed, and its role in the management of antidepressants is discussed. Three main functions of the current version of Knowledge Graphs for Depression (DepressionKG, version 0.6) are presented. Besides, we conduct a semi-structured focus group interview to evaluate this system. The results indicate that DepressionKG has the potential to be a feasible, evidence-based tool for healthcare professionals.

Statins have been widely used for the treatment of cardiovascular diseases. However, the most severe adverse effect of statins is myotoxicity, in the form of myopathy and other similar ones. Identifying whether it is a statins-induced muscle symptoms plays an important role in the use of statins. In this paper, we propose an approach to analyse the neglected influencing factors of statin-induced myopathy in a coronary heart disease case by using the technology of knowledge graphs. Through the n-of-1 trial, we can verify the accuracy of the knowledge graphs for this task. Furthermore, Knowledge graph of adverse reactions and symptoms is expected to assist physicians in determining adverse events in the future.

Brain science is a well-recognized frontier science with extensive and profound contents. For the past hundreds years, biological experimental studies are the main ways to understand the brain, but now, neuro-data and computing have grown up to be almost an equivalent important tool for uncovering the brain and brain disorders. It means that a fuse of “Brain” and “Informatics”, Brainformatics, is becoming an area of science. In this paper, we take the brain research driven by information science as a new discipline – BraInFormatics – with “brain information acquisition”, “Brain information decoding” and “brain information applications” as the main contents, and meso-scale problems are the main space waiting for this discipline. This paper explains the concept, scope and challenges with some examples, such as EEG zero-reference technique, brainwave music etc. developed in our lab.

Biochemical molecules are substantial bases for the function of nervous system. The concentrations of specific molecules are closely linked to the activity and status of nervous system. This paper summarized our previous work on the relationship be-tween some active biochemical molecules and specific activities in nervous system. The results indicated that the levels of some biochemical substances may reflect the function and regulation of the nervous system, which derived easy, fast and cheap methods for assessing the activity and status in nervous system.