Assessment of human operator functional state using a novel differential evolution optimization based adaptive fuzzy model
Introduction
During the modern development of automation technique, highly automated systems require few manual activities for individuals to perform. Human becomes the monitor and decision maker in the automatic system. Consequently, it brings more information processing loads and broader ranges of responsibilities to individuals. Therefore, the increased responsibility and decision making authority to each individual probably cause that the human operators become the weakest links in the operational loop [1], [2], [3]. The operator's performance degradation (because of vigilance decrease, fatigue, inattentiveness, sleepiness, pain, etc.) was the reason of some serious disasters [4]. To reduce this kind of disasters, researchers have begun to turn attention to the study of the performance of operators, particularly in safety-critical applications such as public transportation (railway [5], aviation [6], [7], driving [8]) and manufacturing industries (chemical [9] and nuclear plants [10]).
Adaptive automation has been proposed to solve the above problem. In adaptive automation, the control tasks can be reallocated dynamically between operators and automatic systems according to the operators’ performance level and whole system operation requirements. For example, when high operator mental workload is detected, some of the manual control tasks may be taken away and switched to the automatic system. On the other hand, high level of automation continuing for long time would result in lack of concentration for the operator. Proper manual control tasks should be reallocated to the operator.
There are three primary approaches to generate criteria for task reallocation [1]: (a) critical events logic where automation is engaged consistently in response to an environmental stimulus; (b) model-based approaches where automation is ‘scheduled’ based on a priori models of optimal operator performance; and (c) continuous assessment of operator function and mental state. For the third approach, continuous operator functional state (OFS) assessment is usually achieved through the following ways: (1) subjective assessments of the operator's mental workload; (2) performance of the operator on the primary task, and sometimes secondary tasks; (3) psychophysiological measurements of the operator. The authors’ current work is to investigate accurate and reliable assessment of OFS by psychophysiological measurements.
The psychophysiological signals, including electrocardiogram (ECG), electroencephalogram (EEG), eye blink, respiration, blood pressure, electrodermal activity, etc., have been demonstrated to be sensitive to the changes in OFS. Among those signals, the indices from ECG and EEG have been commonly used to assess the mental workload.
Heart rate (HR) and heart rate variability (HRV) are efficient indices of mental workload [11], [12], [13], [14]. In the previous works of ECG [15], [16], the authors found that HR and HRV1 (ratio of the standard deviation to the average of HR segment) did reflect the changes of operator mental workload.
EEG is another important signal for assessment of OFS. Researchers demonstrated that the oscillations in theta and alpha bands may reflect workload, attention, and performance levels of operator [17], [18], [19]. The increment in theta band power on the midline frontal sites as well as the decrement in alpha band power is relative to the mental workload and attention. Previous works have been done on analyzing the correlation of OFS with variables of EEG [20]. The task load index (TLI) [21], i.e. ratio of theta to alpha power from different sites, is selected for OFS assessment. In this study, HR, HRV1, and TLI (TLI1 and TLI2) are chosen as the inputs to the proposed model for OFS assessment.
Several studies have been done on the psychophysical signals and artificial neural networks (ANNs) to classify the levels of OFS [21], [22], [23]. Wilson and Russell [24] reported high accuracy of OFS classification using ANNs, i.e. 85%, 82%, and 86% for baseline, low and high task difficulty conditions, respectively. However, due to the characteristics of ANNs (the implicit knowledge representation), the relationship between input features and OFS was not explained.
Adaptive-Network-based Fuzzy Inference System (ANFIS) with the explicit knowledge representation was employed to estimate OFS [25], [26]. ANFIS can simulate and analyze the mapping relation between the input and the output data by a learning algorithm to optimize its parameters. It is widely used to build complex and nonlinear relationship between a set of input and output data [27], [28]. In [25], basic ANFIS was used for OFS assessment. The premise parameters and consequent parameters of ANFIS were identified by gradient descent method and least squares estimation, respectively [29]. The results showed that the generalization capability of ANFIS was not satisfied. Because of over-fitting, the basic ANFIS model output is not reliable in real application.
In this paper, a differential evolution algorithm with ant colony search (DEACS) is proposed to optimize the parameters of ANFIS. Differential evolution (DE) [30], [31] is a population-based stochastic optimization algorithm, which has the capability of global optimization and quick convergence. In DE algorithm, there are two significant control parameters: mutation factor and crossover factor. The two factors determine whether DE algorithm can find a good optimal solution. In the original DE algorithm, some recommended settings of the two factors were given through many optimization experiments for test functions. These are empirical settings from statistical results, and would be constant at different stages of evolution. To improve the performance of DE, some researchers developed new strategies to adjust the two factors by linear variation [32] and random selection [33], [34], etc. However, neither linear variation nor random selection is related to the evolution state. Therefore, the idea of ant colony search is employed to find out the proper combination of mutation and crossover factors adaptive to each individual according to current optimization performance. ACS [35] simulates the behavior of ant colony system. When ants prefer in probability to one path (i.e. a combination of mutation and crossover factors) since there are a greater quantity of pheromone on this path, the quantity of pheromone would be further increased. The positive feedback effect of ACS is applicable to selecting the combination of the factors as the most appropriate choice for DE. Instead of being constant or varying by some specific rules, the two factors can be determined adaptively. Applying ACS avoids the inefficiency which results from the improper selection of the two factors. The developed DEACS–ANFIS model is established to assess OFS. The obtained results are compared with ANFIS and DE based ANFIS (DE-ANFIS).
The rest of the paper is arranged as follows. In Section 2, the data acquisition experiment is described in details. Section 3 presents the ANFIS and DEACS algorithm, respectively, and establishes an adaptive fuzzy model based on DEACS–ANFIS for OFS assessment. In Section 4, the experimental results are illustrated, and the discussions are presented. Section 5 includes the conclusions and future works.
Section snippets
Data acquisition equipment
The experiments were carried out on AUTOmation-enhanced Cabin Air Management System (AUTOCAMS), which was developed by Hockey et al. [36] and modified by Lorenz and Parasuraman [37]. AUTOCAMS simulates a life support system of a spacecraft. It requires the subject (operator) to manage a semi-automatic system which is to regulate atmospheric conditions of the cabin so as to maintain comfortable atmospheric condition. There are five sub-systems (Fig. 1) corresponding to five critical parameters
ANFIS based on differential evolution with ant colony search
In this section, an adaptive fuzzy model is proposed. ANFIS is utilized to build the structure of the proposed model. An improved differential evolution with ant colony search is adopted to optimize the parameters of ANFIS.
Training and checking data
The DEACS based ANFIS model (DEACS–ANFIS) is used for OFS assessment. The inputs of the model include HR, HRV1, TLI1 and TLI2, which were found to be most sensitive to the changes of OFS [15], [16], [39]. TIR is selected to be the output.
Different from the works [25], [26] using 7.5 min, the sampling interval of inputs and output data is set to 2 min. The modified sampling interval allows more data information for analysis and facilitates the real-time assessment of OFS. However, it increased the
Conclusions
An adaptive fuzzy model, which is called DEACS–ANFIS, was proposed for operator functional state (OFS) assessment. Four psychophysiological signals, including heart rate (HR), heart rate variability (HRV1), task load indices (TLI1 and TLI2), were selected to be the inputs to the proposed model, and time in range (TIR) was the output. The ANFIS architecture provided a transparent assessment of OFS, and helped to easily understand the relationship between psychophysiological signals and OFS. The
Acknowledgements
The authors would like to thank the editor and anonymous referees for their valuable comments. They would also like to thank Dr. Bei Wang for her English correction and advice. This work was supported in part by the National Natural Science Foundation of China under Grant No. 61074113, 60775033 and 61075070, Shanghai Leading Academic Discipline Project B504, and Fundamental Research Funds for the Central Universities WH0914028. The 2nd author (J. Zhang) would like to gratefully acknowledge
References (41)
- et al.
Psychophysiology and adaptive automation
Biol. Psychol.
(1996) - et al.
Predicting work performance in nuclear power plants
Saf. Sci.
(2008) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis
Brain Res. Rev.
(1999)EEG-alpha rhythms and memory processes
Int. J. Psychophysiol.
(1997)- et al.
A combined adaptive network and fuzzy inference system (ANFIS) approach for overcurrent relay system
Neurocomputing
(2008) - et al.
Adaptive neuro-fuzzy inference system to compute quasi-TEM characteristic parameters of microshield lines with practical cavity sidewall profiles
Neurocomputing
(2006) Theoretical perspectives on adaptive automation
- et al.
A system for monitoring sleep and alertness to maintain effectiveness in the operational environment
- et al.
Engineering Psychology and Human Performance
(1999) Train controllers, interface design and mental workload
Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding
Hum. Factors
Mental workload in air traffic control: an index constructed from field tests
Aviat. Space Environ. Med.
Mental workload while driving: effects on visual search, discrimination, and decision making
J. Exp. Psychol. Appl.
Toward an understanding of DCS control operator workload
ISA Trans.
Fuzzy evaluation of heart rate signals for mental stress assessment
IEEE Trans. Fuzzy Syst.
The effect of mental stress on heart rate variability and blood pressure during computer work
Eur. J. Appl. Physiol.
Effect of mental and physical demands on heart rate variability during computer work
Eur. J. Appl. Physiol.
Level of operator control and changes in heart rate variability during simulated flight maintenance
Hum. Factors
Process control operator's heart rate variability analysis based on wavelet packet transform
J. Nanjing Univ. Sci. Technol.
Use of heart rate variability analysis for quantitatively assessing operator's mental workload
Cited by (45)
Spatially-coded SSVEP BCI without pre-training based on FBCCA
2023, Biomedical Signal Processing and ControlDecoding SSVEP patterns from EEG via multivariate variational mode decomposition-informed canonical correlation analysis
2022, Biomedical Signal Processing and ControlCitation Excerpt :The BCI system does not rely on the conventional muscles or neural pathways surrounding the brain but instead decodes specific neural patterns, typically measured by electroencephalogram (EEG), and translates them to computer commands for controlling external devices [1,2]. BCI has made great progress in medical treatment [3,4], military [5], entertainment [6], driver fatigue detection [7,8], and other fields. It not only brings good news to patients suffering from stroke, epilepsy, and mobility impairment [9–11], but it also provides convenience to people's lives [12,13].
SSVEP signal classification and recognition based on individual signal mixing template multivariate synchronization index algorithm
2021, Biomedical Signal Processing and ControlCitation Excerpt :BCI technology provides a new communication and control method for patients with physical inconvenience to realize wheelchair manipulation [3], prosthetic limb control [4], stimulation of nerve recovery [5] and other functions by using brain ideas. In addition, this technology has also been applied in virtual reality games [6], military communication control [7], operator fatigue state detection [8,9] and other medical fields. BCI systems can induce different EEG signals according to different experimental paradigms.
Operator performance prediction based on Fuzzy mdeling approach
2020, IFAC-PapersOnLine