Abstract

Falls among the elderly comprise a major health problem. Daily activity monitoring and fall detection using wearable sensors provide an important healthcare system for elderly or frail individuals. We investigated the classification accuracy of daily activity and fall data based on surface electromyography (sEMG) and plantar pressure signals. sEMG and plantar pressure signals were collected, and their features were extracted. Suitable features were selected and combined for posture transition, gait, and fall using the Fisher class separability index. A feature-level fusion method, named as the global canonical correlation analysis of weighting genetic algorithm, was proposed to reduce dimensions. For the problem in which the number of daily activities is considerably more than the number of fall activities, Weighted Kernel Fisher Linear Discriminant Analysis (WKFDA) was proposed to classify gait and fall. Double Parameter Kernel Optimization based on Extreme Learning Machine (DPK-OMELM) was used to classify activities. Results showed that the classification accuracy of the posture transition is 100%, and the accuracy of gait and fall classified using WKFDA can reach 98%. For all types of posture transition, gait, and fall, sensitivity, specificity, and accuracy are over 96%.

1. Introduction

The number of elderly who need help in their daily activities is rapidly increasing due to the aging population [13]. As the physical function of the elderly begins to decline, the balance function of the elderly becomes worse, which is more likely to lead to accidents. In case of an accident, if an elderly cannot be found immediately and rescued in time, then serious consequences may occur. Hence, studying daily activity monitoring and fall detection is critical to reducing health and financial burdens [4]. In addition, daily activity recognition is beneficial to rehabilitation engineering [5].

Sensors used for monitoring activities include computer vision, ambient, and wearable sensors [6]. Computer vision-based methods require equipment with multiple-view cameras that must be used indoors [79]. The use of these devices is restricted by many factors, such as lighting conditions, installation location and angle, and occlusion. Moreover, these devices are considered intrusive to the privacy of users [10]. Ambient sensors include infrared sensors, door contacts, radars, and microphones [11]. Although this method can effectively obtain information about daily activities, it exhibits the disadvantages of fixed application scenarios, unstable performance, and cumbersome installation [12, 13]. By contrast, wearable sensors present the advantages of being easy to carry, having rich aware content, and providing controllable invasion of privacy [14].

Wearable sensors include acceleration sensors, gyroscopes, and biosensors. Acceleration sensors can collect information about acceleration and the angle between acceleration and gravity acceleration. Posture and transition activities can be easily distinguished by setting a threshold. However, the different activities included in posture activities cannot be distinguished [15]. Gyroscopes are widely used sensors for monitoring daily activities; they detect activities via triaxial and geomagnetic triangulation. However, gyroscopes can generate false angular velocity due to their drift caused by disturbed torque [16].

Surface electromyography (sEMG) provides information about the movement of different muscle groups during various activities [17]. Compared with inertial sensors, such as acceleration sensors and gyroscopes, the biosensor can accurately reflect the motion of the human body. Such information can be successfully applied in the fields of gesture recognition, gait analysis, and prosthetic limb control [18, 19]. A system developed by Young et al. [20] used sEMG to classify walking, ascending ramps, and climbing stairs, which reported improved recognition accuracy. Cheng et al. [21] proposed a framework for activity monitoring using an accelerometer and sEMG, which achieved over 98% recognition accuracy.

The distribution of plantar pressures provides detailed information about foot movement. It is important in human gait analysis and is widely used in the fields of identification, motion tracking, and motion recognition [22]. Chen et al. [23] fabricated a pressure insole that consisted of a foot-wearable interface and four force-sensing resistor (FSR) pressure sensors. The discrete contact force distribution signal was used to recognize the pattern of motions. Wang et al. [24] used density-based spatial clustering of applications with noise and K-means to determine the position from the plantar pressure image and then extracted the plantar pressure feature parameters to distinguish healthy subjects and patients, showing a significant difference.

Different feature vectors extracted from various activities always reflect the varying characteristics of activities. However, the effort of different features for classification varies. If all features are used to classify activities, then the dimensions of features will increase. Redundant features are eliminated to a certain degree by selecting and combining superior features [25]. In the current study, we propose the class separability index by the Fisher function to select and combine features for posture transition, gait, and fall activities.

Feature fusion has always been an important method for enriching data features in the field of pattern recognition. This method refers to the process of synthesizing the local observation features of the same or different types of sensors, thereby eliminating redundant information. A relatively complete description of features is formed by using complementary information, which can improve the reliability of recognition. Hazarika et al. [26] presented a novel feature fusion method based on twofold feature projection (FP) for electromyography (EMG) classification, which improved the recognition rate. In the current study, the weighting genetic algorithm for global canonical correlation analysis (WGA-GCCA) is proposed as the feature fusion method.

A classifier is a critical aspect of pattern recognition and is the core of the classification problem. Support vector machines (SVMs) and neural networks have also been widely applied to classify human movement based on sEMG [27]. Mishra et al. [28] presented a new method based on five features and an extreme learning machine (ELM) for diagnosing myopathic diseases and obtained a recognition accuracy of 88%. To solve the problem in which the number of daily activities is considerably more than the number of fall activities, a Weighted Kernel Fisher Linear Discriminant Analysis (WKFDA) has been proposed in the current study. In such an analysis, the sample kernel matrix is adjusted by using the corresponding equilibrium parameters. In accordance with the further classification of daily activities and falls, the Double Parameter Kernel Optimization based on ELM (DPK-OMELM) is proposed. The regular parameter C in the weight matrix and the kernel width in the kernel function are selected to optimize the traditional ELM.

The rest of this paper is structured as follows: Section 2 introduces the subjects, along with the data acquisition and proposed feature fusion and classification methods. Section 3 analyzes and discusses the experiments and results. Section 4 presents the conclusions.

2. Materials and Methods

2.1. Subjects and Activities

Twelve healthy subjects (six males and six females; age: 23–27 years; height: 162–180 cm; and weight: 46–70 kg) participated in the experiments. All the subjects read and signed an informed consent form approved by an institutional review board. The experimental activities are divided into four sets: posture, posture transition, gait, and fall. The specific identification activities are listed in Table 1.

Partial experimental activities are shown in Figure 1. The ankle of the participant is covered by an ankle guard to limit its activity and protect it. Walking, going upstairs, and going downstairs are controlled at a speed of about 1 m/s. As it can be seen in Figure 1(d), the participant is instructed to sit on the ground, keep the upper body straight, and the buttocks are about 20 cm apart from the heel. The going upstairs-falling activity is shown in Figure 1(e), in which the test pad is placed on the above steps of the tripped step, and the participants fall forward and land on their knees. The going downstairs-falling is shown in Figure 1(f), in which the hip of the participant is protected by a soft cushion, and the participant slips backward. The walking-falling is shown in Figures 1(g)1(i), in which the participant falls on the ground when tripping over an obstacle on the ground. The running-falling is similar to the walking-falling.

The activities were performed continuously by the subjects for 2 s and were repeated 20 times on different days, thereby ensuring that the samples for each activity were 240.

2.2. Data Acquisition
2.2.1. Acquisition of sEMG

Trigno™ Wireless EMG (Delsys Inc., Natick, MA, USA) is used to record sEMG signals. It provides a resolution of 16 bit, a bandwidth of 20–450 Hz, a baseline noise of less than 1.25 μV, and the motion artifact suppression.

Four sEMG electrodes are used to capture sEMG signals from the vastus lateralis (VL), tibialis anterior (TA), semitendinosus (ST), and gastrocnemius (GT), as shown in Figure 2. The sEMG signals are sampled at 1000 Hz.

2.2.2. Acquisition of Plantar Pressure

Plantar pressure data are collected using an FSR402 pressure sensor, which is a single-area and double-wire force resistor. FSR402 is composed of a solid thick polymer film, and it varies its resistance depending on how much pressure is being applied to the sensing area. FSR402 is a force-sensitive resistor with a round sensing area with a diameter of 14.7 mm. Figure 3 shows a physical diagram of the FSR and the transfer characteristic curve.

Figure 4 presents the area division and distribution of plantar pressures while the subject is standing. The pressures in the Z4/Z5, Z6/Z7, and Z10 regions are the maximum. Thus, three pressure sensors that are fixed to the insole are placed at Z10, at the middle of Z4 and Z5, and at the middle of Z6 and Z7, as shown in Figure 5. Plantar pressure signals are sampled at 200 Hz in this study.

2.3. Data Processing and Analysis
2.3.1. Feature Extraction of sEMG

The feature extraction methods in the feature pool included the following: mean of amplitude (MA), variance (VAR), Wilson amplitude (WAMP), autoregressive coefficient (AR), mean frequency (MF), mean power frequency (MPF), energy of wavelet coefficient (EWT), wavelet packet energy coefficient (EWP), fuzzy entropy (FE), and permutation entropy (PE).

To qualitatively evaluate the extracted features, we convert data samples into the class separability index via the Fisher discriminant function.

Given a sample vector , K is the number of total samples, whereas is the number of samples in class i; , is the vector of class i and C is the number of classes. The between-class scatter matrix and the within-class scatter matrix are as follows:where is the mean value of all the classes and is the mean value of class i.

The class separability index J is calculated aswhere indicates the trace of a matrix, that is, the sum of the diagonal elements of the matrix.

The separability indexes of sEMG features are calculated for posture transition, gait, and fall. The results are presented in Table 2 (the values in bold font indicate the top four separability indexes for one type of activity).

When the separability index of a feature is high, the classification effect is good. Thus, four features with higher separability indexes are selected to comprise a feature group of corresponding activities, as provided in Table 3.

2.3.2. Feature Extraction of Plantar Pressure

The pressure signals of sensors and the total pressure are normalized to form the first (F1) and second (F2) feature subvectors:where is the pressure detected by a single sensor and is the total pressure when a subject is standing.

The subvector F2 formed by the total plantar pressure ratio can be used to identify the contact between foot and ground and then distinguish between posture and gait activities. Figure 6 shows F2 of 10 trials of activities, including standing, sitting on a chair, sitting on the ground, and lying down. The threshold method can be easily used to identify posture activities based on their respective values.

The current values of plantar pressure signals are related to their past values in any motion mode. In the autoregressive (AR) model, the samples are estimated via the linear combination of earlier samples. Therefore, the AR model can be used to form the third feature subvector (F3). The Akaike information criterion [29] is applied to the AR model’s order estimation, and the AR model’s order is set as 3.

2.3.3. Feature Fusion

Canonical correlation analysis (CCA) is a method for measuring the linear relationship between a pair of multidimensional random variables [30]. It is used to find a new base vector for optimal correlation and to calculate the corresponding correlation. That is, a diagonal matrix is formed via CCA, which is the correlation matrix between variable projections and the basic set of maximum correlations. The dimensions of these base vectors must not exceed the smaller dimensions of the two variables [31].

Generalized CCA (GCCA) is an extension of CCA that contains the information of the class matrix. Suppose that and represent the within-class scatter matrix of samples and , respectively:where is the j-th training sample of class i; n is the number of training samples in class i; C is the number of total classes; and denote the mean vectors of samples and in class i, respectively.

The between-class scatter matrix of and is given by

Then, the generalized canonical correlation discriminant criterion is calculated as

From the criterion, and vectors that maximize are called the generalized canonical projective vector (GCPV).

The projection components of the random variables in the original space are not correlated in the correlation subspace, and thus, the dimension of the generated eigenvector is higher than that of the original eigenvector [32]. Therefore, the dimension of the new feature should be reduced. In this study, the weighted genetic algorithm (WGA) is used to reduce the aforementioned dimension.

As shown in Figure 7, GCPV is selected using WGA and is projected onto a new space to obtain a new feature vector, i.e., the GCPV of WGA (WGA-GCPV), which is defined as the GCCA of WGA (i.e., WGA-GCCA).

The chromosomes of a genetic algorithm (GA) are composed of sEMG features, and the initial population is generated randomly. Then, fitness is formed by the Fisher function value under the same feature. Finally, the dynamic weighting method is selected as follows:

In equation (7), and are the recognition rate and weight of the i-th feature set, respectively; and are the recognition rate and weight of the feature set with a minimum recognition rate, respectively. In equation (8), is the weight coefficient of the i-th feature set.

2.3.4. Classification

The flowchart of classification is presented in Figure 8. First, activities whose VAR of the total pressure ratio is less than a certain threshold (Th) are regarded as posture activities; otherwise, activities are considered as other activities. Second, posture transition activities can be distinguished from other activities by determining whether F1 is constantly larger than zero. Third, for gait and fall, the MA and WAMP of sEMG are extracted and inputted into the WKFDA classifier. Lastly, activities, including posture, posture transition, gait, and fall, are further classified. For posture activities, the threshold method is used to classify lying down, sitting on the ground, sitting on a chair, and standing/squatting. For standing and squatting, the MA of sEMG is extracted and Fisher linear discriminant analysis (FDA) is used for recognition. Then, the corresponding feature group of sEMG and the plantar pressure AR coefficient (F3) are fused into a new feature using WGA-GCCA. Lastly, the feature is inputted into DPK-OMELM to recognize activities.

WKFDA is proposed to classify gait and fall because the number of activities of daily living is considerably more than the number of fall activities. For further classification, we propose DPK-OMELM.

WKFDA: FDA involves projecting all samples in one direction and then determining a classification threshold in this 1D space [33]. FDA intends to find the most discriminant projection by maximizing between-class distance and minimizing within-class distance. By using the weighted kernel function, the data processing ability of FDA, now called WKFDA, is considerably improved. The process is described as follows:(1)Suppose that includes two types of training samples. The number of class i is , and is a nonlinear mapping from the input space to the feature space .(2)Kernel function ( is the kernel width) is introduced, and the kernel matrix is defined as(3)Suppose that the mean value vector of the column vector of is . Then, the mean value of is defined as(4)Weight is defined as Then, the weighted kernel matrix is given by(5)The pooled nuclear within-class scatter is calculated bywhere is the identity matrix and is a matrix where all the elements are .(6)The discriminant function based on WKFDA is defined as follows:where is the kernel mean value of class i and is the kernel within-class scatter matrix.(7)The optimal projection vector is obtained by maximizing as follows:(8)In the feature space, the projection transformation of is derived by

DPK-OMELM: ELM is an advanced single hidden layer forward network learning algorithm proposed by Huang et al. [34]. ELM obtains a simple generalized Moore–Penrose inverse operation to determine the hidden output matrix. The input-output relationships of ELM can be expressed aswhere is the state matrix, is the output matrix, and is the output weight coefficient matrix.

Huang et al. [35] proposed the optimization method based on ELM (OMELM). The theory indicates that a smaller module of the output weight of ELM provides better generalization performance of ELM. Thus, OMELM transforms the problem of minimizing the output error of ELM into the problem of minimizing output weight.

By combining the Gaussian kernel function ( is the kernel width), Gaussian kernel ELM requires the selection of regular parameter C in the weight matrix and kernel width in the kernel function. Then, ,where is an n-dimensional input.

By combining the OMELM algorithm, we obtain the minimum output weight value through and C to optimize ELM, which is now called DPK-OMELM:

3. Experimental Results and Discussion

The recognition performance of a classifier is typically evaluated quantitatively using three indicators: sensitivity (SEN), specificity (SPE), and accuracy (ACC).

SEN measures the percentage of correctly identified positive samples:

SPE measures the percentage of correctly identified negative samples:

ACC measures the percentage of all correctly identified samples:

In these equations, TP, FP, TN, and FN denote the numbers of true positive, false positive, true negative, and false negative samples, respectively.

When the number of total samples is insufficient, cross-validation with a low variance and high reliable accuracy can be used. Thus, we conduct fivefold cross-validation on these sample data.

Table 4 provides the classification results of posture, posture transition, gait, and fall activities.

The SEN, SPE, and ACC of posture and posture transition activities are 100%, probably because the signals are simple and evident, thereby providing a good foundation for further classification. Through WKFDA, each recognition index of gait and fall activities can reach 99%. Misclassification is primarily due to the misjudgment of fall and running.

3.1. Posture Recognition

In posture recognition, plantar pressure is divided into three sections using the threshold method, as shown in Figure 9.

When a subject is lying down, the sole is not in contact with the ground and F2 is zero; thus, lying down can be identified accurately. A part of the force is being exerted on the ground or the chair when a subject is sitting on the ground or chair. In general, the hip’s force is greater when sitting on the ground than when sitting on a chair; hence, the F2 of sitting on the ground is relatively small, as shown in Figure 10. As the barycenter will deviate horizontally while sitting on a chair, there is some error in F2 of sitting on a chair.

Therefore, the threshold in this study is as follows:

For standing and squatting, the ratio range is limited within to prevent the misjudgment caused by the subject’s unsteady movement; thus, .

Table 5 presents the recognition results of posture activities. For squatting and standing, we use the MA and WAMP of sEMG to distinguish them via FDA. There are obvious differences in the amplitudes of the sEMG of squatting and standing, particularly GT and VL. Therefore, the MA and WAMP of sEMG are used to distinguish between squatting and standing, and the recognition rate is 100%.

3.2. Posture Transition Recognition

Table 6 presents the recognition results of posture transition using the posture transition feature group and DPK-OMELM. As shown in the table, the reverse processes of posture transition (such as st-sig and sig-st) are likely to be misclassified but the recognition rate can reach 98.95%.

As the Gaussian kernel SVM (GK-SVM) [36] and the fuzzy min-max neural network (FMMNN) [37] show good learning performance in pattern recognition, GK-SVM and FMMNN are introduced to compare with DPK-OMELM. SEN, SPE, and ACC are obtained using the three classifiers, and the results are presented in Table 7.

Table 7 indicates that DPK-OMELM is superior to GK-SVM and FMMNN. The recognition rate of the reverse process of sitting on the ground and lying down is the highest because the total plantar pressure ratio of lying down is 0. For the reverse process of standing-squatting and squatting-standing, the recognition accuracy of GK-SVM is less than 95% and that of FMMNN is less than 90%, whereas that of DPK-OMELM can reach 97%.

3.3. Gait Recognition

The SEN, SPE, and ACC of the three classifiers are obtained and provided in Table 8.

Running is easier to classify because it is faster than the three other gait activities. Therefore, the SEN and SPE of this method are 100% for running. For walking on a flat surface, going upstairs, and going downstairs, the three evaluation values of the classifier algorithm proposed in this study are all close to 100%, thereby demonstrating that DPK-OMELM is superior to the common algorithms.

3.4. Fall Recognition

Similarly, the classification of fall activities is also compared with the GK-SVM and FMMNN classifiers. The SEN, SPE, and ACC of the DPK-OMELM, GK-SVM, and FMMNN classifiers are obtained and presented in Table 9.

As shown in the table, the recognition rate of fall activities is generally lower than that of daily activities. The recognition rate of gd-f is relatively high, partly because gd-f differs from several other fall activities. FMMNN performs poorly in identifying fall activities, and DPK-OMELM remains a better classification algorithm.

4. Conclusions

The objective of this study is to provide a daily activity monitoring and fall detection system based on sEMG and plantar pressure. Research on daily activity monitoring is regarded as both a new and old topic, and considerable progress can still be achieved in this field. In this study, an entire set of activities is divided on the basis of several classic activities included in daily activities. Through the class separability index, these extracted features, including the time domain, frequency domain, time and frequency domain, and entropy, are selected to comprise the feature group. WGA-GCCA (GCPV is obtained via GCCA, and the weighted feature selection of is performed by a GA, which can reduce the dimension by half) is proposed for feature fusion. Then, the feature group and AR of plantar signals are fused using WGA-GCCA.

During classification, FDA is improved by integrating a weighted kernel function. WKFDA is then used to distinguish between gait and fall activities, and a high accuracy is obtained. For posture activities, the threshold method is used to classify lying down, sitting on the ground, sitting on a chair, and standing/squatting. FDA is used to classify standing and squatting. The accuracy is close to 100%. For posture transition, gait, and fall activities, DPK-OMELM is proposed by optimizing ELM using two parameters (C in the output weight matrix of ELM and kernel width in the Gaussian kernel function). The experiment shows that this algorithm achieves higher recognition accuracy than GK-SVM and FMMNN. In a separate on-going study, we considered four sEMG and three plantar pressure sensors to study activity recognition and achieved high recognition accuracy. In a future study, the number of sensors should be reduced to study activity recognition with a high recognition rate. Furthermore, we will also develop a real-time monitoring device, which can be used in the rehabilitation system of human-computer interaction.

Data Availability

All data included in this study are available upon request by contact with the corresponding author.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (nos. 61971169, 61673350, 61671197, and 60903084), Zhejiang Public Welfare Technology Research (no. LGF18F010006), and Jinhua Science and Technology Research Project (no. 2019-3-020).