Introduction
-
We present an automatic HAR system for classifying 33 different physical activities.
-
We analyze several feature extraction strategies to find the one with the best performance and robustness.
-
We study the influence of the type of sensor on the system performance.
-
We propose and evaluate several normalization strategies for dealing with the inter-user variability.
-
We also evaluate and validate the system for Home Care Monitoring using an ADL dataset.
Background
System architecture
Feature extraction
-
XYZ (3 signals): Original accelerometer signals.
-
Mag (1 signal): Magnitude signal computed from the previous three signals. This magnitude is computed as the square root of the sum of squared components (accelerometer signals).
-
Jerk-XYZ (3 signals): Jerk signals (derivative of the accelerometer signals) obtained from the original accelerometer signals.
-
JerkMag (1 signal): Magnitude signal computed from the previous jerk signals (square root of the sum of squared components).
-
fXYZ (3 signals): Fast Fourier transforms (FFTs) from XYZ.
-
fMag (1 signal): FFT from Mag.
-
fJerk-XYZ (3 signals): FFTs from Jerk-XYZ.
-
fJerkMag (1 signal): FFTs from JerkMag.
-
Mean value, standard deviation, median absolute deviation, minimum and maximum values of the samples in a frame.
-
Signal Magnitude Area: The normalized integral of the samples in a frame.
-
Energy measure: Sum of the squares samples divided by the number of samples in a frame.
-
Inter-quartile range: Variability measure obtained by dividing a data set into quartiles.
-
Signal entropy.
-
Auto-regression coefficients with Burg order equal to four correlation coefficients between two signals.
-
Index of the frequency component with largest magnitude.
-
Weighted average of the frequency components to obtain a mean frequency.
-
Skewness and Kurtosis of the frequency domain signal.
-
Energy of 6 equally spaced frequency bands within the 64 bins of the FFT.
Machine learning algorithm
Experiments
REALDISP dataset
-
Ideal-placement all sensors were placed by experts at their optimal place for classification. All the subjects recorded a session in these conditions (17 sessions).
-
Self-placement every subject decides the positions of three sensors by themselves and the remaining sensors were situated by experts. The number of three is considered a reasonable estimate of the proportion of sensors that may be misplaced during the normal wearing. All the subjects recorded a session in these conditions (17 sessions).
-
Mutual-placement where several displacements were intentionally introduced by experts. Three out of the 17 volunteers were recorded for mutual-displacement scenario (subjects 2, 5 and 15). These three subjects recorded one session for every sensor configuration: for the case in which four, five, six or even seven out of the nine sensors are misplaced.
Evaluation methods
Data analysis
Evaluation method
Sensor | Evaluation | # of features | Acc% | F-measure |
---|---|---|---|---|
ACC | Random-part | 954 | 99.1 | 0.991 |
ACC | Subject-wise | 954 | 95.5 | 0.951 |
Type of sensor
Sensor | Evaluation | # of features | Acc% | F-measure |
---|---|---|---|---|
ACC | Subject-wise | 954 | 95.5 | 0.951 |
GYR | Subject-wise | 954 | 94.4 | 0.941 |
MAG | Subject-wise | 954 |
96.3
|
0.952
|
QUAD | Subject-wise | 1224 | 93.0 | 0.924 |
Type of feature
Normalization methods
Method | ACC | GYR | MAG | QUAD | ||||
---|---|---|---|---|---|---|---|---|
Acc% | F-measure | Acc% | F-measure | Acc% | F-measure | Acc% | F-measure | |
None | 95.5 | 0.951 | 94.4 | 0.941 | 96.3 | 0.952 | 93.0 | 0.924 |
SIGN 1 | 94.7 | 0.943 | 93.6 | 0.935 | 96.1 | 0.957 | 92.2 | 0.923 |
SIGN 2 | 94.5 | 0.938 | 93.6 | 0.929 | 93.4 | 0.924 | 92.1 | 0.923 |
SIGN 3 | 89.2 | 0.880 | 88.1 | 0.870 | 93.6 | 0.927 | 87.5 | 0.865 |
SIGN 4 | 92.7 | 0.920 | 91.5 | 0.910 | 94.0 | 0.931 | 90.8 | 0.901 |
SIGN 5
|
96.4
|
0.961
|
95.4
|
0.950
|
97.9
|
0.976
|
94.1
|
0.939
|
SIGN 6 | 96.1 | 0.958 | 95.1 | 0.949 | 94.8 | 0.943 | 93.8 | 0.934 |
FEAT 1 | 95.6 | 0.951 | 94.5 | 0.940 | 95.6 | 0.950 | 93.0 | 0.930 |
FEAT 2 | 95.4 | 0.948 | 94.3 | 0.937 | 95.2 | 0.946 | 93.1 | 0.925 |
FEAT 3 | 94.7 | 0.940 | 93.5 | 0.932 | 95.0 | 0.944 | 92.1 | 0.920 |
FEAT 4 | 94.8 | 0.943 | 93.6 | 0.932 | 95.8 | 0.954 | 92.3 | 0.921 |
FEAT 5 | 92.2 | 0.916 | 91.1 | 0.908 | 96.4 | 0.960 | 90.2 | 0.892 |
FEAT 6 | 93.9 | 0.932 | 94.8 | 0.926 | 96.1 | 0.956 | 91.5 | 0.910 |
Final results and discussion
Ideal placement
System | Accuracy % | F-measure |
---|---|---|
Baseline [24]: Evaluation method: random-partitioning Null-activity: truncated | 97.0 | – |
This paper: Evaluation method: subject-wise Null-activity: truncated | 99.4 | 0.993 |
This paper: Evaluation method: subject-wise Null-activity: included | 99.1 | 0.991 |
Self and mutual placement
Train set | Test set | Baseline [24]: Evaluation method: random-partitioning Null-activity: truncated | This paper: Evaluation method: subject-wise Null-activity: included | ||
---|---|---|---|---|---|
Accuracy % | F-measure | Accuracy % | F-measure | ||
Ideal | Ideal | 97.0 | – | 99.1 | 0.991 |
Self | Self | 88.4 | – | 98.9 | 0.988 |
Mutual4 | Mutual4 | 71.2 | – | 87.9 | 0.847 |
Mutual5 | Mutual5 | 71.6 | – | 93.5 | 0.921 |
Mutual6 | Mutual6 | 77.2 | – | 96.4 | 0.959 |
Mutual7 | Mutual7 | 68.0 | – | 83.2 | 0.799 |
This paper: Evaluation method: subject-wise Null-activity: included | |||||||
---|---|---|---|---|---|---|---|
Train set | Test set | Accuracy % | F-measure | Train set | Test set | Accuracy % | F-measure |
Mutual4 (2 subjects) | Mutual4 (1 subject) | 87.9 | 0.847 | Ideal (16 subjects) | Mutual4 (1 subject) | 99.0 | 0.990 |
Mutual5 (2 subjects) | Mutual5 (1 subject) | 93.5 | 0.921 | Ideal (16 subjects) | Mutual5 (1 subject) | 98.1 | 0.982 |
Mutual6 (2 subjects) | Mutual6 (1 subject) | 96.4 | 0.959 | Ideal (16 subjects) | Mutual6 (1 subject) | 99.0 | 0.990 |
Mutual7 (2 subjects) | Mutual7 (1 subject) | 83.2 | 0.799 | Ideal (16 subjects) | Mutual7 (1 subject) | 94.5 | 0.938 |
Self (16 subjects) | Self (1 subject) | 98.9 | 0.988 | Ideal (16 subjects) | Self (1 subject) | 98.9 | 0.989 |
System analysis in a new domain: home care monitoring
OPPORTUNITY dataset
Experiments on the OPPORTUNITY dataset
Method | F-measure | F-measure (no null class) | ||||||
---|---|---|---|---|---|---|---|---|
S1 | S2 | S3 | S4 | S1 | S2 | S3 | S4 | |
Results from [31] | ||||||||
LDA | 0.62 | 0.64 | 0.68 | 0.43 | 0.73 | 0.70 | 0.74 | 0.53 |
QDA | 0.67 | 0.66 | 0.71 | 0.45 | 0.81 | 0.77 | 0.79 | 0.56 |
NCC | 0.60 | 0.58 | 0.56 | 0.45 | 0.69 | 0.67 | 0.62 | 0.50 |
1 NN | 0.84 | 0.85 |
0.83
| 0.76 | 0.85 | 0.85 | 0.85 | 0.76 |
3 NN | 0.85 | 0.86 |
0.83
| 0.77 | 0.86 | 0.86 | 0.85 | 0.76 |
UP | 0.58 | 0.62 | 0.88 | 0.80 | ||||
NStar | 0.58 | 0.66 | 0.88 | 0.85 | ||||
SStar | 0.61 | 0.68 | 0.87 | 0.83 | ||||
CStar | 0.60 | 0.65 | 0.90 | 0.83 | ||||
NU | 0.54 | 0.49 | 0.83 | 0.63 | ||||
MI | 0.85 | 0.81 | 0.87 | 0.86 | ||||
MU | 0.57 | 0.68 | 0.86 | 0.87 | ||||
UT | 0.48 | 0.55 | 0.74 | 0.72 | ||||
This paper |
0.88
|
0.88
| 0.80 |
0.85
|
0.92
|
0.92
|
0.89
|
0.86
|
Method | F-measure | F-measure (no Null class) | ||||||
---|---|---|---|---|---|---|---|---|
S1 | S2 | S3 | S4 | S1 | S2 | S3 | S4 | |
Results from [31] | ||||||||
LDA | 0.65 | 0.63 | 0.70 | 0.62 | 0.36 | 0.28 | 0.27 | 0.17 |
QDA | 0.60 | 0.57 | 0.69 | 0.64 | 0.34 | 0.29 | 0.34 | 0.22 |
NCC | 0.48 | 0.48 | 0.51 | 0.35 | 0.29 | 0.21 | 0.22 | 0.14 |
1 NN | 0.85 |
0.89
| 0.86 | 0.84 | 0.56 | 0.53 | 0.58 | 0.46 |
3 NN | 0.85 |
0.89
| 0.86 |
0.88
| 0.55 | 0.53 | 0.58 | 0.48 |
NStar | 0.84 | 0.83 | 0.60 | 0.69 | ||||
SStar | 0.87 | 0.84 | 0.65 | 0.72 | ||||
CStar | 0.88 |
0.87
|
0.72
|
0.80
| ||||
UP | 0.64 | 0.64 | 0.64 | 0.23 | 0.19 | 0.16 | ||
NAGS | 0.71 | 0.17 | ||||||
This paper |
0.87
|
0.89
|
0.87
| 0.85 |
0.76
| 0.64 | 0.64 |
0.55
|