Introduction
Background
Ontology driven clinical decision support frameworks
Machine learning driven cardiovascular decision support systems
Methods
MLDPS development based on rapid access chest pain clinic’s clinical case study
Results and discussion
Study group 1 | Study group 2 | |
---|---|---|
Risk factors | Lab test results | |
1 | Smoker | Pathway |
2 | No of cigarettes | Initial assessment |
3 | Number of years smoking | ETT result |
4 | Age | CT result |
5 | Sex | MPS result |
6 | Diabetes type | Angio result |
7 | Hypertension | |
8 | Raised cholesterol |
Study group 1: clinical risk factors
Experimental setup | Selected features | Accuracy | |
---|---|---|---|
1 | LR + FS | 4, 5, 6, 2, 1, 3 | 68.45 |
2 | LR + BS | 1, 3, 4, 5, 6, 8 | 68.99 |
3 | LR + ED | All | 66.12 |
4 | LR + SFFS | 4, 5 ,6 | 67.92 |
5 | LR + P-value | 4, 5, 7, 8, 6, 3, 1, 2 | 66.12 |
6 | LR + mRMR | 4, 5, 7, 6, 8, 3, 1, 2 | 66.12 |
7 | DT + FS | 4, 7, 8, 6, 2 | 65.41 |
8 | DT + BS |
4
|
65.05
|
9 | DT + ED | All | 62.36 |
10 | DT + SFFS | 4 | 65.05 |
11 | DT + P value | 4, 5, 7, 8, 6, 3, 1, 2 | 62.36 |
12 | DT + mRMR | 4, 5, 7, 6, 8, 3, 1, 2 | 62.36 |
14 | SVM + FS |
4, 5,1
|
70.07
|
15 | SVM + BS | 4, 5, 7 | 69.71 |
16 | SVM + ED | All | 68.45 |
17 | SVM + SFFS |
4, 5, 1
|
70.07
|
18 | SVM + P value | 4, 5, 7, 8, 6, 3, 1, 2 | 68.45 |
19 | SVM + mRMR | 4, 5, 7, 6, 8, 3, 1, 2 | 68.45 |
Evaluation
Predicted output | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Actual | LR+FS | LR+BS | LR+ED | LR+SFFS | LR+P | LR+mRMR | ||||||
A | 197 | 87 | 193 | 91 | 188 | 96 | 194 | 90 | 188 | 96 | 188 | 96 |
B | 89 | 185 | 82 | 192 | 93 | 181 | 89 | 185 | 93 | 181 | 93 | 181 |
Accuracy | 68.45 | 68.99 | 66.12 | 67.92 | 66.12 | 66.12 |
LR + BS (%) | DT + FS (%) | SVM + SFFS (%) | |
---|---|---|---|
Weighted accuracy | 68.99 | 65.41 | 70.07 |
Unweighted accuracy | 69.01 | 65.38 | 70.18 |
Precision | 67.96 | 66.90 | 63.73 |
Recall | 70.18 | 65.74 | 73.88 |
Fmeasure | 69.05 | 66.32 | 68.43 |
Matthew’s correlation | 38.03 | 30.78 | 40.67 |
Predicted output | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
DT + FS | DT + BS | DT + ED | DT + SFFS | DT + P | DT + mRMR | |||||||
Actual | ||||||||||||
A | 190 | 94 | 170 | 114 | 169 | 115 | 170 | 114 | 169 | 115 | 169 | 115 |
B | 99 | 175 | 81 | 193 | 95 | 179 | 81 | 193 | 95 | 179 | 95 | 179 |
Accuracy | 65.41 | 65.14 | 62.3656 | 65.05 | 62.36 | 62.36 |
Predicted output | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
SVM + FS | SVM + BS | SVM + ED | SVM + SFFS | SVM + P | SVM + mRMR | |||||||
Actual | ||||||||||||
A | 181 | 103 | 183 | 101 | 179 | 105 | 181 | 103 | 179 | 105 | 179 | 105 |
B | 64 | 210 | 68 | 206 | 71 | 203 | 64 | 210 | 71 | 203 | 71 | 203 |
Accuracy | 70.07 | 69.71 | 68.45 | 70.07 | 68.45 | 64.45 |
Performance evaluation of experimental setups
Anova: single factor | ||||||
---|---|---|---|---|---|---|
Summary | ||||||
Groups | Count | Sum | Average | Variance | ||
Logistic regression | 6 | 403.72 | 67.28 | 1.7478 | ||
Decision tree | 6 | 382.59 | 63.765 | 2.38611 | ||
Support vector machine | 6 | 415.2 | 69.2 | 0.69228 | ||
ANOVA | ||||||
Source of variation | SS |
df
| MS |
F
|
P value
|
F crit
|
Between Groups | 91.20 |
2
| 45.60 |
28.34
|
8.02793E-06
|
3.68
|
Within Groups | 24.13 | 15 | 1.6087 | |||
Total | 115.3354944 | 17 |
Study group 2: lab test results
Evaluation
Clinical variables | P value | ||
---|---|---|---|
Lab test results | |||
1 | Pathway | 1.93e−27 | <0.00000 |
2 | Initial assessment | 1.48e−21 | <0.00000 |
3 | ETT result | 0.04 | <0.05 |
4 | CT result | 0.05 | <0.1 |
5 | MPS result | 0.17 | |
6 | Angio result | 0.9 |
Experimental setup | Selected features | Accuracy (%) | |
---|---|---|---|
1 | LR + FS | 2 | 69.89 |
2 | LR + BS | 1 ,4 ,5, 6 | 72.58 |
3 | LR + ED | All | 67.92 |
4 | LR + SFFS |
2
|
69.89
|
5 | LR + P value | 6,2,5,1,4,3 | 67.92 |
6 | LR + mRMR | 2,6,1,5,4,3 | 67.92 |
7 | DT + FS |
2, 6, 4, 3
|
82.97
|
8 | DT + BS |
2, 3, 4, 6
|
82.97
|
9 | DT + ED |
All
|
81.89
|
10 | DT + SFFS |
2, 6, 4, 3
|
82.97
|
11 | DT + P value |
6,2,5,1,4,3
|
81.89
|
12 | DT + mRMR |
2,6,1,5,4,3
|
81.89
|
14 | SVM + FS |
2,3
|
70.96
|
15 | SVM + BS | 2,4,5 | 70.96 |
16 | SVM + ED | All | 68.63 |
17 | SVM + SFFS | 2,3 | 70.96 |
18 | SVM + P value | 6,2,5,1,4,3 | 68.63 |
19 | SVM + mRMR | 2,6,1,5,4,3 | 68.63 |
DT + FS (%) | DT + SFFS (%) | DT + BS (%) | DT + mRMR (%) | |
---|---|---|---|---|
Weighted accuracy | 82.97 | 81.89 | 82.97 | 82.97 |
Unweighted accuracy | 83.09 | 81.98 | 83.09 | 83.09 |
Precision | 76.41 | 77.46 | 76.41 | 76.41 |
Recall | 88.57 | 85.60 | 88.57 | 88.57 |
Fmeasure | 82.04 | 81.33 | 82.04 | 82.04 |
Matthew’s correlation | 66.68 | 64.15 | 66.68 | 66.68 |
Predicted output | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
LR + FS | LR + BS | LR + ED | LR + SFFS | LR + P | LR + mRMR | |||||||
Actual | ||||||||||||
A | 142 | 142 | 248 | 36 | 206 | 78 | 142 | 142 | 206 | 78 | 208 | 78 |
B | 26 | 248 | 117 | 157 | 101 | 173 | 26 | 248 | 101 | 173 | 101 | 173 |
Accuracy (%) | 69.89 | 72.58 | 67.92 | 69.89 | 67.92 | 67.92 |
Predicted | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
DT + FS | DT + BS | DT + ED | DT + SFFS | DT + P | DT + mRMR | ||||||||
Actual | |||||||||||||
A | 217 | 67 | 217 | 67 | 220 | 64 | 217 | 67 | 220 | 64 | 220 | 64 | |
B | 28 | 246 | 28 | 246 | 37 | 237 | 28 | 246 | 37 | 237 | 37 | 237 | |
Accuracy (%) | 82.97 | 82.97 | 81.89 | 82.97 | 81.89 | 81.89 |
Predicted output | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
SVM + FS | SVM + BS | SVM + ED | SVM + SFFS | SVM + P value | SVM + mRMR | |||||||
Actual | ||||||||||||
A | 142 | 142 | 142 | 142 | 214 | 70 | 142 | 142 | 214 | 70 | 214 | 70 |
B | 20 | 254 | 20 | 254 | 105 | 169 | 20 | 254 | 105 | 169 | 105 | 169 |
Accuracy (%) | 70.96 | 70.96 | 68.63 | 70.96 | 68.63 | 68.63 |
Performance evaluation of experimental setups
Anova: single factor | ||||||
---|---|---|---|---|---|---|
Summary | ||||||
Groups | Count | Sum | Average | Variance | ||
Logistic regression | 6 | 416.12 | 69.35 | 3.4301 | ||
Decision tree | 6 | 494.58 | 82.43 | 0.34992 | ||
Support vector machine | 6 | 418.77 | 69.795 | 1.62867 | ||
ANOVA | ||||||
Source of variation | SS | df | MS | F | P value | F crit |
Between groups | 661.6750111 |
2
| 330.83 |
183.50
|
2.8522E−11
|
3.682
|
Within groups | 27.04368333 |
15
| 1.802912222 | |||
Total | 688.7186944 | 17 |
Implementation of online clinical prognostic models
Best classification setups | ||
---|---|---|
Risk factors and test results | ||
Experimental setups | Selected features | Weighted classification Accuracy (%) |
LR + FS | INA, AGE, ANG, SEX, MPS, YOS, NOC, HPT, PWY, ETT, CT, SMR | 74.68 |
LR + BS | SMR, YOS, AGE, PWY, SEX, HPT, INA, CT, MPS, ANG | 74.68 |
DT + SFFS | ANG, INA, CTT, ETT | 78.63 |
DT + FS | ANG, INA,CT, ETT, DAB, SEX | 77.84 |
SVM + FS | ANG, INA, CT, SEX, ETT, PWY, AGE, MPS, CHL,YOS | 78.16 |
SVM + BS | YOS, AGE, PWY, SEX, HPT, CHL, INA, CT, MPS, ANG | 78.32 |