1 Introduction
2 Theoretical preliminaries
2.1 Data partitioning
2.2 Granular computing
3 Semi-random partitioning of data into training and test sets
3.1 Key features
Weight | Probability for class 1 (%) | Probability for class 2 (%) | Probability for class 3 (%) |
---|---|---|---|
Training set: 70% | 28 | 28 | 14 |
Test set: 30% | 12 | 12 | 6 |
Weight | Probability for class 1 (%) | Probability for class 2 (%) | Probability for class 3 (%) |
---|---|---|---|
Training set: 70% | 70 | 70 | 70 |
Test set: 30% | 30 | 30 | 30 |
3.2 Justification
Data set | Feature types | #Attributes | #Instances | #Classes |
---|---|---|---|---|
Anneal | Discrete, continuous | 38 | 798 | 6 |
Autos | Discrete, continuous | 26 | 205 | 7 |
Credit-a | Discrete, continuous | 15 | 690 | 2 |
Heart-stalog | Continuous | 13 | 270 | 2 |
Iris | Continuous | 4 | 150 | 3 |
kr-vs-kp | Discrete | 36 | 3196 | 2 |
Labor | Discrete, continuous | 17 | 57 | 2 |
Segment | Continuous | 19 | 2310 | 7 |
Sonar | Continuous | 60 | 208 | 2 |
Tae | Discrete, continuous | 6 | 151 | 3 |
Vote | Discrete | 16 | 435 | 2 |
Wine | Continuous | 13 | 178 | 3 |
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
SS | ||||||||
Precision | 0.00 | 0.88 | 0.99 | 0.00 | 1.00 | 1.00 | 0.98 | |
Recall | 0.00 | 1.00 | 0.98 | 0.00 | 1.00 | 1.00 | ||
SR | ||||||||
Precision | 0.00 | 0.97 | 0.99 | 0.00 | 1.00 | 1.00 | 0.99 | |
Recall | 0.00 | 1.00 | 1.00 | 0.00 | 1.00 | 1.00 | ||
Autos | ||||||||
SS | ||||||||
Precision | 0.00 | 0.00 | 0.63 | 0.88 | 0.72 | 0.80 | 0.80 | 0.77 |
Recall | 0.00 | 0.00 | 0.71 | 0.7 | 0.81 | 0.80 | 1.00 | |
SR | ||||||||
Precision | 0.00 | 0.50 | 1.00 | 0.95 | 0.69 | 0.55 | 0.89 | 0.79 |
Recall | 0.00 | 1.00 | 0.57 | 0.95 | 0.69 | 0.60 | 1.00 | |
Credit-a | ||||||||
SS | ||||||||
Precision | 0.80 | 0.85 | 0.83 | |||||
Recall | 0.82 | 0.83 | ||||||
SR | ||||||||
Precision | 0.82 | 0.97 | 0.89 | |||||
Recall | 0.97 | 0.83 | ||||||
Heart-statlog | ||||||||
SS | ||||||||
Precision | 0.80 | 0.68 | 0.74 | |||||
Recall | 0.71 | 0.78 | ||||||
SR | ||||||||
Precision | 0.79 | 0.89 | 0.83 | |||||
Recall | 0.93 | 0.69 | ||||||
Iris | ||||||||
SS | ||||||||
Precision | 1.00 | 0.93 | 0.93 | 0.96 | ||||
Recall | 1.00 | 0.93 | 0.93 | |||||
SR | ||||||||
Precision | 1.00 | 1.00 | 0.94 | 0.98 | ||||
Recall | 1.00 | 0.93 | 1.00 | |||||
kr-vs-kp | ||||||||
SS | ||||||||
Precision | 0.99 | 1.00 | 0.99 | |||||
Recall | 1.00 | 0.99 | ||||||
SR | ||||||||
Precision | 0.99 | 1.00 | 0.99 | |||||
Recall | 1.00 | 0.99 | ||||||
Labor | ||||||||
SS | ||||||||
Precision | 0.80 | 0.85 | 0.83 | |||||
Recall | 0.67 | 0.92 | ||||||
SR | ||||||||
Precision | 0.83 | 0.91 | 0.88 | |||||
Recall | 0.83 | 0.91 | ||||||
Segment | ||||||||
SS | ||||||||
Precision | 0.98 | 1.00 | 0.89 | 0.92 | 0.84 | 0.99 | 1.00 | 0.95 |
Recall | 0.98 | 1.00 | 0.90 | 0.92 | 0.83 | 1.00 | 0.99 | |
SR | ||||||||
Precision | 0.97 | 1.00 | 0.89 | 0.99 | 0.88 | 1.00 | 1.00 | 0.96 |
Recall | 0.97 | 1.00 | 0.89 | 0.94 | 0.93 | 1.00 | 1.00 | |
Sonar | ||||||||
SS | ||||||||
Precision | 0.65 | 0.72 | 0.68 | |||||
Recall | 0.69 | 0.68 | ||||||
SR | ||||||||
Precision | 0.81 | 0.87 | 0.84 | |||||
Recall | 0.86 | 0.82 | ||||||
Tae | ||||||||
SS | ||||||||
Precision | 0.40 | 0.39 | 0.46 | 0.41 | ||||
Recall | 0.53 | 0.33 | 0.38 | |||||
SR | ||||||||
Precision | 0.55 | 0.67 | 0.55 | 0.57 | ||||
Recall | 0.73 | 0.27 | 0.69 | |||||
Vote | ||||||||
SS | ||||||||
Precision | 0.94 | 0.98 | 0.96 | |||||
Recall | 0.96 | 0.96 | ||||||
SR | ||||||||
Precision | 0.97 | 0.94 | 0.96 | |||||
Recall | 0.96 | 0.96 | ||||||
Wine | ||||||||
SS | ||||||||
Precision | 1.00 | 0.96 | 0.93 | 0.96 | ||||
Recall | 1.00 | 0.96 | 0.93 | |||||
SR | ||||||||
Precision | 1.00 | 0.91 | 1.00 | 0.96 | ||||
Recall | 0.94 | 1.00 | 0.93 |
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
SS | ||||||||
Precision | 1.00 | 0.87 | 0.98 | 0.00 | 1.00 | 1.00 | 0.93 | |
Recall | 1.00 | 0.87 | 0.92 | 0.00 | 1.00 | 0.50 | ||
SR | ||||||||
Precision | 0.50 | 0.79 | 0.99 | 0.00 | 1.00 | 0.30 | 0.86 | |
Recall | 1.00 | 1.00 | 0.82 | 0.00 | 1.00 | 0.92 | ||
Autos | ||||||||
SS | ||||||||
Precision | 0.00 | 0.00 | 1.00 | 0.46 | 0.65 | 0.44 | 0.50 | 0.53 |
Recall | 0.00 | 0.00 | 0.14 | 0.6 | 0.81 | 0.40 | 0.38 | |
SR | ||||||||
Precision | 0.00 | 1.00 | 0.42 | 0.80 | 0.55 | 0.20 | 0.67 | 0.53 |
Recall | 0.00 | 1.00 | 0.71 | 0.40 | 0.69 | 0.20 | 0.75 | |
Credit-a | ||||||||
SS | ||||||||
Precision | 0.77 | 0.87 | 0.82 | |||||
Recall | 0.85 | 0.79 | ||||||
SR | ||||||||
Precision | 0.91 | 0.78 | 0.83 | |||||
Recall | 0.67 | 0.95 | ||||||
Heart-statlog | ||||||||
SS | ||||||||
Precision | 0.91 | 0.82 | 0.86 | |||||
Recall | 0.84 | 0.89 | ||||||
SR | ||||||||
Precision | 0.86 | 0.94 | 0.89 | |||||
Recall | 0.96 | 0.81 | ||||||
Iris | ||||||||
SS | ||||||||
Precision | 1.00 | 0.93 | 0.88 | 0.93 | ||||
Recall | 1.00 | 0.87 | 0.93 | |||||
SR | ||||||||
Precision | 1.00 | 1.00 | 1.00 | 1.00 | ||||
Recall | 1.00 | 1.00 | 1.00 | |||||
kr-vs-kp | ||||||||
SS | ||||||||
Precision | 0.86 | 0.89 | 0.88 | |||||
Recall | 0.91 | 0.84 | ||||||
SR | ||||||||
Precision | 0.88 | 0.89 | 0.89 | |||||
Recall | 0.91 | 0.87 | ||||||
Labor | ||||||||
SS | ||||||||
Precision | 1.00 | 0.86 | 0.89 | |||||
Recall | 0.67 | 1.00 | ||||||
SR | ||||||||
Precision | 1.00 | 1.00 | 1.00 | |||||
Recall | 1.00 | 1.00 | ||||||
Segment | ||||||||
SS | ||||||||
Precision | 1.00 | 1.00 | 0.68 | 0.53 | 0.49 | 1.00 | 1.00 | 0.75 |
Recall | 0.48 | 1.00 | 0.87 | 0.85 | 0.56 | 0.51 | 0.99 | |
SR | ||||||||
Precision | 0.79 | 1.00 | 0.57 | 0.90 | 0.43 | 0.95 | 1.00 | 0.80 |
Recall | 0.97 | 1.00 | 0.12 | 0.87 | 0.68 | 0.97 | 1.00 | |
Sonar | ||||||||
SS | ||||||||
Precision | 0.92 | 0.66 | 0.71 | |||||
Recall | 0.41 | 0.97 | ||||||
SR | ||||||||
Precision | 0.73 | 0.83 | 0.77 | |||||
Recall | 0.83 | 0.73 | ||||||
Tae | ||||||||
SS | ||||||||
Precision | 0.41 | 0.44 | 0.46 | 0.44 | ||||
Recall | 0.47 | 0.53 | 0.31 | |||||
SR | ||||||||
Precision | 0.65 | 0.63 | 0.69 | 0.65 | ||||
Recall | 0.73 | 0.67 | 0.56 | |||||
Vote | ||||||||
SS | ||||||||
Precision | 0.84 | 0.96 | 0.91 | |||||
Recall | 0.94 | 0.89 | ||||||
SR | ||||||||
Precision | 0.97 | 0.83 | 0.91 | |||||
Recall | 0.88 | 0.96 | ||||||
Wine | ||||||||
SS | ||||||||
Precision | 1.00 | 0.92 | 1.00 | 0.96 | ||||
Recall | 0.94 | 1.00 | 0.93 | |||||
SR | ||||||||
Precision | 0.94 | 0.95 | 1.00 | 0.98 | ||||
Recall | 0.97 | 1.00 | 1.00 |
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
SS | ||||||||
Precision | 0.00 | 0.63 | 0.86 | 0.00 | 0.75 | 0.83 | 0.83 | |
Recall | 0.50 | 1.00 | 0.98 | 0.00 | 1.00 | 0.62 | ||
SR | ||||||||
Precision | 1.00 | 0.90 | 0.99 | 0.00 | 1.00 | 0.69 | 0.96 | |
Recall | 1.00 | 0.93 | 0.96 | 0.00 | 1.00 | 0.92 | ||
Autos | ||||||||
SS | ||||||||
Precision | 0.00 | 0.00 | 0.00 | 0.33 | 0.54 | 0.11 | 0.00 | 0.32 |
Recall | 0.00 | 0.00 | 0.00 | 0.60 | 0.44 | 0.10 | 0.00 | |
SR | ||||||||
Precision | 0.00 | 0.00 | 0.71 | 0.58 | 0.55 | 0.50 | 0.67 | 0.58 |
Recall | 0.00 | 0.00 | 0.71 | 0.55 | 0.75 | 0.40 | 0.50 | |
Credit-a | ||||||||
SS | ||||||||
Precision | 0.66 | 0.71 | 0.69 | |||||
Recall | 0.63 | 0.74 | ||||||
SR | ||||||||
Precision | 0.91 | 0.88 | 0.89 | |||||
Recall | 0.84 | 0.93 | ||||||
Heart-statlog | ||||||||
SS | ||||||||
Precision | 0.64 | 0.54 | 0.59 | |||||
Recall | 0.60 | 0.58 | ||||||
SR | ||||||||
Precision | 0.84 | 0.88 | 0.85 | |||||
Recall | 0.91 | 0.79 | ||||||
Iris | ||||||||
SS | ||||||||
Precision | 1.00 | 1.00 | 0.94 | 0.98 | ||||
Recall | 1.00 | 0.88 | 0.92 | |||||
SR | ||||||||
Precision | 1.00 | 0.88 | 1.00 | 0.96 | ||||
Recall | 1.00 | 1.00 | 0.87 | |||||
kr-vs-kp | ||||||||
SS | ||||||||
Precision | 0.52 | 0.00 | 0.52 | |||||
Recall | 1.00 | 0.00 | ||||||
SR | ||||||||
Precision | 0.94 | 0.97 | 0.96 | |||||
Recall | 0.97 | 0.94 | ||||||
Labor | ||||||||
SS | ||||||||
Precision | 0.86 | 1.00 | 0.94 | |||||
Recall | 1.00 | 0.92 | ||||||
SR | ||||||||
Precision | 1.00 | 0.92 | 0.94 | |||||
Recall | 0.83 | 1.00 | ||||||
Segment | ||||||||
SS | ||||||||
Precision | 0.96 | 1.00 | 0.88 | 0.93 | 0.89 | 1.00 | 1.00 | 0.95 |
Recall | 0.96 | 1.00 | 0.92 | 0.89 | 0.88 | 1.00 | 1.00 | |
SR | ||||||||
Precision | 0.96 | 1.00 | 0.85 | 0.99 | 0.87 | 0.96 | 1.00 | 0.95 |
Recall | 0.98 | 1.00 | 0.95 | 0.86 | 0.83 | 1.00 | 1.00 | |
Sonar | ||||||||
SS | ||||||||
Precision | 0.88 | 0.82 | 0.84 | |||||
Recall | 0.76 | 0.91 | ||||||
SR | ||||||||
Precision | 0.84 | 0.78 | 0.81 | |||||
Recall | 0.72 | 0.88 | ||||||
Tae | ||||||||
SS | ||||||||
Precision | 0.25 | 0.43 | 0.40 | 0.37 | ||||
Recall | 0.20 | 0.40 | 0.50 | |||||
SR | ||||||||
Precision | 0.54 | 0.57 | 0.63 | 0.59 | ||||
Recall | 0.47 | 0.53 | 0.75 | |||||
Vote | ||||||||
SS | ||||||||
Precision | 0.89 | 0.97 | 0.94 | |||||
Recall | 0.96 | 0.93 | ||||||
SR | ||||||||
Precision | 0.96 | 0.90 | 0.94 | |||||
Recall | 0.94 | 0.94 | ||||||
Wine | ||||||||
SS | ||||||||
Precision | 0.84 | 0.65 | 0.44 | 0.65 | ||||
Recall | 0.94 | 0.50 | 0.53 | |||||
SR | ||||||||
Precision | 0.95 | 1.00 | 0.93 | 0.96 | ||||
Recall | 1.00 | 0.90 | 1.00 |
Data set | Original distribution | Training set | Test set |
---|---|---|---|
Anneal | |||
# | 8:99:684:0:67:40 | 6:69:479:0:47:28 | 2:30:205:0:20:12 |
% | 1:11:76:0:7:4 | 1:11:76:0:7:4 | 1:11:76:0:7:4 |
Autos | |||
# | 0:3:22:67:54:32:27 | 0:2:15:47:38:22:19 | 0:1:7:20:16:10:8 |
% | 0:1:11:33:26:16:13 | 0:1:10:33:27:15:13 | 0:2:11:32:26:16:13 |
Credit-a | |||
# | 307:383 | 215:268 | 92:115 |
% | 44:56 | 45:55 | 44:56 |
Heart-statlog | |||
# | 150:120 | 105:84 | 45:36 |
% | 56:44 | 56:44 | 56:44 |
Iris | |||
# | 50:50:50 | 35:35:35 | 15:15:15 |
% | 33:33:33 | 33:33:33 | 33:33:33 |
kr-vs-kp | |||
# | 1669:1527 | 1168:1069 | 501:458 |
% | 52:48 | 52:48 | 52:48 |
Labor | |||
# | 20:37 | 14:26 | 6:11 |
% | 35:65 | 35:65 | 35:65 |
Segment | |||
# | 330:330:330:330:330:330:330 | 231:231:231:231:231:231:231 | 99:99:99:99:99:99:99 |
% | 14:14:14:14:14:14:14 | 14:14:14:14:14:14:14 | 14:14:14:14:14:14:14 |
Sonar | |||
# | 97:111 | 68:78 | 29:33 |
% | 47:53 | 47:53 | 47:53 |
Tae | |||
# | 49:50:52 | 34:35:36 | 15:15:16 |
% | 32:33:34 | 32:33:34 | 33:33:35 |
Vote | |||
# | 267:168 | 187:118 | 80:50 |
% | 61:39 | 61:39 | 62:38 |
Wine | |||
# | 59:71:48 | 41:50:34 | 18:21:14 |
% | 33:40:27 | 33:40:27 | 34:40:26 |
Data set | Original distribution | Training set | Test set |
---|---|---|---|
Anneal | |||
# | 8:99:684:0:67:40 | 7:73:483:0:39:27 | 1:26:201:0:28:13 |
% | 1:11:76:0:7:4 | 1:12:77:0:6:4 | 0:10:75:0:10:5 |
Autos | |||
# | 0:3:22:67:54:32:27 | 0:3:17:41:43:23:17 | 0:0:5:26:11:9:10 |
% | 0:1:11:33:26:16:13 | 0:2:12:28:30:16:12 | 0:0:8:43:18:15:16 |
Credit-a | |||
# | 307:383 | 211:272 | 96:111 |
% | 44:56 | 44:56 | 46:54 |
Heart-statlog | |||
# | 150:120 | 99:90 | 51:30 |
% | 56:44 | 52:48 | 63:37 |
Iris | |||
# | 50:50:50 | 38:30:37 | 12:20:13 |
% | 33:33:33 | 36:29:35 | 27:44:29 |
kr-vs-kp | |||
# | 1669:1527 | 1196:1041 | 473:486 |
% | 52:48 | 53:47 | 49:51 |
Labor | |||
# | 20:37 | 13:27 | 7:10 |
% | 35:65 | 33:68 | 41:59 |
Segment | |||
# | 330:330:330:330:330:330:330 | 223:223:230:239:242:229:231 | 107:107:100:91:88:101:99 |
% | 14:14:14:14:14:14:14 | 14:14:14:15:15:14:14 | 15:15:14:13:13:15:14 |
Sonar | |||
# | 97:111 | 62:84 | 35:27 |
% | 47:53 | 42:58 | 56:44 |
Tae | |||
# | 49:50:52 | 34:32:40 | 15:18:12 |
% | 32:33:34 | 32:30:38 | 33:40:27 |
Vote | |||
# | 267:168 | 186:119 | 81:49 |
% | 61:39 | 61:39 | 62:38 |
Wine | |||
# | 59:71:48 | 42:55:28 | 17:16:20 |
% | 33:40:27 | 34:44:22 | 32:30:38 |
Data set | Original distribution | Training set | Test set |
---|---|---|---|
Anneal | |||
# | 8:99:684:0:67:40 | 4:67:484:0:44:30 | 4:32:200:0:23:10 |
% | 1:11:76:0:7:4 | 1:11:77:0:7:5 | 1:12:74:0:9:4 |
Autos | |||
# | 0:3:22:67:54:32:27 | 0:2:15:45:39:23:20 | 0:1:7:22:15:9:7 |
% | 0:1:11:33:26:16:13 | 0:1:10:31:27:16:14 | 0:2:11:36:25:15:11 |
Credit-a | |||
# | 307:383 | 216:267 | 91:116 |
% | 44:56 | 45:55 | 44:56 |
Heart-statlog | |||
# | 150:120 | 111:78 | 39:42 |
% | 56:44 | 59:41 | 48:52 |
Iris | |||
# | 50:50:50 | 37:31:37 | 13:19:13 |
% | 33:33:33 | 35:30:35 | 29:42:29 |
kr-vs-kp | |||
# | 1669:1527 | 1164:1073 | 505:454 |
% | 52:48 | 52:48 | 53:47 |
Labor | |||
# | 20:37 | 16:24 | 4:13 |
% | 35:65 | 40:60 | 24:76 |
Segment | |||
# | 330:330:330:330:330:330:330 | 245:228:229:220:245:218:232 | 85:102:101:110:85:112:98 |
% | 14:14:14:14:14:14:14 | 15:14:14:14:15:13:14 | 12:15:15:16:12:16:14 |
Sonar | |||
# | 97:111 | 60:86 | 37:25 |
% | 47:53 | 41:59 | 60:40 |
Tae | |||
# | 49:50:52 | 34:31:41 | 15:19:11 |
% | 32:33:34 | 32:29:39 | 33:42:24 |
Vote | |||
# | 267:168 | 183:122 | 84:46 |
% | 61:39 | 60:40 | 65:35 |
Wine | |||
# | 59:71:48 | 48:43:34 | 11:28:14 |
% | 33:40:27 | 38:34:27 | 21:53:26 |
Data set | Original distribution | Training set | Test set |
---|---|---|---|
Anneal | |||
# | 8: 99:684:0:67:40 | 4:64:484:0:50:27 | 4:35:200:0:17:13 |
% | 1:11:76:0:7:4 | 1:10:77:0:8:4 | 1:13:74:0:6:5 |
Autos | |||
# | 0:3:22:67:54:32:27 | 0:3:16:49:38:21:17 | 0:0:6:18:16:11:10 |
% | 0:1:11:33:26:16:13 | 0:2:11:34:26:15:12 | 0:0:10:30:26:18:16 |
Credit-a | |||
# | 307:383 | 224:259 | 83:124 |
% | 44:56 | 46:54 | 40:60 |
Heart-statlog | |||
# | 150:120 | 106:83 | 44:37 |
% | 56:44 | 56:44 | 54:46 |
Iris | |||
# | 50:50:50 | 35:34:36 | 15:16:14 |
% | 33:33:33 | 33:32:34 | 33:36:31 |
kr-vs-kp | |||
# | 1669:1527 | 1177:1060 | 492:467 |
% | 52:48 | 53:47 | 51:49 |
Labor | |||
# | 20:37 | 15:25 | 5:12 |
% | 35:65 | 38:63 | 29:71 |
Segment | |||
# | 330:330:330:330:330:330:330 | 220:223:231:238:241:234:230 | 110:107:99:92:89:96:100 |
% | 14:14:14:14:14:14:14 | 14:14:14:15:15:14:14 | 16:15:14:13:13:14:14 |
Sonar | |||
# | 97:111 | 64:82 | 33:29 |
% | 47:53 | 44:56 | 53:47 |
Tae | |||
# | 49:50:52 | 33:35:38 | 16:15:14 |
% | 32:33:34 | 31:33:36 | 36:33:31 |
Vote | |||
# | 267:168 | 186:119 | 81:49 |
% | 61:39 | 61:39 | 62:38 |
Wine | |||
# | 59:71:48 | 34:55:36 | 25:16:12 |
% | 33:40:27 | 27:44:29 | 47:30:23 |
4 Experiments, results, and discussion
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
R | ||||||||
Precision | 0.00 | 0.96 | 0.99 | 0.00 | 1.00 | 1.00 | 0.99 | |
Recall | 0.00 | 1.00 | 1.00 | 0.00 | 1.00 | 0.85 | ||
SR | ||||||||
Precision | 0.00 | 0.97 | 0.99 | 0.00 | 1.00 | 1.00 | 0.99 | |
Recall | 0.00 | 1.00 | 1.00 | 0.00 | 1.00 | 1.00 | ||
Autos | ||||||||
R | ||||||||
Precision | 0.00 | 0.00 | 1.00 | 0.92 | 0.50 | 0.78 | 0.75 | 0.77 |
Recall | 0.00 | 0.00 | 0.80 | 0.85 | 0.73 | 0.78 | 0.60 | |
SR | ||||||||
Precision | 0.00 | 0.50 | 1.00 | 0.95 | 0.69 | 0.55 | 0.89 | 0.79 |
Recall | 0.00 | 1.00 | 0.57 | 0.95 | 0.69 | 0.60 | 1.00 | |
Credit-a | ||||||||
R | ||||||||
Precision | 0.82 | 0.90 | 0.86 | |||||
Recall | 0.90 | 0.83 | ||||||
SR | ||||||||
Precision | 0.82 | 0.97 | 0.89 | |||||
Recall | 0.97 | 0.83 | ||||||
Heart-statlog | ||||||||
R | ||||||||
Precision | 0.97 | 0.67 | 0.81 | |||||
Recall | 0.73 | 0.97 | ||||||
SR | ||||||||
Precision | 0.79 | 0.89 | 0.83 | |||||
Recall | 0.93 | 0.69 | ||||||
Iris | ||||||||
R | ||||||||
Precision | 1.00 | 0.94 | 0.81 | 0.91 | ||||
Recall | 0.92 | 0.85 | 1.00 | |||||
SR | ||||||||
Precision | 1.00 | 1.00 | 0.94 | 0.98 | ||||
Recall | 1.00 | 0.93 | 1.00 | |||||
kr-vs-kp | ||||||||
R | ||||||||
Precision | 0.99 | 0.99 | 0.99 | |||||
Recall | 0.99 | 0.99 | ||||||
SR | ||||||||
Precision | 0.99 | 1.00 | 0.99 | |||||
Recall | 1.00 | 0.99 | ||||||
Labor | ||||||||
R | ||||||||
Precision | 0.67 | 0.64 | 0.65 | |||||
Recall | 0.29 | 0.90 | ||||||
SR | ||||||||
Precision | 0.83 | 0.91 | 0.88 | |||||
Recall | 0.83 | 0.91 | ||||||
Segment | ||||||||
R | ||||||||
Precision | 0.97 | 0.98 | 0.92 | 0.99 | 0.84 | 1.00 | 0.99 | 0.96 |
Recall | 0.99 | 1.00 | 0.91 | 0.88 | 0.90 | 1.00 | 1.00 | |
SR | ||||||||
Precision | 0.97 | 1.00 | 0.89 | 0.99 | 0.88 | 1.00 | 1.00 | 0.96 |
Recall | 0.97 | 1.00 | 0.89 | 0.94 | 0.93 | 1.00 | 1.00 | |
Sonar | ||||||||
R | ||||||||
Precision | 0.85 | 0.79 | 0.82 | |||||
Recall | 0.83 | 0.81 | ||||||
SR | ||||||||
Precision | 0.81 | 0.87 | 0.84 | |||||
Recall | 0.86 | 0.82 | ||||||
Tae | ||||||||
R | ||||||||
Precision | 0.50 | 0.56 | 0.60 | 0.56 | ||||
Recall | 0.40 | 0.56 | 0.75 | |||||
SR | ||||||||
Precision | 0.55 | 0.67 | 0.55 | 0.57 | ||||
Recall | 0.73 | 0.27 | 0.69 | |||||
Vote | ||||||||
R | ||||||||
Precision | 0.95 | 0.96 | 0.95 | |||||
Recall | 0.98 | 0.92 | ||||||
SR | ||||||||
Precision | 0.97 | 0.94 | 0.96 | |||||
Recall | 0.96 | 0.96 | ||||||
Wine | ||||||||
R | ||||||||
Precision | 0.94 | 0.83 | 0.94 | 0.91 | ||||
Recall | 0.94 | 0.94 | 0.85 | |||||
SR | ||||||||
Precision | 1.00 | 0.91 | 1.00 | 0.96 | ||||
Recall | 0.94 | 1.00 | 0.93 |
-
For initially balanced data sets such as ‘iris’, ‘segment’, and ‘tae’, the random partitioning may lead to a loss of balance within the training and test sets; this loss can be observed for C4.5 on the ‘iris’ and ‘tae’ data sets, while for the ‘segment’ data set, the variation is smaller; similarly, for NB, the loss of balance can be noticed for the ‘iris’ and ‘tae’ data sets, while for the ‘segment’ data set, the variation is smaller, but more noticeable than for C4.5; for K-NN, a loss of balance can be observed for the ‘tae’ data set, while for the iris data set, the imbalance is very small, and for the ‘segment’ data set, the variation is small and similar to the variation for C4.5.
-
For slightly imbalanced data sets, the random partitioning may lead to a more balanced distribution in the training set, but a more imbalanced one in the test set, i.e., for C4.5., ‘heart-statlog’; for NB, labor, and vote; for K-NN, ‘credit-a’, ‘labor’, and ‘sonar’. Sometimes, the imbalance in the test set may mean that the majority class from the training set becomes minority class in the test set— this occurs only for one data set, i.e., ‘sonar’ with K-NN, which is probably due to the fact that the distribution in this data set is very close to perfect balance (47:53).
-
For slightly imbalanced data sets, the random partitioning may lead to a more balanced distribution in the test set, but a more imbalanced distribution in the training set, i.e., for C4.5, ‘kr-vs-kp’, and ‘labor’ by C4.5; for NB, ‘heart-statlog’. For two of these, C4.5—‘kr-vs-kp’ and NB—‘heart-statlog’, in the test set, the majority class is reversed in comparison with the training set.
-
For slightly imbalanced data sets, the random partitioning may lead to both the training and test sets to become more imbalanced, with a different class being the majority class in the training and test sets; for example, in the ‘sonar’ data set with C4.5, class 2 is the majority class in the training set, while class 1 is the majority class in the test set. This situation occurs on the ‘sonar’ data set for C4.5 and NB, and on the ‘wine’ data set for all algorithms (C4.5, NB, and K-NN).
-
For the data sets with a high number of classes and an imbalanced distribution, e.g., anneal and autos, the random partitioning may preserve the original distribution for some classes, while for others, there is an imbalance in the training set, the test set or both, i.e., the ‘autos’ data set for all algorithms (C4.5, NB, and K-NN); sometimes, the majority class in the training set is no longer the majority class in the test set, e.g., for C4.5—‘autos’, class 5 is the majority class in the training set, while class 4 is the majority class in the test set (as well as the original data set). For the anneal data set, the distribution changes slightly, but the majority of the changes are less than 2%—for this reason, we consider that the distribution for this data set with all algorithms is very similar to the original distribution.
-
For all data sets, the random partitioning may lead to a very similar distribution in the training and test sets as in the original data set. i.e., for C4.5, ‘anneal’, ‘credit-a’, and ‘vote’; for NB, ‘anneal’, ‘credit-a’, and ‘kr-vs-kp’; for K-NN, ‘anneal’, ‘heart-statlog’, ‘kr-vs-kp’, and ‘vote’.
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
R | ||||||||
Precision | 0.66 | 0.76 | 0.99 | 0.00 | 1.00 | 0.24 | 0.84 | |
Recall | 0.50 | 1.00 | 0.79 | 0.00 | 1.00 | 1.00 | ||
SR | ||||||||
Precision | 0.50 | 0.79 | 0.99 | 0.00 | 1.00 | 0.30 | 0.86 | |
Recall | 1.00 | 1.00 | 0.82 | 0.00 | 1.00 | 0.92 | ||
Autos | ||||||||
R | ||||||||
Precision | 0.00 | 1.00 | 0.30 | 0.50 | 0.59 | 0.50 | 0.67 | 0.52 |
Recall | 0.00 | 1.00 | 0.43 | 0.32 | 0.87 | 0.44 | 0.57 | |
SR | ||||||||
Precision | 0.00 | 1.00 | 0.42 | 0.80 | 0.55 | 0.20 | 0.67 | 0.53 |
Recall | 0.00 | 1.00 | 0.71 | 0.40 | 0.69 | 0.20 | 0.75 | |
Credit-a | ||||||||
R | ||||||||
Precision | 0.89 | 0.79 | 0.82 | |||||
Recall | 0.68 | 0.93 | ||||||
SR | ||||||||
Precision | 0.91 | 0.78 | 0.83 | |||||
Recall | 0.67 | 0.95 | ||||||
Heart-statlog | ||||||||
R | ||||||||
Precision | 0.77 | 0.94 | 0.84 | |||||
Recall | 0.95 | 0.74 | ||||||
SR | ||||||||
Precision | 0.86 | 0.94 | 0.89 | |||||
Recall | 0.96 | 0.81 | ||||||
Iris | ||||||||
R | ||||||||
Precision | 1.00 | 1.00 | 0.87 | 0.96 | ||||
Recall | 1.00 | 0.89 | 1.00 | |||||
SR | ||||||||
Precision | 1.00 | 1.00 | 1.00 | 1.00 | ||||
Recall | 1.00 | 1.00 | 1.00 | |||||
kr-vs-kp | ||||||||
R | ||||||||
Precision | 0.87 | 0.89 | 0.88 | |||||
Recall | 0.91 | 0.85 | ||||||
SR | ||||||||
Precision | 0.88 | 0.89 | 0.89 | |||||
Recall | 0.91 | 0.87 | ||||||
Labor | ||||||||
R | ||||||||
Precision | 0.75 | 0.92 | 0.88 | |||||
Recall | 0.75 | 0.92 | ||||||
SR | ||||||||
Precision | 1.00 | 1.00 | 1.00 | |||||
Recall | 1.00 | 1.00 | ||||||
Segment | ||||||||
R | ||||||||
Precision | 0.76 | 1.00 | 0.69 | 0.91 | 0.41 | 0.98 | 1.00 | 0.81 |
Recall | 0.99 | 1.00 | 0.18 | 0.83 | 0.71 | 0.97 | 0.99 | |
SR | ||||||||
Precision | 0.79 | 1.00 | 0.57 | 0.90 | 0.43 | 0.95 | 1.00 | 0.80 |
Recall | 0.97 | 1.00 | 0.12 | 0.87 | 0.68 | 0.97 | 1.00 | |
Sonar | ||||||||
R | ||||||||
Precision | 0.77 | 0.79 | 0.77 | |||||
Recall | 0.89 | 0.60 | ||||||
SR | ||||||||
Precision | 0.73 | 0.83 | 0.77 | |||||
Recall | 0.83 | 0.73 | ||||||
Tae | ||||||||
R | ||||||||
Precision | 0.63 | 0.50 | 0.25 | 0.47 | ||||
Recall | 0.80 | 0.26 | 0.36 | |||||
SR | ||||||||
Precision | 0.65 | 0.63 | 0.69 | 0.65 | ||||
Recall | 0.73 | 0.67 | 0.56 | |||||
Vote | ||||||||
R | ||||||||
Precision | 0.96 | 0.81 | 0.90 | |||||
Recall | 0.88 | 0.93 | ||||||
SR | ||||||||
Precision | 0.97 | 0.83 | 0.91 | |||||
Recall | 0.88 | 0.96 | ||||||
Wine | ||||||||
R | ||||||||
Precision | 1.00 | 1.00 | 1.00 | 1.00 | ||||
Recall | 1.00 | 1.00 | 1.00 | |||||
SR | ||||||||
Precision | 0.94 | 0.95 | 1.00 | 0.98 | ||||
Recall | 0.97 | 1.00 | 1.00 |
Data set | Class 1 | Class 2 | Class 3 | Class 4 | Class 5 | Class 6 | Class 7 | Accuracy |
---|---|---|---|---|---|---|---|---|
Anneal | ||||||||
R | ||||||||
Precision | 0.67 | 0.95 | 0.97 | 0.00 | 1.00 | 0.89 | 0.96 | |
Recall | 0.50 | 1.00 | 0.98 | 0.00 | 1.00 | 0.62 | ||
SR | ||||||||
Precision | 1.00 | 0.90 | 0.99 | 0.00 | 1.00 | 0.69 | 0.96 | |
Recall | 1.00 | 0.93 | 0.96 | 0.00 | 1.00 | 0.92 | ||
Autos | ||||||||
R | ||||||||
Precision | 0.00 | 0.00 | 0.67 | 0.60 | 0.65 | 0.29 | 1.00 | 0.52 |
Recall | 0.00 | 0.00 | 0.33 | 0.67 | 0.69 | 0.36 | 0.30 | |
SR | ||||||||
Precision | 0.00 | 0.00 | 0.71 | 0.58 | 0.55 | 0.50 | 0.67 | 0.58 |
Recall | 0.00 | 0.00 | 0.71 | 0.55 | 0.75 | 0.40 | 0.50 | |
Credit-a | ||||||||
R | ||||||||
Precision | 0.85 | 0.88 | 0.87 | |||||
Recall | 0.82 | 0.90 | ||||||
SR | ||||||||
Precision | 0.91 | 0.88 | 0.89 | |||||
Recall | 0.84 | 0.93 | ||||||
Heart-statlog | ||||||||
R | ||||||||
Precision | 0.71 | 0.70 | 0.70 | |||||
Recall | 0.77 | 0.62 | ||||||
SR | ||||||||
Precision | 0.84 | 0.88 | 0.85 | |||||
Recall | 0.91 | 0.79 | ||||||
Iris | ||||||||
R | ||||||||
Precision | 1.00 | 0.93 | 0.87 | 0.93 | ||||
Recall | 1.00 | 0.88 | 0.92 | |||||
SR | ||||||||
Precision | 1.00 | 0.88 | 1.00 | 0.96 | ||||
Recall | 1.00 | 1.00 | 0.87 | |||||
kr-vs-kp | ||||||||
R | ||||||||
Precision | 0.93 | 0.98 | 0.95 | |||||
Recall | 0.98 | 0.92 | ||||||
SR | ||||||||
Precision | 0.94 | 0.97 | 0.96 | |||||
Recall | 0.97 | 0.94 | ||||||
Labor | ||||||||
R | ||||||||
Precision | 1.00 | 0.92 | 0.94 | |||||
Recall | 0.80 | 1.00 | ||||||
SR | ||||||||
Precision | 1.00 | 0.92 | 0.94 | |||||
Recall | 0.83 | 1.00 | ||||||
Segment | ||||||||
R | ||||||||
Precision | 0.98 | 1.00 | 0.87 | 0.94 | 0.80 | 0.98 | 1.00 | 0.94 |
Recall | 0.97 | 1.00 | 0.91 | 0.88 | 0.82 | 1.00 | 0.99 | |
SR | ||||||||
Precision | 0.96 | 1.00 | 0.85 | 0.99 | 0.87 | 0.96 | 1.00 | 0.95 |
Recall | 0.98 | 1.00 | 0.95 | 0.86 | 0.83 | 1.00 | 1.00 | |
Sonar | ||||||||
R | ||||||||
Precision | 0.86 | 0.74 | 0.79 | |||||
Recall | 0.73 | 0.86 | ||||||
SR | ||||||||
Precision | 0.84 | 0.78 | 0.81 | |||||
Recall | 0.72 | 0.88 | ||||||
Tae | ||||||||
R | ||||||||
Precision | 0.44 | 0.36 | 0.50 | 0.44 | ||||
Recall | 0.50 | 0.27 | 0.57 | |||||
SR | ||||||||
Precision | 0.54 | 0.57 | 0.63 | 0.59 | ||||
Recall | 0.47 | 0.53 | 0.75 | |||||
Vote | ||||||||
R | ||||||||
Precision | 0.97 | 0.87 | 0.93 | |||||
Recall | 0.91 | 0.96 | ||||||
SR | ||||||||
Precision | 0.96 | 0.90 | 0.94 | |||||
Recall | 0.94 | 0.94 | ||||||
Wine | ||||||||
R | ||||||||
Precision | 1.00 | 1.00 | 0.92 | 0.98 | ||||
Recall | 1.00 | 0.93 | 1.00 | |||||
SR | ||||||||
Precision | 0.95 | 1.00 | 0.93 | 0.96 | ||||
Recall | 1.00 | 0.90 | 1.00 |