Introduction
Related work
The framework of transfer learning
Framework of TSL applied to principal component analysis (PCA)
Convolutional neural networks
Proposed three-phase training algorithm for CNN architecture using transfer learning approach
Comparison of traditional algorithm (conventional) with proposed algorithm
Proposed database | |
---|---|
Source | COEP and MIT Pune |
Purpose | Designed for studying the problem of transfer subspace learning |
Number of subjects | 50 |
Number of images/videos | 20,000 |
Static/videos | Static |
Single/multiple faces | Single |
Gray/color | Color |
Resolution |
\(640 \times 480\) and \(2816 \times 2112\)
|
Face pose | Frontal view |
Facial expression | Neutral |
Illumination | Controlled illumination |
Ground truth | Identification of subjects under transfer subspace learning |
Testing images | Training images | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S1 | S1sh1 | S1sh2 | S2 | S2sh1 | S2sh2 | S3 | S4 | S5 | S6 | S7 | S7r1 | S7r2 | S8 | S8r1 | S8r2 | |
S1 |
96
| 64 | 20 | 0 | 8 | 8 | 8 | 32 | 76 | 76 | 88 | 88 | 92 | 84 | 88 | 84 |
S1sh1 | 44 |
92
| 4 | 20 | 12 | 12 | 16 | 36 | 48 | 52 | 60 | 56 | 36 | 64 | 68 | 36 |
S1sh2 | 16 | 48 |
96
| 8 | 12 | 12 | 8 | 28 | 44 | 52 | 56 | 56 | 32 | 56 | 56 | 36 |
S2 | 4 | 4 | 12 |
100
| 56 | 48 | 20 | 16 | 24 | 20 | 24 | 28 | 16 | 16 | 12 | 4 |
S2sh1 | 24 | 16 | 16 | 36 |
100
| 56 | 20 | 72 | 88 | 84 | 88 | 84 | 76 | 80 | 84 | 68 |
S2sh2 | 36 | 20 | 16 | 48 | 52 |
100
| 12 | 44 | 68 | 92 | 88 | 92 | 80 | 88 | 96 | 80 |
S3 | 12 | 8 | 8 | 8 | 12 | 12 |
96
| 24 | 88 | 12 | 16 | 8 | 12 | 24 | 20 | 76 |
S4 | 36 | 20 | 48 | 20 | 28 | 20 | 28 |
100
| 56 | 44 | 52 | 40 | 48 | 44 | 44 | 40 |
S5 | 28 | 20 | 28 | 20 | 40 | 24 | 12 | 20 |
96
| 40 | 8 | 12 | 12 | 0 | 12 | 20 |
S6 | 12 | 24 | 20 | 24 | 24 | 16 | 0 | 8 | 28 |
100
| 52 | 40 | 44 | 24 | 20 | 20 |
S7 | 20 | 24 | 16 | 24 | 20 | 20 | 4 | 4 | 20 | 44 |
100
| 68 | 48 | 60 | 64 | 40 |
S7r1 | 16 | 20 | 20 | 32 | 28 | 44 | 16 | 4 | 16 | 36 | 68 |
100
| 64 | 48 | 52 | 32 |
S7r2 | 12 | 16 | 12 | 0 | 4 | 0 | 12 | 8 | 8 | 52 | 60 | 56 |
96
| 24 | 24 | 44 |
S8 | 0 | 4 | 8 | 0 | 4 | 4 | 8 | 4 | 8 | 4 | 60 | 32 | 16 |
100
| 84 | 40 |
S8r1 | 16 | 16 | 20 | 16 | 28 | 24 | 8 | 4 | 4 | 16 | 68 | 44 | 40 | 76 |
96
| 72 |
S8r2 | 44 | 16 | 12 | 0 | 4 | 0 | 12 | 8 | 8 | 52 | 60 | 56 | 96 | 24 | 24 |
100
|
Dataset
Testing images | Training images | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S1 | S1sh1 | S1sh2 | S2 | S2sh1 | S2sh2 | S3 | S4 | S5 | S6 | S7 | S7r1 | S7r2 | S8 | S8r1 | S8r2 | |
S1 |
92
| 22 | 13 | 12 | 8 | 17 | 12 | 8 | 20 | 4 | 41 | 29 | 50 | 30 | 21 | 90 |
S1sh1 | 28 |
94
| 47 | 14 | 14 | 12 | 11 | 11 | 30 | 2 | 47 | 44 | 42 | 38 | 36 | 88 |
S1sh2 | 16 | 26 |
89
| 13 | 15 | 15 | 9 | 13 | 27 | 3 | 55 | 48 | 46 | 50 | 49 | 85 |
S2 | 8 | 5 | 5 |
96
| 38 | 9 | 4 | 4 | 9 | 2 | 69 | 36 | 32 | 48 | 37 | 81 |
S2sh1 | 4 | 6 | 4 | 24 |
88
| 38 | 4 | 6 | 11 | 3 | 52 | 24 | 33 | 41 | 30 | 71 |
S2sh2 | 4 | 6 | 3 | 23 | 62 |
69
| 2 | 7 | 7 | 4 | 55 | 31 | 29 | 36 | 33 | 73 |
S3 | 21 | 2 | 28 | 7 | 6 | 6 |
91
| 5 | 5 | 2 | 18 | 11 | 13 | 12 | 25 | 21 |
S4 | 22 | 23 | 72 | 6 | 11 | 22 | 8 |
88
| 9 | 3 | 10 | 2 | 3 | 6 | 8 | 4 |
S5 | 28 | 28 | 10 | 27 | 24 | 36 | 9 | 7 |
97
| 24 | 4 | 2 | 11 | 3 | 6 | 2 |
S6 | 21 | 24 | 14 | 17 | 50 | 58 | 9 | 4 | 33 |
97
| 29 | 21 | 23 | 12 | 8 | 2 |
S7 | 9 | 21 | 14 | 13 | 14 | 10 | 18 | 4 | 11 | 31 |
97
| 46 | 34 | 36 | 29 | 12 |
S7r1 | 26 | 20 | 16 | 15 | 17 | 14 | 24 | 6 | 10 | 24 | 47 |
97
| 29 | 19 | 37 | 10 |
S7r2 | 21 | 11 | 12 | 18 | 13 | 14 | 19 | 2 | 14 | 41 | 38 | 24 |
97
| 21 | 20 | 24 |
S8 | 18 | 17 | 18 | 14 | 12 | 9 | 37 | 5 | 6 | 30 | 40 | 22 | 19 |
96
| 71 | 20 |
S8r1 | 28 | 10 | 18 | 15 | 13 | 10 | 40 | 7 | 6 | 14 | 31 | 26 | 20 | 59 |
94
| 25 |
S8r2 | 22 | 11 | 17 | 10 | 12 | 17 | 33 | 3 | 8 | 15 | 16 | 12 | 33 | 31 | 43 |
98
|
Experiments, parameter settings, and observations
Testing images | Training images | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S1 | S1sh1 | S1sh2 | S2 | S2sh1 | S2sh2 | S3 | S4 | S5 | S6 | S7 | S7r1 | S7r2 | S8 | S8r1 | S8r2 | |
S1 |
84
| 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 |
S1sh1 | 92 |
92
| 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 | 92 |
S1sh2 | 88 | 88 |
88
| 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 |
S2 | 80 | 80 | 80 |
80
| 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
S2sh1 | 84 | 84 | 84 | 84 |
84
| 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 |
S2sh2 | 80 | 80 | 80 | 80 | 80 |
80
| 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
S3 | 82 | 82 | 82 | 82 | 82 | 82 |
82
| 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 |
S4 | 84 | 84 | 84 | 84 | 84 | 84 | 84 |
84
| 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 |
S5 | 88 | 88 | 88 | 88 | 88 | 88 | 88 | 88 |
88
| 88 | 88 | 88 | 88 | 88 | 88 | 88 |
S6 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
80
| 80 | 80 | 80 | 80 | 80 | 80 |
S7 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 | 84 |
84
| 84 | 84 | 84 | 84 | 84 |
S7r1 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
80
| 80 | 80 | 80 | 80 |
S7r2 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
80
| 80 | 80 | 80 |
S8 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 | 82 |
82
| 82 | 82 |
S8r1 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
80
| 80 |
S8r2 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 | 80 |
80
|
Testing images | Training images | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S1 | S1sh1 | S1sh2 | S2 | S2sh1 | S2sh2 | S3 | S4 | S5 | S6 | S7 | S7r1 | S7r2 | S8 | S8r1 | S8r2 | |
S1 |
65
| 65 | 65 | 65 | 65 | 64 | 65 | 65 | 64 | 64 | 65 | 65 | 65 | 64 | 65 | 63 |
S1sh1 | 65 |
66
| 65 | 65 | 65 | 65 | 65 | 65 | 65 | 67 | 66 | 64 | 65 | 65 | 64 | 63 |
S1sh2 | 64 | 55 |
64
| 65 | 64 | 68 | 65 | 64 | 67 | 64 | 63 | 65 | 65 | 59 | 64 | 65 |
S2 | 66 | 66 | 61 |
66
| 66 | 67 | 65 | 65 | 64 | 67 | 65 | 65 | 66 | 66 | 65 | 65 |
S2sh1 | 64 | 66 | 66 | 65 |
65
| 65 | 66 | 65 | 63 | 65 | 64 | 65 | 64 | 65 | 64 | 65 |
S2sh2 | 66 | 66 | 67 | 66 | 65 |
65
| 65 | 66 | 65 | 63 | 63 | 62 | 64 | 62 | 64 | 63 |
S3 | 65 | 65 | 65 | 66 | 65 | 65 |
67
| 64 | 65 | 67 | 63 | 64 | 65 | 66 | 67 | 63 |
S4 | 66 | 65 | 65 | 65 | 64 | 67 | 64 |
65
| 65 | 66 | 64 | 65 | 63 | 65 | 63 | 55 |
S5 | 65 | 66 | 66 | 64 | 64 | 64 | 66 | 62 |
67
| 63 | 65 | 63 | 63 | 55 | 64 | 67 |
S6 | 65 | 65 | 62 | 66 | 65 | 65 | 65 | 66 | 65 |
65
| 66 | 65 | 66 | 65 | 64 | 64 |
S7 | 65 | 64 | 63 | 67 | 66 | 64 | 54 | 64 | 65 | 64 |
62
| 64 | 64 | 64 | 69 | 68 |
S7r1 | 66 | 65 | 66 | 65 | 65 | 66 | 64 | 66 | 65 | 63 | 66 |
63
| 65 | 64 | 66 | 63 |
S7r2 | 66 | 66 | 63 | 64 | 66 | 65 | 65 | 65 | 62 | 65 | 62 | 63 |
64
| 64 | 63 | 64 |
S8 | 68 | 65 | 66 | 64 | 65 | 65 | 65 | 64 | 65 | 59 | 65 | 65 | 64 |
66
| 64 | 62 |
S8r1 | 65 | 66 | 62 | 67 | 66 | 66 | 67 | 62 | 63 | 62 | 65 | 64 | 63 | 64 |
65
| 65 |
S8r2 | 59 | 66 | 64 | 68 | 65 | 66 | 65 | 67 | 62 | 64 | 65 | 64 | 66 | 65 | 62 |
62
|
Algorithm | Various layers of CNN | Total trainable weights | ||||||
---|---|---|---|---|---|---|---|---|
C\(_{1}\)
| S\(_{2}\)
| C\(_{3}\)
| S\(_{4}\)
| C\(_{5}\)
| F\(_{6}\)
| Output layer trainable weights | ||
trainable weights | trainable weights | trainable weights | trainable weights | trainable weights | trainable weights | |||
Conventional CNN algorithm (trained for 25 users) | 732 | – | 1952 | – | 14,640 | 10,080 | 2100 | 29,504 |
Conventional CNN algorithm (trained for 50 users) | 732 | – | 1952 | – | 14,640 | 10,080 | 4200 | 31,604 |
Proposed three-phase training algorithm for CNN (25 users) | – | – | – | – | – | 10,080 | 2100 | 12,180 |
Proposed three-phase training algorithm for CNN (50 users) | – | – | – | – | – | 10,080 | 4200 | 14,280 |