Introduction
Related works
-
using local binary patterns to form an image feature vector;
-
sequential gender, race and age classification;
-
using a two-stage scheme for age classification that is based on support vector regression using cumulative features.
Proposed modifications
Modifying the method of the image feature vector formation
Modifying the procedure for classifying images of human faces by gender and race
-
a gender classifier (assigns labels “M” and “F” to images);
-
a classifier by race among men that identifies Caucasians among the images that carry the “M” label, i.e. the images receive the label “CM”;
-
a classifier by race among men that identifies Asians among the images that are labeled “M” and have not been classified as Caucasian, i.e. the images receive the label “MM”, and the rest of the images are classified as Black and receive the label BM;
-
a classifier by race among women that identifies Caucasians among the images that carry the “F” label, i.e. the images receive the label “CF”;
-
a classifier by race among women that identifies Asians among the images that are labeled “F” and have not been classified as Caucasian, i.e. the images receive the label “MF”, and the rest of the images are classified as Black and receive the label BF.
Modification of the two-stage scheme for human age classification
-
the set \(\widehat{G}^{l}\) is randomly divided into two parts \(\widehat{G}^{l}_{1}\) and \(\widehat{G}^{l}_{2 }\)with a ratio of 1 to 2;
-
a decision function is generated for the set \(\widehat{G}^{l}_{1}\) using SVR;
-
the set \(\widehat{G}^{l}_{2}\) is used for training tests with identification of “hard” examples (examples are considered “hard” if they do not fall within the sensitivity band \(\varepsilon _{0})\);
-
the set \(\widehat{G}^{l}_{1}\) is expanded with “hard” examples, i.e. the set \(\widehat{G}^{l}_{1\mathrm{new}}\) is generated;
-
the set \(\widehat{G}^{l}_{1\mathrm{new}}\) is used for final determination of the decision function \(f_0 (\widehat{a},\varepsilon _{0},\widehat{G}^{l})\).
Summary of the proposed algorithm
-
selection of half of a normalized face image and application of LBP to this half-image in order to generate a feature vector containing only uniform patterns;
-
usage of the Adaboost method for the “gender” attribute to exclude insignificant features from the image feature vector (the sum of weight coefficients of the boosted features is equal to 0.95).
-
function \(f_F^C (\widehat{x})\) identifies the objects belonging to the “Caucasian” class among the objects classified as “Female”;
-
function \(f_F^{AB} (\widehat{x})\) identifies the objects belonging to the “Asian” class among the objects classified as “Female”; the rest of the objects are classified as “Black”;
-
function \(f_M^C (\widehat{x})\) identifies the objects belonging to the “Caucasian” class among the objects classified as “Male”;
-
function \(f_M^{AB} (\widehat{x})\) identifies the objects belonging to the “Asian” class among the objects classified as “Male”; the rest of the objects are classified as “Black”.
-
the cumulative features are generated \(\widehat{\varvec{A}}^{l_p }\) based on boosted features \(\widehat{\varvec{X}}^{l_p }\) of the training set for each combination of gender (“Male”, “Female”) and race (“Caucasian”, “Asian”, “Black”) groups \(p; p=(1, 6);\)
-
the cumulative features \(\widehat{\varvec{A}}^{l_p}\) are used for regression analysis by SVR on the whole training set of the gender-race group p with a predefined value of \(\varepsilon \)-sensitivity \(\varepsilon _0^p \), i.e. a decision function is generated \(f_0^p ({\hat{a}}^{p},\varepsilon _0^p )\) (in general, the sensitivity \(\varepsilon \) of the loss function may depend on the gender-race group p) for a preliminary estimation of age (stage 3.1, global regression) for each gender-race group p, \(p=(1, 6).\) As a result, six decision functions are generated, each capable of calculating an approximate value of the “age” attribute for each object with the attributes of “gender” and “race” determined at the previous stages using the boosted features of this object. A bootstrapping procedure is used on the entire training set for generating decision functions \(f_0^p (\widehat{a}^{p},\varepsilon _0^p )\) for each gender-race group p;
-
for each age value \(y^{p}\) in each gender-race group p an age range \(({y^{p}-d^{p-}({y^{p}}),y^{p}+d^{p+}({y^{p}})})\) is selected with values of \(d^{p-}({y^{p}})\) and \(d^{p+}({y^{p}})\) (in general, the breadth of the range may depend on the gender-race group p), that are defined by the expert on the basis of the specific features of the problem being solved; within each range belonging to value \(y^{p}\), for the corresponding subset of precedents (\(x_i^p \), \(y_i^p)\): \(y_i^p \in ( {y^{p}-d^{p-}({y^{p}}),y^{p}+d^{p+}({y^{p}})})\); \(p=(1,6);i = (1,n^{p})\). \(n^{p}\) is the number of precedents of the training set in the gender-race group p, SVR is used for regression analysis (stage 3.2, local regression) where the sensitivity of the loss function \(\varepsilon ^{p}(y^{p})\) depends on the value of \(y^{p}\) and is also defined by the expert based on the features of the problem being solved (in general, the sensitivity \(\varepsilon \) of the loss function may depend on the gender-race group p). As a result, a set of decision functions \(f_y^p [\widehat{a}^{p},\varepsilon ^{p}({y^{p}}),d^{p+}({y^{p}} ),d^{p-}({y^{p}})]\) is generated for the purpose of estimating the refined age values for each gender-race group p and age \(y^{p}\): \(p=(1, 6)\), \(y^{p}\in (y_{\min },y_{\max })\). A bootstrapping procedure is used for generating a decision function \(f_y^p [\widehat{a}^{p},\varepsilon ^{p}({y^{p}}),d^{p+}( {y^{p}}),d^{p-}({y^{p}})]\) for each gender-race group p in every age range \((y^{p}-d^{p-}({y^{p}}),y^{p}+d^{p+}({y^{p}}))\).
Attribute | Contents |
---|---|
Gender | Images of women and men of different races and ages |
Race | Images of men of different races and ages |
Images of women of different races and ages | |
Approximate value of age (global regression) | Images of Caucasian men of different ages |
Images of Caucasian women of different ages | |
Images of Asian men of different ages | |
Images of Asian women of different ages | |
Images of Black men of different ages | |
Images of Black women of different ages | |
Refined value of the age attribute (local regression) | Images of Caucasian men in the age range \((y_1-d_1^{-} ,y_{1} +d_1^{+} )\)
|
. . . | |
Images of Caucasian men in the age range \((y_{r}-d_r^{-},y_{r}+d_r^{+} )\)
| |
Images of Caucasian women in the age range \((y_{1}-d_1^- ,y_1 +d_1^{+} )\)
| |
. . . | |
Images of Caucasian women in the age range \((y_{r} -d_r^{-} ,y_r +d_r^{+})\)
| |
Images of Asian men in the age range \((y_1 -d_1^{-},y_1 +d_1^+)\)
| |
. . . | |
. . . | |
Images of Asian women in the age range \((y_{r}-d_r^{-} ,y_r +d_r^+ )\)
|
Experimental results
No. | Base of images | Gender | Race | Age |
---|---|---|---|---|
1 | FERET |
\(+\)
| ||
2 | MORPH |
\(+\)
|
\(+\)
| |
3 | FG-NET |
\(+\)
|
\(+\)
| |
4 | Mall |
\(+\)
|
\(+\)
| |
5 | LFW |
\(+\)
|
\(+\)
|
\(+\)
|
Men | Women | Total number of images | |
---|---|---|---|
Caucasian | 50,210 | 42,419 | 92,629 |
Asian | 22,740 | 19,682 | 42,422 |
Black | 18,045 | 16,533 | 34,578 |
Total number of images | 90,995 | 78,634 | 169,629 |
Algorithm | SVM | SVM | SVM \(+\) bootstrapping |
---|---|---|---|
Features | LBP | Boosted LBP | Boosted LBP |
Accuracy | |||
FERET (%) | 86 | 86 | 94 |
MORPH (%) | 87 | 87 | 95 |
BigSample (%) | 79 | 79 | 92 |
Algorithm | SVM | SVM \(+\) bootstrapping | SVM \(+\) bootstrapping in a selected gender group |
---|---|---|---|
Features | LBP | Boosted LBP | Boosted LBP |
Accuracy | |||
Mall (%) | 81 | 90 | 96 |
BigSample (%) | 74 | 85 | 89 |
Algorithm | Two stages SVR | Two stages SVR \(+\) “floating” limits \(+\) bootstrapping | Two stages SVR \(+\) “floating” limits \(+\) bootstrapping within a predefined gender-race group | Age classification after gender-race classification |
---|---|---|---|---|
Features | LBP | Boosted LBP | Boosted LBP | Boosted LBP |
MAE | ||||
FG-NET | 4.65 | 4.43 | 4.30 | 4.31 |
MORPH | 4.61 | 4.42 | 4.28 | 4.29 |
BigSample | 8.07 | 6.72 | 6.08 | 6.09 |
-
the accuracy of gender classification using a conventional SVM based on LBP on the BigSample image database virtually coincides with the published results [5, 22] obtained from actual images from the Internet (the difference in accuracy is lower than 1%); a slightly more accurate gender classification on the FERET and MORTH databases is due to the fact that these databases, unlike BigSample, contain specially prepared images;
-
the transition to boosted LBP features does not decrease the accuracy of gender and race classification either in the case of specially prepared databases of face images (FERET, MORPH, Mall), or in the case of real face images from the Internet (BigSample);
-
the application of a bootstrapping procedure leads to an increase in accuracy of gender and race classification both for databases of specially prepared face images and for databases of real face images from the Internet by \(\sim \)12% in comparison to the basic method.
-
the simultaneous application of boosted LBP features, a bootstrapping procedure and sequential classification of race after the classification of gender lead to an increase in the accuracy of race classification by \(\sim \)15% in comparison to the basic method.
-
the implementation of the proposed modification of the two-stage regression scheme for age classification in combination with the use of boosted image description features leads to an increase in accuracy of estimation of this attribute in real face images by about 2 years using the MAE metric, based on Internet face image analysis.
-
the higher accuracy of age estimation in face images from FG-NET and MORPH databases is explained by the use of specially prepared face images in these databases
-
a slight decrease in the accuracy of age estimation with the use of the proposed method is explained by the fact that sometimes the classifier commits errors in determining the gender or race; this leads to the use of wrong decision function trained on a “foreign” gender-race group at the age estimation stage.