Image skin segmentation based on multi-agent learning Bayesian and neural network

https://doi.org/10.1016/j.engappai.2014.03.002Get rights and content

Abstract

Skin colour is considered to be a useful and discriminating spatial feature for many skin detection-related applications, but it is not sufficiently robust to address complex image environments because of light-changing conditions, skin-like colours and reflective glass or water. These factors can create major difficulties in face pixel-based skin detectors when the colour feature is used. Thus, this paper proposes a multi-agent learning method that combines the Bayesian method with a grouping histogram (GH) technique and the back-propagation neural network with a segment adjacent-nested (SAN) technique based on the YCbCr and RGB colour spaces, respectively, to improve skin detection performance. The findings from this study have shown that the proposed multi-agent learning for skin detector has produced significant true positive (TP) and true negative (TN) average rates (i.e. 98.44% and 99.86% respectively). In addition, it has achieved a significantly lower average rate for the false negative (FN) and false positive (FP) (i.e. only 1.56% and 0.14% respectively). The experimental results show that multi-agent learning in the skin detector is more efficient than other approaches.

Introduction

Skin detection is one of the most important techniques in image processing and is the most distinctive and widely used approach for many applications (Wadud et al., 2008, Mohamed et al., 2008), such as face detection (Dargham et al., 2009), human motion analysis (Peer et al., 2009), pornographic image filters (Daxiang et al., 2013), and blocking objectionable content (Ottinger et al., 2009). Correspondingly, applications such as content-aware video compression and image colour balancing can also benefit from automatic skin detection in images. Skin detection is used to determine the image pixels that are related to human skin. Colour is a useful cue for extracting skin pixels, and choosing a skin colour as an element in detecting human presence is a relatively simple and straightforward task; in addition, skin colour has a processing time advantage, given that colour processing is faster compared to other processing of other features (Taqa and Jalab, 2010b); nevertheless, using colour information for skin detection is challenging because the appearance of skin in images is affected by various factors, such as the illumination, background, camera characteristics, and reflections from glass or water (Liu et al., 2011, Shoyaib et al., 2012, Khan et al., 2012, Mohair et al., 2012). Many techniques have been presented in the literature for skin detection using colour (Bhoyar and Kakde, 2010). Skin detection is the process of finding skin-coloured pixels and regions in an image or a video (Elgammal et al., 2009). This process is typically used as a pre-processing step to finding regions that potentially have human faces and limbs in the images. A skin colour detector typically transforms a given pixel into an appropriate colour space and then uses a skin classifier to label the pixel, whether it is a skin or a non-skin pixel (Berbar, 2011). A skin classifier defines a decision boundary of the skin colour class in the colour space based on a training database of skin-coloured pixels. The main goal of a skin colour detection algorithm is to build a decision rule that will discriminate between skin and non-skin pixels (Zolfaghari et al., 2011). This goal is accomplished by introducing a metric that measures the distance between pixel colours in relation to the skin tone. This metric type is defined by the skin colour modelling method. Different methods for discriminating between skin and non-skin pixels are available in the literature (Vezhnevets et al., 2003, Kakumanu et al., 2007). However, it was only in the last 15 years that the value of skin colour segmentation has been fully realised in various computer vision applications (Naji et al., 2012). These applications can be grouped into three types of skin modelling, namely, explicitly defined skin regions, non-parametric skin, and parametric skin approaches.

The first category uses explicit rules based on colour values. The explicit skin cluster classifier experimentally defines the boundaries of the skin cluster in a certain colour space (Vezhnevets et al., 2003). These methods use explicit rules that are based on colour values, which is an approach that many studies have taken (Kovac et al., 2003). These methods are very modest to implement, and the simplicity of the skin detection rules leads to a quick and inexpensive construction of a classifier. However, the main difficulty of achieving high recognition rates with this method is the need to find empirically both good colour space and adequate decision rules. Recently, a method that uses machine learning (ML) algorithms to find a suitable colour space and a simple decision rule and thus achieves high recognition rates was proposed (Sisodia and Verma, 2011).

The second category uses non-parametric models. These modelling methods are better suited for constructing classifiers in the cases of limited training and expected target datasets (Shoyaib et al., 2012). The generalisation and interpolation ability of these methods facilitate the construction of classifiers that have acceptable performance using incomplete training data (Surendiran and Alagarsamy, 2010, Paul et al., 2011). These methods are expressed with a small number of parameters and need very little storage space; however, it requires more time to compute the skin probability model in comparison to non-parametric methods. Thus, they can be very slow; a mixture of Gaussians in both training and work, as well as in their performance, depends strongly on the skin distribution shape. Moreover, most of the parametric skin modelling methods ignores the non-skin colour statistics. This aspect, together with the dependence on the skin cluster shapes, results in higher false positive rates compared to non-parametric methods (Vezhnevets et al., 2003). Neural Networks are considered to be the most suitable method to use in parametric skin modelling compared with other methods (Bhoyar and Kakde, 2010, Doukim et al., 2010, Wylie, 2010, Zolfaghari et al., 2011).

The third category uses parametric models for skin colour distributions. The key idea of non-parametric skin modelling methods is to estimate skin colour distributions from the training data without deriving an explicit model of the skin colour. The results of these methods are sometimes referred to as the skin probability map, which assigns a probability value to each point in a discrete colour space (Brand and Mason, 2000, Gomez, 2002). Vezhnevets et al. (2003) and Kakumanu et al. (2007) demonstrated three clear advantages of non-parametric methods: fast training, classification, and usage. Their performance depends heavily on the training set selection and is theoretically independent of the shape of the skin distribution, which is not the case for explicit skin cluster definitions and parametric skin modelling. These methods usually have high true positive and low false positive rates, which indicates that they can find most of the skin regions when the number of non-skin pixels marked as skin pixels is low (Kakumanu et al., 2007). The disadvantages of these methods are that they need a very large amount of storage space and cannot interpolate or generalise the training data. The storage of the (skin|RGB) table, also known as the lookup table (LUT), requires a large amount of memory (Vezhnevets et al., 2003). A solution to this problem is to use a smaller colour space, such as a colour space with 323 colours instead of 2563 colours (Phung et al., 2005). A Naïve Bayesian classifier, however, is considered to be the most suitable method to be used in non-parametric skin modelling compared with the others (Zhanyu and Leijon, 2010, Cao and Liu, 2011, Liu et al., 2011).

This paper proposes a multi-agent learning system that combines the Bayesian method with a grouping histogram technique and the back-propagation neural network with a segment adjacent-nested technique based on YCbCr and RGB colour spaces, to improve the skin detection performance.

The remaining sections are organised as follows. Section 2 covers the related research and objectives of the present study. The proposed system design is described in Section 3. In Section 4, the experiments conducted for the proposed skin detector are described. An evaluation and discussion of the results are provided in Section 5, followed by the research limitations, the research contributions, the future works and conclusions in 6 Research limitations, 7 Research contributions, 8 Potential future works, 9 Conclusions, respectively.

Section snippets

Literature review

A number of skin colour segmentation approaches have been proposed based on neural networks or Naïve Bayesian classifiers (Bhoyar and Kakde, 2010).

Duan et al. (2009) proposed a method of human skin region detection that is based on a pulse-coupled neural network (PCNN). The original input image was translated from RGB colour space to YIQ colour space, and then, a channelled image is obtained. Elgammal et al. (2009) used the Bayesian approach for skin detection based on an RGB colour space.

Proposed system design

In developing a reliable skin detector that has a high detection rate, a number of issues pertaining to skin must be resolved. These issues include a skin-like, lighting condition and reflection in relation to water and glass. In attempting to resolve these issues, we proposed a system that works and that has two machine learning methods. Multi-agent learning is used to extract the skin regions from the image. The skin detector then goes through a number of processes, as described in Fig. 1.

The

Proposed system implementation

The multi-agent learning system for skin detection involves three phases. These phases are the pre-processing phase, the training phase and the testing phase, as described in 4.1 Pre-processing phase, 4.2 Training phase, 4.3 Detection phase.

Evaluation and discussion of the results

An evaluation and discussion of the results of the proposed skin detector will be presented in this section; it will focus on the ability of the proposed techniques to resolve the problems mentioned in Section 1. This section includes a discussion of the results related to the multi-agent learning system for skin detection. Then, we provide a comparison of the test results between the proposed multi-agent system and among other existing standards in the skin detection application, followed by a

Research limitations

The multi-agent learning system for skin detection is a combination between a Bayesian method with a grouping histogram technique based on the YCbCr colour space and the back-propagation neural network with a segment adjacent-nested technique based on RGB colour space; this system was implemented. The proposed module considers skin-like colour pixels, the lighting conditions, and water and glass reflection problems. The simulation of a skin detector system that is discussed in this paper,

Research contributions

The contributions of the proposed skin detector are summarised in the following points:

  • Proposed a novel technique called grouping histogram based on the YCbCr colour space. This approach is the pre-processing technique used for the Bayesian method.

  • Proposed a novel technique called adjacent nested segment based on the RGB colour space. This approach is the pre-processing technique used for the back-propagation neural network method.

  • Proposed a novel multi-agent method that combines the Bayesian

Potential future works

  • Skin segmentation plays a large role in human computer interaction (HCI) based on many different applications. Therefore, the proposed skin detector can be used in these applications. For example, this method can be used in the analysis of gestures, the tracking of human faces and the tracing of human emotions. Other image processing applications that are related to humans in computer vision and multimedia, such as the filtering of web contents, retrieving multimedia databases, video

Conclusions

This paper proposes a skin detector that uses a hybrid method involving the technique of a grouping histogram for the Bayesian method and a back-propagation neural network with an adjacent segment-nested technique. These techniques are based on two colour spaces, namely, YCbCr and RGB, respectively. The results produced by this hybrid method are significant because a number of goals have been obtained; as a result, we have the following conclusions:

  • The grouping histogram technique with the

References (38)

  • J.A. Dargham et al.

    Data fusion for skin detection

    Artif. Life Robot.

    (2009)
  • L. Daxiang et al.

    Pornographic images filtering using PMK-based MIL algorithm

    Int. J. Comput. Sci.

    (2013)
  • Demuth, H., Beale, M., Hagan, M., 2005. Neural network toolbox 5. The MathWorks products. U.S. Patents....
  • C.A. Doukim et al.

    Combining neural networks for skin detection

    Int. J. Signal Image Process.

    (2010)
  • L. Duan et al.

    A method of human skin region detection based on PCNN

    Lecture Notes in Computer Science

    (2009)
  • A. Elgammal et al.

    Skin detection—a short tutorial

    (2009)
  • P.A. Flach et al.

    Naïve Bayesian classification of structured data

    Mach. Learn.

    (2004)
  • G. Gomez

    On selecting color components for skin detection

    IEEE Int. Conf. Pattern Recognit.

    (2002)
  • J. Kovac et al.

    Human skin color clustering for face detection

    IEEE Int. Conf. Comput. Tool

    (2003)
  • Cited by (25)

    • Helping doctors hasten COVID-19 treatment: Towards a rescue framework for the transfusion of best convalescent plasma to the most critical patients based on biological requirements via ml and novel MCDM methods

      2020, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      Section 3.2.2 discusses the prioritisation of patients with COVID-19 and CPs by using the MCDM approach. The classification processes involve testing and development phases [75–79]. In the testing phase, donors with patient repository in the development phase are classified (Fig. 2).

    • Systematic review of artificial intelligence techniques in the detection and classification of COVID-19 medical images in terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodological aspects

      2020, Journal of Infection and Public Health
      Citation Excerpt :

      As mentioned in Section “Dataset and pre-processing”, the dataset is divided into training and testing sets. However, each individual instance is supposed to belong to a predefined class [18,97,102,103]. In the testing portion, if the classification technique performance looks ‘acceptable’, then the classification technique can be used to classify future data for which the class label is unknown.

    • Comprehensive insights into evaluation and benchmarking of real-time skin detectors: Review, open issues & challenges, and recommended solutions

      2018, Measurement: Journal of the International Measurement Confederation
      Citation Excerpt :

      Several studies have argued that this relationship is based on the complex background of the algorithm, and other researchers have claimed that high-quality and accurate skin detection relies on the computation features of the processor. Chang et al. [19], Kawulok et al. [56], and Zaidan et al. [149] proposed three criteria for skin detection and explored the influence of each criterion on the output. The present study provides a brief review of the basic criteria for computing reliability (i.e., reliability group), training and testing (i.e., dataset group), and time complexity to evaluate and benchmark real-time skin detection techniques [91].

    • Human skin detection through correlation rules between the YCb and YCr subspaces based on dynamic color clustering

      2017, Computer Vision and Image Understanding
      Citation Excerpt :

      In Al-Mohair et al. (2012), a survey of artificial neural networks or semi-parametric methods is presented. In Zaidan et al. (2014) a skin detection method based on a multi-agent learning Bayesian classifier and a neural network in the RGB and YCbCr color spaces is presented. Concerning the rule-based approaches, many studies operate in the HSV and YCbCr color spaces, some take into account the luminance component (see Dominguez et al., 2006; Tsekeridou and Pitas, 1998 for the HSV space and Brancati et al. (2016); Hsu et al. (2002); Hwang et al. (2013) for the YCbCr space), while others ignore the luminance component (see Dardas and Georganas, 2011; Guo et al., 2012; Sobottka and Pitas, 1998 for the HSV space and Alsoos and Joukhadar (2014); Basilio et al. (2011); Bilal et al. (2015); Chai and Ngan (1999); Khan et al. (2008); Liensberger et al. (2009) for the YCbCr space).

    • Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm

      2015, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      Artificial neural networks (ANNs) have risen from 1960s as a novel way to mimic the human brain. The learning ability of ANNs makes them a powerful tool for various applications, such as classification (Zhang, 2000; Cantu-Paz and Kamath, 2005; Castellani and Rowlands, 2009; Rivero et al., 2010; Castellani, 2013), clustering (Du, 2010), vision (Weimer et al., 2013; Zaidan et al., 2014), control systems (Li et al., 2014; Czajkowski et al., 2014), prediction (Muttil and Chau, 2006; Chen et al., 2008; Chen and Wang, 2010; Taormina et al., 2012; Weizhong, 2012; Khashei and Bijari, 2012; Coban, 2013), and many others (Rabuñal and Dorado, 2005). Designing the network architecture and training it are the most important problems with exploiting ANNs.

    View all citing articles on Scopus
    View full text