Neuron Pruning-Based Discriminative Extreme Learning Machine for Pattern Classification

Extreme learning machine (ELM), as a newly developed learning paradigm for the generalized single hidden layer feedforward neural networks, has been widely studied due to its unique characteristics, i.e., fast training, good generalization, and universal approximation/classification ability. A novel framework of discriminative extreme learning machine (DELM) is developed for pattern classification. In DELM, the margins between different classes are enlarged as much as possible through a technique called ε-dragging. DELM is further extended to pruning DELM (P-DELM) using L_2,1-norm regularization. The performance of DELM is compared with several state-of-the-art methods on public face databases. The simulation results show the effectiveness of DELM for face recognition when there are posture, facial expression, and illumination variations. P-DELM can distinguish the importance of different hidden neurons and remove the worthless ones. The model can achieve promising performance with fewer hidden neurons and less prediction time on several benchmark datasets. In DELM model, the margins between different classes are enlarged by learning a nonnegative label relaxation matrix. The experiments validate the effectiveness of DELM. Furthermore, DELM is extended to P-DELM based on L_2,1-norm regularization. The developed P-DELM can naturally distinguish the importance of different hidden neurons, which will lead to a more compact network by neuron pruning. Experimental validations on some benchmark datasets show the advantages of the proposed P-DELM method.