Skip to main content
main-content

Über dieses Buch

This book contains some selected papers from the International Conference on Extreme Learning Machine 2015, which was held in Hangzhou, China, December 15-17, 2015. This conference brought together researchers and engineers to share and exchange R&D experience on both theoretical studies and practical applications of the Extreme Learning Machine (ELM) technique and brain learning.

This book covers theories, algorithms ad applications of ELM. It gives readers a glance of the most recent advances of ELM.

Inhaltsverzeichnis

Frontmatter

Large-Scale Scene Recognition Based on Extreme Learning Machines

For intelligent robots, scene recognition aims to find a semantic explanation of a scene, i.e., it helps the robots to know where they are. It can be widely applied into various robotic tasks, e.g, topological localization, simultaneous localization and mapping and autonomous navigation. Many of existing methods for scene recognition focused on how to build scene features, such as holistic representations and bags of visual words. However, less attention is put on the classification. Due to the huge number of scene classes in the real world, the variances within each class and the shared features between classes, the classification becomes a challenging issue for scene recognition. This paper proposes an ensemble method for large-scale scene recognition. This proposed method builds a three-level hierarchy for recognizing 397 classes of scenes in the real world. At each level, an ensemble-based classifier is built by using 13 types of features. Extreme learning machine is employed as the basic classifier in each ensemble-based classifier. Experimental results have shown that this proposed method outperforms other state-of-the-art methods in terms of recognition accuracy.

Yuanlong Yu, Lingying Wu, Kai Sun, Jason Gu

Partially Connected ELM for Fast and Effective Scene Classification

Scene classification is often solved as a machine learning problem, where a classifier is first learned from training data, and class labels are then assigned to unlabelled testing data based on the outputs of the classifier. Generally, image descriptors are represented in high-dimensional space, where classifiers such as support vector machine (SVM) show good performance. However, SVM classifiers demand high computational power during model training. Extreme learning machine (ELM), whose synaptic weight matrix from the input layer to the hidden layer are randomly generated, has demonstrated superior computational efficiency. But the weights thus generated may not yield enough discriminative power for hidden layer nodes. Our recent study shows that the random mapping from the input layer to the hidden layer in ELM can be replaced by semi-random projection (SRP) to achieve a good balance between computational complexity and discriminative power of the hidden nodes. The application of SRP to ELM yields the so-called partially connected ELM (PC-ELM) algorithm. In this study, we apply PC-ELM to multi-class scene classification. Experimental results show that PC-ELM outperforms ELM in high-dimensional feature space at the cost of slightly higher computational complexity.

Dongzhe Wang, Rui Zhao, Kezhi Mao

Two-Layer Extreme Learning Machine for Dimension Reduction

The extreme learning machine (ELM), which was originally proposed for “generalized” single-hidden layer feedforward neural networks (SLFNs), provides efficient unified learning solutions for the applications of regression and classification. It presents competitive accuracy with superb efficiency in many applications. However, due to its single-layer architecture, feature selection using ELM may not be effective for natural signals. To address this issue, this paper proposes a new ELM-based multi-layer learning framework for dimension reduction. The novelties of this work are as follows: (1) Unlike the existing multi-layer ELM methods in which all hidden nodes are generated randomly, in this paper some hidden layers are calculated by replacement technologies. By doing so, more important information can be exploited for feature learning, which lead to a better generalization performance. (2) Unlike the existing multi-layer ELM methods which only work for sparse representation, the proposed method is designed for dimension reduction. Experimental results on several classification datsets show that, compared to other feature selection methods, the proposed method performs competitively or much better than other feature selection methods with fast learning speed.

Yimin Yang, Q. M. Jonathan Wu

Distributed Extreme Learning Machine with Alternating Direction Method of Multiplier

Extreme learning machine, as a generalized single-hidden-layer feedforward networks has achieved much attention for its extremely fast learning speed and good generalization performance. However, big data often makes a challenge in large scale learning of ELM due to the limitation of memory of single machine as well as the distributed manner of large scale data storage and collection in many applications. For the purpose of relieving the limitation of memory with big data, in this paper, we exploit a novel distributed extreme learning machine to implement the extreme learning machine algorithm in parallel for large-scale data set. A corresponding distributed algorithm is also developed on the basis of alternating direction method of multipliers which shows effectiveness in distributed convex optimization. Finally, some numerical experiments on well-known benchmark data sets are carried out to illustrate the effectiveness of the proposed DELM method and provide an analysis on the performance of speedup, scaleup and sizeup.

Minnan Luo, Qinghua Zheng, Jun Liu

An Adaptive Online Sequential Extreme Learning Machine for Real-Time Tidal Level Prediction

An adaptive variable-structure online sequential extreme learning machine (OS-ELM) is proposed by incorporating a hidden nodes pruning strategy. As conventional OS-ELM increases network dimensionality by adding newly-received data samples, the resulted dimension would expand dramatically and result in phenomenon of “dimensionality curse” finally. As the measurement samples may come endlessly, there is a practical need to adjust the dimension of OS-ELM not only by adding hidden units but also by pruning superfluous units simultaneously. To evaluate the contribution of existing hidden units and locate the superfluous units, an index is implemented referred to as normalized error reduction ratio. As the OS-ELM adds new samples in hidden units, those existing units contribute less to current dynamics would be deleted from network, thus the resulted parsimonious network can represent current system dynamics more efficiently. This online dimension adjustment approach can handle samples which are presented one-by-one or chuck-by-chuck with variable chuck size. The adaptive variable-structure OS-ELM was implemented for online tidal level prediction purpose. To evaluate the efficiency of the adaptive variable structure OS-ELM, tidal prediction simulations was conducted based on the actual measured tidal data and meteorological data of Old Port Tampa in the United States. Simulation results reveal that the proposed variable-structure OS-ELM demonstrates its effectiveness in short term tidal predictions in respect of accuracy and rapidness.

Jianchuan Yin, Lianbo Li, Yuchi Cao, Jian Zhao

Optimization of Outsourcing ELM Problems in Cloud Computing from Multi-parties

In this letter, we introduce a secure and practical multi-parties cooperating mechanism of outsourcing extreme learning machines (ELM) in Cloud Computing. This outsourcing mechanism enables original ELM to perform over large-scale dataset in which multi-parties are involved. We propose a optimized partition policy in Cloud Computing to significantly improve the training speed and dramatically reduce the communication overhead. According to the partition policy, cloud servers are mainly responsible for calculating the inverse of an intermediate matrix derived from the hidden layer output matrix, which is the heaviest computation. Although most of the computation is outsourced in Cloud Computing, the confidentiality of the input/output is assured because the randomness of the hidden layer is fully exploited. Theoretical analysis and experiments have shown that the proposed multi-parties cooperating mechanism for outsourcing ELM can effectively release customers from heavy computations.

Jiarun Lin, Tianhang Liu, Zhiping Cai, Xinwang Liu, Jianping Yin

H-MRST: A Novel Framework for Support Uncertain Data Range Query Using ELM

Probabilistic range query is a typical problem in the domain of probabilistic database management system. There exist many efforts for supporting such query. However, the state of arts approaches can not efficiently index uncertain data when their probability density function are discrete. In this paper, we propose a general framework to construct summary for uncertain data with any type of PDF. Especially, if the PDF of uncertain data is discrete, we employ a novel machine learning technique named ELM to learn its distribution type and fit the specific function. If this method does not work, we propose a hybrid algorithm to construct its summary. Besides the hybrid summary construction algorithm, we propose a bitwise-based accessing algorithm to speed up the query. Theoretical analysis and extensive experimental results demonstrate the effectiveness of the proposed algorithms.

Bin Wang, Rui Zhu, Guoren Wang

The SVM-ELM Model Based on Particle Swarm Optimization

Extreme learning machine (ELM) is a simple and effective SLFNs single hidden layer feedforward neural network learning algorithm, in recent years, it has become one of the hot areas in machine learning research. But single hidden layer node lacks of judgement ability, to some extent the classification accuracy depends on the number of hidden layer nodes. In order to improve the judgement ability of single hidden layer node, Support Vector Machine (SVM) is combined with ELM, and a simplified SVM-ELM model is established. At the same time, in order to avoid the subjectivity of human to choose parameters, the SVM-ELM model uses Particle Swarm Optimization (PSO) algorithm to automatically select the parameters, finally PSO-SVM-ELM model is proposed. Experiments show that classification accuracy of the model is higher than the SVM-ELM and ELM, and it also has good robustness and adaptive generation ability.

Miao-miao Wang, Shi-fei Ding

ELM-ML: Study on Multi-label Classification Using Extreme Learning Machine

Extreme learning machine (ELM) techniques have received considerable attention in computational intelligence and machine learning communities, because of the significantly low computational time. ELM provides solutions to regression, clustering, binary classification, multiclass classifications and so on, but not to multi-label learning. A thresholding method based ELM is proposed in this paper to adapted ELM for multi-label classification, called extreme learning machine for multi-label classification (ELM-ML). In comparison with other multi-label classification methods, ELM-ML outperforms them in several standard data sets in most cases, especially for applications which only have small labeled data set.

Xia Sun, Jiarong Wang, Changmeng Jiang, Jingting Xu, Jun Feng, Su-Shing Chen, Feijuan He

Sentiment Analysis of Chinese Micro Blog Based on DNN and ELM and Vector Space Model

Analysis of Chinese micro blog has great commercial value and social value. Based on the depth analysis of the language style of the Chinese micro blog, this paper makes a deep research on the sentiment analysis of Chinese microblog based on and DNN ELM and vector space model. First of all, the micro blog Abstract sentiment feature extraction technology is studied in depth. Based on the traditional text representation model, DNN algorithm is used to extract the feature of the abstract emotion. Combined with the characteristics of the short text of micro blog, this paper uses SAE to construct the DNN. In the construction process of vector space, in order to fully and effectively said microblogging text emotional information, in this paper, we introduce the emotional factor and structure factor of information gain feature selection method is improved and introduced the location information of the feature words of TF-IDF weighting calculation method to improve. Then, the sentiment classification of micro blog is deeply studied. In this paper, we use the concept model to express the emotion category of micro blog, and propose the spatial expansion algorithm based on the concept model (ESA). The experimental results show that the presented in this paper, based on DNN microblogging Abstract emotional feature extraction algorithm and the algorithm of conceptual model of spatial development based on the microblogging text emotional other identification is effective.

Huilin Liu, Shan Li, Chunfeng Jiang, He Liu

Self Forward and Information Dissemination Prediction Research in SINA Microblog Using ELM

With the popularity of social network, information propagation prediction based on social network is also becoming popular. As far as we know, people do not concern the user who forwards its own microblog in information propagation prediction. In our investigation the self forward behavior can cause the further spreading of the information. Thus in this paper we propose a self forward prediction model to predict the self forward behavior. We use ELM to train and predict self forward behavior. Based on this model we proposed an algorithm to predict the information dissemination. The experiment results show that our algorithm is real and effective and it significantly improves the forecast accuracy. It also can be seen in the experimental results that the results of ELM has a better performance than SVM.

Huilin Liu, Yao Li, He Liu

Sparse Coding Extreme Learning Machine for Classification

As one of supervised learning algorithms, extreme learning machine (ELM) has been proposed for single-hidden-layer feedforward neural networks (SLFN) and shown great generalization performance. ELM randomly assigns the weights and biases between the input and hidden layers and trains the weights between hidden and output layers. Physiological research has shown that neurons at the same layer are laterally inhibited to each other such that the output of each layer is a type of sparse codings. However, it is difficult to accommodate the lateral inhibition by directly using random feature mapping in ELM. Therefore, this paper proposes a sparse coding ELM (ScELM) algorithm, which can map the input feature vector into a sparse representation such that the mapped feature is sparse. In this proposed ScELM algorithm, an unsupervised way is used for sparse coding in the sense that dictionary is randomly assigned rather than learned. Gradient projection (GP) based method is used for the sparse coding. The output weights are trained in the same supervised way which ELM presents. Experimental results on benchmark databases have shown that this proposed ScELM algorithm can outperform other state-of-the art methods in terms of classification accuracy.

Zhenzhen Sun, Yuanlong Yu

Continuous Top-K Remarkable Comments over Textual Streaming Data Using ELM

The increasing popularity of location-based social networks encourages more and more users to share their experience. It deeply impact the decision of the other users. In this paper, we study the problem of top-K remarkable comments over textual streaming data. We first study how to efficiently identify the mendacious comments. Through using a novel machine learning technique named ELM, we could filter most of mendacious comments. We then study how to maintain these vital comments. For one thing, we propose a two-level index to maintain their position information. For another, we employ domination transitivity to remove meaningless comments. Theoretical analysis and extensive experimental results demonstrate the effectiveness of the proposed algorithms.

Rui Zhu, Bin Wang, Guoren Wang

ELM Based Representational Learning for Fault Diagnosis of Wind Turbine Equipment

The data preprocessing, feature extraction, classifier training and testing play as the key components in a typical fault diagnosis system. This paper proposes a new application of extreme learning machines (ELM) in an integrated manner, where multiple ELM layers play correspondingly different roles in the fault diagnosis framework. The ELM based representational learning framework integrates functions including data preprocessing, feature extraction and dimension reduction. In the novel framework, an ELM based autoencoder is trained to get a hidden layer output weight matrix, which is then used to transform the input data into a new feature representation. Finally, a single layered ELM is applied for fault classification. Compared with existing feature extraction methods, the output weight matrix is treated as the mapping result with weighted distribution of input vector. It avoids wiping off “insignificant” feature information that may convey some undiscovered knowledge. The proposed representational learning framework does not need parameters fine-tuning with iterations. Therefore, the training speed is much faster than the traditional back propagation-based DL or support vector machine method. The experimental tests are carried out on a wind turbine generator simulator, which demonstrates the advantages of this method in both speed and accuracy.

Zhixin Yang, Xianbo Wang, Pak Kin Wong, Jianhua Zhong

Prediction of Pulp Concentration Using Extreme Learning Machine

Pulp concentration is one of the most important production parameters during ore dressing process. Generally, pulp concentration not only affects concentrate grade and recovery rate, but also has a major influence on the chemical and power consumptions during the flotation process. Recently, there has been a growing interest in the study of prediction for pulp concentration to improve the productivity and reduce consumption of various resources. Since the pulp concentration and other production parameters are nonlinearly related, it imposes very challenging obstacles to the prediction for this parameter. Because extreme learning machine (ELM) has the advantages of extremely fast learning speed, good generalization performance, and the smallest training errors, we employ ELM to predict pulp concentration in this paper. Pulp concentration data is first preprocessed using phase space reconstruction method. Then time series prediction model is adjusted from one dimension to multiple dimensions and thus it is established by several improved ELM algorithms, including traditional ELM, kernel-based ELM (Kernel-ELM), regularized ELM (R-ELM), and $$L_2$$L2-norm based ELM (ELM-L2). The experiments are conducted with a real-world production data set from a mine. The experimental results show the effectiveness of ELM-based prediction approaches, and we can also find that ELM-L2 has better prediction effects than other algorithms with the increase of sample size. Both training speed and prediction accuracy are improved by employing ELM-L2 to the prediction of pulp concentration.

Changwei Jiang, Xiong Luo, Xiaona Yang, Huan Wang, Dezheng Zhang

Rational and Self-adaptive Evolutionary Extreme Learning Machine for Electricity Price Forecast

Electricity price forecast is of great importance to electricity market participants. Given the sophisticated time-series of electricity price, various approaches of extreme learning machine (ELM) have been identified as effective prediction approaches. However, in high dimensional space, evolutionary extreme learning machine (E-ELM) is time-consuming and difficult to converge to optimal region when just relying on stochastic searching approaches. In the meanwhile, due to the complicated functional relationship, objective function of E-ELM seems difficult also to be mined directly for some useful mathematical information to guide the optimum exploring. This paper proposes a new differential evolution (DE) like algorithm to enhance E-ELM for more accurate and reliable prediction of electricity price. An approximation model for producing DE-like trail vector is the key mechanism, which can use simpler mathematical mapping to replace the original yet complicated functional relationship within a small region. Thus, the evolutionary procedure frequently dealt with some rational searching directions can make the E-ELM more robust and faster than supported only by the stochastic methods. Experimental results show that the new method can improve the performance of E-ELM more efficiently.

Chixin Xiao, Zhaoyang Dong, Yan Xu, Ke Meng, Xun Zhou, Xin Zhang

Contractive ML-ELM for Invariance Robust Feature Extraction

Extreme Learning Machine (ELM), a single hidden layer feedforward neural networks works efficiently in many areas such as machine learning, pattern recognition, natural language processing, et al. due to its powerful universal approximation capability and classification capability. This paper uses multiply layer ELM (ML-ELM) which stacks many ELMs based on Auto Encoder (ELM-AE) as main framework. ELM-AE lets the input data as output data and chooses orthogonal random weights and random biases of the hidden nodes to perform unsupervised learning. To extract more invariance robust feature, we propose Contractive ML-ELM (C-ML-ELM referring to the work of Rifai et al.). Contractive ML-ELM applys a penalty term corresponding to the Frobenius norm of the Jacobian matrix of the encoder activations with respect to the input in each layer of ML-ELM. Experiment results show that Contractive ML-ELM achieves state of the art classification error on Mnist dataset.

Xibin Jia, Hua Du

Automated Human Facial Expression Recognition Using Extreme Learning Machines

Facial expressions form a vital component of our daily interpersonal communication. The automation of the recognition of facial expressions has been studied in depth and experiments have been performed to recognize the six basic facial expressions as defined by Paul Ekman. The Facial Action Coding System (FACS) defines Action Units, which are movements in muscle groups on the face. Combinations of Action Units yield expressions. In this paper, we propose an approach to perform automated facial expression recognition involving two stages. Stage one involves training Extreme Learning Machines (ELMs) to recognize Action Units present in a face (one ELM per AU), using Local Binary Patterns as features. Stage two deduces the expression based on the set of Action Units present.

Abhilasha Ravichander, Supriya Vijay, Varshini Ramaseshan, S. Natarajan

Multi-modal Deep Extreme Learning Machine for Robotic Grasping Recognition

Learning rich representations efficiently plays an important role in multi-modal recognition task, which is crucial to achieve high generalization performance. To address this problem, in this paper, we propose an effective Multi-Modal Deep Extreme Learning Machine (MM-DELM) structure, while maintaining ELM’s advantages of training efficiency. In this structure, unsupervised hierarchical ELM is conducted for feature extraction for all modalities separately. Then, the shared layer is developed by combining these features from all of modalities. Finally, the Extreme Learning Machine (ELM) is used as supervised feature classifier for final decision. Experimental validation on Cornell grasping dataset illustrates that the proposed multiple modality fusion method achieves better grasp recognition performance.

Jie Wei, Huaping Liu, Gaowei Yan, Fuchun Sun

Denoising Deep Extreme Learning Machines for Sparse Representation

In last decade, a large number of research has focused on the sparse representation for signal. As a dictionary learning algorithm, K-SVD, is introduced to efficiently learn an redundant dictionary from a set of training signals. In the mean time, there is an interesting technique named extreme learning machines (ELM), which is an single-layer feed-forward neural networks (SLFNs) with a fast learning speed, good generalization and universal classification capability. In this paper, we propose an denoising deep extreme learning machines based on autoencoder (DDELM-AE) for sparse representation. It makes the conventional K-SVD algorithm perform better. Finally, we show the experimental rusults on our optimized method and the typical K-SVD algorithm.

Xiangyi Cheng, Huaping Liu, Xinying Xu, Fuchun Sun

Extreme Learning Machine Based Point-of-Interest Recommendation in Location-Based Social Networks

Researches on Point-of-Interests (POIs) have attracted a lot of attentions in Location-based Social Networks (LBSNs) in recent years. Existing studies on this topic most treat this kind of recommendation as just a type of point recommendation according to its similar properties for collaborative filtering. We argue that this recommending strategy could yield inaccuracy because these properties could not illustrate complete information of POIs for users. In this paper, we propose a novel Extreme Learning Machine (ELM) based approach named ELM Based POI Recommendation (EPR), which takes into account user preference, periodical movement and social relationship to discover the correlation of a user and a certain POI. Furthermore, we model recommendation in EPR as the problem of binary-class classification for each individual user and POI pair. To our best knowledge, this is the first work on POI recommendation in LBSNs by exploring the preference property, social property and periodicity property simultaneously. We show through comprehensive evaluation that the proposed approach delivers excellent performance and outperforms existing state-of-the-art POI recommendation methods, especially for cold start users.

Mo Chen, Feng Li, Ge Yu, Dan Yang

The Granule-Based Interval Forecast for Wind Speed

With the increasing penetration of wind power in modern power systems, sound challenges have emerged for system operators due to the uncertain nature of wind power. Deterministic point forecasting has become less effective to power system operators in terms of information accuracy and reliability. Unlike the conventional methods, a granule-based interval forecasting approach is proposed in this paper, which effectively considers the uncertainties involved in the original time series and regression models, other than only generating a plausible yet less reliable value. By incorporating Extreme Learning Machine (ELM) into the granular model construction, a specific interval can be simply obtained by granular outputs at extremely fast speed. Case studies based on 1-min wind speed time series demonstrate the feasibility of this approach.

Songjian Chai, Youwei Jia, Zhao Xu, Zhaoyang Dong

KELMC: An Improved K-Means Clustering Method Using Extreme Learning Machine

As a critical step for unsupervised learning, clustering is widely used in scientific data analysis and engineering systems. However, the shortage of categories information makes clustering an inconvenient issue. As an efficient and effective supervised learning algorithm, Extreme Learning Machines (ELMs) can be also adaptive for clustering tasks by constructing class labels properly. In this paper, we present a new clustering algorithm, K-means ELM Clustering (KELMC), which uses the output of an extreme learning machine instead of the similarity metrics in k-means. Extreme learning machine in KELMC is trained from potential cluster centers with its categories artificially labeled. For further improvement, we tried KELMC on an ELM-AE-PCA feature space and proposed another algorithm called EP-KELMC. Empirical study on UCI data sets demonstrates that the proposed algorithms are competitive with the state-of-the-art clustering algorithms.

Lijuan Duan, Bin Yuan, Song Cui, Jun Miao, Wentao Zhu

Wind Power Ramp Events Classification Using Extreme Learning Machines

Wind power is becoming increasingly popular as a renewable source of energy. Being a non-dispatchable energy resource, wind power facilities entail efficient forecast mechanisms to estimate the production of various wind power utilities available. In an integrated grid system, a balance must be maintained between production and consumption. Given that wind power is directly affected by meteorological factors (wind speed etc.) accurately predicting such fluctuations becomes extremely important. These events of fluctuation are termed as ramp events. Forecast of wind power is important but accurate prediction of ramp events is much more crucial to the safety of the grid as well as the security and reliability of the grid. In this paper we employ the ELM (Extreme Learning Machine) technique on wind power data of 2012 Alberta, Canada market for different sampling times to predict wind power ramp events. We also try to compare it with respect to other existing standard algorithms of feed-forward Neural Networks to analyze the efficacy of the technique in the area. ELM is shown to outperform other techniques in terms of computation time whereas prediction performance is at par with other neural network algorithms.

Sujay Choubey, Anubhav Barsaiyan, Nitin Anand Shrivastava, Bijaya Ketan Panigrahi, Meng-Hiot Lim

Facial Expression Recognition Based on Ensemble Extreme Learning Machine with Eye Movements Information

Facial expression recognition has become a very active research in computer vision, behavior interpretation of emotions, human computer interaction, cognitive science and intelligent control. Traditional facial expression analysis methods mainly focuses on the facial muscle movement and basic expression features of face image. In this paper, we propose a novel method for facial expression recognition based on ensemble extreme learning machine with eye movements information. Here, the eye movements information is regarded as explicit clue to improve the performance of facial expression recognition. Firstly, we extract eye movements features from eye movements information which recorded by Tobii eye tracker. The histogram of orientation gradient (HOG) features are simultaneously obtained from the face images by dividing it into a number of small cells. Secondly, we combine the eye movements features together with the HOG features of face images by using a tensor kernel. Finally, the fusion features are trained by ensemble extreme learning machine and a bagging algorithm is explored for producing the results. Extensive experiment on the two widely available datasets of facial expressions demonstrate that our proposal effectively improves the accuracy and efficiency of face expression recognition and achieve performance at extremely high speed.

Bo Lu, Xiaodong Duan, Ye Yuan

Correlation Between Extreme Learning Machine and Entorhinal Hippocampal System

In recent years there has been a considerable interest in exploring the nature of learning and memory system among artificial intelligence researchers and neuroscientists about the neural mechanisms, simulation and enhancement. While a number of studies have investigated the artificial neural networks inspired by biological learning and memory systems, for example the extreme learning machine and support vector machine, seldom research exists examining and comparing the recording neural data and these neural networks. Therefore, the purpose of this exploratory qualitative study is to investigate the extreme learning machine proposed by Huang as a novel method to analyze and explain the biological learning process in the entorhinal hippocampal system, which is thought to play an important role in animal learning, memory and spatial navigation. Data collected from multiunit recordings of different rat hippocampal regions in multiple behavioral tasks was used to analyze the relationship between the extreme learning machine and the biological learning. The results demonstrated that there was a correlation between the biological learning and the extreme learning machine which can contribute to a better understanding of biological learning mechanism.

Lijuan Su, Min Yao, Nenggan Zheng, Zhaohui Wu

RNA Secondary Structure Prediction Using Extreme Learning Machine with Clustering Under-Sampling Technique

This paper gives a machine learning method for the subject of RNA secondary structure prediction. The method is based on extreme learning machine for its outstanding performance in classification problem, and use under-sampling technique to solve the problem of data imbalance. Feature vector in the classifier includes covariation score and inconsistent sequence penalty. The proposed method is compared with SVM and ELM without under-sampling, as well as classical method RNAalifold in terms of sensitivity, specificity, Matthews correlation coefficient and G-mean. The training and testing data are 68 RNA aligned families from Rfam, version 11.0. The results show that the proposed method can achieve highest scores in sensitivity, MCC and G-mean, which means that it is an effective method for RNA secondary structure prediction.

Tianhang Liu, Jiarun Lin, Chengkun Wu, Jianping Yin

Multi-instance Multi-label Learning by Extreme Learning Machine

Multi-instance Multi-label learning is a learning framework, where every object is represented by a bag of instances and associated with multiple labels simultaneously. The existing degeneration strategy based methods often suffer from some common drawbacks: (1) the user-specific parameter for the number of clusters may incur the effective problem; (2) utilizing SVM as the classifiers builder may bring the high computational cost. In this paper, we propose an algorithm, namely MIML-ELM, to address the problems. To our best knowledge, we are the first utilizing ELM in MIML problem and conducting the comparison of ELM and SVM on MIML. Extensive experiments are conducted on the real datasets and the synthetic datasets. The results show that MIML-ELM tends to achieve better generalization performance at a higher learning speed.

Chenguang Li, Ying Yin, Yuhai Zhao, Guang Chen, Libo Qin

A Randomly Weighted Gabor Network for Visual-Thermal Infrared Face Recognition

In this paper, a novel three-layer Gabor-based network is proposed for heterogeneous face recognition. The input layer of our proposed network consists of pixel-wise image patches. At the hidden layer, a set of Gabor features are extracted by a projection operation and a magnitude function. Subsequently, a non-linear activation function is utilized after weighting the extracted Gabor features with random weight vectors. Finally, the output weights are deterministically learned similarly to that in extreme learning machine. Some experimental results on private BERC visual-thermal infrared database are observed and discussed. The proposed method shows promising results based on the average test recognition accuracy.

Beom-Seok Oh, Kangrok Oh, Andrew Beng Jin Teoh, Zhiping Lin, Kar-Ann Toh

Dynamic Adjustment of Hidden Layer Structure for Convex Incremental Extreme Learning Machine

Extreme Learning Machine (ELM) is a learning algorithm based on generalized single-hidden-layer feed-forward neural network. Since ELM has an excellent performance on regression and classification problems, it has been paid more and more attention recently. The determination of structure of ELM plays a vital role in ELM applications. Essentially, determination of the structure of ELM is equivalent to the determination of the hidden layer structure. Utilizing a smaller scale of the hidden layer structure can promote faster running speed. In this paper, we propose algorithm PCI-ELM (Pruned-Convex Incremental Extreme Learning Machine) based on CI-ELM (Convex Incremental Extreme Learning Machine). Furthermore, we also present an improved PCI-ELM algorithm, EPCI-ELM (Enhanced Pruned-Convex Incremental Extreme Learning Machine), which introduces a filtering strategy for PCI-ELM during the neurons adding process. In order to adjust the single-hidden-layer feed-forward neural network more flexibly and achieve the most compact form of the hidden layer structure, in this paper, we propose a algorithm which can dynamically determine hidden layer structure, DCI-ELM (Dynamic Convex Incremental Extreme Learning Machine). At the end of this paper, we verify the performance of PCI-ELM, EPCI-ELM and DCI-ELM. The results show that PCI-ELM, EPCI-ELM and DCI-ELM control hidden layer structure very well and construct the more compact single-hidden-layer feed-forward neural network.

Yongjiao Sun, Yuangen Chen, Ye Yuan, Guoren Wang

ELMVIS+: Improved Nonlinear Visualization Technique Using Cosine Distance and Extreme Learning Machines

This paper presents ELMVIS+, a significant improvement in ELMVIS methodology that enables faster computation, more stable results and a wider application range. The novel cost function and a fast way of estimating it speeds up the method compared to ELMVIS, especially in large-dimensional datasets. The included Genetic Algorithms add global optimization that helps ELMVIS+ to find a better optimum. The improved methodology shows state-of-the-art performance in three different benchmark datasets.

Anton Akusok, Yoan Miche, Kaj-Mikael Björk, Rui Nian, Paula Lauren, Amaury Lendasse

On Mutual Information over Non-Euclidean Spaces, Data Mining and Data Privacy Levels

In this paper, we propose a framework for measuring the impact of data privacy techniques, in information theoretic and in data mining terms. The need for data privacy and anonymization is often hampered by the fact that the privacy functions alter the data in non-measurable amounts and details. We propose here to use Mutual Information over non-Euclidean spaces as a means of measuring this distortion. In addition, and following the same principle, we also propose to use Machine Learning techniques in order to quantify the impact of the data obfuscation in terms of further data mining goals.

Yoan Miche, Ian Oliver, Silke Holtmanns, Anton Akusok, Amaury Lendasse, Kaj-Mikael Björk

Probabilistic Methods for Multiclass Classification Problems

In this paper, two approaches for probability-based class prediction are presented. In the first approach, the output of Extreme Learning Machines algorithm is used as an input for Gaussian Mixture models. In this case, ELM performs as dimensionality reduction technique. The second approach is based on ELM and a newly proposed Histogram Probability method. Detailed description and analysis of these methods are presented. To evaluate these methods five datasets from UCI Machine Learning Repository are used.

Andrey Gritsenko, Emil Eirola, Daniel Schupp, Edward Ratner, Amaury Lendasse

A Pruning Ensemble Model of Extreme Learning Machine with L 1/2 Regularizer

Extreme learning machine (ELM) as an emerging branch of machine learning has shownits good generalization performance at a fast learning speed. Nevertheless, the preliminary ELM and other evolutional versions based on ELM cannot provide the optimal solution of parameters between the hidden and output layer and cannot determine the suitable number of hidden nodes automatically. In this paper, a pruning ensemble model of ELM with L1/2 regularizer (PE-ELMR) is proposed to solve above problems. It involves two stages. First, we replace the original solving method of the output parameter in ELM to a minimum squared-error problem with sparse solution by combining ELM with L1/2 regularizer. In addition, L1/2 regularizerguarantees the sparse solution with less computational cost. Second, in order to get the required minimum number for good performance, we prune the nodes in hidden layer with the ensemble model, which reflects the superiority in searching the reasonable hidden nodes. Experimental results present the performance of L1 and L1/2 regularizers used in our model PE-ELMR, compared with ELM and OP-ELM, for regression and classification problems under a variety of benchmark datasets.

Bo He, Tingting Sun, Tianhong Yan, Yue Shen, Rui Nian

Evaluating Confidence Intervals for ELM Predictions

This paper proposes a way of providing more useful and interpretable results for ELM models by adding confidence intervals to predictions. Unlike a usual statistical approach with Mean Squared Error (MSE) that evaluates an average performance of an ELM model over the whole dataset, the proposed method computed particular confidence intervals for each data sample. A confidence for each particular sample makes ELM predictions more intuitive to interpret, and an ELM model more applicable in practice under task-specific requirements. The method shows good results on both toy and a real skin segmentation datasets. On a toy dataset, the predicted confidence intervals accurately represent a variable magnitude noise. On a real dataset, classification with a confidence interval improves the precision at the cost of recall.

Anton Akusok, Yoan Miche, Kaj-Mikael Björk, Rui Nian, Paula Lauren, Amaury Lendasse

Real-Time Driver Fatigue Detection Based on ELM

Driver fatigue is a serious road safety issue that results in thousands of road crashes every year. Image-based fatigue monitoring is one of the most important methods of avoiding fatigue-related accidents. In this paper, a vision-based real-time driver fatigue detection system based on ELM is proposed. The system has three main stages. The first stage performs facial features localization and tracking, by using the Viola–Jones face detector and the KLT algorithm. The second stage is the judgement of facial and fatigue status, applying twice ELM with an extremely fast learning speed. The last one is online learning, which can continuously improve ELM accuracy according to the user’s feedback. Multiple facial features (including the movement of eyes, head and mouth) are used to comprehensively assess the driver vigilance state. In comparison to backpropagation (BP), the experimental results showed that applying ELM has a better performance with much faster training speed.

Hengyu Liu, Tiancheng Zhang, Haibin Xie, Hongbiao Chen, Fangfang Li

A High Speed Multi-label Classifier Based on Extreme Learning Machines

In this paper a high speed neural network classifier based on extreme learning machines for multi-label classification problem is proposed and discussed. Multi-label classification is a superset of traditional binary and multi-class classification problems. The proposed work extends the extreme learning machine technique to adapt to the multi-label problems. As opposed to the single-label problem, both the number of labels the sample belongs to, and each of those target labels are to be identified for multi-label classification resulting in increased complexity. The proposed high speed multi-label classifier is applied to six benchmark datasets comprising of different application areas such as multimedia, text and biology. The training time and testing time of the classifier are compared with those of the state-of-the-arts methods. Experimental studies show that for all the six datasets, our proposed technique have faster execution speed and better performance, thereby outperforming all the existing multi-label classification methods.

Meng Joo Er, Rajasekar Venkatesan, Ning Wang

Image Super-Resolution by PSOSEN of Local Receptive Fields Based Extreme Learning Machine

Image super-resolution aims at generating high-resolution images from low-resolution inputs. In this paper, we propose a novel learning-based and efficient image super-resolution approach called particle swarm optimization based selective ensemble (PSOSEN) of local receptive fields based extreme learning machine (ELM-LRF). ELM-LRF is locally connected ELM, which can directly process information including strong correlations such as images. PSOSEN is a selective ensemble used to optimize the output of ELM-LRF. This method constructs an end-to-end mapping of which the input is a single low-resolution image and the output is a high resolution image. Experiments show that our method is better in terms of accuracy and speed with different magnification factors compared to the state-of-the-art methods.

Yan Song, Bo He, Yue Shen, Rui Nian, Tianhong Yan

Sparse Extreme Learning Machine for Regression

Extreme learning machine (ELM) solves regression and classification problems efficiently. However, the solution provided is dense and requires plenty of storage space and testing time. A sparse ELM has been proposed for classification in [1]. However, it is not applicable for regression problems. In this paper, we propose a sparse ELM for regression, which significantly reduces the storage space and testing time. In addition, we develop an efficient training algorithm based on iterative computation, which scales quadratically with regard to the number of training samples. Therefore, the proposed sparse ELM is advantageous over other ELM methods when facing large data sets for achieving faster training and testing speed, while requiring less storage space. In addition, sparse ELM outperforms support vector regression (SVR) in the aspects of generalization performance, training speed and testing speed.

Zuo Bai, Guang-Bin Huang, Danwei Wang

WELM: Extreme Learning Machine with Wavelet Dynamic Co-Movement Analysis in High-Dimensional Time Series

In this paper, we propose a fast and efficient learning approach called WELM based on Extreme Learning Machine and 3-D Wavelet Dynamic Co-Movement Analysis to enhance the speed and precision of big data prediction. 3-D Wavelet Dynamic Co-Movement Analysis is firstly employed to transform optimization problems from an original higher-dimensional space to a new lower-dimensional space while preserving the optimum of the original function, and then ELM is utilized to train and forecast the whole process. WELM model is used in the volatility of time series prediction. The forecasts obtained by WELM has been compared with ELM, PCA-ELM, ICA-ELM, KPCA-ELM, SVM and GARCH type models in terms of closeness to the realized volatility. The computational results demonstrate that the WELM provides better time series forecasts and it shows the excellent performance in the accuracy and efficiency.

Heng-Guo Zhang, Rui Nian, Yan Song, Yang Liu, Xuefei Liu, Amaury Lendasse

Imbalanced Extreme Learning Machine for Classification with Imbalanced Data Distributions

Due to its much faster speed and better generalization performance, extreme learning machine (ELM) has attracted many attentions as an effective learning approach. However, ELM rarely involves strategies for imbalanced data distributions which may exist in many fields. In this paper, we will propose a novel imbalanced extreme learning machine (Im-ELM) algorithm for binary classification problems, which is applicable to the cases with both balanced and imbalanced data distributions, by addressing the classification errors for each class in the performance index, and determining the design parameters through a two-stage heuristic search method. Detailed performance comparison for Im-ELM is done based on a number of benchmark datasets for binary classification. The results show that Im-ELM can achieve better performance for classification problems with imbalanced data distributions.

Wendong Xiao, Jie Zhang, Yanjiao Li, Weidong Yang

Backmatter

Weitere Informationen

Premium Partner

BranchenIndex Online

Die B2B-Firmensuche für Industrie und Wirtschaft: Kostenfrei in Firmenprofilen nach Lieferanten, Herstellern, Dienstleistern und Händlern recherchieren.

Whitepaper

- ANZEIGE -

Best Practices für die Mitarbeiter-Partizipation in der Produktentwicklung

Unternehmen haben das Innovationspotenzial der eigenen Mitarbeiter auch außerhalb der F&E-Abteilung erkannt. Viele Initiativen zur Partizipation scheitern in der Praxis jedoch häufig. Lesen Sie hier  - basierend auf einer qualitativ-explorativen Expertenstudie - mehr über die wesentlichen Problemfelder der mitarbeiterzentrierten Produktentwicklung und profitieren Sie von konkreten Handlungsempfehlungen aus der Praxis.
Jetzt gratis downloaden!

Bildnachweise