Skip to main content

Über dieses Buch

This book includes the proceedings of the International Conference on Artificial Neural Networks (ICANN 2006) held on September 10-14, 2006 in Athens, Greece, with tutorials being presented on September 10, the main conference taking place during September 11-13 and accompanying workshops on perception, cognition and interaction held on September 14, 2006. The ICANN conference is organized annually by the European Neural Network Society in cooperation with the International Neural Network Society, the Japanese Neural Network Society and the IEEE Computational Intelligence Society. It is the premier European event covering all topics concerned with neural networks and related areas. The ICANN series of conferences was initiated in 1991 and soon became the major European gathering for experts in these fields. In 2006 the ICANN Conference was organized by the Intelligent Systems Laboratory and the Image, Video and Multimedia Systems Laboratory of the National Technical University of Athens in Athens, Greece. From 475 papers submitted to the conference, the International Program Committee selected, following a thorough peer-review process, 208 papers for publication and presentation to 21 regular and 10 special sessions. The quality of the papers received was in general very high; as a consequence, it was not possible to accept and include in the conference program many papers of good quality.



Neural Networks, Semantic Web Technologies and Multimedia Analysis (Special Session)

The Core Method: Connectionist Model Generation

Knowledge based artificial networks networks have been applied quite successfully to propositional knowledge representation and reasoning tasks. However, as soon as these tasks are extended to structured objects and structure-sensitive processes it is not obvious at all how neural symbolic systems should look like such that they are truly connectionist and allow for a declarative reading at the same time. The core method aims at such an integration. It is a method for connectionist model generation using recurrent networks with feed-forward core. After an introduction to the core method, this paper will focus on possible connectionist representations of structured objects and their use in structure-sensitive reasoning tasks.

Sebastian Bader, Steffen Hölldobler

A Neural Scheme for Robust Detection of Transparent Logos in TV Programs

In this paper, we present a connectionist approach for detecting and precisely localizing transparent logos in TV programs. Our system automatically synthesizes simple problem-specific feature extractors from a training set of logo images, without making any assumptions or using any hand-made design concerning the features to extract or the areas of the logo pattern to analyze. We present in detail the design of our architecture, our learning strategy and the resulting process of logo detection. We also provide experimental results to illustrate the robustness of our approach, that does not require any local preprocessing and leads to a straightforward real time implementation.

Stefan Duffner, Christophe Garcia

A Neural Network to Retrieve Images from Text Queries

This work presents a neural network for the retrieval of images from text queries. The proposed network is composed of two main modules: the first one extracts a global picture representation from local block descriptors while the second one aims at solving the retrieval problem from the extracted representation. Both modules are trained jointly to minimize a loss related to the retrieval performance. This approach is shown to be advantageous when compared to previous models relying on unsupervised feature extraction: average precision over


queries reaches 26.2% for our model, which should be compared to 21.6% for PAMIR, the best alternative.

David Grangier, Samy Bengio

Techniques for Still Image Scene Classification and Object Detection

In this paper we consider the interaction between different semantic levels in still image scene classification and object detection problems. We present a method where a neural method is used to produce a tentative higher-level semantic scene representation from low-level statistical visual features in a bottom-up fashion. This emergent representation is then used to refine the lower-level object detection results. We evaluate the proposed method with data from Pascal VOC Challenge 2006 image classification and object detection competition. The proposed techniques for exploiting global classification results are found to significantly improve the accuracy of local object detection.

Ville Viitaniemi, Jorma Laaksonen

Adaptation of Weighted Fuzzy Programs

Fuzzy logic programs are a useful framework for handling uncertainty in logic programming; nevertheless, there is the need for modelling adaptation of fuzzy logic programs. In this paper, we first overview weighted fuzzy programs, which bring fuzzy logic programs and connectionist models closer together by associating significance weights with the atoms of a logic rule: by exploiting the existence of weights, it is possible to construct a neural network model that reflects the structure of a weighted fuzzy program. Based on this model, we then introduce the weighted fuzzy program adaptation problem and propose an algorithm for adapting the weights of the rules of the program to fit a given dataset.

Alexandros Chortaras, Giorgos Stamou, Andreas Stafylopatis

Classified Ranking of Semantic Content Filtered Output Using Self-organizing Neural Networks

Cosmos-7 is an application that can create and filter MPEG-7 semantic content models with regards to objects and events, both spatially and temporally. The results are presented as numerous video segments that are all relevant to the user’s consumption criteria. These results are not ranked to the user’s ranking of relevancy, which means the user must now laboriously sift through them. Using self organizing networks we rank the segments to the user’s preferences by applying the knowledge gained from similar users’ experience and use content similarity for new segments to derive a relative ranking.

Marios Angelides, Anastasis Sofokleous, Minaz Parmar

Classifier Fusion: Combination Methods For Semantic Indexing in Video Content

Classifier combination has been investigated as a new research field to improve recognition reliability by taking into account the complementarity between classifiers, in particular for automatic semantic-based video content indexing and retrieval. Many combination schemes have been proposed in the literature according to the type of information provided by each classifier as well as their training and adaptation abilities. This paper presents an overview of current research in classifier combination and a comparative study of a number of combination methods. A novel training technique called Weighted Ten Folding based on Ten Folding principle is proposed for combining classifier. Experiments are conducted in the framework of the TRECVID 2005 features extraction task that consists in ordering shots with respect to their relevance to a given class. Finally, we show the efficiency of different combination methods.

Rachid Benmokhtar, Benoit Huet

Bridging the Semantic Gap in Multimedia Machine Learning Approaches (Special Session)

Retrieval of Multimedia Objects by Combining Semantic Information from Visual and Textual Descriptors

We propose a method of content-based multimedia retrieval of objects with visual, aural and textual properties. In our method, training examples of objects belonging to a specific semantic class are associated with their low-level visual descriptors (such as MPEG-7) and textual features such as frequencies of significant keywords. A fuzzy mapping of a semantic class in the training set to a class of similar objects in the test set is created by using Self-Organizing Maps (SOMs) trained from automatically extracted low-level descriptors. We have performed several experiments with different textual features to evaluate the potential of our approach in bridging the gap from visual features to semantic concepts by the use textual presentations. Our initial results show a promising increase in retrieval performance.

Mats Sjöberg, Jorma Laaksonen, Matti Pöllä, Timo Honkela

A Relevance Feedback Approach for Content Based Image Retrieval Using Gaussian Mixture Models

In this paper a new relevance feedback (RF) methodology for content based image retrieval (CBIR) is presented. This methodology is based on Gaussian Mixture (GM) models for images. According to this methodology, the GM model of the query is updated in a probabilistic manner based on the GM models of the relevant images, whose relevance degree (positive or negative) is provided by the user. This methodology uses a recently proposed distance metric between probability density functions (pdfs) that can be computed in closed form for GM models. The proposed RF methodology takes advantage of the structure of this metric and proposes a method to update it very efficiently based on the GM models of the relevant and irrelevant images characterized by the user. We show with experiments the merits of the proposed methodology.

Apostolos Marakakis, Nikolaos Galatsanos, Aristidis Likas, Andreas Stafylopatis

Video Representation and Retrieval Using Spatio-temporal Descriptors and Region Relations

This paper describes a novel methodology for video summarization and representation. The video shots are processed in space-time as 3D volumes of pixels. Pixel regions with consistent color and motion properties are extracted from these 3D volumes by a space-time segmentation technique based on a novel machine learning algorithm. Each region is then described by a high-dimensional point whose components represent the average position, motion velocity and color of the region. Subsequently, the spatio-temporal relations of the regions are deduced and a concise, graph-based description of them is generated. This graph-based description of the video shot’s content, along with the region centroids, comprises a concise yet powerful description of the video-shot and is used for retrieval applications. The retrieval problem is formulated as an inexact graph matching problem between the data video shots and the query input which is also a video segment. Experimental results on action recognition and video retrieval are illustrated and discussed.

Sotirios Chatzis, Anastasios Doulamis, Dimitrios Kosmopoulos, Theodora Varvarigou

Bridging the Syntactic and the Semantic Web Search

This paper proposes an information system, which aims to bridge the semantic gap in web search. The system uses multiple domain ontological structures expanding the user’s query with semantically related concepts, enhancing in parallel the quality of retrieval to a large extend. Query analyzers broaden the user’s information needs from classical term-based to conceptually representations, using knowledge from relevant ontologies and theirs’ properties. Besides the use of semantics, the system employs machine learning techniques from the field of swarm intelligence through the Ant Colony algorithm, where ants are considered as web agents capable of collecting and processing relevant information. Furthermore, the effectiveness of the approach is verified experimentally, by observing that the retrieval precision for the enhanced queries is in higher levels, in comparison with the results derived from the classical term-based retrieval procedure.

Georgios Kouzas, Ioannis Anagnostopoulos, Ilias Maglogiannis, Christos Anagnostopoulos

Content-Based Coin Retrieval Using Invariant Features and Self-organizing Maps

During the last years, Content-Based Image Retrieval (CBIR) has developed to an important research domain within the context of multimodal information retrieval. In the coin retrieval application dealt in this paper, the goal is to retrieve images of coins that are similar to a query coin based on features extracted from color or grayscale images. To assure improved performance at various scales, orientations or in the presence of noise, a set of global and local invariant features is proposed. Experimental results using a Euro coin database show that color moments as well as edge gradient shape features, computed at five concentric equal-area rings, compare favorably to wavelet features. Moreover, combinations of the above features using L1 or L2 similarity measures lead to excellent retrieval capabilities. Finally, color quantization of the database images using self-organizing maps not only leads to memory savings but also it is shown to even improve retrieval accuracy.

Nikolaos Vassilas, Christos Skourlas

Signal and Time Series Processing (I)

Learning Time-Series Similarity with a Neural Network by Combining Similarity Measures

Within this paper we present the approach of learning the non-linear combination of time-series similarity values through a neural network. A wide variety of time-series comparison methods, coefficients and criteria can be found in the literature that are all very specific, and hence apply only for a small fraction of applications. Instead of designing a new criteria we propose to combine the existing ones in an intelligent way by using a neural network. The approach aims to the goal of making the neural network to learn to compare the similarity between two time-series as a human would do. Therefore, we have implemented a set of comparison methods, the neural network and an extension to the learning rule to include a human as a teacher. First results are promising and show that the approach is valuable for learning human judged time-series similarity with a neural network.

Maria Sagrebin, Nils Goerke

Prediction Improvement via Smooth Component Analysis and Neural Network Mixing

In this paper we derive a novel smooth component analysis algorithm applied for prediction improvement. When many prediction models are tested we can treat their results as multivariate variable with the latent components having constructive or destructive impact on prediction results. The filtration of those destructive components and proper mixing of those constructive should improve final prediction results. The filtration process can be performed by neural networks with initial weights computed from smooth component analysis. The validity and high performance of our concept is presented on the real problem of energy load prediction.

Ryszard Szupiluk, Piotr Wojewnik, Tomasz Ząbkowski

Missing Value Estimation for DNA Microarrays with Mutliresolution Schemes

The expression pattern of a gene across time can be considered as a signal; a microarray experiment is collection of thousands of such signals where due to instrument failure, human errors and technology limitations, values at some time instances are usually missing. Furthermore, in some microarray experiments the gene signals are not sampled at regular time intervals, which renders the direct use of well established frequency-temporal signal analysis approaches such as the wavelet transform problematic. In this work we evaluate a novel multiresolution method, known as the lifting transform to estimate missing values in time series microarray data. Though the lifting transform has been developed to deal with irregularly spaced data its usefulness for the estimation of missing values in microarray data has not been examined in detail yet. In this framework we evaluate the lifting transform against the wavelet transform, a moving average method and a zero imputation on 5 data sets from the cell cycle and the sporulation of the

saccharomyces cerevisiae


Dimitrios Vogiatzis, Nicolas Tsapatsoulis

Applying REC Analysis to Ensembles of Sigma-Point Kalman Filters

The Sigma-Point Kalman Filters (SPKF) is a family of filters that achieve very good performance when applied to time series. Currently most researches involving time series forecasting use the Sigma-Point Kalman Filters, however they do not use an ensemble of them, which could achieve a better performance. The REC analysis is a powerful technique for visualization and comparison of regression models. The objective of this work is to advocate the use of REC curves in order to compare the SPKF and ensembles of them and select the best model to be used.

Aloísio Carlos de Pina, Gerson Zaverucha

Analysis of Fast Input Selection: Application in Time Series Prediction

In time series prediction, accuracy of predictions is often the primary goal. At the same time, however, it would be very desirable if we could give interpretation to the system under study. For this goal, we have devised a fast input selection algorithm to choose a parsimonious, or sparse set of input variables. The method is an algorithm in the spirit of backward selection used in conjunction with the resampling procedure. In this paper, our strategy is to select a sparse set of inputs using linear models and after that the selected inputs are also used in the non-linear prediction based on multi-layer perceptron networks. We compare the prediction accuracy of our parsimonious non-linear models with the linear models and the regularized non-linear perceptron networks. Furthermore, we quantify the importance of the individual input variables in the non-linear models using the partial derivatives. The experiments in a problem of electricity load prediction demonstrate that the fast input selection method yields accurate and parsimonious prediction models giving insight to the original problem.

Jarkko Tikka, Amaury Lendasse, Jaakko Hollmén

A Linguistic Approach to a Human-Consistent Summarization of Time Series Using a SOM Learned with a LVQ-Type Algorithm

The purpose of this paper is to propose a new, human consistent way to capture the very essence of a dynamic behavior of some sequences of numerical data. Instead of using traditional, notably statistical type analyses, we propose the use of fuzzy logic based linguistic summaries of data(bases) in the sense of Yager, later developed by Kacprzyk and Yager, and Kacprzyk, Yager and Zadrożny. Our main interest is in the summarization of trends characterized by: dynamics of change, duration and variability. To define the dynamic of change of the time series we propose to use for a preprocessing of data a SOM (self-organizing maps) learned with a LVQ (Learning Vector Quantization) algorithm, and then our approach for linguistic summaries of trends.

Janusz Kacprzyk, Anna Wilbik, Sławomir Zadrożny

Signal and Time Series Processing (II)

Long-Term Prediction of Time Series Using State-Space Models

State-space models offer a powerful modelling tool for time series prediction. However, as most algorithms are not optimized for long-term prediction, it may be hard to achieve good prediction results. In this paper, we investigate Gaussian linear regression filters for parameter estimation in state-space models and we propose new long-term prediction strategies. Experiments using the EM-algorithm for training of nonlinear state-space models show that significant improvements are possible with no additional computational cost.

Elia Liitiäinen, Amaury Lendasse

Time Series Prediction Using Fuzzy Wavelet Neural Network Model

The fuzzy wavelet neural network (FWNN) for time series prediction is presented in this paper. Using wavelets the fuzzy rules are constructed. The gradient algorithm is applied for learning parameters of fuzzy system. The application of FWNN for modelling and prediction of complex time series and prediction of electricity consumption is considered. Results of simulation of FWNN based prediction system is compared with the simulation results of other methodologies used for prediction. Simulation results demonstrate that FWNN based system can effectively learn complex nonlinear processes and has better performance than other models.

Rahib H. Abiyev

OFDM Channel Equalization Based on Radial Basis Function Networks

The universal approximation property makes neural networks very attractive for system modelling and identification. Channel estimation and equalization for digital communications are good examples. We explore the application of a Radial Basis Function Network to approximate the frequency response of a wireless channel, under the settings established by the IEEE 802.11 family of standards for wireless LAN architecture. We aim to exploit the channel impulse response correlation in the frequency domain to reduce the effect of noise. We obtain a smoother reconstructed function than by using a single tap Zero Forcing frequency domain equalizer. This is achieved by using a smaller number of basis functions, in the approximating Radial Basis Function Network, than the number of sub-carriers used by the OFDM modulation technique adopted in the transmission system. Although the training of the network following the Least Squares criterion requires the inversion of a matrix, this is feasible given the relatively small number of sub-carriers in the WLAN. Simulations show that the proposed algorithm behaves considerably better with respect to a simple single tap Zero Forcing algorithm, by reducing the bit error rate by more than a half. We also outline a possible solution based on the Kalman filter to update the network parameters adaptively and thus exploit any time correlation of the channel impulse response.

Giuseppina Moffa

A Quasi-stochastic Gradient Algorithm for Variance-Dependent Component Analysis

We discuss the blind source separation problem where the sources are not independent but are dependent only through their variances. Some estimation methods have been proposed on this line. However, most of them require some additional assumptions: a parametric model for their dependencies or a temporal structure of the sources, for example. In previous work, we have proposed a generalized least squares approach using fourth-order moments to the blind source separation problem in the general case where those additional assumptions do not hold. In this article, we develop a simple optimization algorithm for the least squares approach, or a quasi-stochastic gradient algorithm. The new algorithm is able to estimate variance-dependent components even when the number of variables is large and the number of moments is computationally prohibitive.

Aapo Hyvärinen, Shohei Shimizu

Two ICA Algorithms Applied to BSS in Non-destructive Vibratory Tests

Two independent component analysis (ICA) algorithms have been applied for blind source separation (BSS) in a synthetic, multi-sensor scenario, within a non-destructive pipeline test. The first one, CumICA, is based in the computation of the cross-cumulants of the mixed observed signals, and needs the aid of a digital high-pass filter to achieve the same SNR (up to -40 dB) as the second algorithm, Fast-ICA. Vibratory signals were acquired by a wide frequency range transducer (100-800 kHz) and digitalized by a 2.5 MHz, 8-bit ADC. Different types of commonly observed source signals are linearly mixed, involving acoustic emission (AE) sequences, impulses and other parasitic signals modelling human activity. Both ICA algorithms achieve to separate the impulse-like and the AE events, which often are associated to cracks or sudden non-stationary vibrations.

Juan-José González de-la-Rosa, Carlos G. Puntonet, R. Piotrkowski, I. Lloret, Juan-Manuel Górriz

Reference-Based Extraction of Phase Synchronous Components

Phase synchronisation is a phenomenon observed in measurements of dynamic systems, composed of several interacting oscillators. It can be quantified by the phase locking factor (


), which requires knowledge of the instantaneous phase of an observed signal. Linear sources separation methods treat scenarios in which measurements do not represent direct observations of the dynamics, but rather superpositions of underlying latent processes. Such a mixing process can cause spuriously high


s between the measurements, and camouflage the phase locking to a provided reference signal. The


between a linear projection of the data and a reference can be maximised as an optimisation criterion revealing the most synchronous source component present in the data, with its corresponding amplitude. This is possible despite the amplitude distributions being Gaussian, or the signals being statistically dependent, common assumptions in blind sources separation techniques without




in form of a reference signal.

Jan-Hendrik Schleimer, Ricardo Vigário

Data Analysis (I)

Analytic Solution of Hierarchical Variational Bayes in Linear Inverse Problem

In singular models, the Bayes estimation, commonly, has the advantage of the generalization performance over the maximum likelihood estimation, however, its accurate approximation using Markov chain Monte Carlo methods requires huge computational costs. The variational Bayes (VB) approach, a tractable alternative, has recently shown good performance in the automatic relevance determination model (ARD), a kind of hierarchical Bayesian learning, in brain current estimation from magnetoencephalography (MEG) data, an ill-posed linear inverse problem. On the other hand, it has been proved that, in three-layer linear neural networks (LNNs), the VB approach is asymptotically equivalent to a positive-part James-Stein type shrinkage estimation. In this paper, noting the similarity between the ARD in a linear problem and an LNN, we analyze a simplified version of the VB approach in the ARD. We discuss its relation to the shrinkage estimation and how ill-posedness affects learning. We also propose the algorithm that requires simpler computation than, and will provide similar performance to, the VB approach.

Shinichi Nakajima, Sumio Watanabe

Nonnegative Matrix Factorization for Motor Imagery EEG Classification

In this paper, we present a method of feature extraction for motor imagery single trial EEG classification, where we exploit nonnegative matrix factorization (NMF) to select discriminative features in the time-frequency representation of EEG. Experimental results with motor imagery EEG data in BCI competition 2003, show that the method indeed finds meaningful EEG features automatically, while some existing methods should undergo cross-validation to find them.

Hyekyoung Lee, Andrzej Cichocki, Seungjin Choi

Local Factor Analysis with Automatic Model Selection: A Comparative Study and Digits Recognition Application

A further investigation is made on an adaptive local factor analysis algorithm from Bayesian Ying-Yang (BYY) harmony learning, which makes parameter learning with automatic determination of both the component number and the factor number in each component. A comparative study has been conducted on simulated data sets and several real problem data sets. The algorithm has been compared with not only a recent approach called Incremental Mixture of Factor Analysers (IMoFA) but also the conventional two-stage implementation of maximum likelihood (ML) plus model selection, namely, using the EM algorithm for parameter learning on a series candidate models, and selecting one best candidate by AIC, CAIC, and BIC. Experiments have shown that IMoFA and ML-BIC outperform ML-AIC or ML-CAIC while the BYY harmony learning considerably outperforms IMoFA and ML-BIC. Furthermore, this BYY learning algorithm has been applied to the popular MNIST database for digits recognition with a promising performance.

Lei Shi, Lei Xu

Interpolating Support Information Granules

We develop a hybrid strategy combing thruth-functionality, kernel, support vectors and regression to construct highly informative regression curves. The idea is to use statistical methods to form a confidence region for the line and then exploit the structure of the sample data falling in this region for identifying the most fitting curve. The fitness function is related to the fuzziness of the sampled points and is regarded as a natural extension of the statistical criterion ruling the identification of the confidence region within the Algorithmic Inference approach. Its optimization on a non-linear curve passes through kernel methods implemented via a smart variant of support vector machine techniques. The performance of the approach is demonstrated for three well-known benchmarks.

B. Apolloni, S. Bassis, D. Malchiodi, W. Pedrycz

Feature Selection Based on Kernel Discriminant Analysis

For two-class problems we propose two feature selection criteria based on kernel discriminant analysis. The first one is the objective function of kernel discriminant analysis (KDA) and the second one is the KDA-based exception ratio. We show that the objective function of KDA is monotonic for the deletion of features, which ensures stable feature selection. The KDA-based exception ratio defines the overlap between classes in the one-dimensional space obtained by KDA. The computer experiments show that the both criteria work well to select features but the former is more stable.

Masamichi Ashihara, Shigeo Abe

Local Selection of Model Parameters in Probability Density Function Estimation

Here we present a novel probability density estimation model. The classical Parzen window approach builds a spherical Gaussian density around every input sample. Our proposal selects a Gaussian specifically tuned for each sample, with an automated estimation of the local intrinsic dimensionality of the embedded manifold and the local noise variance. This leads to outperform other proposals where local parameter selection is not allowed, like the manifold Parzen windows.

Ezequiel López-Rubio, Juan Miguel Ortiz-de-Lazcano-Lobato, Domingo López-Rodríguez, Enrique Mérida-Casermeiro, María del Carmen Vargas-González

The Sphere-Concatenate Method for Gaussian Process Canonical Correlation Analysis

We have recently developed several ways of using Gaussian Processes to perform Canonical Correlation Analysis. We review several of these methods, introduce a new way to perform Canonical Correlation Analysis with Gaussian Processes which involves sphering each data stream separately with probabilistic principal component analysis (PCA), concatenating the sphered data and re-performing probabilistic PCA. We also investigate the effect of sparsifying this last method. We perform a comparative study of these methods.

Pei Ling Lai, Gayle Leen, Colin Fyfe

Theory of a Probabilistic-Dependence Measure of Dissimilarity Among Multiple Clusters

We introduce novel dissimilarity to properly measure dissimilarity among multiple clusters when each cluster is characterized by a probability distribution. This measure of dissimilarity is called redundancy-based dissimilarity among probability distributions. From aspects of source coding, a statistical hypothesis test and a connection with Ward’s method, we shed light on the theoretical reasons that the redundancy-based dissimilarity among probability distributions is a reasonable measure of dissimilarity among clusters.

Kazunori Iwata, Akira Hayashi

Kernel PCA as a Visualization Tools for Clusters Identifications

Kernel PCA has been proven to be a powerful technique as a nonlinear feature extractor and a pre-processing step for classification algorithms. KPCA can also be considered as a visualization tool; by looking at the scatter plot of the projected data, we can distinguish the different clusters within the original data. We propose to use visualization given by KPCA in order to decide the number of clusters. K-means clustering algorithm on both data and projected space is then applied using synthetic and real datasets. The number of clusters discovered by the user is compared to the Davies-Bouldin index originally used as a way of deciding the number of clusters.

Alissar Nasser, Denis Hamad, Chaiban Nasr

Data Analysis (II)

A Fast Fixed-Point Algorithm for Two-Class Discriminative Feature Extraction

We propose a fast fixed-point algorithm to improve the Relevant Component Analysis (RCA) in two-class cases. Using an objective function that maximizes the predictive information, our method is able to extract more than one discriminative component of data for two-class problems, which cannot be accomplished by classical Fisher’s discriminant analysis. After prewhitening the data, we apply Newton’s optimization method which automatically chooses the learning rate in the iterative training of each component. The convergence of the iterative learning is quadratic, i.e. much faster than the linear optimization by gradient methods. Empirical tests presented in the paper show that feature extraction using the new method resembles RCA for low-dimensional ionosphere data and significantly outperforms the latter in efficiency for high-dimensional facial image data.

Zhirong Yang, Jorma Laaksonen

Feature Extraction with Weighted Samples Based on Independent Component Analysis

This study investigates a new method of feature extraction for classification problems with a considerable amount of outliers. The method is a weighted version of our previous work based on the independent component analysis (ICA). In our previous work, ICA was applied to feature extraction for classification problems by including class information in the training. The resulting features contain much information on the class labels producing good classification performances. However, in many real world classification problems, it is hard to get a clean dataset and inherently, there may exist outliers or dubious data to complicate the learning process resulting in higher rates of misclassification. In addition, it is not unusual to find the samples with the same inputs to have different class labels. In this paper, Parzen window is used to estimate the correctness of the class information of a sample and the resulting class information is used for feature extraction.

Nojun Kwak

Discriminant Analysis by a Neural Network with Mahalanobis Distance

We propose a neural network which can approximate Mahalanobis discriminant functions after being trained. It can be realized if a Bayesian neural network is equipped with two additional subnetworks. The training is performed sequentially and, hence, the past teacher signals need not be memorized. In this paper, we treat the two-category normal-distribution case. The results of simple simulations are included.

Yoshifusa Ito, Cidambi Srinivasan, Hiroyuki Izumi

Assessment of an Unsupervised Feature Selection Method for Generative Topographic Mapping

Feature selection (FS) has long been studied in classification and regression problems. In comparison, FS for unsupervised learning has received far less attention. For many real problems concerning unsupervised data clustering, FS becomes an issue of paramount importance. An unsupervised FS method for Gaussian Mixture Models, based on Feature Relevance Determination (FRD), was recently defined. Unfortunately, the data visualization capabilities of general mixture models are limited. Generative Topographic Mapping (GTM), a constrained mixture model, was originally defined to overcome such limitation. In this brief study, we test in some detail the capabilities of a recently described FRD method for GTM that allows the clustering results to be intuitively visualized and interpreted in terms of a reduced subset of selected relevant features.

Alfredo Vellido

A Model Selection Method Based on Bound of Learning Coefficient

To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIC are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.

Keisuke Yamazaki, Kenji Nagata, Sumio Watanabe, Klaus-Robert Müller

Pattern Recognition

Sequential Learning with LS-SVM for Large-Scale Data Sets

We present a subspace-based variant of LS-SVMs (i.e. regularization networks) that sequentially processes the data and is hence especially suited for online learning tasks. The algorithm works by selecting from the data set a small subset of basis functions that is subsequently used to approximate the full kernel on arbitrary points. This subset is identified online from the data stream. We improve upon existing approaches (esp. the kernel recursive least squares algorithm) by proposing a new, supervised criterion for the selection of the relevant basis functions that takes into account the approximation error incurred from approximating the kernel as well as the reduction of the cost in the original learning task. We use the large-scale data set ’forest’ to compare performance and efficiency of our algorithm with greedy batch selection of the basis functions via orthogonal least squares. Using the same number of basis functions we achieve comparable error rates at much lower costs (CPU-time and memory wise).

Tobias Jung, Daniel Polani

A Nearest Features Classifier Using a Self-organizing Map for Memory Base Evaluation

Memory base learning is one of main fields in the area of machine learning. We propose a new methodology for addressing the classification task that relies on the main idea of the k – nearest neighbors algorithm, which is the most important representative of this field. In the proposed approach, given an unclassified pattern, a set of neighboring patterns is found, but not necessarily using all input feature dimensions. Also, following the concept of the naïve Bayesian classifier, we adopt the hypothesis of the independence of input features in the outcome of the classification task. The two concepts are merged in an attempt to take advantage of their good performance features. In order to further improve the performance of our approach, we propose a novel weighting scheme of the memory base. Using the self-organizing maps model during the execution of the algorithm, dynamic weights of the memory base patterns are produced. Experimental results have shown superior performance of the proposed method in comparison with the aforementioned algorithms and their variations.

Christos Pateritsas, Andreas Stafylopatis

A Multisensor Fusion System for the Detection of Plant Viruses by Combining Artificial Neural Networks

Several researchers have shown that substantial improvements can be achieved in difficult pattern recognition problems by combining the outputs of multiple neural networks. In this work, we present and test a multi-net system for the detection of plant viruses, using biosensors. The system is based on the Bioelectric Recognition Assay (BERA) method for the detection of viruses, developed by our team. BERA sensors detect the electric response of culture cells suspended in a gel matrix, as a result to their interaction with virus’s cells, rendering thus feasible his identification. Currently this is achieved empirically by examining the biosensor’s response data curve. In this paper, we use a combination of specialized Artificial Neural Networks that are trained to recognize plant viruses according to biosensors’ responses. Experiments indicate that the multi-net classification system exhibits promising performance compared with the case of single network training, both in terms of error rates and in terms of training speed (especially if the training of the classifiers is done in parallel).

Dimitrios Frossyniotis, Yannis Anthopoulos, Spiros Kintzios, Antonis Perdikaris, Constantine P. Yialouris

A Novel Connectionist-Oriented Feature Normalization Technique

Feature normalization is a topic of practical relevance in real-world applications of neural networks. Although the topic is sometimes overlooked, the success of connectionist models in difficult tasks may depend on a proper normalization of input features. As a matter of fact, the relevance of normalization is pointed out in classic pattern recognition literature. In addition, neural nets require input values that do not compromise numerical stability during the computation of partial derivatives of the nonlinearities. For instance, inputs to connectionist models should not exceed certain ranges, in order to avoid the phenomenon of “saturation” of sigmoids. This paper introduces a novel feature normalization technique that ensures values that are distributed over the (0,1) interval in a uniform manner. The normalization is obtained starting from an estimation of the probabilistic distribution of input features, followed by an evaluation (over the feature that has to be normalized) of a “mixture of Logistics” approximation of the cumulative distribution. The approach turns out to be compliant with the very nature of the neural network (it is realized via a mixture of sigmoids, that can be encapsulated within the network itself). Experiments on a real-world continuous speech recognition task show that the technique is effective, comparing favorably with some standard feature normalizations.

Edmondo Trentin

An Evolutionary Approach to Automatic Kernel Construction

Kernel-based learning presents a unified approach to machine learning problems such as classification and regression. The selection of a kernel and associated parameters is a critical step in the application of any kernel-based method to a problem. This paper presents a data-driven evolutionary approach for constructing kernels, named KTree. An application of KTree to the Support Vector Machine (SVM) classifier is described. Experiments on a synthetic dataset are used to determine the best evolutionary strategy, e.g. what fitness function to use for kernel evaluation. The performance of an SVM based on KTree is compared with that of standard kernel SVMs on a synthetic dataset and on a number of real-world datasets. KTree is shown to outperform or match the best performance of all the standard kernels tested.

Tom Howley, Michael G. Madden

A Leave-K-Out Cross-Validation Scheme for Unsupervised Kernel Regression

We show how to employ leave-K-out cross-validation in Unsupervised Kernel Regression, a recent method for learning of nonlinear manifolds. We thereby generalize an already present regularization method, yielding more flexibility without additional computational cost. We demonstrate our method on both toy and real data.

Stefan Klanke, Helge Ritter

Neural Network Clustering Based on Distances Between Objects

We present an algorithm of clustering of many-dimensional objects, where only the distances between objects are used. Centers of classes are found with the aid of neuron-like procedure with lateral inhibition. The result of clustering does not depend on starting conditions. Our algorithm makes it possible to give an idea about classes that really exist in the empirical data. The results of computer simulations are presented.

Leonid B. Litinskii, Dmitry E. Romanov

Rotation-Invariant Pattern Recognition: A Procedure Slightly Inspired on Olfactory System and Based on Kohonen Network

A computational scheme for rotation-invariant pattern recognition based on Kohonen neural network is developed. This scheme is slightly inspired on the vertebrate olfactory system, and its goal is to recognize spatiotemporal patterns produced in a two-dimensional cellular automaton that would represent the olfactory bulb activity when submitted to odor stimuli. The recognition occurs through a multi-layer Kohonen network that would represent the olfactory cortex. The recognition is invariant to rotations of the patterns, even when a noise lower than 1% is added.

M. B. Palermo, L. H. A. Monteiro

Pattern Classification Using Composite Features

In this paper, we propose a new classification method using composite features, each of which consists of a number of primitive features. The covariance of two composite features contains information on statistical dependency among multiple primitive features. A new discriminant analysis (C-LDA) using the covariance of composite features is a generalization of the linear discriminant analysis (LDA). Unlike LDA, the number of extracted features can be larger than the number of classes in C-LDA. Experimental results on several data sets indicate that C-LDA provides better classification results than other methods.

Chunghoon Kim, Chong-Ho Choi

Visual Attention Algorithms and Architectures for Perceptional Understanding and Video Coding (Special Session)

Towards a Control Theory of Attention

An engineering control approach to attention is developed here, based on the original CODAM (COrollary Discharge of Attention Movement) model. Support for the existence in the brain of the various modules thereby introduced is presented, especially those components involving an observer. The manner in which the model can be extended to executive functions involving the prefrontal cortices is then outlined, Finally the manner in which conscious experience may be supported by the architecture is described.

John G. Taylor

Localization of Attended Multi-feature Stimuli: Tracing Back Feed-Forward Activation Using Localized Saliency Computations

This paper demonstrates how attended stimuli may be localized even if they are complex items composed of elements from several different feature maps and from different locations within the Selective Tuning (ST) model. As such, this provides a step towards the solution of the ‘binding problem’ in vision. The solution relies on a region-based winner-take-all algorithm, a definition of a featural receptive field for neurons where several representations provide input from different spatial areas, and a localized, distributed saliency computation specialized for each featural receptive field depending on its inputs. A top-down attentive mechanism traces back the connections activated by feed-forward stimuli to localize and bind features into coherent wholes.

John K. Tsotsos

An Attention Based Similarity Measure for Colour Images

Much effort has been devoted to visual applications that require effective image signatures and similarity metrics. In this paper we propose an attention based similarity measure in which only very weak assumptions are imposed on the nature of the features employed. This approach generates the similarity measure on a trial and error basis; this has the significant advantage that similarity matching is based on an unrestricted competition mechanism that is not dependent upon a priori assumptions regarding the data. Efforts are expended searching for the best feature for specific region comparisons rather than expecting that a fixed feature set will perform optimally over unknown patterns. The proposed method has been tested on the BBC open news archive with promising results.

Li Chen, F. W. M. Stentiford

Learning by Integrating Information Within and Across Fixations

In this work we introduce a Bayesian Integrate And Shift (BIAS) model for learning object categories. The model is biologically inspired and uses Bayesian inference to integrate information within and across fixations. In our model, an object is represented as a collection of features arranged at specific locations with respect to the location of the fixation point. Even though the number of feature detectors that we use is large, we show that learning does not require a large amount of training data due to the fact that between an object and features we introduce an intermediate representation, object views, and thus reduce the dependence among the feature detectors. We tested the system on four object categories and demonstrated that it can learn a new category from only a few training examples.

Predrag Neskovic, Liang Wu, Leon N Cooper

Feature Conjunctions in Visual Search

Selective Tuning (ST) [1] presents a framework for modeling attention and in this paper we show how it performs in visual search tasks. Two types of tasks are presented, a motion search task and an object search task. Both tasks are successfully tested with different feature and conjunction visual searches.

Antonio J. Rodríguez-Sánchez, Evgueni Simine, John K. Tsotsos

A Biologically Motivated System for Unconstrained Online Learning of Visual Objects

We present a biologically motivated system for object recognition that is capable of online learning of several objects based on interaction with a human teacher. The training is unconstrained in the sense that arbitrary objects can be freely presented in front of a stereo camera system and labeled by speech input. The architecture unites biological principles such as appearance-based representation in topographical feature detection hierarchies and context-driven transfer between different levels of object memory. The learning is fully online and thus avoids an artificial separation of the interaction into training and test phases.

Heiko Wersing, Stephan Kirstein, Michael Götting, Holger Brandl, Mark Dunn, Inna Mikhailova, Christian Goerick, Jochen Steil, Helge Ritter, Edgar Körner

Second-Order (Non-Fourier) Attention-Based Face Detection

We present an attention-based face detection and localization system. The system is biologically motivated, combining face detection based on second-order circular patterns with the localization capabilities of the Selective Tuning (ST) model of visual attention [1]. One of the characteristics of this system is that the face detectors are relatively insensitive to the scale and location of the face, and thus additional processing needs to be performed to localize the face for recognition. We extend ST’s ability to recover spatial information to this object recognition system, and show how this can be used to precisely localize faces in images. The system presented in this paper exhibits temporal characteristics that are qualitatively similar to those of the primate visual system in that detection and categorization is performed early in the processing cycle, while detailed information needed for recognition is only available after additional processing, consistent with experimental data and with certain theories of visual object recognition [2].

Albert L. Rothenstein, Andrei Zaharescu, John K. Tsotsos

Requirements for the Transmission of Streaming Video in Mobile Wireless Networks

The ability to transmit video and support related real-time multimedia applications is considered important in mobile networks. Video streaming, video conferencing, online interactive gaming, and mobile TV are only a few of the applications expected to support the viability, and survival, of next generation mobile wireless networks. It is, therefore, significant to analyze the interaction of the particular media and applications. This paper presents the characteristics of mobile wireless networks and relates these characteristics to the requirements of video transmission. The relationship derived is based not only on the objective QoS metrics measured in the network, but also on the subjective quality measures obtained by video viewers at end hosts. Through this work we establish guidelines for the transmission of video based on the limits of mobile and wireless networks. We believe that the results help researchers and professionals in the fields of video production and encoding to create videos of high efficiency, in terms of resource utilization, and of high performance, in terms of end-user satisfaction.

Vasos Vassiliou, Pavlos Antoniou, Iraklis Giannakou, Andreas Pitsillides

Wavelet Based Estimation of Saliency Maps in Visual Attention Algorithms

This paper deals with the problem of saliency map estimation in computational models of visual attention. In particular, we propose a wavelet based approach for efficient computation of the topographic feature maps. Given that wavelets and multiresolution theory are naturally connected the usage of wavelet decomposition for mimicking the center surround process in humans is an obvious choice. However, our proposal goes further. We utilize the wavelet decomposition for inline computation of the features (such as orientation) that are used to create the topographic feature maps. Topographic feature maps are then combined through a sigmoid function to produce the final saliency map. The computational model we use is based on the Feature Integration Theory of Treisman

et al

and follows the computational philosophy of this theory proposed by Itti

et al

. A series of experiments, conducted in a video encoding setup, show that the proposed method compares well against other implementations found in the literature both in terms of visual trials and computational complexity.

Nicolas Tsapatsoulis, Konstantinos Rapantzikos

Vision and Image Processing (I)

Selective Tuning: Feature Binding Through Selective Attention

We present a biologically plausible computational model for solving the visual binding problem. The binding problem appears due to the distributed nature of visual processing in the primate brain, and the gradual loss of spatial information along the processing hierarchy. The model relies on the reentrant connections so ubiquitous in the primate brain to recover spatial information, and thus allow features represented in different parts of the brain to be integrated in a unitary conscious percept. We demonstrate the ability of the Selective Tuning (ST) model of visual attention [1] to recover spatial information, and based on this propose a general solution to the binding problem. The solution is demonstrated on two classic problems: recovery of form from motion and binding of shape and color. We also demonstrate how the method is able to handle difficult situations such as occlusions and transparency. The model is discussed in relation to recent results regarding the time course and processing sequence for form-from-motion in the primate visual system.

Albert L. Rothenstein, John K. Tsotsos

Rotation Invariant Recognition of Road Signs with Ensemble of 1-NN Neural Classifiers

The paper presents a parallel system of two compound classifiers for recognition of the circular shape road signs. Each of the two classifiers is built of an ensemble of 1-nearest-neighbour (1-NN) classifiers and the arbitration unit operating in the winner-takes-all mode. For the 1-NN we employed the Hamming neural network (HNN) which accepts the binary input. Each HNN is responsible for classification within a single group of deformable prototypes of the road signs. Each of the two compound classifiers has the same structure, however they accept features from different domains: the spatial and the log-polar spaces. The former has an ability of precise classification for shifted but non-rotated objects. The latter exhibits good abilities to register the rotated shapes and also to reject the non road sign objects due to its high false negative detection properties. The combination of the two outperformed each of the single versions what was verified experimentally. The system is characterized by fast learning and recognition rates.

Bogusław Cyganek

Computer Aided Classification of Mammographic Tissue Using Independent Component Analysis and Support Vector Machines

In this paper a robust regions-of-suspicion (ROS) diagnosis system on mammograms, recognizing all types of abnormalities is presented and evaluated. A new type of statistical descriptors, based on Independent Component Analysis (ICA), derive the source regions that generate the observed ROS in mammograms. The reduced set of linear transformation coefficients, estimated from ICA after principal component analysis (PCA), compose the features vector that describes the observed regions in an effective way. The ROS are diagnosed using support-vector-machines (SVMs) with polynomial and radial basis function kernels. Taking into account the small number of training data, the PCA preprocessing step reduces the dimensionality of the features vector and consequently improves the classification accuracy of the SVM classifier. Extensive experiments using the Mammographic Image Analysis Society (MIAS) database have given high recognition accuracy above 87%.

Athanasios Koutras, Ioanna Christoyianni, George Georgoulas, Evangelos Dermatas

Growing Neural Gas for Vision Tasks with Time Restrictions

Self-organizing neural networks try to preserve the topology of an input space by means of their competitive learning. This capacity is being used for the representation of objects and their motion. In addition, these applications usually have real-time constraints. In this work, diverse variants of a self-organizing network, the Growing Neural Gas, that allow an acceleration of the learning process are considered. However, this increase of speed causes that, in some cases, topology preservation is lost and, therefore, the quality of the representation. So, we have made a study to quantify topology preservation using different measures to establish the most suitable learning parameters, depending on the size of the network and on the available time for its adaptation.

José García, Francisco Flórez-Revuelta, Juan Manuel García

A Fixed-Point Algorithm of Topographic ICA

Topographic ICA is a well-known ICA-based technique, which generates a topographic mapping consisting of edge detectors from natural scenes. Topographic ICA uses a complicated criterion derived from a two-layer generative model and minimizes it by a gradient descent algorithm. In this paper, we propose a new simple criterion for topographic ICA and construct a fixed-point algorithm minimizing it. Our algorithm can be regarded as an expansion of the well-known fast ICA algorithm to topographic ICA, and it does not need any tuning of the stepsize. Numerical experiments show that our fixed-point algorithm can generate topographic mappings similar to those in topographic ICA.

Yoshitatsu Matsuda, Kazunori Yamaguchi

Image Compression by Vector Quantization with Recurrent Discrete Networks

In this work we propose a recurrent multivalued network, generalizing Hopfield’s model, which can be interpreted as a vector quantifier. We explain the model and establish a relation between vector quantization and sum-of-squares clustering. To test the efficiency of this model as vector quantifier, we apply this new technique to image compression. Two well-known images are used as benchmark, allowing us to compare our model to standard competitive learning. In our simulations, our new technique clearly outperforms the classical algorithm for vector quantization, achieving not only a better distortion rate, but even reducing drastically the computational time.

Domingo López-Rodríguez, Enrique Mérida-Casermeiro, Juan M. Ortiz-de-Lazcano-Lobato, Ezequiel López-rubio

Vision and Image Processing (II)

Feature Extraction Using Class-Augmented Principal Component Analysis (CA-PCA)

In this paper, we propose a novel feature extraction method called Class-Augmented PCA (CA-PCA) which uses class information. The class information is augmented to data and influences the extraction of features so that the features become more appropriate for classification than those from original PCA. Compared to other supervised feature extraction methods LDA and its variants, this scheme does not use the scatter matrix including inversion and therefore it is free from the problems of LDA originated from this matrix inversion. The performance of the proposed scheme is evaluated by experiments using two well-known face database and as a result we can show that the performance of the proposed CA-PCA is superior to those of other methods.

Myoung Soo Park, Jin Hee Na, Jin Young Choi

A Comparative Study of the Objectionable Video Classification Approaches Using Single and Group Frame Features

This paper deals with the methods for classifying whether a video is harmful or not and also evaluates their performance. The objectionable video classification can be performed using two methods. One can be practiced by judging whether each frame included in the video is harmful, and the other be obtained by using the features reflecting the entire characteristics of the video. The former is a single frame-based feature and the latter is a group frame-based feature. Experimental results show that the group frame-based feature outperforms the single frame-based feature and is robust to the objectionable video classification.

Seungmin Lee, Hogyun Lee, Taekyong Nam

Human Facial Expression Recognition Using Hybrid Network of PCA and RBFN

In this paper, we propose a hybrid architecture combining radial basis function network (RBFN) and Principal Component Analysis (PCA) re-constructure model to perform facial expression recognition from static images. The resultant framework is a two stages coarse to fine discrimination model based on local features extracted from eyes and face images by applying PCA technique . It decomposes the acquired data into a small set of characteristic features. The objective of this research is to develop a more efficient approach to classify between seven prototypic facial expressions, such as neutral, joy, anger, surprise, fear, disgust, and sadness. A constructive procedure is detailed and the system performance is evaluated on a public database ”Japanese Females Facial Expression (JAFFE)”. As anticipated, the experimental results demonstrate the potential capabilities of the proposed approach.

Daw-Tung Lin

Extracting Motion Primitives from Natural Handwriting Data

For the past 10 years it has become clear that biological movement is made up of sub-routine type blocks, or

motor primitives

, with a central controller timing the activation of these blocks, creating synergies of muscle activation. This paper shows that it is possible to use a factorial hidden Markov model to infer primitives in handwriting data. These primitives are not predefined in terms of location of occurrence within the handwriting, and they are not limited or defined by a particular character set. Also, the variation in the data can to a large extent be explained by timing variation in the triggering of the primitives. Once an appropriate set of primitives has been inferred, the characters can be represented as a set of timings of primitive activations, along with variances, giving a very compact representation of the character. Separating the motor system into a motor primitive part, and a timing control gives us a possible insight into how we might create scribbles on paper.

Ben H. Williams, Marc Toussaint, Amos J. Storkey

Including Metric Space Topology in Neural Networks Training by Ordering Patterns

In this paper a new approach to the problem of ordering data in neural network training is presented. According to conducted research, generalization error visibly depends on the order of the training examples. Construction of an order gives some possibility to incorporate knowledge about structure of input and output space into the training process. Simulation results conducted for the isolated handwritten digit recognition problem confirmed the above claims.

Cezary Dendek, Jacek Mańdziuk

Computational Finance and Economics (Special Session)

A Technical Trading Indicator Based on Dynamical Consistent Neural Networks

In econometrics, the behaviour of financial markets is described by quantitative variables. Mathematical and statistical methods are used to explore economic relationships and to forecast the future market development. However, econometric modeling is often limited to a single financial market. In the age of globalisation financial markets are highly interrelated and thus, single market analyses are misleading. In this paper we present a new way to model the dynamics of coherent financial markets. Our approach is based on so-called dynamical consistent neural networks (DCNN), which are able to map multiple scales and different sub-dynamics of the coherent market movement. Unlikely to standard econometric methods, small market movements are not treated as noise but as valuable market information. We apply the DCNN to forecast monthly movements of major foreign exchange (FX) rates. Based on the DCNN forecasts we develop a technical trading indicator to support investment decisions.

Hans Georg Zimmermann, Lorenzo Bertolini, Ralph Grothmann, Anton Maximilian Schäfer, Christoph Tietz

Testing the Random Walk Hypothesis with Neural Networks

Although, there is an ongoing belief in the investment community that technical analysis can be used to infer the direction of future prices, the academic community always treated it (at best) with skepticism. However, if there is a degree of effectiveness in technical analysis, that necessarily lies in direct contrast with the efficient market hypothesis. In this paper, we use neural network estimators to infer from technical trading rules how to extrapolate future price movements. To the extend that the total return of a technical trading strategy can be regarded as a measure of predictability, technical analysis can be seen as a test of the independent increments version of random walk.

Achilleas Zapranis

Financial Application of Neural Networks: Two Case Studies in Greece

In the past few years, many researchers have used Artificial Neural Networks (ANNs) to analyze traditional classification and prediction problems in accounting and finance. This paper explores the efficacy of ANNs in detecting firms that issue fraudulent financial statements (FFS) and in predicting corporate bankruptcy. To this end, two experiments have been conducted using representative ANNs algorithms. During the first experiment, ANNs algorithms were trained using a data set of 164 fraud and non-fraud Greek firms in the recent period 2001-2002. During the second experiment, ANNs algorithms were trained using a data set of 150 failed and solvent Greek firms in the recent period 2003-2004. It was found that ANNs could enable experts to predict bankruptcies and fraudulent financial statements with satisfying accuracy.

S. Kotsiantis, E. Koumanakos, D. Tzelepis, V. Tampakas

Credit Risk Analysis Using a Reliability-Based Neural Network Ensemble Model

Credit risk analysis is an important topic in the financial risk management. Due to recent financial crises and regulatory concern of Basel II, credit risk analysis has been the major focus of financial and banking industry. An accurate estimation of credit risk could be transformed into a more efficient use of economic capital. In this study, we try to use a triple-phase neural network ensemble technique to design a credit risk evaluation system to discriminate good creditors from bad ones. In this model, many diverse neural network models are first created. Then an uncorrelation maximization algorithm is used to select the appropriate ensemble members. Finally, a reliability-based method is used for neural network ensemble. For further illustration, a publicly credit dataset is used to test the effectiveness of the proposed neural ensemble model.

Kin Keung Lai, Lean Yu, Shouyang Wang, Ligang Zhou

Competitive and Collaborative Mixtures of Experts for Financial Risk Analysis

We compare the performance of competitive and collaborative strategies for mixtures of autoregressive experts with normal innovations for conditional risk analysis in financial time series. The prediction of the mixture of collaborating experts is an average of the outputs of the experts. If a competitive strategy is used the prediction is generated by a single expert. The expert that becomes activated is selected either deterministically (hard competition) or at random, with a certain probability (soft competition). The different strategies are compared in a sliding window experiment for the time series of log-returns of the Spanish stock index IBEX 35, which is preprocessed to account for the heteroskedasticity of the series. Experiments indicate that the best performance for risk analysis is obtained by mixtures with soft competition, where the experts have a probability of activation given by the output of a gating network of softmax units.

José Miguel Hernández-Lobato, Alberto Suárez

Neural Computing in Energy Engineering (Special Session)

Kernel Regression Based Short-Term Load Forecasting

Electrical load forecasting is an important tool in managing transmission and distribution facilities, financial resources, manpower, and materials at electrical power utility companies. A simple and accurate electrical load forecasting scheme is required. Short-term load forecasting (STLF) involves predicting the load from few hours to a week ahead. A simple non-parametric kernel regression (KR) approach for STLF is presented. Kernel regression is a linear approach with the ability to handle nonlinear information. A Gaussian kernel whose bandwidth selected by the Direct Plug-in (DPI) method is utilized. The performance comparison of the proposed method with artificial neural network (ANN), ordinary least squares (OLS), and ridge regression (RR) predictions on the same data set is presented. Experimental results show that kernel regression performs better than ANN forecaster on the given data set. The method proposed provides analytical solution, features optimal bandwidth selection, which is more instructive compared to ANN architecture and its other parameters.

Vivek Agarwal, Anton Bougaev, Lefteri Tsoukalas

Electricity Load Forecasting Using Self Organizing Maps

Electricity load forecasting has become increasingly important for the industry. However, the accurate load prediction remains a challenging task due to several issues such as the nonlinear character of the time series or the seasonal patterns it exhibits.

Several non-linear techniques such as the SVM have been applied to this problem. However, the properties of the load time series change strongly with the seasons, holidays and other factors. Therefore global models such as the SVM are not suitable to predict accurately the load demand.

In this paper we propose a model that first splits the time series into homogeneous regions using the Self Organizing Maps (SOM). Next, an SVM is locally trained in each region.

The algorithm proposed has been applied to the prediction of the maximum daily electricity demand. The experimental results show that our model outperforms several statistical and machine learning forecasting techniques.

Manuel Martín-Merino, Jesus Román

A Hybrid Neural Model in Long-Term Electrical Load Forecasting

A novel hierarchical hybrid neural model to the problem of long-term electrical load forecasting is proposed in this paper. The neural model is made up of two self-organizing map nets — one on top of the other —, and a single-layer perceptron. It has application into domains which require time series analysis. The model is compared to a multilayer perceptron. Both the hierarchical and the multilayer perceptron models are endowed with time windows in their input layers. They are trained and assessed on load data extracted from a North-American electric utility. The models are required to predict once every week the electric peak-load and mean-load during the next two years. The results are presented and evaluated in the paper.

Otávio A. S. Carpinteiro, Isaías Lima, Rafael C. Leme, Antonio C. Zambroni de Souza, Edmilson M. Moreira, Carlos A. M. Pinheiro

Application of Radial Basis Function Networks for Wind Power Forecasting

In this paper, an advanced system based on artificial intelligence and fuzzy logic techniques is developed to predict the wind power output of a wind farm. A fuzzy logic model is applied first to check the reliability of the numerical weather predictions (NWPs) and to split them in two sub-sets, of good and bad quality NWPs, respectively. Two Radial Basis Function (RBF) neural networks, one for each sub-set are trained next to estimate the wind power. Results from a real wind farm are presented and the added value of the proposed method is demonstrated by comparison with alternative methods.

George Sideratos, N. D. Hatziargyriou

The Application of Neural Networks to Electric Power Grid Simulation

A neural network approach is being developed to enable real time simulations for large scale dynamic system simulations of the electric power grid. If the grid is decomposed into several subsystems, neural networks can be utilized to simulate computationally intensive subsystems. An electrical generator sub-system was created in MATLAB using the SIMULINK interface. The SIMULINK model provided corresponding input/output pairs by varying parameters in sample transmission lines. A feed-forward backpropagation neural network was created from this data. Integration of the generator neural network into the SIMULINK interface was also performed. The original SIMULINK model requires about 342,000 iterations to simulate a 30 second simulation and consumes about 27 minutes of execution time. Conversely, the neural network based system is able to determine accurate solutions in less than 75 seconds and 300 iterations, which is more than an order of magnitude reduction in the execution time.

Emily T. Swain, Yunlin Xu, Rong Gao, Thomas J. Downar, Lefteri H. Tsoukalas

Early Detection of Winding Faults in Windmill Generators Using Wavelet Transform and ANN Classification

This paper introduces the Wavelet Transform (WT) and Artificial Neural Networks (ANN) analysis to the diagnostics of electrical machines winding faults. A novel application is presented, exploring the potential of automatically identifying short circuits of windings that can appear during machine manufacturing and operation. Such faults are usually the result of the influence of electrodynamics forces generated during the flow of large short circuit currents, as well as of the forces occurring when the transformers or generators are transported. The early detection and classification of winding failures is of particular importance, as these kinds of defects can lead to winding damage due to overheating, imbalance, etc. Application results on investigations of windmill generator winding faults are presented. The ANN approach is proven effective in classifying faults based on features extracted by the WT.

Zacharias Gketsis, Michalis Zervakis, George Stavrakakis

Combining Artificial Neural Networks and Heuristic Rules in a Hybrid Intelligent Load Forecast System

In this work, an Artificial Neural Network (ANN) is combined to Heuristic Rules producing a powerful hybrid intelligent system for short and mid-term electric load forecasting. The Heuristic Rules are used to adjust the ANN output to improve the system performance. The study was based on load demand data of Energy Company of Pernambuco (CELPE), which contain the hourly load consumption in the period from January-2000 until December-2004. The more critical period of the rationing in Brazil was eliminated from the data file, as well as the consumption of the holidays. For this reason, the proposed system forecasts a holiday as one Saturday or Sunday based on the specialist’s information. The result obtained with the proposed system is compared with the currently system used by CELPE to test its effectiveness. In addition, it was also compared to the result of the ANN acting alone.

Ronaldo R. B. de Aquino, Aida A. Ferreira, Manoel A. Carvalho, Milde M. S. Lira, Geane B. Silva, Otoni Nóbrega Neto

New Phenemenon on Power Transformers and Fault Identification Using Artificial Neural Networks

In this paper voltage recovery after voltage dip that cause magnetizing inrush current which is a new phenomenon in power transformers are discussed and a new technique is proposed to distinquish internal fault conditions from no-fault conditions that is also containing these new phenomenons. The proposed differential algorithm is based on Artificial Neural Network (ANN). The training and testing data sets are obtained using SIMPOW-STRI power system simulation program and laboratory transformer. A novel neural network is designed and trained using back-propagation algorithm. It is seen that the proposed network is well trained and able to discriminate no-fault examples from fault examples with high accuracy.

Mehlika Şengül, Semra Öztürk, Hasan Basri Çetinkaya, Tarık Erfidan

Applications to Biomedicine and Bioinformatics

Neural Network Based Algorithm for Radiation Dose Evaluation in Heterogeneous Environments

An efficient and accurate algorithm for radiation dose evaluation is presented in this paper. Such computations are useful in the radiotherapic treatment planning of tumors. The originality of our approach is to use a neural network which has been trained with several homogeneous environments to deduce the doses in any kind of environment (possibly heterogeneous). Our algorithm is compared in several representative contexts to a reference simulation code in the domain.

Jacques M. Bahi, Sylvain Contassot-Vivier, Libor Makovicka, Éric Martin, Marc Sauget

Exploring the Intrinsic Structure of Magnetic Resonance Spectra Tumor Data Based on Independent Component Analysis and Correlation Analysis

Analysis on magnetic resonance spectra (MRS) data gives a deep insight into pathology of many types of tumors. In this paper, a new method based on independent component analysis (ICA) and correlation analysis is proposed for MRS tumour data structure analysis. First, independent components and their coefficients are derived by ICA. Those components are interpreted in terms of metabolites, which interrelate with each other in tissues. Then correlation analysis is performed to reveal the interrelationship on coefficient of ICs, where residue dependence of components of metabolites remains. The method was performed on MRS data of hepatic encephalopathy. Experimental results reveal the intrinsic data structure and describe the pathological interrelation between parts of the structure successfully.

Jian Ma, Zengqi Sun

Fusing Biomedical Multi-modal Data for Exploratory Data Analysis

Data analysis in modern biomedical research has to integrate data from different sources, like microarray, clinical and categorical data, so called multi-modal data. The reef SOM, a metaphoric display, is applied and further improved such that it allows the simultaneous display of biomedical multi-modal data for an exploratory analysis. Visualizations of microarray, clinical, and category data are combined in one informative and entertaining image. The U-matrix of the SOM trained on microarray data is visualized as an underwater sea bed using color and texture. The clinical data and category data are integrated in the form of fish shaped glyphs. The resulting images are intuitive, entertaining and can easily be interpreted by the biomedical collaborator, since specific knowledge about the SOM algorithm is not required. Visual inspection enables the detection of interesting structural patterns in the multi-modal data when browsing through and zooming into the image. Results of such an analysis are presented for the van’t Veer data set.

Christian Martin, Harmen grosse Deters, Tim W. Nattkemper

Semi-supervised Significance Score of Differential Gene Expressions

In gene expression analyses for DNA microarray data, various statistical scores have been proposed for evaluating significance of genes exhibiting differential expression between two or more controlled conditions. To consider an unsupervised case or a semi-supervised case rather than a well-studied supervised case, we assume a latent variable model and apply the optimal discovery procedure (ODP) proposed by Storey (2005) to the model. Theoretical consideration leads to two different interpretations of the hidden variable, i.e., it only implicitly affects the alternative model through the model parameters, or is explicitly included in the alternative model, so that they correspond to two different implementations of ODP. By comparing the two implementations through experiments with simulation data, we found that sharing the latent variable estimation as in the latter case is effective in increasing the detectability of truly active genes. We also propose unsupervised and semi-supervised rating of genes and show its effectiveness as a significance score.

Shigeyuki Oba, Shin Ishii

Semi Supervised Fuzzy Clustering Networks for Constrained Analysis of Time-Series Gene Expression Data

Clustering analysis of time series data from DNA microarray hybridization studies is essential for identifying biological relevant groups of genes. Microarrrays provide large datasets that are currently primarily analyzed using crisp clustering techniques. Crisp clustering methods such as K-means or self organizing maps assign each gene to one cluster, thus omitting information concerning the multiple roles of genes. One of the major advantages of fuzzy clustering is that genes can belong to more than one group, revealing this way more profound information concerning the function and regulation of each gene. Additionally, recent studies have proven that integrating a small amount of information in purely unsupervised algorithms leads to much better performance. In this paper we propose a new semi-supervised fuzzy clustering algorithm which we apply in time series gene expression data. The clustering that was performed on simulated as well as experimental microarray data proved that the proposed method outperformed other clustering techniques.

Ioannis A. Maraziotis, Andrei Dragomir, Anastasios Bezerianos

Evolutionary Optimization of Sequence Kernels for Detection of Bacterial Gene Starts

Oligo kernels for biological sequence classification have a high discriminative power. A new parameterization for the


-mer oligo kernel is presented, where all oligomers of length


are weighted individually. The task specific choice of these parameters increases the classification performance and reveals information about discriminative features. For adapting the multiple kernel parameters based on cross-validation the covariance matrix adaptation evolution strategy is proposed. It is applied to optimize the trimer oligo kernel for the detection of prokaryotic translation initiation sites. The resulting kernel leads to higher classification rates, and the adapted parameters reveal the importance for classification of particular triplets, for example of those occurring in the Shine-Dalgarno sequence.

Britta Mersch, Tobias Glasmachers, Peter Meinicke, Christian Igel

Tree-Dependent Components of Gene Expression Data for Clustering

Tree-dependent component analysis (TCA) is a generalization of independent component analysis (ICA), the goal of which is to model the multivariate data by a linear transformation of latent variables, while latent variables fit by a tree-structured graphical model. In contrast to ICA, TCA allows dependent structure of latent variables and also consider non-spanning trees (forests). In this paper, we present a TCA-based method of clustering gene expression data. Empirical study with yeast cell cycle-related data, yeast metabolic shift data, and yeast sporulation data, shows that TCA is more suitable for gene clustering, compared to principal component analysis (PCA) as well as ICA.

Jong Kyoung Kim, Seungjin Choi

Applications to Security and Market Analysis

A Neural Model in Anti-spam Systems

The paper proposes the use of the multilayer perceptron model to the problem of detecting ham and spam e-mail patterns. It also proposes an intensive use of data pre-processing and feature selection methods to simplify the task of the multilayer perceptron in classifying ham and spam e-mails. The multilayer perceptron is trained and assessed on patterns extracted from the SpamAssassin Public Corpus. It is required to classify novel types of ham and spam patterns. The results are presented and evaluated in the paper.

Otávio A. S. Carpinteiro, Isaías Lima, João M. C. Assis, Antonio C. Zambroni de Souza, Edmilson M. Moreira, Carlos A. M. Pinheiro

A Neural Model in Intrusion Detection Systems

The paper proposes the use of the multilayer perceptron model to the problem of detecting attack patterns in computer networks. The multilayer perceptron is trained and assessed on patterns extracted from the files of the Third International Knowledge Discovery and Data Mining Tools Competition. It is required to classify novel normal patterns and novel categories of attack patterns. The results are presented and evaluated in the paper.

Otávio A. S. Carpinteiro, Roberto S. Netto, Isaías Lima, Antonio C. Zambroni de Souza, Edmilson M. Moreira, Carlos A. M. Pinheiro

Improved Kernel Based Intrusion Detection System

Computer security has become a critical issue with the rapid development of business and other transaction systems over the Internet. The application of artificial intelligence, machine learning and data mining techniques to intrusion detection systems has been increasing recently. But most research is focused on improving the classification performance of a classifier. Selecting important features from input data leads to simplification of the problem, and faster and more accurate detection rates. Thus selecting important features is an important issue in intrusion detection. Another issue in intrusion detection is that most of the intrusion detection systems are performed by off-line and it is not a suitable method for a real-time intrusion detection system. In this paper, we develop the real-time intrusion detection system, which combines an on-line feature extraction method with the on-line Least Squares Support Vector Machine classifier. Applying the proposed system to KDD CUP 99 data, experimental results show that it has a remarkable feature feature extraction, classification performance and reducing detection time compared to existing off-line intrusion detection system.

Byung-Joo Kim, Il Kon Kim

Testing the Fraud Detection Ability of Different User Profiles by Means of FF-NN Classifiers

Telecommunications fraud has drawn the attention in research due to the huge economic burden on companies and to the interesting aspect of users’ behavior characterization. In the present paper, we deal with the issue of user characterization. Several real cases of defrauded user accounts for different user profiles were studied. Each profile’s ability to characterize user behavior in order to discriminate normal activity from fraudulent one was tested. Feed-forward neural networks were used as classifiers. It is found that summary characteristics of user’s behavior perform better than detailed ones towards this task.

Constantinos S. Hilas, John N. Sahalos

Predicting User’s Movement with a Combination of Self-Organizing Map and Markov Model

In the development of location-based services, various location-sensing techniques and experimental/commercial services have been used. We propose a novel method of predicting the user’s future movements in order to develop advanced location-based services. The user’s movement trajectory is modeled using a combination of recurrent self-organizing maps (RSOM) and the Markov model. Future movement is predicted based on past movement trajectories. To verify the proposed method, a GPS dataset was collected on the Yonsei University campus. The results were promising enough to confirm that the application works flexibly even in ambiguous situations.

Sang-Jun Han, Sung-Bae Cho

Learning Manifolds in Forensic Data

Chemical data related to illicit cocaine seizures is analyzed using linear and nonlinear dimensionality reduction methods. The goal is to find relevant features that could guide the data analysis process in chemical drug profiling, a recent field in the crime mapping community. The data has been collected using gas chromatography analysis. Several methods are tested: PCA, kernel PCA, isomap, spatio-temporal isomap and locally linear embedding. ST-isomap is used to detect a potential time-dependent nonlinear manifold, the data being sequential. Results show that the presence of a simple nonlinear manifold in the data is very likely and that this manifold cannot be detected by a linear PCA. The presence of temporal regularities is also observed with ST-isomap. Kernel PCA and isomap perform better than the other methods, and kernel PCA is more robust than isomap when introducing random perturbations in the dataset.

Frédéric Ratle, Anne-Laure Terrettaz-Zufferey, Mikhail Kanevski, Pierre Esseiva, Olivier Ribaux

A Comparison of Target Customers in Asian Online Game Markets: Marketing Applications of a Two-Level SOM

The purpose of our research is to identify the critical variables, to implement a new methodology for Asian online game market segmentation, and to compare target customers in Asian online game markets; Korea, Japan and China. Conclusively, the critical segmentation variables and the characteristics of target customers were different among countries. Therefore, online game companies should develop diverse marketing strategies based on characteristics of their target customers.

Sang-Chul Lee, Jae-Young Moon, Yung-Ho Suh

Real World Applications (I)

A Neural Network Approach to Study O3 and PM10 Concentration in Environmental Pollution

In this paper two artificial neural networks are trained to determine Ozone and PM10 concentrations trying to model the environmental system. Then a method to partition the connection weights is used to calculate a relative importance index which returns the relative contribution of each chemical and meteorological input to the concentrations of Ozone and PM10. Moreover, an investigation of the variances of the input in the observation time contribute to understand which input mainly influence the output. Therefore a neural network trained only by the variables with higher values of relative importance index and low variability is used to improve the accuracy of the proposed model. The experimental results show that this approach could help to understand the environmental system.

Giuseppe Acciani, Ernesto Chiarantoni, Girolamo Fornarelli

ROC Analysis as a Useful Tool for Performance Evaluation of Artificial Neural Networks

In many applications of neural networks, the performance of the network is given by the classification accuracy. While obtaining the classification accuracies, the total true classification is computed, but the number of classification rates of the classes and fault classification rates are not given. This would not be enough for a problem having fatal importance. As an implementation example, a dataset having fatal importance is classified by MLP, RBF, GRNN, PNN and LVQ networks and the real performances of these networks are found by applying ROC analysis.

Fikret Tokan, Nurhan Türker, Tülay Yıldırım

NewPR-Combining TFIDF with Pagerank


was widely used in IR system based on the vector space model (VSM). Pagerank was used in systems based on hyperlink structure such as Google. It was necessary to develop a technique combining the advantages of two systems. In this paper, we drew up a framework by using the content of web pages and the out-link information synchronously. We set up a matrix M, which composed of out-link information and the relevant value of web pages with the given query. The relevant value was denoted by


. We got the NewPR (New Pagerank) by solving the equation with the coefficient M. Experimental results showed that more pages, which were more important both in content and hyper-link sides, were selected.

Hao-ming Wang, Martin Rajman, Ye Guo, Bo-qin Feng

A Fast Algorithm for Words Reordering Based on Language Model

What appears to be given in all languages is that words can not be randomly ordered in sentences, but that they must be arranged in certain ways, both globally and locally. The “scrambled” words into a sentence cause a meaningless sentence. Although the use of manually collected grammatical rules can boost the performance of grammar checker in word order diagnosis, the repairing task is still very difficult. This work proposes a method for repairing word order errors in English sentences by reordering words in a sentence and choosing the version that maximizes the number of trigram hits according to a language model. The novelty of this method concerns the use of a permutations’ filtering approach in order to reduce the search space among the possible sentences with reordered words. The filtering method is based on bigrams’ probabilities. In this work the search space is further reduced using a threshold over bigrams’ probabilities. The experimental results show that more than 95% of the test sentences can be repaired using this technique. The comparative advantage of this method is that it is not restricted into a specific set of words, and avoids the laborious and costly process of collecting word order errors for creating error patterns. Unlike most of the approaches, the proposed method is applicable to any language (language models can be simply computed in any language) and does not work only with a specific set of words. The use of parser and/or tagger is not necessary.

Theologos Athanaselis, Stelios Bakamidis, Ioannis Dologlou

Phonetic Feature Discovery in Speech Using Snap-Drift Learning

This paper presents a new application of the


algorithm [1]: feature discovery and clustering of speech waveforms from non-stammering and stammering speakers. The learning algorithm is an unsupervised version of snap-drift which employs the complementary concepts of fast, minimalist learning (


) & slow


(towards the input pattern) learning. The


Neural Network (SDNN) is toggled between snap and drift modes on successive epochs. The speech waveforms are drawn from a phonetically annotated corpus, which facilitates phonetic interpretation of the classes of patterns discovered by the SDNN.

Sin Wee Lee, Dominic Palmer-Brown

A New Neuro-Dominance Rule for Single Machine Tardiness Problem with Unequal Release Dates

We present a neuro-dominance rule for single machine total weighted tardiness problem with unequal release dates. To obtain the neuro-dominance rule (NDR), backpropagation artificial neural network (BPANN) has been trained using 10000 data and also tested using 10000 another data. The proposed neuro-dominance rule provides a sufficient condition for local optimality. It has been proved that if any sequence violates the neuro-dominance rule then violating jobs are switched according to the total weighted tardiness criterion. The proposed neuro-dominance rule is compared to a number of competing heuristics and meta heuristics for a set of randomly generated problems. Our computational results indicate that the neuro-dominance rule dominates the heuristics and meta heuristics in all runs. Therefore, the neuro-dominance rule can improve the upper and lower bounding schemes.

Tarık Çakar

Real World Applications (II)

A Competitive Approach to Neural Device Modeling: Support Vector Machines

Support Vector Machines (SVM) are a system for efficiently training linear learning machines in the kernel induced feature spaces, while respecting the insights provided by the generalization theory and exploiting the optimization theory. In this work, Support Vector Machines are employed for the nonlinear regression. The nonlinear regression ability of the Support Vector Machines has been demonstrated by forming the SVM model of a microwave transistor and it has been compared with its neural model.

Nurhan Türker, Filiz Güneş

Non-destructive Testing for Assessing Structures by Using Soft-Computing

A hybrid system which combines Self Organizing Maps and Case Based Reasoning is presented and apply to Structural Assessment. Self Organizing Maps are trained as a classification tool in order to organize the old cases in memory with the purpose of speeding up the Case Based Reasoning process. Three real structures have been used: An aluminium beam, a pipe section and a long pipe.

Luis Eduardo Mujica, Josep Vehí, José Rodellar

Neural Unit Element Application for in Use Microwave Circuitry

In this work, a Neural Unit Element (NUE) is defined to be used in the analysis and synthesis of the microwave circuits. For this purpose, analysis of impedance transformation property of a transmission line segment with the parameters (


ℓ , Z


) is defined as the problem in the forward direction and synthesis of the transmission line to obtain the target impedance is also defined the problem in the reverse direction. This problem is solved using Multilayer Perceptron (MLP) with efficient training algorithm. Finally, NUE driven by 50Ω. and complex source which is very common in microwave applications and the short-circuited NUE (Stub) are given as the worked examples.

M. Fatih Çağlar, Filiz Güneş

An Artificial Neural Network Based Simulation Metamodeling Approach for Dual Resource Constrained Assembly Line

The main objective of this study is to find the optimum values of design and operational parameters related to worker flexibility in a Dual Resource Constrained (


) assembly line considering the performance measures of Hourly Production Rate (


), Throughput Time (


) and Number of Worker Transfers (


). We used Artificial Neural Networks (


) as a simulation metamodel to estimate


assembly line performances for all possible alternatives. All alternatives were evaluated with respect to a utility function which consists of weighted sum of normalized performance measures.

Gokalp Yildiz, Ozgur Eski

A Packet Routing Method Using Chaotic Neurodynamics for Complex Networks

We propose a new packet routing method for a computer network using chaotic neurodynamics. We first compose a basic neural network which routes packets using information of shortest path lengths from a node to the other nodes. When the computer network topology is regular, the routing method works well, however, when the computer network topology becomes irregular, the basic routing method doesn’t work well. The reason is that most of packets cannot be transmitted to their destinations because of packet congestion in the computer network. To avoid such an undesirable problem, we extended the basic method to employ chaotic neurodynamics. We confirm that our proposed method exhibits good performance for computer networks with various topologies. Furthermore, we analyze why the proposed routing method is effective: we introduce the method of surrogate data which is often used in the field of nonlinear time-series analysis. In consequence of introducing such a statistical control, we confirm that using chaotic neurodynamics is the most effective policy to decentralize the congestion of the packets in the computer network.

Takayuki Kimura, Tohru Ikeguchi


Weitere Informationen