Neural Information Processing. Theory and Algorithms

Mammalian visual cortex is known to have various neuronal response properties that depend on stimuli outside classical receptive fields. In this article, we give a probabilistic explanation to one such property called border-ownership signals, by interpreting them as posterior joint probabilities of a low-level edge property and a high-level figure property. We show that such joint probabilities can be found in a hierarchical Bayesian network mimicking visual cortex, and indeed they exhibit simulational responses qualitatively similar to physiological data.

In natural scenes, objects are often partially occluded. Nonetheless, our visual system can readily complete an object shape from available information and perceive it as a whole, a process known as amodal completion. Although implementation of this completion process is an important issue, visual computation for completion, based on both the local continuity of contours and on global regularities, such as symmetry, has received little attention. Here, we show a novel neurocomputational model based on recent physiological findings, in particular those in visual area V4. The model enables amodal completion through the evaluation of a global constellation of features describing a shapefs contours.

We present a mathematical model to quantitatively describe the neuronal dynamics in

Caenorhabditis elegans

. Since calcium imaging is a popular technique to visualize the neuronal activity in

C. elegans

, the model includes the variable of the fluorescence intensity in addition to the membrane potential and the intracellular calcium concentration. The fluorescence intensity is a quantity which is comparable with the experimental data. The parameters in the model are determined to reproduce the neurophysiological experimental data. Our model exhibits good agreement with the data. We apply the model to a neural circuit for chemotaxis and find that the neuronal activity measured by the fluorescence intensity shows quantitatively different behavior from that measured by the membrane potential in some neurons. The difference is discussed from the viewpoint of neuronal mechanisms.

This paper analyzes the performance of the human localization by a spiking neural network in informationally structured space based on sensor networks. First, we discuss the importance of information structuralization. Next, we apply a spiking neural network to extract the human position in a room equipped with sensor network devices. Next, we propose how to update the base value as a method of preprocessing to generate input values to the spiking neurons, and the learning method of the spiking neural network based on the time series of measured data. Finally, we show several experimental results, and discuss the effectiveness of the proposed method.

The authors have proposed a computational model of the cerebral cortex, called the BESOM model, that combines a Bayesian network and Self-Organizing Maps. In this paper, we add another model of the cerebral cortex, called sparse coding, into our model in a biologically plausible way. In the BESOM model, hyper-columns in the cerebral cortex are interpreted as random variables in a Bayesian network. We extend our model so that random variables can become “inactive.” In addition, we apply bias at the time of recognition so that almost all of the random variables may become inactive. This mechanism realizes sparse coding without breaking the theoretical framework of the model based on the Bayesian networks.

Cortical activity exhibits complex, persistent self-sustained dynamics, which is hypothesised to support the brain’s sophisticated processing capabilities. Prior studies have shown how complex activity can be sustained for some time in spiking neural network models, but network activity in these models resembled high firing rate seizure which would eventually fail, leading to indefinite quiescence. We present a spiking network model of cortex innervated by diffuse thalamic projections, called the Complex Spiking Model (CSM). The model exhibits persistent, self-sustained, non-periodic, complex dynamics at low firing rates. Multiple network configurations were tested, systematically varying diffuse excitation from the thalamus, strength of the local cortical inhibition and excitation, neighbourhood diameters, synaptic efficacies and synaptic current time constants. Complex activity in all the network configurations depended strongly upon the strength of the diffuse excitation from the thalamus. We propose that diffuse thalamic projections to cortex facilitate complex cortical dynamics and are likely to be an important factor in the support of cognitive functions.

The exponentially decaying tabu search, which exhibits high performance in solving quadratic assignment problems (QAPs), has been implemented on a neural network with chaotic neurodynamics. To exploit the inherent parallel processing capability of analog hardware systems, a synchronous updating scheme, in which all neurons in the network are updated simultaneously, has also been proposed. However, several neurons may fire simultaneously with the synchronous updating. As a result, we cannot determine only one candidate for the 2-opt exchange from among the many fired neurons. To solve this problem, several neuron selection methods, which select a specific neuron from among the fired neurons, have been devised. These neuron selection methods improved the performance of the synchronous updating scheme; however, the dynamics of the chaotic neural network driven by these heuristic algorithms cannot be intuitively understood. In this paper, we analyze the dynamics of a chaotic neural network driven by the neuron selection methods by considering the spatial and temporal mutual information.

This paper describes how an emerging standard neural network modelling language can be used to configure a general-purpose neural multi-chip system by describing the process of writing and loading neural network models on the SpiNNaker neuromimetic hardware. It focuses on the implementation of a SpiNNaker module for PyNN, a simulator-independent language for neural networks modelling. We successfully extend PyNN to deal with different non-standard (eg. Izhikevich) cell types, rapidly switch between them and load applications on a parallel hardware by orchestrating the software layers below it, so that they will be abstracted to the final user. Finally we run some simulations in PyNN and compare them against other simulators, successfully reproducing single neuron and network dynamics and validating the implementation.

This paper proposes an optimization algorithm, which utilizes ideal spatiotemporal chaotic dynamics for solution search in a high dimensional solution space. Such chaotic dynamics is generated by the Lebesgue spectrum filter, which has been applied to the chaotic CDMA in previous researches to minimize the cross-correlation among the sequences. In the proposed method, such a filter is applied to the output functions of optimization neural networks to realize an ideal chaotic search, which generates ideally complicated searching dynamics. The proposed scheme is applied to two combinatorial optimization approaches, the Hopfield-Tank neural network with additive noise and a heuristic algorithm driven by neural networks, which solve the Traveling Salesman Problems and the Quadratic Assignment Problems. The simulation results show that the proposed approach using the ideal chaotic dynamics simply improves the performance of the chaotic algorithms without searching appropriate parameter values even for large-scale problems.

Many applications would emerge from the development of artificial systems able to accurately localize and identify sound sources. However, one of the main difficulties of such kind of system is the natural presence of mixed sound sources in real environments. This paper proposes a pulsed neural network based system for extraction and recognition of objective sound sources from background sound source. The system uses the short term depression, that implements by the weight’s decay in the output layer and changing the weight by frequency component in the competitive learning network. Experimental results show that objective sounds could be successfully extracted and recognized.

So far it is widely believed that mammalian navigation is based on map-like place representations created by hippocampal place cells. Models supporting this compute returning path to origin as a sequence of places, or as a sequence of firing place cells. However, these models often fail to compute the shortest returning path to start point. Moreover, one big constraint of these models is that the space has to be well explored to find a returning path from a certain endpoint. Here we propose a computational model for path integration and navigation based on Head Direction (HD) cells and grid cells. We show that an ensemble of HD cells can accumulate distance and compute the direction from the start point to the present location during exploration. Based on this vector navigation, and with the help of properties of grid cells in entorhinal cortex, it is possible to navigate the way back to the origin. We show that our recent model of rat entorhinal cortex provides these functional mechanisms of path integration and vector navigation.

We studied possibilities of classification of single spike neuron models by their intrinsic timescale parameters, because little is known about changes of timescale on spiking dynamics, and its influence on other spike properties and network dynamics such as synchronization. Using both FitzHugh-Nagumo (FHN) type and Terman-Wang (TW) type of theoretically tractable models, analysis of the phase response curve (PRC) found common and unique dynamic characteristics with respect to two parameters of timescale and injected current amplitude in the models. Also, a scheme of synchronization transition in the identical pair systems, in which two identical models mutually interact through the same model of synaptic response, was systematically explained by controlling these parameters. Then we found their common and unique synchronous behaviors.

The border ownership (BO) that indicates which side of the contour owns the border plays a fundamental role in object perception[1]. The responses of BO-selective cells exhibit rapid transition when a stimulus is fliped along its classical receptive field so that the opposite BO is presented, while the transition is significantly slower when a clear BO is turned into an ambiguous edge such as when a square is enlarged extensively[2]. This phenomenon appears to be a crucial clue for understanding the neural mechanims underlying the credibility of BO. We hypothesize that dynamics of BO-selective cells and networks behind them play a crucial role in the credibility, and that the credibility is related to early visual areas as an appearance of a salient object evokes bottom-up attention. To investigate these hypotheses, we examined the dynamics of BO-selective cells with a computational model that include recurent pathways among V1, V2 and Posterior Parietal (PP) areas[3]. The model cells have been shown to reproduce effects of spatial attention. Simulations of the model exhibited distinct response time depending on the ambiguity of BO, indicating a crucial role of dynamics in the credibility. The recurrent network between PP and V1 appear to play a crucial role for the time course of BO-selective cells that governs simultaneously both credibility of BO and bottom-up attention.

A digital spiking neuron is a wired system of shift registers that can mimic nonlinear dynamics of simplified spiking neuron models. In this paper, we present a novel pulse-coupled system of the digital spiking neurons. The coupled neurons can exhibit various pseudo-periodic synchronizations, non-periodic synchronizations, and related bifurcations. We derive theoretical parameter conditions that guarantee occurrence of typical synchronization phenomena. Also, the theoretical results are validated by numerical simulations.

We investigated the network topology organized through spike-timing-dependent plasticity (STDP) using pair- and triad-connectivity patterns, considering difference of excitatory and inhibitory neurons.

As a result, we found that inhibitory synaptic strength affects statistical properties of the network topology organized through STDP more strongly than the bias of the external input rate for the excitatory neurons. In addition, we also found that STDP leads highly nonrandom structure to the neural network. Our analysis from a viewpoint of connectivity pattern transitions reveals that STDP does not uniformly strengthen and depress excitatory synapses in neural networks. Further, we also found that the significance of triad-connectivity patterns after the learning results from the fact that the probability of triad-connectivity-pattern transitions is much higher than that of combinations of pair-connectivity-pattern transitions.

Most species of bats making echolocation use Doppler-shifted frequency of ultrasonic echo pulse to measure the velocity of target. The neural circuits for detecting the target velocity are specialized for fine-frequency analysis of the second harmonic constant frequency (CF2) component of Doppler-shifted echoes. To perform the fine-frequency analysis, the feedback signals from cortex to subcortical and peripheral areas are needed. The feedback signals are known to modulate the tuning property of subcortical neurons. However, it is not yet clear the neural mechanism for the modulation of the tuning property. We present here a neural model for detecting Doppler-shifted frequency of echo sound reflecting a target. We show that the model reproduce qualitatively the experimental results on the modulation of tuning shifts of subcortical neurons. We also clarify the neural mechanism by which the tuning property is changed depending on the feedback connections between cortex and subcortex neurons.

Neurons in the primary visual cortex (V1) encode natural images that are exposed. As a candidate encoding principle, the efficient coding hypothesis was proposed by Attneave (1954) and Barlow (1961). This hypothesis emphasizes that the primary role of neurons in the sensory area is to reduce the redundancy of the external signal and to produce a statistically efficient representation. However, the outputs of neurons in V1 are statistically dependent because their classical receptive fields largely overlap and natural images have structures such as edges and textures. As described in this paper, we propose that the computational role of horizontal connections (HCs) is to decrease statistical dependency and attempt to self-organize the spatial distribution of HCs from natural images. In addition, we show that our neural network model with self-organized HCs can reproduce some nonlinear properties of V1 neurons,

e.g.

size-tuning and contextual modulation. These results support the efficient coding hypothesis and imply that HCs serve an important role in decreasing statistical dependency in V1.

Bionic vision devices aiming to restore visual perception in the vision impaired rely on microelectrode arrays that can be implanted under the diseased retina. Arrays used today in first human trials are high density monopolar arrays comprising up to 1500 electrodes in a 3x3 mm a simple field calculations demonstrate that such high density arrays suffer from degradation of contrast between those 1500 stimulation sites when driven simultaneously. This effect can be described as electric crosstalk between the electrodes that strongly depends on the number of electrodes on such an array and proximity of electrodes to the target cells. The limit of spatial frequency of visual patterns that could be resolved by such arrays can be assessed to be 4.5; 1.2; and 0.7 cycles/mm, for an anticipated distance of target neurons of 20 (m, 200 (m, and 400 (m, respectively. This relates to a theoretically best achievable visual acuity of 2%, 0.6%, and 0.3% of normal vision, respectively (logMAR 1.7; 2.2; 2.5). These data suggest that novel strategies have to be pursued to either get closer to target structures, e.g. by the use of penetrating electrode arrays, or to create more confined stimulating fields within the retina, e.g. by the use of hexagonal arrays with multiple electrodes guarding one active electrode.

Spike timing dependent synaptic plasticity (STDP) is found in various areas of the brain, visual cortex, hippocampus and hindbrain of electric fish, etc. The synaptic modification by STDP depends on time difference between pre- and postsynaptic firing time. If presynaptic neuron fires earlier than postsynaptic neuron dose, synaptic weight is strengthened. If postsynaptic neuron fires earlier than presynaptic neuron dose, synaptic weight is weakened. This learning rule is one example of various rules (hippocampal type). The learning rule of electric fish type is reversed to the rule of hippocampal type. Changes of synaptic efficiency precisely depend on timing of pre- and postsynaptic spikes under STDP. Because of this precise dependence, it is thought that STDP plays the important role in temporal processing. Temporal processing by STDP is well known. However, the role of STDP in spatial processing is not enough understood. In present study, we propose two type spatial filter by STDP on interconnected network. One is high-pass filter when the learning rule is hippocampal type. Another is low-pass filter when the learning rule is electric fish type. We show that synaptic modification based on STDP may play important role in spatial processing.

Memory is often considered to be embedded into one of the attractors in neural dynamical systems, which provides an appropriate output depending on the initial state specified by an input. However, memory is recalled only under the presence of external inputs. Without such inputs, neural states do not provide such memorized outputs. Hence, each of memories do not necessarily correspond to an attractor of the dynamical system without input and do correspond to an attractor of the dynamics system with input. With this background, we propose that memory recall occurs when the neural activity changes to an appropriate output activity upon the application of an input. We introduce a neural network model that enables learning of such memories. After the learning process is complete, the neural dynamics is shaped so that it changes to the desired target with each input. This change is analyzed as bifurcation in a dynamical system. Conditions on timescales for synaptic plasticity are obtained to achieve the maximal memory capacity.

An extension of an evolving spiking neural network (eSNN) is proposed that enables the method to process spatio-temporal information. In this extension, an additional layer is added to the network architecture that transforms a spatio-temporal input pattern into a single intermediate high-dimensional network state which in turn is mapped into a desired class label using a fast one-pass learning algorithm. The intermediate state is represented by a novel probabilistic reservoir computing approach in which a stochastic neural model introduces a non-deterministic component into a liquid state machine. A proof of concept is presented demonstrating an improved separation capability of the reservoir and consequently its suitability for an eSNN extension.

Applying electrical stimulation to the visual cortex has been shown to produce dot-like visual perceptions called phosphenes. Artificial prosthetic vision is based on the concept that patterns of phosphenes can be used to convey visual information to blind patients. We designed a system that performs real-time simulation of phosphene perceptions evoked by cortical electrical stimulation. Phosphenes are displayed as Gaussian circular and ellipsoid spots on a randomised grid based on existing neurophysiological models of cortical retinotopy and magnification factor. The system consists of a silicon retina camera (analogue integrated vision sensor), desktop computer and headmounted display.

The Pulse Coupled Neural Network (PCNN) had been proposed as a model of visual cortex and a lot of applications to the image processing have been proposed recently. Authors also have been proposed Inhibitory Connected PCNN (IC-PCNN) which shows good performances for the color image processing. In our recent study, we had been shown that the IC-PCNN can obtain successful results for the color image segmentation. In this study, we show the effect of the inhibitory connections to the characteristics of synchronous firing assembly. Here we consider that the results will be a key to find appropriate values of inhibitory connections for the image processing using IC-PCNN. In simulations, we show that the valid domains of inhibitory connections for the color image segmentation exists.

Stacy

et.al.

demonstrated that an array of simulated hippocampal CA1 neurons exhibited stochastic resonance-like behaviors where an optimal correlation value between the sub-threshold input and the output was obtained by tuning both the noise intensity and the connection strength between the CA1 neurons. Based on this model, we proposed a simple neural network model for semiconductor devices. We carried out simulations using internal noise sources and confirmed that the correlation value between input and output in the network increased as the coupling strength increased.

Collective decision making involves on the one hand individual mental states such as beliefs, emotions and intentions, and on the other hand interaction with others with possibly different mental states. Achieving a satisfactory common group decision on which all agree requires that such mental states are adapted to each other by social interaction. Recent developments in Social Neuroscience have revealed neural mechanisms by which such mutual adaptation can be realised. These mechanisms not only enable intentions to converge to an emerging common decision, but at the same time enable to achieve shared underlying individual beliefs and emotions. This paper presents a computational model for such processes.

Congruency or incongruency between a psychological distance and a spatial distance for given visual stimuli can affect reaction time to corresponding visual stimuli in the course of visual perception. Human reacts more rapidly for congruent stimuli than incongruent one. More rapid reaction to visual stimuli is related with higher selectivity property in terms of visual selective attention. Based on this psychological evidence, we propose a new visual selective attention model reflecting the congruency or incongruency between a psychological distance and a spatial distance of visual stimuli as well as bottom-up saliency generated by spatial relativity of primitive visual features. The proposed visual selective attention model can generate more human like visual scan path by considering a psychological factor, which can focus on congruent visual stimuli with higher priority than incongruent one.

Free-energy based reinforcement learning was proposed for learning in high-dimensional state and action spaces, which cannot be handled by standard function approximation methods in reinforcement learning. In the free-energy reinforcement learning method, the action-value function is approximated as the negative free energy of a restricted Boltzmann machine. In this paper, we test if it is feasible to use free-energy reinforcement learning for real robot control with raw, high-dimensional sensory inputs through the extraction of task-relevant features in the hidden layer. We first demonstrate, in simulation, that a small mobile robot could efficiently learn a vision-based navigation and battery capturing task. We then demonstrate, for a simpler battery capturing task, that free-energy reinforcement learning can be used for on-line learning in a real robot. The analysis of learned weights showed that action-oriented state coding was achieved in the hidden layer.

In this paper, we investigate the dependence on the size and the number of memory pattern in the sensitive response to memory pattern fragments in chaotic wandering states among three types of chaotic neural network (CNN) models. From the computer experiments, the three types of chaotic neural network model show that the success ratio is high and the accessing time is short without depending on the size and the number of the memory patterns. The feature is introduced in chaotic wandering states with weaker instability of orbits and stronger randomness in memory pattern space. Thus, chaos in the three model is practical in the memory pattern search.

The stimulus-response circuits in the brain need to have flexible and fast characteristics and these circuits should be activated during the preparatory period for movement. We propose a fundamental neural network model, which can trigger the movement in response to a specific sensory input using top-down signals. When the top-down signal is received in the 1st layer, this circuit waiting for the specific sensory input is in the ready state to move. In response to the specific sensory input, synchrony is emitted and quickly transmitted to the 2nd layer. Because of more convergent connections from 1st to 2nd layers, some synchronous spikes are stably transferred and others are suppressed by the 2nd top-down signal. Thus, appropriate pairing of top-down signals to 1st and 2nd layers enables the circuits to execute an arbitrary stimulus-response behavior.

In this paper, a novel feature for shape-based image classification is proposed, in which a set of randomly dilating discs are distributed on an unknown regular or irregular shape, and the radius histogram of discs is used to represent the target shape. By such doing, a shape can be modeled by the radius histogram. The proposed feature is rather effective for shape retrieval with rotation and scaling invariance. The proposed feature is particularly effective in the retrieval of string-linked objects in which conventional approaches may fail badly. Experimental results on seven shapes and string-linked objects show that our proposed new feature is very effective in shape classification and shape-based image retrieval.

This paper addresses the problem of statistically learning typical features which characterize object categories and particular features which characterize individual objects in the categories. For this purpose, we propose a probabilistic learning method of object categories and their composition based on a bag of feature representation of co-occurring segments of objects and their context. In this method, multi-class classifiers are learned based on intra-categorical probabilistic latent component analysis with variable number of classes and inter-categorical typicality analysis. Through experiments by using images of plural categories in an image database, it is shown that the method learns probabilistic structures which characterize not only object categories but also object composition of categories, especially typical and non-typical objects and context in each category.

Forgetfulness results in interference and/or deletion. Visual short-term memory (VSTM) gradually decays as the retention time elapses, causing forgetfulness. Little is known about forgetfulness in VSTM, while substantial studies on VSTM have focused on the process of memory encoding, often with control of attention. Evidences suggest that the prefrontal cortex may contribute to maintain short-term memory during extended retention periods while the posterior parietal cortex may support the capacity-limited store of visual items. Here we conduct a visual memory experiment to measure the levels and source of memory decay. In particular, multiple retention intervals were used between the presentation of a study array and a cue. The results show that the correct response to cued objects decreased as retention interval increased while that to uncued and novel objects remain unchanged. These data indicate that forgetfulness in VSTM is primarily due to interference rather than memory deletion.

Alternative least squares (ALS) algorithm is considered as a “work-horse” algorithm for general tensor factorizations. For nonnegative tensor factorizations (NTF), we usually use a nonlinear projection (rectifier) to remove negative entries during the iteration process. However, this kind of ALS algorithm often fails and cannot converge to the desired solution. In this paper, we proposed a novel algorithm for NTF by recursively solving nonnegative quadratic programming problems. The validity and high performance of the proposed algorithm has been confirmed for difficult benchmarks, and also in an application of object classification.

Physical activity is often considered an important factor in handling mood regulation and depression. This paper presents a computational model of this role of physical activity in mood regulation. It is shown on the one hand how a developing depression can go hand in hand with a low level of physical activity, and on the other hand, how Exercise Therapy is able to reverse this pattern and make the depression disappear. Simulation results are presented, and properties are formally verified against these simulation runs.

Three methods for representation of hypertext based on links, terms and text compressibility have been compared to check their usefulness in document classification. Documents for classification have been selected from the Wikipedia articles taken from five distinct categories. For each representation dimensionality reduction by Principal Component Analysis has been performed, providing rough visual presentation of the data. Compression-based feature space representation needed about 5 times less PCA vectors than the term or link-based representations to reach 90% cumulative variance, giving comparable results of classification by Support Vector Machines.

In recent years, in order to cope with spam based attacks, there have been many efforts made towards the clustering of spam emails. During the clustering process, many statistical features (

e.g.

, the size of emails) are used for calculating similarities between spam emails. In many cases, however, some of the features may be redundant or contribute little to the clustering process. Feature selection is one of the most typical methods used to identify a subset of key features from an initial set. In this paper, we propose a heuristic-based feature selection method for clustering spam emails. Unlike the existing methods in that they make the combinations of given features and evaluate them using data mining and machine learning techniques, our method focuses on evaluating each feature according to only its value distribution in spam clusters. With our method, we identified 4 significant features which yielded a clustering accuracy of 86.33% with low time complexity.

In this paper, an extension of Subjective Logic (SL) is presented which uses semantic information from a document to find ‘opinions’ about a sentence. This method computes semantic overlap of events (words or sentences) using Hierarchical Document Signature (HDS) and uses it as evidence to formulate SL belief measures to order sentences according to their importance. Stronger the opinion, more is the significance. These significant sentences then form extractive summaries of the document. The experimental results show that summaries generated by this method are more similar to human generated ones have outperformed the baseline summaries on average over all the data sets considered.

Semantic web is gaining popularity as a candidate for next generation World Wide Web. In recent years, there has been a tremendous increase of using Internet on mobile devices and search engines are considered essential for Internet users. The existing search engines have been designed for powerful computers and are highly resource hungry, while mobiles have limited computational resources. In this paper, we study the use of comprehension for aiding the search engine results for mobile users. Our preliminary evaluation shows the promising results for comprehension generation.

A novel text classification approach is proposed in this paper based on deep belief network. Deep belief network constructs a deep architecture to obtain the high level abstraction of input data, which can be used to model the semantic correlation among words of documents. After basic features are selected by statistical feature selection measures, a deep belief network with discriminative fine tuning strategy is built on basic features to learn high level deep features. A support vector machine is then trained on the learned deep features. The proposed method outperforms traditional classifier based on support vector machine. As a dimension reduction strategy, the deep belief network also outperforms the traditional latent semantic indexing method. Detailed experiments are also made to show the effect of different fine tuning strategies and network structures on the performance of deep belief network.

User click behaviors reflect his preference in Web search processing objectively, and it is very important to give a proper interpretation of user click for improving search results. Previous click models explore the relationship between user examines and latent clicks web document obtained by search result page via multiple-click model, such as the independent click model(ICM) or the dependent click model(DCM),which the examining-next probability only depends on the current click. However, user examination on a search result page is a continuous and relevant procedure. In this paper, we attempt to explore the historical clicked data using a probability click tracking model(PCTM). In our approach, the examine-next probability is decided by the click variables of each clicked result. We evaluate the proposed model on a real-world data set obtained from a commercial search engine. The experiment results illustrate that PCTM can achieve the competitive performance compared with the existing click models under standard metrics.

Identifying intentions of users plays a crucial role in providing better user services, such as web-search and automated message-handling. There is a significant literature on extracting speakers’ intentions and speech acts from spoken words, and this paper proposes a novel approach on extracting intentions from non-spoken words, such as web-search query texts, and text messages. Unlike spoken words, such as in a telephone conversation, text messages often contain longer and more descriptive sentences than conversational speech. In addition, text messages contain a mix of conversational speech and non-conversational contents such as documents.

The experiments describe a first attempt to extracting writers’ intentions from Usenet text messages. Messages are segmented into sentences, and then each sentence is converted into a tuple (

performative, proposition

) using a dialogue act classifier. The writers’ intentions are then formulated from the tuples using constraints on felicitous human communication.

In many real world applications, different features (or multiview data) can be obtained and how to duly utilize them in dimension reduction is a challenge. Simply concatenating them into a long vector is not appropriate because each view has its specific statistical property and physical interpretation. In this paper, we propose a multiview stochastic neighbor embedding (m-SNE) that systematically integrates heterogeneous features into a unified representation for subsequent processing based on a probabilistic framework. Compared with conventional strategies, our approach can automatically learn a combination coefficient for each view adapted to its contribution to the data embedding. Also, our algorithm for learning the combination coefficient converges at a rate of

$O\left(1/k^2\right)$

, which is the optimal rate for smooth problems. Experiments on synthetic and real datasets suggest the effectiveness and robustness of m-SNE for data visualization and image retrieval.

This paper proposes a novel approach to learn highly scalable Control Policies (CPs) of basis movement skills from

multiple

demonstrations. In contrast to conventional studies with a

single

demonstration, i.e., Dynamic Movement Primitives (DMPs) [1], our approach efficiently encodes multiple demonstrations by shaping a parametric-attractor landscape in a set of differential equations. This approach allows the learned CPs to synthesize novel movements with novel motion styles by specifying the linear coefficients of the bases as parameter vectors without losing useful properties of DMPs, such as stability and robustness against perturbations. For both discrete and rhythmic movement skills, we present a unified learning procedure for learning a parametric-attractor landscape from multiple demonstrations. The feasibility and highly extended scalability of DMPs are demonstrated on an actual dual-arm robot.

Adaboost, one of the most famous boosting algorithms, has been used in various fields of machine learning. With its success, many people focus on the improvement of this algorithm in different ways. In this paper, we propose a new algorithm to improve the performance of adaboost by the theory of multi-view learning, which is called Embedded Multi-view Adaboost (EMV-Adaboost). Different from some approaches used by other researchers, we not only blend multi-view learning into adaboost thoroughly, but also output the final hypothesis in a new form of the combination of multi-learners. These theories are combined into a whole in this paper. Furthermore, we analyze the effectiveness and feasibility of EMV-Adaboost. Experimental results with our algorithm validate its effectiveness.

A method of speaker recognition which uses feature vectors of pole distribution derived from piecewise linear predictive coefficients obtained by bagging CAN2 (competitive associative net 2) is presented and analyzed. The CAN2 is a neural net for learning efficient piecewise linear approximation of nonlinear function, and the bagging CAN2 (bootstrap aggregating version of CAN2) is used to obtain statistically stable multiple linear predictive coefficients. From the coefficients, the present method obtains a number of poles which are supposed to reflect the shape of the speaker’s vocal tract. Then, the pole distribution is used as a feature vector for speaker recognition. The effectiveness is analyzed and validated using real speech data.

Several authors have recently proposed sparse estimation techniques for

time-varying

Markov networks, in which both graph structures and model parameters may change with time. In this study, we extend a previous approach with a low-rank assumption on the matrix of parameter sequence, utilizing a recent technique of nuclear norm regularization. This can potentially improve the estimation performance particularly in such cases that the local smoothness assumed in previous studies do not really hold. Then, we derive a simple algorithm based on the alternating direction method of multipliers (ADMM) which can effectively utilize the separable structure of our convex minimization problem. With an artificially-generated dataset, its superior performance in structure learning is demonstrated.

Metric learning is the task of learning a distance metric from training data that reasonably identifies the important relationships between the data. An appropriate distance metric is of considerable importance for building accurate classifiers. In this paper, we propose a novel supervised metric learning method, nearest hit-misses component analysis. In our method, the margin is first defined with respect to the nearest hits (nearest neighbors from the same class) and the nearest misses (nearest neighbors from the different class), and then the distance metric is trained by maximizing the margin while minimizing the distance between each sample and its nearest hits. We further introduce a regularization term to alleviate overfitting. Moreover, the proposed method can perform metric learning and dimensionality reduction simultaneously. Comparative experiments with the state-of-the-art metric learning methods on various real-world data sets demonstrate the effectiveness of the proposed method.

In compressed sensing and statistical society, dozens of algorithms have been developed to solve ℓ

1

penalized least square regression, but

constrained sparse quadratic optimization

(SQO) is still an open problem. In this paper, we propose

backward-forward least angle shrinkage

(BF-LAS), which provides a scheme to solve general SQO including sparse eigenvalue minimization. BF-LAS starts from the dense solution, iteratively shrinks unimportant variables’ magnitudes to zeros in the backward step for minimizing the ℓ

1

norm, decreases important variables’ gradients in the forward step for optimizing the objective, and projects the solution on the feasible set defined by the constraints. The importance of a variable is measured by its correlation w.r.t the objective and is updated via least angle shrinkage (LAS). We show promising performance of BF-LAS on sparse dimension reduction.

Recommendation, in the filed of machine learning, is known as a technique of identifying user preferences to new items with ratings from recommender systems. Recently, one novel recommendation model using Green’s function treats recommendation as the process of label propagation. Although this model outperforms many standard recommendation methods, it suffers from information loss during graph construction because of data sparsity. In this paper, aiming at solving this problem and improving prediction accuracy, we propose an enhanced semi-supervised Green’s function recommendation model. The main contributions are two-fold: 1) To reduce information loss, we propose a novel graph construction method with global and local consistent similarity; 2) We enhance the recommendation algorithm with the multi-class semi-supervised learning framework. Finally, experimental results on real world data demonstrate the effectiveness of our model.

In this paper, we propose a reinforcement learning method using Kohonen Feature Map Probabilistic Associative Memory based on Weights Distribution and Area Neuron and Increase and Decrease (KFMPAM-WD-NID). The proposed method is based on the actor-critic method, and the actor is realized by the KFMPAM-WD-NID. The KFMPAM-WD-NID is based on the self-organizing feature map, and it can realize successive learning and one-to-many associations. Moreover, the weights distribution in the Map Layer can be modified by the increase and decrease of neurons in each area. The proposed method makes use of these properties in order to realize the learning during the practice of task. We carried out a series of computer experiments, and confirmed the effectiveness of the proposed method in the pursuit problem.

The purpose of this article is to present a novel learning paradigm that extracts reward-related low-dimensional state space by combining correlation-based learning like Input Correlation Learning (ICO learning) and reward-based learning like Reinforcement Learning (RL). Since ICO learning can quickly find a correlation between a state and an unwanted condition (e.g., failure), we use it to extract low-dimensional feature space in which we can find a failure avoidance policy. Then, the extracted feature space is used as a prior for RL. If we can extract proper feature space for a given task, a model of the policy can be simple and the policy can be easily improved. The performance of this learning paradigm is evaluated through simulation of a cart-pole system. As a result, we show that the proposed method can enhance the feature extraction process to find the proper feature space for a pole balancing policy. That is it allows a policy to effectively stabilize the pole in the largest domain of initial conditions compared to only using ICO learning or only using RL without any prior knowledge.

Understanding non-stationary effects is one of the key challenges in data analysis. However, in many settings the observation is a mixture of stationary and non-stationary sources. The aim of Stationary Subspace Analysis (SSA) is to factorize multivariate data into its stationary and non-stationary components. In this paper, we propose a novel SSA algorithm (ASSA) that extracts stationary sources from multiple time series blocks. It has a globally optimal solution under certain assumptions that can be obtained by solving a generalized eigenvalue problem. Apart from the numerical advantages, we also show that compared to the existing method, fewer blocks are required in ASSA to guarantee the identifiability of the solution. We demonstrate the validity of our approach in simulations and in an application to domain adaptation.

We propose a new object classification model, which is applied to a computer-vision-based traffic surveillance system. The main issue in this paper is to recognize various objects on a road such as vehicles, pedestrians and unknown backgrounds. In order to achieve robust classification performance against translation and scale variation of the objects, we propose new C1-like features which modify the conventional C1 features in the Hierarchical MAX model to get the computational efficiency. Also, we develop a new adaptively boosted Gaussian mixture model to build a classifier for multi-class objects recognition in real road environments. Experimental results show the excellence of the proposed model for multi-class object recognition and can be successfully used for constructing a traffic surveillance system.

The aim of this paper is to present an alternative ensemble-based drift learning method that is applicable to cascaded ensemble classifiers. It is a hybrid of

detect-and-retrain

and

constant-update

approaches, thus being equally responsive to both gradual and abrupt concept drifts. It is designed to address the issues of concept forgetting, experienced when altering weights of individual ensembles, as well as real-time adaptability limitations of classifiers that are not always possible with ensemble structure-modifying approaches. The algorithm achieves an effective trade-off between accuracy and speed of adaptations in time-evolving environments with unknown rates of change and is capable of handling large volume data-streams in real-time.

In the high dimensional regression and classification, we often need the feature selection and the dimensionality reduction to cope with the huge computational cost and the over-fitting of the parameters. Canonical Correlation Analysis (CCA) and their hierarchical extension (includes Bayesian method) was proposed for this purpose. However, the real data set often violates the assumption of the linearity of CCA. Thus, we need the non-linear extension of them. To solve this problem, we propose the Bayesian mixture of CCA and give the efficient inference algorithm by Gibbs sampling. We show that the proposed method is the scalable natural extension of CCA and RBF type neural networks for the high dimensional non-linear problems.

Following the introduction by Tibshirani of the LASSO technique for feature selection in regression, two algorithms were proposed by Osborne et al. for solving the associated problem. One is an homotopy method that gained popularity as the LASSO modification of the LARS algorithm. The other is a finite-step descent method that follows a path on the constraint polytope, and seems to have been largely ignored. One of the reason may be that it solves the constrained formulation of the LASSO, as opposed to the more practical regularized formulation. We give here an adaptation of this algorithm that solves the regularized problem, has a simpler formulation, and outperforms state-of-the-art algorithms in terms of speed.

Label propagation (LP) is used in the framework of semi-supervised learning. In this paper, we propose a novel method of logistic label propagation (LLP). The proposed method employs logistic functions for accurately estimating the label values as the posterior probabilities. In LLP, the label of newly input sample is efficiently estimated by using the optimized coefficients in the logistic function, without such recomputation of all label values as in original LP. In the experiments on classification, the proposed method produced more reliable label values at the high degree of confidence than LP and ordinary logistic regression. In addition, even for a small portion of the labeled samples, the error rates by LLP were lower than those by the logistic regression.

In this paper a holistic method and a local method based on decision template ensemble are investigated. In addition by combining both methods, a new hybrid method for boosting the performance of the system is proposed and evaluated with respect to robustness against small sample size problem in face recognition. Inadequate and substantial variations in the available training samples are the two challenging obstacles in classification of an unknown face image. At first in this novel multi learner framework, a decision template is designed for the global face and a set of decision templates is constructed for each local part of the face as a complement to the previous part. The prominent results demonstrate that, the new hybrid method based on fusion of weighted multiple decision templates is superior to the other classic combining schemes for both ORL and Yale data sets. In addition when the global and the local components of the face are combined together the best performance is achieved.

In reinforcement learning, Markov decision processes are the most popular stochastic sequential decision processes. We frequently assume stationarity or ergodicity, or both to the process for its analysis, but most stochastic sequential decision processes arising in reinforcement learning are in fact, not necessarily Markovian, stationary, or ergodic. In this paper, we give an information-spectrum analysis of return maximization in more general processes than stationary or ergodic Markov decision processes. We also present a class of stochastic sequential decision processes with the necessary condition for return maximization. We provide several examples of best sequences in terms of return maximization in the class.

We report relationships between the effects of noise and applied constant currents on the behavior of a system of excitable elements. The analytical approach based on the nonlinear Fokker-Planck equation of a mean-field model allows us to study the effects of noise

without approximations

only by dealing with deterministic nonlinear dynamics . We find the similarity, with respect to the occurrence of oscillations involving subcritical Hopf bifurcations, between the systems of an excitable element with applied constant currents and mean-field coupled excitable elements with noise.

In this paper, we present a first study which learns the ECOC matrix as well as dichotomizers simultaneously from data; these two steps are usually conducted independently in previous methods. We formulate our learning model as a sequence of concave-convex programming problems and develop an efficient alternative minimization algorithm to solve it. Extensive experiments over eight real data sets and one image analysis problem demonstrate the advantage of our model over other state-of-the-art ECOC methods in multi-class classification.

Any natural or biological signal can be seen as a linear combination of meaningful and non-meaningful structures. According to the theory of multiresolution wavelet expansions, one can quantify the degree of information those structures using entropy and then select the most meaningful ones. Herein we propose to use adaptive time and frequency transform (ATFT) to measure wavelet entropy, where one line of approach to ATFT is to use a matching pursuit (MP) framework. The proposed method is tested on a set of heartbeat intervals whose population is composed of healthy and pathological subjects. Our results show that wavelet entropy measure based on MP decomposition can capture significant differences between the analyzed cardiac states that are intrinsically related to the structure of the signal.

Rough set theory has been applied in many areas such as knowledge discovery and has the ability to deal with incomplete, imprecise or inconsistent information. The traditional association rule which should be fixed in order to avoid both that only trivial rules are retained and also that interesting rules are not discarded. In this paper, the new data mining techniques applied to ordinal scale data, which has the ability to handle the uncertainty in the classing process. The aim of the research is to provide a new association rule concept, which is using ordinal scale data.

According to the high accuracy of load model in power system, a novel dynamic population variation genetic programming with Kalman operator for load model in power system is proposed. First, an evolution load model called initial model in power system evolved by dynamic variation population genetic programming is obtained which has higher accuracy than traditional models. Second, parameters in initial model are optimized by Kalman operator for higher accuracy and an optimization model is obtained. Experiments are used to illustrate that evolved model has higher accuracy 4.6~48% than traditional models and It is also proved the performance of evolved model is prior to RBF network. Furthermore, the optimization model has higher accuracy 7.69~81.3% than evolved model.

Today, iris recognition is reported as one of the most reliable biometric approaches. With the strength of contactless, the hygienic issue is therefore minimized and the possibility of disease infection through the device as a medium is low. In this paper, a MMU2 iris database with such consideration is created for this study. Moreover, the proposed iris segmentation method has shown its robustness with intelligent fuzzy supports. Furthermore, it has been tested with 18414 iris images across different databases available in the public without changing any threshold values and parameters. The experiment results show a total of 17915 or 97.30%.of correct iris segmentation.

Due to the combinatorial nature of the facility layout problem (FLP), several heuristic and meta-heuristic approaches have been developed to obtain good rather than optimal solutions. Unfortunately, most of these approaches are predominantly on a single objective. However, the real-world FLPs are multi-objective by nature and only very recently have meta-heuristics been designed and used in multi-objective FLP. These most often use the weighted sum method to combine the different objectives and thus, inherit the well-known problems of this method. This paper presents an adaptive local search based genetic algorithm (GA) for solving the multi-objective FLP that presents the layouts as a set of Pareto-optimal solutions optimizing both quantitative and qualitative objectives simultaneously. Unlike the conventional local search, the proposed adaptive local search scheme automatically determines whether local search is used in a GA loop or not. The results obtained show that the proposed algorithm outperforms the other competing algorithms and can find near-optimal and non-dominated solutions by optimizing multiple criteria simultaneously.

In this paper we present an approach to generate ensemble of classifiers using non–uniform layered clustering. In the proposed approach the dataset is partitioned into variable number of clusters at different layers. A set of base classifiers is trained on the clusters at different layers. The decision on a pattern at each layer is obtained from the classifier trained on the nearest cluster and the decisions from the different layers are fused using majority voting to obtain the final verdict. The proposed approach provides a mechanism to obtain the optimal number of layers and clusters using a Genetic Algorithm. Clustering identifies difficult–to–classify patterns and layered non–uniform clustering approach brings in diversity among the base classifiers at different layers. The proposed method performs relatively better than the other state–of–art ensemble classifier generation methods as evidenced from the experimental results.

In Recommendation System, Collaborative Filtering by Clustering is a technique to predict interesting items from users with similar preferences. However, misleading prediction could be taken place by items with very rare ratings. These missing data could be considered as noise and influence the cluster’s centroid by shifting its position. To overcome this issue, we proposed a new novel fuzzy algorithm that formulated objective function with Exponential equation (XFCM) in order to enhance ability to assign degree of membership. XFCM embeds noise filtering and produces membership for noisy data differently to other Fuzzy Clustering. Thus the centroid is robust in the noisy environment. The experiments on Collaborative Filtering dataset show that centroid produced by XFCM is robust by the improvement of prediction accuracy 6.12% over Fuzzy C-Mean (FCM) and 9.14% over Entropy based Fuzzy C-Mean (FCME).

This paper deals with the characterization of the fitness landscape of multimodal functions and how it can be used to choose the most appropriate evolutionary algorithm for a given problem. An algorithm that obtains a general description of real valued multimodal fitness landscapes in terms of the relative number of optima, their sparseness, the size of their attraction basins and the evolution of this size when moving away from the global optimum is presented and used to characterize a set of well-known multimodal benchmark functions. To illustrate the relevance of the information obtained and its relationship to the performance of evolutionary algorithms over different fitness landscapes, two evolutionary algorithms, Differential Evolution and Covariance Matrix Adaptation, are compared over the same benchmark set showing their behavior depending on the multimodal features of each landscape.

Ensemble learning refers to the methods that combine multiple models to improve the performance. Ensemble methods, such as stacking, have been intensively studied, and can bring slight performance improvement. However, there is no guarantee that a stacking algorithm outperforms all base classifiers. In this paper, we propose a new stacking algorithm, where the predictive scores of each possible class label returned by the base classifiers are firstly collected by the meta-learner, and then all possible class labels are reranked according to the scores. This algorithm is able to find the best linear combination of the base classifiers on the training samples, which make sure it outperforms all base classifiers during training process. The experiments conducted on several public datasets show that the proposed algorithm outperforms the baseline algorithms and several state-of-the-art stacking algorithms.

Constrained Optimization is one of the most active research areas in the computer science, operation research and optimization fields. The Differential Evolution (DE) algorithm is widely used for solving continuous optimization problems. However, no single DE algorithm performs consistently over a range of Constrained Optimization Problems (COPs). In this research, we propose a Self-Adaptive Operator Mix Differential Evolution algorithm, indicated as SAOMDE, for solving a variety of COPs. SAOMDE utilizes the strengths of three well-known DE variants through an adaptive learning process. SAOMDE is tested by solving 13 test problems. The results showed that SAOMDE is not only superior to three single mutation based DE, but also better than the stateof- the-art algorithms.

We previously proposed multiple particle swarm optimizers with diversive curiosity (MPSO

α

/DC). Its main features are to introduce diversive curiosity and localized random search into MPSO to comprehensively manage the trade-off between exploitation and exploration for preventing stagnation and improving the search efficiency. In this paper, we further extend these features to multiple particle swarm optimizers with inertia weight and multiple canonical particle swarm optimizers to create two analogues, called MPSOIW

α

/DC and MCPSO

α

/DC. To demonstrate the effectiveness of these proposals, computer experiments on a suite of multidimensional benchmark problems are carried out. The obtained results show that the search performance of the MPSO

α

/DC is superior to that of both the MPSOIW

α

/DC and MCPSO

α

/DC, and they have better search efficiency compared to other methods such as the convenient cooperative PSO and a real-coded genetic algorithm.

Automatic patent classification is of great practical value for saving a lot of resources and manpower. As real patent classification tasks are often very-large scale and serious imbalanced such as patent classification, using traditional pattern classification techniques has shown inefficient and ineffective. In this paper, an adaptive ensemble learning strategy using an assistant classifier is proposed to improve generalization accuracy and the efficiency. The effectiveness of the method is verified on a group of real patent classification tasks which are decomposed in multiple ways by using different algorithms as the assistant classifiers.

Ant Colony System is a viable method for routing problems such as TSP, because it provides a dynamic parallel discrete search algorithm. Ants in the conventional ACS are unable to learn as they are. In the present paper, we propose to combine ACS with reinforcement learning to make decision adaptively. We succeeded in making decision adaptively in the Ant Colony system and in improving the performance of exploration.

Cloud computing has become a main medium for Software as a Service (SaaS) hosting as it can provide the scalability a SaaS requires. One of the challenges in hosting the SaaS is the placement process where the placement has to consider SaaS interactions between its components and SaaS interactions with its data components. A previous research has tackled this problem using a classical genetic algorithm (GA) approach. This paper proposes a cooperative coevolutionary algorithm (CCEA) approach. The CCEA has been implemented and evaluated and the result has shown that the CCEA has produced higher quality solutions compared to the GA.

In this paper we present an artificial bee colony (ABC) algorithm to solve the quadratic multiple knapsack problem (QMKP) which can be considered as an extension of two well known knapsack problems viz. multiple knapsack problem and quadratic knapsack problem. In QMKP, profit values are associated not only with individual objects but also with pairs of objects. Profit value associated with a pair of objects is added to the total profit if both objects of the pair belong to the same knapsack. The objective of this problem is to assign each object to at most one knapsack in such a way that the total weight of the objects in each knapsack should not exceed knapsack’s capacity and the total profit of all the objects included into the knapsacks is maximized. We have compared our approach with three genetic algorithms and a stochastic hill climber. Computational results show the effectiveness of our approach.

Of the consumers who often patronize retail stores, 87 of 100 respondents visited a convenience store in the past three months. The superstore/hypermarket and the supermarket came in second and third, respectively. This demonstrates that retail channels are essential to the day-to-day life of the common populace. With the social and economic evolution, not only have product sales and shopping habits changed, but the current marketing concepts have also changed from being product-oriented to being consumer-oriented. In this research, we first provide new algorithms modified from the Apriori algorithm. The new approach can be applied in finding association rules, which can handle an uncertainty, combined with the rough set theory, and then to find the influence degree of the consumer preferences variables for the marketing decision-makers used.

In today’s volatile manufacturing scenario, manufacturing facilities must operate in a dynamic and market-driven environment in which production rates and production mixes are continuously changing. To operate efficiently within such an environment, the facility layout needs to be adaptable to changes. The dynamic facility layout problem (DFLP) deals with changes of layout over time. DFLPs are usually solved just considering quantitative aspect of layout alone, ignoring the qualitative aspect. Few attempts have been made to date to deal with the multi-objective DFLP. These most often use the weighted-sum method to combine different objectives and thus, inherit the well-known problems of this method. The objective of this paper is to introduce an evolutionary approach for solving multi-objective DFLP that presents the layout as a set of Pareto-optimal solutions optimizing both quantitative and qualitative objectives simultaneously. Experimental results obtained with the proposed approach are promising.

We propose in this paper a new approach for topological hierarchical tree clustering inspired from self-assembly behavior of artificial ants. Our method called THT (Topologial Hierarchical Tree) builds, autonomously and simultaneously, a topological and hierarchical partitioning of data. Each ”cluster” associated to one cell of a 2D grid is modeled by a tree. The artificial ants that we define dissimilarly build a tree where each ant represents a node/data. The benefit of this novel approach is the intuitive representation of hierarchical relations in the data. This is especially appealing in explorative data mining applications, allowing the inherent structure of the data unfold in highly intuitive fashion.

We introduce a compositional probabilistic model for tree-structured data that defines a bottom-up generative process from the leaves to the root of a tree. Contextual state transitions are introduced from the joint configuration of the children to the parent nodes, allowing hidden states to model the co-occurrence of substructures among the child subtrees. A mixed memory approximation is proposed to factorize the joint transition matrix as a mixture of pairwise transitions. A comparative experimental analysis shows that the proposed approach is able to better model deep structures with respect to top-down approaches.

With the rapid growth of the Internet, more and more people interact with their friends in online social networks like Facebook. Currently, the privacy issue of online social networks becomes a hot and dynamic research topic. Though some privacy protecting strategies are implemented, they are not stringent enough. Recently, Semi-Supervised Learning (SSL), which has the advantage of utilizing the unlabeled data to achieve better performance, attracts much attention from the web research community. By utilizing a large number of unlabeled data from websites, SSL can effectively infer hidden or sensitive information on the Internet. Furthermore, graph-based SSL is much more suitable for modeling real-world objects with graph characteristics, like online social networks. Thus, we propose a novel Community-based Graph (CG) SSL model that can be applied to exploit security issues in online social networks, then provide two consistent algorithms satisfying distinct needs. In order to evaluate the effectiveness of this model, we conduct a series of experiments on a synthetic data and two real-world data from StudiVZ and Facebook. Experimental results demonstrate that our approach can more accurately and confidently predict sensitive information of online users, comparing to previous models.

In this paper, a novel hierarchical lossless image coding scheme using the cellular neural network (CNN) is proposed. The coding architecture of the proposed method is based on the lifting scheme that is one of the scalable coding framework for still images, and its coding performance strongly depends on the prediction ability. To cope with this spontaneously characteristic, an image interpolation is modeled by an optimal problem that minimizes the prediction error. To achieve the high accuracy prediction with a low coding rate, two types of templates are used for dealing with the local structure of the image, and the CNN parameters are decided by the minimum coding rate learning. In the coding layer, the arithmetic coder with context modeling is used for obtaining a high coding efficiency. Experimental results in various standard test images suggest that the coding performance of our proposed method is better than that of conventional hierarchical coding schemes.

In reinforcement learning, when dimensionality of the state space increases, making use of state abstraction seems inevitable. Among the methods proposed to solve this problem, decision tree based methods could be useful as they provide automatic state abstraction. But existing methods use univariate, therefore axis-aligned, splits in decision nodes, imposing hyper-rectangular partitioning of the state space. In some applications, multivariate splits can generate smaller and more accurate trees. In this paper, we use oblique decision trees as an instance of multivariate trees to implement state abstraction for reinforcement learning agents. Simulation results on mountain car and puddle world tasks show significant improvement in the average received rewards, average number of steps to finish the task, and size of the trees both in learning and test phases.

We present a classifier-combination experimental framework for part-of-speech (POS) tagging in which four different POS taggers are combined in order to get a better result for sentence similarity using Hierarchical Document Signature (HDS). It is important to abstract information available to form humanly accessible structures. The way people think and talk is hierarchical with limited information presented in any one sentence, and that information is always linked together to further information. As such, HDS is a significant way to represent sentences when finding their similarity. POS tagging plays an important role in HDS. But POS taggers available are not perfect in tagging words in a sentence and tend to tag words improperly if they are either not properly cased or do not match the corpus dataset by which these taggers are trained. Thus, different weighted voting strategies are used to overcome some of these drawbacks of these existing taggers. Comparisons between individual taggers and combined taggers under different voting strategies are made. Their results show that the combined taggers provide better results than the individual ones.

Hierarchical clustering is very versatile in real world applications. However, due to the issue of higher computational complexity from which automated hierarchical clustering algorithms suffer, the user can hardly correct possible misclassifications from the tree-structured nature of clusters. Visualization is a powerful technique for data analysis, however, most of the existing cluster visualization techniques are mainly used for displaying clustering results. In order for the user to be directly involved in the process of discovering nested cluster structures, we introduce a visualization technique, called HOV

3

, to detect clusters and their internal cluster structure. As a result, our approach provides the user an effective method for the discovery of nested cluster structures by visualization.