Intelligent Data Engineering and Automated Learning — IDEAL 2002

Many practical applications are related to frequent sequential pattern mining, ranging from Web Usage Mining to Bioinformatics. To ensure an appropriate extraction cost for useful mining tasks, a key issue is to push the user-defined constraints deep inside the mining algorithms. In this paper, we study the search for frequent sequential patterns that are also similar to an user-defined reference pattern. While the effective processing of the frequency constraints is well-understood, our contribution concerns the identification of a relaxation of the similarity constraint into a convertible anti-monotone constraint. Both constraints are then used to prune the search space during a levelwise search. Preliminary experimental validations have confirmed the algorithm efficiency.

The automatic induction of classification rules from examples in the form of a classification tree is an important technique used in data mining. One of the problems encountered is the overfitting of rules to training data. In some cases this can lead to an excessively large number of rules, many of which have very little predictive value for unseen data. This paper describes a means of reducing overfitting known as J-pruning, based on the J-measure, an information theoretic means of quantifying the information content of a rule. It is demonstrated that using J-pruning generally leads to a substantial reduction in the number of rules generated and an increase in predictive accuracy. The advantage gained becomes more pronounced as the proportion of noise increases.

A resource manager (RM), a fuzzy logic based expert system, has been developed. The RM automatically allocates resources in real-time over many dissimilar agents. A new data mining algorithm that uses a genetic program, an algorithm that evolves other computer programs, as a data mining function has been developed to evolve fuzzy decision trees for the resource manager. It not only determines the fuzzy decision tree structure it also creates fuzzy rules while mining scenario databases. The genetic program’s structure is discussed as well as the terminal set, function set, the operations of cross-over and mutation, and the construction of the database used for data mining. Finally, an example of a fuzzy decision tree generated by this algorithm is discussed.

A fuzzy logic based expert system has been developed that automatically allocates resources in real-time over many dissimilar platforms. An approach is being explored that involves embedding the resource manager in an electronic game environment. The game allows a human expert to play against the resource manager in a simulated battlespace with each of the defending platforms being exclusively directed by the fuzzy resource manager and the attacking platforms being controlled by the human expert or operating autonomously under their own logic. This approach automates the data mining problem. The game automatically creates a database reflecting the domain expert’s knowledge, it calls a data mining function, a genetic algorithm, for data mining of the data base as required. The game allows easy evaluation of the information mined in the second step. The criterion for re-optimization is discussed. The mined information is extremely valuable as indicated by demanding scenarios.

We introduce a method for finding temporal and atemporal relations in nominal, causal data. This method searches for relations among variables that characterize the behavior of a single system. Data are gathered from variables of the system, and used to discover relations among the variables. In general, such rules could be causal or acausal. We formally characterize the problem and introduce RFCT, a hybrid tool based on the C4.5 classification software. By performing appropriate preprocessing and postprocessing, RFCT extends C4.5’s domain of applicability to the unsupervised discovery of temporal relations among temporally ordered nominal data.

Automated personalisation offers a major improvement in the use of large Web sites. These systems learn from a user and suggest where on the Web site a user might move. Self-organising maps (SOM) may also be considered as a potential tool for Web data analysis. In this paper, the use of SOM analysis for automated personalisation of Internet users is demonstrated. The map was obtained by training a self-organising network with user demographics; click stream data were used to calculate the probabilities of user behaviour on the Web site. Thus, the map can be used for personalisation of users and to calculate the probabilities of each neuron in predicting where the user will next move on the Web site. The results indicate that SOM analysis can successfully process Web information.

In this paper, we investigate data abstractions for mining association rules with numerical conditions and boolean consequents as a target class. The act of our abstraction corresponds to joining some consecutive primitive intervals of a numerical attribute. If the interclass variance for two adjacent intervals is less than a given admissible upper-bound ∈, then they are combined together into an extended interval. Intuitively speaking, a low value of the variance means that the two intervals can provide almost the same posterior class distributions. This implies few properties or characteristics about the class would be lost by combining such intervals together. We discuss a bottom-up method for finding maximally extended intervals, called maximal appropriate abstraction. Based on such an abstraction, we can reduce the number of extracted rules, still preserving almost the same quality of the rules extracted without abstractions. The usefulness of our abstraction method is shown by preliminary experimental results.

This paper proposes, for query optimizing on Data warehouses, the use of range-encoded bitmap index to calculate aggregates. By using space optimal range-encoded bitmap index for range and aggregate predicates, the need of separate indexes for these operations can be eliminated. The proposed algorithm also uses the population ratio of 1’s in a bitmap to decide whether the bitmap has to be scanned from the disk at all; thus exploiting the opportunity of skipping many bitmap scans since processing them does not affect the solution. These optimizations result in significant improvement in query evaluation time.

This paper describes and evaluates T3, an algorithm that builds trees of depth at most three, and results in high accuracy whilst keeping the size of the tree reasonably small. T3 is an improvement over T2 in that it builds larger trees and adopts a less greedy approach. T3 gave better results than both T2 and C4.5 when run against publicly available data sets: T3 decreased classification error on average by 47% and generalisation error by 29%, compared to T2; and T3 resulted in 46% smaller trees and 32% less classification error compared to C4.5. Due to its way of handling unknown values, T3 outperforms C4.5 in generalisation by 99% to 66%, on a specific medical dataset.

Image retrieval by subjective content has been recently addressed by the Kansei engineering community in Japan. Such information retrieval systems aim to include subjective aspects of the users in the querying criteria. While many techniques have been proposed in modeling such users’ aspects, little attention has been placed on analyzing the amount of information involved in this modeling process and the multi-interpretation of such information. We propose a data warehouse as a support for the mining of the multimedia user feedback. A unique characteristic of our data warehouse lays in its ability to manage multiple hierarchical descriptions of images. Such characteristic is necessary to allow the mining of such data, not only at different levels of abstraction, but also according to multiple interpretation of their content. The proposed data warehouse has been used to support the adaptation of web-based image retrieval systems by impression words.

This paper presents a methodology for applying the principles of evolutionary computation to knowledge discovery in databases by evolving SQL queries that describe datasets. In our system, the fittest queries are rewarded by having their attributes being given a higher probability of surviving in subsequent queries. The advantages of using SQL queries include their readability for non-experts and ease of integration with existing databases. The evolutionary algorithm (EA) used in our system is very different from existing EAs, but seems to be effective and efficient according to the experiments to date with three different testing data sets.

Previous studies on frequent pattern discovery from temporal sequence mainly consider finding global patterns, where every record in a sequence contributes to support the patterns. In this paper, we present a novel problem class that is the discovery of local sequential patterns, which only a subsequence of the original sequence exhibits. The problem has a two-dimensional solution space consisting of patterns and temporal features, therefore it is impractical that use traditional methods on this problem directly in terms of either time complexity or result validity. Our approach is to maintain a suffix-tree-like index to support efficiently locating and counting of local patterns. Based on the index, a method is proposed for discovering such patterns. We have analyzed the behavior of the problem and evaluated the performance of our algorithm on both synthetic and real data. The results correspond with the definition of our problem and verify the superiority of our method.

In this paper, a new method for discovering knowledge from empirical data is proposed. This model consists of five steps. Firstly, we find the centers of fuzzy membership functions using adapted self-organizing feature map (SOFM). Secondly, we use the centers of Gaussian membership functions derived from previous step to determine the widths of Gaussian membership functions by means of the first-nearest-neighbor heuristic. Thirdly, it builds a weight network of Hopfield network so that weights reflect the importance of the network’s connections. Fourthly, Hopfield network is operated to get output values. The final step is to extract rules or knowledge via our proposed algorithm. In this algorithm, the irrelevant candidate rules are deleted so that the number of fuzzy rules and the number of antecedents can be defined. Therefore, it extracts fuzzy rules from the network. The experiments on wine recognition data show good performance concerning predictive accuracy.

This paper presents an approach to reasoning with learnt and expert information where inconsistencies are present. Information is represented as an uncertain taxonomical hierarchy where each class is a concept specification either defined by an expert or learnt from data. We present this as a good framework within which to perform information fusion. We show through a simple example how learnt information and uncertain expert knowledge can be represented and how conclusions can be reasoned from the fused hierarchy. This reasoning mechanism relies on a default assumption to rank conclusions based on the position of contributing information in the class hierarchy.

Nowadays, document management is a challenging necessity, especially for businesses. A particularly difficult but essential document management task is that of email management within corporate mailing lists. This paper will show how information extraction, retrieval and integration can be combined to deliver a powerful software tool to extract information from such lists. While they will never replace humans, systems such as the one illustrated in this paper do help alleviate information overload. Moreover, they are important tools in organisational knowledge management, as they enable the discovery of important knowledge within and about an organisation.

A software concept for automated design of a multi-spindle drilling gear machine used in furniture production process is proposed. To find an optimised design of the target-machine, this means to find the minimum number of supports and gears as well as the optimised configuration of the multi-spindle drilling gears, an automated system based on pattern identification, knowledge discovery and automated decision process is explained. The transfer of acquired manual design experience from the human expert to a software strategy to solve the multi-criteria optimisation problem will achieve cost reductions during the machine design.

We describe a Classifier of email queries, which executes text categorization by topic. The specifics of our Classifier is that it allows accurate categorization of short messages containing only a few words. This advantage is achieved by executing morphological and semantic analyses of an incoming text. Specifically, the Classifier provides an efficient information extraction and takes the meaning of words into consideration. By using the hierarchically structured subject domain and classification rules, the Classifier’s engine assigns an email query to the most relevant category or categories.

This paper presents a scheme of knowledge-based wrapper generation for semi-structured and labeled documents. The implementation of an agent-oriented information extraction system, XTROS, is described. In contrast with previous wrapper learning agents, XTROS represents both the domain knowledge and the wrappers by XML documents to increase modularity, flexibility, and interoperability. XTROS shows good performance on several Web sites in the domain of real estate, and it is expected to be easily adaptable to different domains by plugging in appropriate XML-based domain knowledge.

This paper investigates the determination of semantic similarity by the incorporation of structural semantic knowledge from a lexical database and the learning ability of neural networks. The lexical database is assumed to be organised in a hierarchical structure. The extracted lexical knowledge contains the relative location of the concerned words in the lexical hierarchy. The neural network then processes available lexical knowledge to provide semantic similarity for words. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed method is effective in measuring semantic similarity between words.

One of the main limitation when accessing web is the lack of explicit structure, whose presence may help in understanding data semantics. Here, an approach to extract logical schema from web pages is presented, defining a page model where its contents is divided into “logical” sections, i.e. parts of a page each collecting related information. This model aims to take into account both traditional, static HTML pages, as well as dynamic pages content.

In this paper we propose an effective method to cluster documents into a dynamically built taxonomy of topics, directly extracted from the documents. We take into account short contextual information within the text corpus, which is weighted by importance and used as input to a set of independently spun growing Self-Organising Maps (SOM). This work shows an increase in precision and labelling quality in the hierarchy of topics, using these indexing units. The use of the tree structure over sets of conventional two-dimensional maps creates topic hierarchies that are easy to browse and understand, in which the documents are stored based on their content similarity.

Web data are typically Semi-structured data and lack explicit external schema information, which makes querying and browsing the web data inefficient. In this paper, we present an approach to discover the inherent schema(s) in semi-structured, hierarchical data sources fast and efficiently, based on OEM model and efficient pruning strategy. The schema discovered by our algorithm is a kind of data path expressions and can be transformed into schema tree easily.

The amount of information available on the Internet is currently growing at an incredible rate. However, the lack of efficient indexing is still a major barrier to effective information retrieval on the web. This paper presents a new technique for capturing the semantic of the document to be used for indexing and retrieval of relevant document from the Internet. It performs the conventional keyword based indexing and introduces a thematic relationship between parts of text using natural language understanding (NLU) and a linguistics theory called rhetorical structure theory (RST).

The recognition and extraction of semantic/logical structures in HTML documents are substantially important and difficult tasks for intelligent document processing. In this paper, we show that alignment is appropriate for recognizing characteristic semantic/logical structures of a series of HTML documents, within a framework of case-based reasoning. That is, given a series of HTML documents and a sample transformation from an HTML document into an XML format, then the alignment can identify semantic/logical structures in the remaining HTML documents of the series, by matching the text-block sequence of the remaining document with the one of the sample transformation. Several important properties of texts, such as continuity and sequentiality, can naturally be utilized by the alignment. The alignment technology can significantly improve the ability of the case-based transformation method which transforms a spatial/temporal series of HTML documents into machine-readable XML formats. Throughout experimental evaluations, we show that the case-based method with alignment achieved a highly accurate transformation of HTML documents into XML.

We present a representation for XML documents, derived from Warren’s representation of Prolog terms in the Warren Abstract Machine (WAM), which permits very efficient access and update operations. Our scheme is implemented in CXMLParser, a non-validating XML processor. We present the results of a performance comparison with two other XML processors, which show that CXMLParser is faster by a factor of 2.2 to 4.3 than its nearest competitor.

Automatic clustering of documents is a task that has become increasingly important with the explosion of online information. The Self Organising Map (SOM) has been used to cluster documents effectively, but efforts to date have used a single or a series of 2-dimensional maps. Ideally, the output of a document-clustering algorithm should be easy for a user to interpret. This paper describes a method of clustering documents using a series of 1-dimensional SOM arranged hierarchically to provide an intuitive tree structure representing document clusters. Wordnet is used to find the base forms of words and only cluster on words that can be nouns.

One way to find information that may be required, is to approach a person who is believed to possess it or to identify a person who knows where to look for it. Technical support, which automatically compiles individual expertise and makes this accessible, may be centred on an expert finder system. A central component of such a system is a user profile, which describes user expertise level in discussed subjects. Previous works have made attempts to weight user expertise by using content-based methods, which associate the expertise level with the analysis of keyword usage, irrespective of any semantic meanings conveyed. This paper explores the idea of using a natural language processing technique to understand given information from both a structural and semantic perspective in building user profiles. With its improved interpretation capability compared to prior works, it aims to enhance the performance accuracy in ranking the order of names of experts, returned by a system against a help-seeking query. To demonstrate its efficiency, e-mail communication is chosen as an application domain, since its closeness to a spoken dialog, makes it possible to focus on the linguistic attributes of user information in the process of expertise modelling. Experimental results from a case study show a 23% higher performance on average over 77% of the queries tested with the approach presented here.

In this paper we introduce a branch and bound algorithm for finding tree solution of minimum cost network flow problem. We consider different situations such as unit and non-unit traffic demand. The methods used to prune the searching tree in different situations are emphasized respectively but the complete searching process is not interpreted in detail due to limited space. Programming test results show the efficiency of these techniques.

The aim of this study is to investigate relationship between past users’ behavior (described by access patterns) and future one. The two main ideas are first to explore the possible users’ characterization that can be extracted from access pattern. This allows to measure and to have a better understanding of users’ behavior. This knowledge allows us to build new services as building interest communities based on a comparative approach and clustering. The second idea is to see if these characterizations can be useful to forecast future access. This could be useful to prefetch web data in proxy-cache. We show that there are some partial mathematical models binding the users’ behavior to the repetition of queries.

Mobile commerce (MC) is expected to occupy an important part in electronic commerce in spite of a number of problems such as restriction of device, limited contents and expensive charge system. But the functions that can automatically extend the limited contents, which are provided for the MC and efficiently support commercial transaction on the expensive charge system, are essential for the activation of the MC. In this paper we propose a next generation intelligence mobile commerce system, which enables a series of e-commerce activities like searching, ordering and settlement on the mobile device including the functions mentioned above. The proposed system has been actually designed, implemented and confirmed its effectiveness through experiments.

A new probabilistic approach for focused crawling in hierarchically ordered information repositories is presented in this paper. The model is suitable for searching the World Wide Web and it is based on the fictitious play model from the theory of learning in games. The leading idea of the play is that players (software agents) are competing of the resources so that the search focuses on areas where relevant information can be found more likely. The model is basically a coordination model of the agent population but it is also possible to plug different features into the model, e.g. features for the user’s relevance feedback or semantic links between documents. Additionally, the method is highly scalable and the efficient parallel implementation of the method is possible.

Mobile Commerce (MC) has some common critical problems such as constraints on small mobile device, expensive charge system, limited contents, and so on. In this paper we propose some solutions for solving the problems as follows: 1) personalized contents providing to increase the usability of the device by reducing the amount of information transferred to the device with personalization policy. 2) A middlet application to support user’s purchase activities such as products searching, ordering and settlement on the device with a minimum network connection. The solutions make us to overcome the both problems of the device with small screen and memory and expensive charge system.

This paper presents a knowledge-based product retrieval and recommendation system for e-commerce. The system is based on the observation that, in Business to Customer (B2C) e-commerce, customers’ preferences naturally cluster into groups. Customers belonging to the same cluster have very similar preferences for product selection. The system is primarily based on product classification hierarchy. The hierarchy contains weight vectors. The system learns from experience. The learning is in the form of weight refinement based on customer selections. The learning resembles radioactive decay in some situations. Labor profile domain has been taken up for system implementation. The results are at the preliminary stage, and the system is not yet evaluated completely.

Along with the enterprise globalization and Internet popularization, the Internet-based Data Warehouse System (DWS) has gradually replaced the traditional DWS and becomes its mainstream structure. The manager can easily obtain and share the data on the distribution system using the Internet. Through the multiple data source collections, the quality and broad base of DWS can be increased and thus help managers to make more decisive policies. However, utilizing the basic client/server structure of DWS can increase many tolerances and cost based problems. This paper uses the XML to establish the Internet-based DWS and utilize the advantage of its flexibility, self-definition, self-description and low cost to improve the unavoidable defect of the client/server DWS. We also use pull and push method approaches to determine what information can be shared on the Internet or delivered through e-mail. In this work, we show that the DWS architecture can not only improve the scalability and speed but also enhance the system security. In addition, it can be applied for both traditional client/server DWS and web-based DWS. We present a case study to prove the validity of this system architecture and create a prototype system to show the feasibility of this system architecture.

In order to establish a useful data warehouse, it must be correct and consistent. Hence, when selecting the data sources for building the data warehouse, it is essential know exactly about the concept and structure of all possible data sources and the dependencies between them. In a perfect world, this knowledge stems from an integrated, enterprize-wide data model. However, the reality is different and often an explicit model is not available.This paper proposes an approach for identifying data sources for a data warehouse, even without having detailed knowledge about interdependencies of data sources. Furthermore, we are able to confine the number of potential data sources. Hence, our approach reduces the time needed to build and maintain a data warehouse and it increases the data quality of the data warehouse.

In this paper, a coordination technique is described for fully cooperative learning based multiagent systems, based on the Collective Intelligence work by Wolpert et al. Our work focuses on a practical implementation of these approaches within a FIPA compliant agent system, using the FIPA-OS agent development toolkit. The functionality of this system is illustrated with a simple buyer/seller agent application, where it is shown that the buyer agents are capable of self-organising behaviour in order to maximise their contribution to the global utility of the system.

This paper describes AGILE, an agent-assisted infrastructure with the aim of structuring information and processes in a distributed learning environment. AGILE is based on a generalized producer-consumer schema: agents are providers of educational resources consumed by users (learners). The model assumes that agents support learning tasks and facilitate delivery and the collaborative share of learning resources. User agents are responsible for the student representation, tutorial agents facilitate the access to educational material and supervisor agents route and distribute information. Agent interactions are modelled as asynchronous occurrences of internal events triggered by user requests. Meta-knowledge expresses user roles in defining groups and user competence levels.

A fuzzy logic expert system has been developed that automatically allocates resources in real-time over a collection of autonomous agents. Genetic algorithm based optimization is conducted to determine the form of the membership functions for the fuzzy root concepts. The resource manager is made up of four trees, the isolated platform tree, the multi-platform tree, the fuzzy parameter selection tree and the fuzzy strategy tree. The isolated platform tree provides a fuzzy decision tree that allows an individual platform to respond to a threat. The multi-platform tree allows a group of platforms to respond to a threat in a collaborative self-organizing fashion. The fuzzy parameter selection tree is designed to make optimal selections of root concept parameters. The strategy tree is a fuzzy tree that an agent uses to try to predict the behavior of an enemy. Finally, the five approaches to validating the expert system are discussed.

In a Multi-agent system (MAS), agents work to achieve common goals. This paper gives a flexible model for writing agents and a mechanism to guarantee correctness of the MAS when agents operate on multiple databases.

eCommerce is a faceless business arrangement where the process of creating trust towards merchants, hereby referred to as “merchant trust”, is still a big challenge. Merchant trust framework can be created by using factors such as existence, affiliation, performance and policy. In this paper, we present an information model for a merchant trust based on the previously cited factors. We then provide a framework for the implementation of the model using intelligent agents. Gathering the required information on the merchant is the first step in helping consumers to evaluate merchant trust in an eCommerce setting.

The project MASIVE (Multi-Agent Systems Interactive Virtual Environments) is the multi-agent extension of our Interactivist-Expectative Theory on Agency and Learning (IETAL). The agents in the environment learn expectations from their interactions with the environment. In addition to that, they are equipped with special sensors for sensing akin agents, and interchange their knowledge of the environment (their intrinsic representations) during their imitation conventions. In this paper we discuss the basics of the theory, and the social consequences of such an environment from the perspective of learning, knowledge dissemination, and emergence of language.

An alternative approach to learning decision strategies in multi-state multiple agent systems is presented here. The method, which uses a game theoretic construction which is model free and does not rely on direct communication between the agents in the system. Limited experiments show that the method can find Nash equilibrium point for 3 player multi-stage game and converges more quickly than a comparable co-evolution method.

Distributed learning is the learning process of multiple autonomous agents in a varying environment, where each agent has only partial information about the global task. In this paper, we investigate the influence of different reinforcement signals (local and global) and team diversity (homogeneous and heterogeneous agents) on the learned solutions. We compare the learned solutions with those obtained by systematic search in a simple case study in which pairs of agents have to collaborate in order to solve the task without any explicit communication. The results show that policies which allow teammates to specialize find an adequate diversity of the team and, in general, achieve similar or better performances than policies which force homogeneity. However, in this specific case study, the achieved team performances appear to be independent of the locality or globality of the reinforcement signal.

Most inherently distributed systems require self diagnosis and on-line monitoring. This is especially true in the domains of power transmission and mobile communication. Much effort has been expended in developing on-site monitoring systems for distributed power transformers and mobile communication base stations. In this paper, a new approach has been employed to implement the autonomous self diagnosis and on-site monitoring using multi-agents on mobile communication base stations.

In this paper we follow an “evolutionary approach” to solve the ‘’Busy Beaver Game” using agent based techniques.

As many systems and organisations are migrating from centralised control to a distributed one, there is a need for a better understanding of organisational behaviour. The implications of such a trend are discussed with reference to current agent technology. A custom simulation, together with some preliminary results, for the investigation of organisational scenarios is presented. Finally, the development of suitable metrics to quantify the efficiency and effectiveness of differing control mechanisms is considered.

Mining various dependence structures from data are important to many data mining applications. In this paper, several major dependence structure mining tasks are overviewed from statistical learning perspective, with a number of major results on unsupervised learning models that range from a single-object world to a multi-object world. Moreover, efforts towards a key challenge to learning have been discussed in three typical streams, based on generalization error bounds, Ockham principle, and BYY harmony learning, respectively.

This paper presents a new clustering technique named STep- wise Automatic Rival- penalized (STAR) k-means algorithm (denoted as k*-means), which is actually a generalized version of the conventional k-means (MacQueen 1967). Not only is this new algorithm applicable to ellipse-shaped data clusters rather than just to ball-shaped ones like the k-means algorithm, but also it can perform appropriate clustering without knowing cluster number by gradually penalizing the winning chance of those extra seed points during learning competition. Although the existing RPCL (Xu et al. 1993) can automatically select the cluster number as well by driving extra seed points far away from the input data set, its performance is much sensitive to the selection of the de-learning rate. To our best knowledge, there is still no theoretical result to guide its selection as yet. In contrast, the proposed k*-means algorithm need not determine this rate. We have qualitatively analyzed its rival-penalized mechanism with the results well-justified by the experiments.

In this paper, we present an autonomous pattern search approach to solving Satisfiability Problems (SATs). Our approach is essentially a multiagent system. To solve a SAT problem, we first divide variables into groups, and represent each variable group with an agent. Then, we randomly place each agent onto a position in the correspoding local space which is composed of the domains of the variables that are represented by this agent. Thereafter, all agents will autonomously make search decisions guided by some reactive rules in their local spaces until a special pattern (i.e., solution) is found or a time step threshold is reached. Experimental results on some benchmark SAT test-sets have shown that by employing the MASSAT approach, we can obtain performances comparable to those of other popular algorithms.

E-mail messages can be modeled as semi-structured documents that consist of a set of classes and a number of variable length free-text. Thus, many text mining techniques can be used to develop a personal e-mail filtering and management system. This paper addresses a text mining agents based architecture, in which two kinds of text mining agents: USPC (uncertainty sampling based probabilistic classifier) and R2L (rough relation learning) are used cooperatively, for personal e-mail filtering and management.

How to increase both autonomy and versatility of a knowledge discovery system is a core problem and a crucial aspect of KDD (Knowledge Discovery and Data Mining). We have been developing a multi-agent based KDD methodology/system called GLS (Global Learning Scheme) for performing multi-aspect intelligent data analysis as well as multi-level conceptual abstraction and learning. With multi-level and multi-phase process, GLS increases versatility and autonomy. This paper presents our recent development on the GLS methodology/system.

We have previously described trading systems based on un-supervised learning approaches such as reinforcement learning and genetic algorithms which take as input a collection of commonly used technical indicators and generate profitable trading decisions from them. This article demonstrates the advantages of applying evolutionary algorithms to the reinforcement learning problem using a hybrid credit assignment approach. In earlier work, the temporal difference reinforcement learning approach suffered from problems with overfitting the in-sample data. This motivated the present approach.Technical analysis has been shown previously to have predictive value regarding future movements of foreign exchange prices and this article presents methods for automated high-frequency FX trading based on evolutionary reinforcement learning about signals from a variety of technical indicators. These methods are applied to GBPUSD, USDCHF and USDJPY exchange rates at various frequencies. Statistically significant profits are made consistently at transaction costs of up to 4bp for the hybrid system while the standard RL is only able to trade profitably up to about 1bp slippage per trade.

We propose a neural network based up-trend detector. An auto-associative neural network was trained with “up-trend” data obtained from the KOSPI 200 future price. It was then used to predict an up-trend. Simple investment strategies based on the detector achieved a two year return of 19.8 % with no leverage.

Viewed as a promising application of neural networks, financial time series forecasting was studied in the literature of neural nets and machine learning. The recently developed Temporal Factor Analysis (TFA) model mainly targeted at further study of the Arbitrage Pricing Theory (APT) is found to have potential application in the prediction of stock price and index. In this paper, we aim to illustrate the superiority of using the APT-based Gaussian TFA model as compared to three conventional approaches which are not financial model-based.

In this paper we comparatively study three MAP factor estimate approaches, i.e., iterative fixed posteriori approximation, gradient descent approach, and conjugate gradient algorithm, for the non-Gaussian factor analysis (NFA). With the so-called Gaussian approximation as initialization, the iterative fixed posteriori approximation is empirically found to be the best one among them.

In this paper, we propose a way of effective fraud detection to improve the detection efficiency. We focus on the bias of the training dataset, which is typically caused by the skewed distribution and highly overlapped classes of credit card transaction data and leads to lots of mis-detections. To reduce mis-detections, we take the fraud density of real transaction data as a confidence value and generate the weighted fraud score in the proposed scheme. The effectiveness of our proposed scheme is examined with experimental results on real data.

In the framework of competitive markets, the market’s participants need energy price forecasts in order to determine their optimal bidding strategies and maximize their benefits. Therefore, if generation companies have a good accuracy in forecasting hourly prices they can reduce the risk of over/underestimating the income obtained by selling energy. This paper presents and compares two energy price forecasting tools for day-ahead electricity market: a k Weighted Nearest Neighbours (kWNN) the weights being estimated by a genetic algorithm and a Dynamic Regression (DR). Results from realistic cases based on Spanish electricity market energy price forecasting are reported.

Recently, Support Vector Regression (SVR) has been introduced to solve regression and prediction problems. In this paper, we apply SVR to financial prediction tasks. In particular, the financial data are usually noisy and the associated risk is time-varying. Therefore, our SVR model is an extension of the standard SVR which incorporates margins adaptation. By varying the margins of the SVR, we could reflect the change in volatility of the financial data. Furthermore, we have analyzed the effect of asymmetrical margins so as to allow for the reduction of the downside risk. Our experimental results show that the use of standard deviation to calculate a variable margin gives a good predictive result in the prediction of Hang Seng Index.

We compare pre-processing time series data using Complexity Pursuit (CP) 2 and Logarithm Complexity Pursuit (LCP) [2] with a view to subsequently using a multi-layer perceptron (MLP) to forecast on the data set. Our rationale [1] is that forecasting the underlying factors will be easier than forecasting the original time series which is a combination of these factors. The projections of the data onto the filters found by the pre-processing method were fed into the MLP and it was trained to find Least Mean Square Error (LMSE). Both methods find interesting structure in the time series but LCP is more robust and achieves the best (in terms of least mean square error) performance.

Artificial intelligence supports decision by analyzing enormous information. This paper introduces an intelligent portfolio management system (IPMS) that applies artificial intelligence to assist investors in planning their investments. A prototype is developed based on the portfolio management process, involving stock selection and asset allocation optimization. A genetically optimised fuzzy rule-base is developed for stock selection. Genetic algorithm is used to optimize asset allocation according to investor’s risk aversion.

We introduce and formalize the multilevel classification problem, in which each category can be subdivided into different levels. We analyze the framework in a Bayesian setting using Normal class conditional densities. Within this framework, a natural monotonicity hint converts the problem into a nonlinear programming task, with non-linear constraints. We present Monte Carlo and gradient based techniques for addressing this task, and show the results of simulations. Incorporation of monotonicity yields a systematic improvement in performance.

The volatility of a speculative asset is a fundamental ingredient of many financial pricing algorithms, therefore, accurate forecasts of volatility are essential to financial practioners. Autoregressive Conditional Heteroscekdastic models and their generalisations (GARCH) have been shown to provide reasonable forecasts of volatility with relatively few parameters. Recent evidence suggests, however, that financial volatility is a multiplicative process whereby dominant time frames can be located by the derivative of volatility [14]. Using the Widrow Hoff learning rule, this paper shows how the GARCH(1,1) forecast can be improved by filtering the volatility derivative against the residual forecast error from a GARCH(1,1) model. Information criterion is used to evaluate the contribution of the new adaptive parameter.

Frequency shift differences in 1H MRSI spectra due to magnetic field inhomogeneities pose a problem, if automated lineshape fitting routines (LF) or artificial neural network (ANN) methods are used for spectral quantification. Use of self-organizing map (SOM) analysis for automated shift correction of long echo time (TE=270 ms) in vivo1H NMR spectra of human brain is demonstrated. The map is obtained by training a SOM with proton spectra and the chemical shifts of the reference vectors were calibrated. The maps were then used for classification of spectroscopic imaging data and the calibration information for corrections of chemical shifts.

In this paper we present a method for the automatic discovery and tuning of term similarities. The method is based on the automatic extraction of significant patterns in which terms tend to appear. Beside that, we use lexical and functional similarities between terms to define a hybrid similarity measure as a linear combination of the three similarities. We then present a genetic algorithm approach to supervised learning of parameters that are used in this linear combination. We used a domain specific ontology to evaluate the generated similarity measures and set the direction of their convergence. The approach has been tested and evaluated in the domain of molecular biology.

We present the functional and architectural specification of BIKMAS, a Bioinformatics Knowledge Management System. BIKMAS contains an interactive user interface, a database in which several sources of knowledge are registered and a nucleus of knowledge management implemented with an algorithm that filters scientific information and assists the user in the task of using knowledge. BIKMAS is an active information system capable of retrieving, processing and filtering scientific information, checking for consistency and structuring the relevant information for its efficient distribution and convenient use. Two of the most important aspects of BIKMAS are that the system is based on an object-oriented database and it has been developed in JAVA tightly integrated in Internet.

We present a method for automatically decomposing magnetic resonance (MR) spectra of different types of human brain tumours into components which directly reflect their different chemical compositions. The automatic analysis of in vivo MR spectra can be problematic due to their large dimensionality and the low signal to noise ratio. Principal Component Analysis allows an economic representation of the data but the extracted components themselves may bear little relationship to the underlying metabolites represented by the spectra. The Principal Components can be rotated in order to make them more meaningful but this requires expertise to decide on the transformation. In this study, we use Independent Component Analysis and show that this technique can overcome these two drawbacks and provide meaningful and representative components without requiring prior knowledge.

Data cleaning is an important part of the knowledge discovery process. The principal causes of data anomalies include incomplete information, absence of a unique identifier across multiple databases, inconsistent data, existence of data entry errors and logically incorrect data. This situation is further exacerbated while integrating data from multiple, disparate data sources. Since data quality is directly related with the quality of services in data-driven applications, such as medical informatics, a reliable data cleaning solution, which allows rapid and precise detection of invalid data, is needed. Most existing data cleaning solutions are domain specific, time-consuming and do not easily accommodate logical validations. In this paper, we propose a Fuzzy rule-based framework, which is domain independent, flexible and easily accommodates physical as well as logical validations. We have implemented existing cleaning strategies (i.e. Sorted Neighborhood Method), and enhanced them by using state-of-the-art algorithms (i.e. Rete, Bigram). As proof-of-concept, our prototype system was applied to real patient data. Experimental results illustrate that our framework is extensible and allows rapid detection of invalid data with high precision.

The purpose of a classification algorithm is to predict the class label of a new instance based on the analysis of a training dataset. Many classification algorithms work most naturally with nominal attributes. However, numerical data are very common in real-life applications. In this paper, we present a classification learning for numerical datasets. We adopt the idea of function decomposition, which is an approach used to represent a complex function by simple and smaller subfunctions. We modify the decomposition and apply it to decompose numerical datasets for classification learning. The proposed method is also implemented to evaluate the classification accuracy. The experimental evaluation shows the proposed method is a relatively effective method for classification learning.

The k-nearest neighbor (k-NN) classification is a simple and effective classification approach. However, it suffers from over-sensitivity problem due to irrelevant and noisy features. In this paper, we propose an algorithm to improve the effectiveness of k-NN by combining these two approaches. Specifically, we select all relevant features firstly, and then assign a weight to each one. Experimental results show that our algorithm achieves the highest accuracy or near to the highest accuracy on all test datasets. It also achieves higher generalization accuracy compared with the well-known algorithms IB1-4 and C4.5.

SVMs tend to take a very long time to train with a large data set. If “redundant” patterns are identified and deleted in pre-processing, the training time could be reduced significantly. We propose a k-nearest neighbors(k-NN) based pattern selection method. The method tries to select the patterns that are near the decision boundary and that are correctly labeled. The simulations over synthetic data sets showed promising results: (1) By converting a non-separable problem to a separable one, the search for an optimal error tolerance parameter became unnecessary. (2) SVM training time decreased by two orders of magnitude without any loss of accuracy. (3) The redundant SVs were substantially reduced.

The performance of a classification process depends heavily on the feature used in it. The traditional features/variables selection schemes are mostly developed from the model fitting point of view, which may not be good or efficient for classification purpose. Here we propose a graphical selection method, which allows us to integrate the information in the test data set, and it is suitable for selection useful features from high dimensional data set. We applied it to the Thrombin data set, which was used in KDD CUP 2001. By using the selected features from our graphical method and a SVM classifier, we obtained the higher classification accuracy than the results reported in KDD Cup 2001.

The predominate weakness in the creation of decision trees is the strict partitions which are selected by the induction algorithm. To overcome this problem the theories of fuzzy logic have been applied to generate soft thresholds leading to the creation of fuzzy decision trees, thus allowing cases passing through the tree for classification to be assigned partial memberships down all paths. A challenging task is how these resultant membership grades are combined to produce an overall outcome. A number of theoretical fuzzy inference techniques exist, yet they have not been applied extensively in practical situations and are often domain dependent. Thus the overall classification success of the fuzzy trees has a high dependency on the optimization of the strength of the fuzzy intersection and union operators that are applied. This paper investigates a new, more general approach to combining membership grades using neural-fuzzy inference. Comparisons are made between using the fuzzy-neural approach and the use of pure fuzzy inference trees.

A decision tree can be used not only as a classifier but also as a clustering method. One of such applications can be found in automatic speech recognition using hidden Markov models (HMMs). Due to the insufficient amount of training data, similar states of triphone HMMs are grouped together using a decision tree to share a common probability distribution. At the same time, in order to predict the statistics of unseen triphones, the decision tree is used as a classifier as well. In this paper, we study several cluster split criteria in decision tree building algorithms for the case where the instances to be clustered are probability density functions. Especially, when Gaussian probability distributions are to be clustered, we have found that the Bhattacharyya distance based measures are more consistent than the conventional log likelihood based measure.

We introduce a technique to compute new summary information estimations (information and entropy) for fuzzy sets. Special features for these estimations are investigated. We give an algorithm for determine various information measures for fuzzy sets and fuzzy decision trees. Finally, we are using our estimations for induction of fuzzy decision trees from a group of training examples.

A genetic algorithm based on the quantum probability representation (GAQPR) is proposed, in which each individual evolves independently; a new crossover operator is designed to integrate searching processes of multiple individuals into a more efficient global searching process; a new mutation operator is also proposed and analyzed. Optimization capability of GAQPR is studied via experiments on function optimization, results of experiments show that, for multi-peak optimization problem, GAQPR is more efficient than GQA[4]

A discretization technique converts continuous attribute values into discrete ones. Discretization is needed when classification algorithms require only discrete attributes. It is also useful to increase the speed and the accuracy of classification algorithms. This paper presents a dynamic discretization method, whose main characteristic is to detect interdependencies between all continuous attributes. Empirical evaluation on 12 datasets from the UCI repository shows that the proposed algorithm is a relatively effective method for discretization.

We investigate an extension of the learning rules in a Principal Component Analysis network which has been derived to be optimal for a specific probability density function(pdf). We note that this probability density function is one of a family of pdfs and investigate the learning rules formed in order to be optimal for several members of this family. We show that, whereas previous authors [5] have viewed the single member of the family as an extension of PCA, it is more appropriate to view the whole family of learning rules as methods of performing Exploratory Projection Pursuit(EPP). We explore the performance of our method first in response to an artificial data type, then to a real data set.

We present a general framework for interactive visualization and analysis of multi-dimensional data points. The proposed model is a hierarchical extension of the latent trait family of models developed in [4] as a generalization of GTM to noise models from the exponential family of distributions. As some members of the exponential family of distributions are suitable for modeling discrete observations, we give a brief example of using our methodology in interactive visualization and semantic discovery in a corpus of text-based documents. We also derive formulas for computing local magnification factors of latent trait projection manifolds.

In this paper we propose the use of dominant point method for Chinese character recognition. We compare the performance of three classifiers on the same inputs; a statistical linear classifier, a machine learning C4.5 classifier, and a fuzzy nearest neighbour method of classification. Such a comparison highlights the degree of advantage of correct recognition method for our problem.

In this paper, we propose a new measure within the framework of reinforcement learning, by describing a model of an information source as a representation of a learning process. We confirm in experiments that Lempel-Ziv coding for a string of episode sequences provides a quality measure to describe the degree of complexity for learning. In addition, we discuss functions comparing expected return and its variance.

In this paper, we present a method to efficiently extract a human’s face from a given image sequence. The method consists of two steps: image segmentation and facial region extraction. In the image segmentation, the input frames are segmented using watershed algorithms segmenting the frame into an appropriate set of arbitrary regions. In the facial region extraction, the facial regions are extracted by integrating the results of facial region detection using a skin-color model and the results of facial region identification using a Neural Network (NN). The results of the image segmentation and facial region extraction are integrated to provide facial regions with accurate and closed boundaries. In our experiments, the presented method detected 92.2% of the faces and the average run time ranged from 0.31 to 0.48 sec per frame.

The classification problem is one of the typical problems encountered in data mining and machine learning. In this paper, a rough genetic algorithm (RGA) is applied to the classification problem in an undetermined environment based on a fuzzy distance function by calculating attribute weights. The RGA, a genetic algorithm based on rough values, can complement the existing tools developed in rough computing. Computational experiments are conducted on benchmark problems downloaded from UCI machine learning databases. Experimental results, compared with the usual GA [1] and C4.5 algorithms, verify the efficiency of the developed algorithm. Furthermore, the weights acquired by the proposed learning method are applicable not only to fuzzy similarity functions but also to any similarity functions. As an application, a new distance metric called weighted discretized value difference metric (WDVDM) is proposed. Experimental results show that WDVDM is an improvement on the discretized value difference metric (DVDM).

In this paper, recursive fast Fourier transform is presented for two-dimensional signals. When applying to real-time analysis, the computational efficiency is highly improved by integrating a recursive procedure. The recursive procedure highly reduces the number of complex arithmetic operations, and provide detailed spectral analysis for one or two-dimensional signals.In the first stage, the recursive algorithm is realized for one-dimensional signals. Then, recursive fast Fourier transform is presented for two-dimensional signals. The advantages of the recursive algorithm are presented by giving examples for one and two-dimensional signals.

Visual images synchronized with audio signals can provide user-friendly interface for man machine interactions. The visual speech can be represented as a sequence of visemes, which are the generic face images corresponding to particular sounds. We use HMMs (hidden Markov models) to convert audio signals to a sequence of visemes. In this paper, we compare two approaches in using HMMs. In the first approach, an HMM is trained for each triviseme which is a viseme with its left and right context, and the audio signals are directly recognized as a sequence of trivisemes. In the second approach, each triphone is modeled with an HMM, and a general triphone recognizer is used to produce a triphone sequence from the audio signals. The triviseme or triphone sequence is then converted to a viseme sequence. The performances of the two viseme recognition systems are evaluated on the TIMIT speech corpus.

In this paper, a satisfactory method is described for the extraction of staff lines in which there are some inclinations, discontinuities, and curvatures. The extraction calls for four processes: (1) Extraction of specific points on a stave on vertical scan lines, (2) Connection of the points using DP matching, (3) Composition of stave groups using labeling, and (4) Extraction and adjustment of the edges of lines. The experiment resulted in an extraction rate of 99.4% for 71 printed music scores that included lines with some inclinations, discontinuities, and curvatures.

In this paper, we propose EMACF (Expectation-Maximization Algorithm for Clustering Features) to generate clusters from data summaries rather than data items directly. Incorporating with an adaptive grid-based data summarization procedure, we establish a scalable clustering algorithm: gEMACF. The experimental results show that gEMACF can generate more accurate results than other scalable clustering algorithms. The experimental results also indicate that gEMACF can run two order of magnitude faster than the traditional expectation-maximization algorithm with little loss of accuracy.

Many researchers have devoted into video retrieval research and given lots of efficient video retrieval algorithms. Most of past algorithms were done in pixel domain, which needed lots of decoding calculations. What’s more, the same matching algorithm was used to all the video clips, which also wasted many unnecessary calculations. This paper presents a new approach to hierarchically retrieve video by example video in MPEG compressed domain. Firstly, the dct_dc_size field in I frames is analyzed to filter out the obviously different video clips quickly, and then the precise matching analysis of DC image is used to get the final retrieval result. Our experiment results show that this approach needs a few calculations and has high precision ratio.

An algorithm, Aspiration Scout/MTD(f), is derived from an analysis of alpha-beta (α-β) tree search. When compared to its predecessors, it results in an average 10.1% reduction in search effort with a best case of 17.4%. 1 The search space is usually referred to as a tree, however in games such as chess and draughts it is actually a directed acyclic graph (DAG). Programs search a dynamically unfolding DAG.

Identifying and quantifying relevance of input features are particularly useful in data mining when dealing with ill-understood real-world data defined problems. The conventional methods, such as statistics and correlation analysis, appear to be less effective because the data of such type of problems usually contains high-level noise and the actual distributions of attributes are unknown. This papers presents a neural-network based method to identify relevant input features and quantify their general and specified relevance. An application to a real-world problem, i.e. osteoporosis prediction, demonstrates that the method is able to quantify the impacts of risk factors, and then select the most salient ones to train neural networks for improving prediction accuracy.