Machine Learning and Its Applications

This chapter has two goals. The first goal is to compare Machine Learning (ML) and Knowledge Discovery in Data (KDD, also often called Data Mining, DM) insisting on how much they actually differ. In order to make my ideas somewhat easier to understand, and as an illustration, I will include a description of several research topics that I find relevant to KDD and to KDD only. The second goal is to show that the definition I give of KDD can be almost directly applied to text analysis, and that will lead us to a very restrictive definition of Knowledge Discovery in Texts (KDT). I will provide a compelling example of a real-life set of rules obtained by what I call KDT techniques.

In concept learning and data mining, a typical objective is to determine concept descriptions or patterns that will classify future data points as correctly as possible. If one can assume that the data contain no noise, then it is desirable that descriptions are complete and consistent with regard to all the data, i.e., they characterize all data points in a given class (positive examples) and no data points outside the class (negative examples).

Since the field’s inception, most research in machine learning has focused on the problem of supervised induction from labeled training cases. If anything, this trend has been strengthened by the creation of data repositories that, typically, include class information. But this emphasis is misguided if we want to understand the nature of learning in intelligent agents like humans. Clearly, children acquire many concepts about the world before they learn names for them, and scientists regularly discover patterns without any clear supervision from an outside source. Even the availability of class labels in public data sets can be misleading; many such domains are medical in nature, and medical researchers first had to discover a disease before they could diagnose it for particular patients.

To solve a complex problem, one of the effective general approaches is to decompose it into smaller, less complex and more manageable subproblems. In machine learning, this principle is a foundation for structured induction [44]: instead of learning a single complex classification rule from examples, define a concept hierarchy and learn rules for each of the (sub)concepts. Shapiro [44] used structured induction for the classification of a fairly complex chess endgame and demonstrated that the complexity and comprehensiveness (“brain-compatibility”) of the obtained solution was superior to the unstructured one. Shapiro was helped by a chess master to structure his problem domain. Typically, applications of structured induction involve a manual development of the hierarchy and a manual selection and classification of examples to induce the subconcept classification rules; usually this is a tiresome process that requires an active availability of a domain expert over long periods of time. Therefore, it would be very desirable to automate the problem decomposition task.

Current machine learning systems are often distinguished on the basis of their representation, which can either be propositional or first order logic. Systems belonging to the first category are often called attribute value learners, systems of the second category are called relational learners or inductive logic programming systems.

This chapter contains an overview of Case-Based Reasoning (CBR). The main goal is to have a balance between brevity and expressiveness and to provide helpful pointers to literature in the field. To do so, we first describe the CBR types and the CBR cycle, then we briefly review a representative set of systems, next we discuss the connections between CBR and learning. The main part of the chapter analyses the most important issues and problems of the CBR components, such as indexing/retrieval/selection, memory organization, adaptation/evaluation, forgetting, and integration with other techniques. Finally, we discuss the added value of incorporating fuzzy techniques in CBR and briefly describe some representative Fuzzy-CBR systems.

Genetic algorithms are stochastic search algorithms which act on a population of possible solutions. They are loosely based on the mechanics of population genetics and selection. The potential solutions are encoded as ‘genes’ — strings of characters from some alphabet. New solutions can be produced by ‘mutating’ members of the current population, and by ‘mating’ two solutions together to form a new solution. The better solutions are selected to breed and mutate and the worse ones are discarded. They are probabilistic search methods; this means that the states which they explore are not determined solely by the properties of the problems. A random process helps to guide the search. Genetic algorithms are used in artificial intelligence like other search algorithms are used in artificial intelligence — to search a space of potential solutions to find one which solves the problem.

Pattern Recognition (PR) is a fast growing field with applications in many diverse areas such as optical character recognition (OCR), computer – aided diagnosis and speech recognition, to name but a few.

Machine Learning was primarily inspired by human learning. In a branch of Artificial Intelligence scientists tried to build systems that reproduce forms of human learning. Currently the methods that were discovered in this way have been elaborated and are applied to tasks that are not performed by humans at all. For example, one of the most popular applications is the analysis of consumer data to predict buying behaviour. This has not traditionally been viewed as an interesting form of human intelligence.

With the growing complexity of Machine Learning applications, the need for using integrated or hybrid (or multistrategy) approaches becomes more and more imperative, and an increasing amount of research effort is devoted to this issue. The increasing complexity of applications is not the only reason making multistrategic approaches appealing: as it is well known, no single approach/system can claim to be uniformly superior to any other, so that hybridisation seems a natural and viable way of compensating drawbacks and enhancing advantages. Even though there is no common agreement on what integration exactly means in Machine Learning, in a broad sense an integrated architecture can be defined as one which is organised or structured so that its constituent units function co-operatively.

The process of scientific discovery has long been viewed as the pinnacle of creative thought. Thus, to many people, including some scientists themselves, it seems an unlikely candidate for automation by computer. However, over the past two decades, researchers in artificial intelligence have repeatedly questioned this attitude and attempted to develop intelligent artifacts that replicate the act of discovery. The computational study of scientific discovery has made important strides in its short history, some of which we review in this paper.

This chapter presents a summary of the issues discussed during the one day workshop on “Support Vector Machines (SVM) Theory and Applications” organized as part of the Advanced Course on Artificial Intelligence (ACAI ’99) in Chania, Greece [19]. The goal of the chapter is twofold: to present an overview of the background theory and current understanding of SVM, and to discuss the papers presented as well as the issues that arose during the workshop.

Knowledge discovery in databases (KDD) has become a very attractive discipline both for research and industry within the last few years. Its goal is to extract "pieces" of knowledge or "patterns" from usually very large databases. It portrays a robust sequence of procedures or steps that have to be carried out so as to derive reasonable and understandable results. One of its components is the process which induces the above “pieces” of knowledge; usually this is a machine learning (ML) algorithm. However, most of the machine learning algorithms require perfect data in a specific format. The data that are to be processed by a knowledge acquisition (inductive) algorithm are usually noisy and often inconsistent. Many steps are involved before the actual data analysis starts. Moreover, many ML systems do not easily allow processing of numerical attributes as well as numerical (continuous) classes. Therefore, certain procedures have to precede the actual data analysis process. Next, a result of an ML algorithm, such as a decision tree, a set of decision rules, or weights and topology of a neural net, may not be appropriate from the view of custom or commercial applications. As a result, a concept description (model, knowledge base) produced by an inductive process has to be usually postprocessed. Postprocessing procedures usually include various pruning routines, rule quality processing, rule filtering, rule combination, model combination, or even knowledge integration. All these procedures provide a kind of symbolic filter for noisy, imprecise, or non-user-friendly knowledge derived by an inductive algorithm. Therefore, some preprocessing routines as well as postprocessing ones should fill up the entire chain of data processing. The pre- and post-processing tools always help to investigate databases as well as to refine the acquired knowledge. Usually, these tools exploit techniques that are not genuinely symbolic/logical, e.g., statistics, neural nets, and others.

The undoubted usefulness of present-day information systems is only moderated by the fact that people have to invest substantial effort and training time in order to learn how to use them. Even modern applications with Graphical-User Interfaces (which are considered user-friendly), built-in wizards and on-line context-sensitive help, require a considerable self-training period, thus discouraging most people from fully exploiting their capabilities. In the years to come we expect that information systems will gradually become more and more complex and since the training period is usually proportional to the system complexity, with the usual Human Computer Interaction methods less and less people will have the time to learn how to use a new piece of software.

As the volume of electronically stored information continues to expand across computer networks, the need for intelligent access to on-line collections of multimedia documents becomes imperative. Examples of such collections are the World Wide Web, digital libraries and enterprise-wide information repositories. Machine learning offers an invaluable corpus of techniques, tools and systems that can help to solve effectively related problems, such as semantic indexing, contentbased search, semantic querying, integration of ontologies/knowledge bases into Internet search technologies, in order to develop a new generation of intelligent search engines. There has been a growing interest in augmenting or replacing traditional information filtering and retrieval approaches with machine learning techniques in order to build systems that can scale to the intrinsic complexity of the task. This issue was addressed in the workshop on “Machine Learning for Intelligent Information Access”, which was organized as part of the Advanced Course on Artificial Intelligence (ACAI ’99).

The purpose of this chapter is to provide an introduction to the field of machine learning techniques for intelligent agents based on the contributions in the workshop of ’Machine Learning and Intelligent Agents’ [20], which was held in conjunction with the Advanced Course on Artificial Intelligence (ACAI ’99) on Machine Learning & Applications, at Chania, Greece.

It is generally recognized that information systems are becoming more complex and, therefore, intelligent user interfaces are needed to improve user interaction with these systems. Furthermore, the exponential growth of the Internet makes it difficult for the users to cope with the huge amount of available on-line information. The challenge that information providers and system engineers face is the creation of adaptive (Webbased) applications, as well as the development of “personalized” retrieval and filtering mechanisms. Responses to this challenge come from various disciplines including machine learning and data mining, intelligent agents and multi-agent systems, intelligent tutoring, information retrieval, etc.

Data Mining has become a buzzword in industry in recent years. It is something that everyone is talking about but few seem to understand. There are two reasons for this lack of understanding: First is the fact that Data Mining researchers have very diverse backgrounds such as machine learning, psychology and statistics. This means that the research is often based on different methodologies and communication links e.g. notation is often unique to a particular research area which hampers the exchange of ideas and the dissemination to the wider public. The second reason for the lack of understanding is that the main ideas behind Data Mining are often completely opposite to mainstream statistics and as many companies interested in Data Mining already employ statisticians, such a change of view can create opposition.

Machine Learning (ML) provides methods, techniques, and tools that can help solving diagnostic and prognostic problems in a variety of medical domains. ML is being used for the analysis of the importance of clinical parameters and their combinations for prognosis, e.g. prediction of disease progression, extraction of medical knowledge for outcome research, therapy planning and support, and for the overall patient management. ML is also being used for data analysis, such as detection of regularities in the data by appropriately dealing with imperfect data, interpretation of continuous data used in the Intensive Care Unit, and intelligent alarming resulting in effective and efficient monitoring. It is argued that the successful implementation of ML methods can help the integration of computer-based systems in the healthcare environment providing opportunities to facilitate and enhance the work of medical experts and ultimately to improve the efficiency and quality of medical care. Below, we summarize some major ML applications in medicine.

The recent developments in the power system area, i.e. the on-going liberalization of the energy markets, the pressing demands for power system efficiency and power quality, the increase of dispersed, renewable generation and the growing number of interconnections and power exchanges among utilities, dictate the need for improvements in the power system planning, operation and control. At the same time, the power equipment industry faces new challenges in nowadays ever-increasing competition. Artificial Intelligence techniques together with traditional analytical techniques can significantly contribute in the solution of the related problems. Indeed, during the last 15 years, pattern recognition, expert systems, artificial neural networks, fuzzy systems, evolutionary programming, and other artificial intelligence methods have been proposed in an impressive number of publications in the power system community.