Computational Intelligence: Soft Computing and Fuzzy-Neuro Integration with Applications

The essence of soft computing is that, unlike the traditional, hard computing, it is aimed at an accommodation with the pervasive imprecision of the real world. Thus, the guiding principle of soft computing is: ‘….exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality.’ In the final analysis, the role model for soft computing is the human mind.

Soft computing is not a single methodology. Rather, it is a partnership. The principal partners at this juncture are fuzzy logic, neurocomputing, genetic computing and probabilistic computing, with the latter subsuming chaotic systems, belief networks and parts of learning theory.

In coming years, the ubiquity of intelligent systems is certain to have a profound impact on the ways in which man-made intelligent systems are conceived, designed, manufactured, employed and interacted with. It is in this perspective that the basic issues relating to soft computing and intelligent systems are addressed in this paper.

Here is the abstract from my 1992 paper about Computational Intelligence (CI) [1]: This paper concerns the relationship between neural-like computational networks, numerical pattern recognition and intelligence. Extensive research that proposes the use of neural models for a wide variety of applications has been conducted in the past few years. Sometimes the justification for investigating the potential of neural nets (NNs) is obvious. On the other hand, current enthusiasm for this approach has also led to the use of neural models when the apparent rationale for their use has been justified by what is best described as “feeding frenzy”. In this latter instance there is at times a concomitant lack of concern about many “side issues” connected with algorithms (e.g., complexity, convergence, stability, robustness and performance validation) that need attention before any computational model becomes part of an operational system. These issues are examined with a view towards guessing how best to integrate and exploit the promise of the neural approach with other efforts aimed at advancing the art and science of pattern recognition and its applications in fielded systems in the next decade. A further purpose of the present paper is to characterize the notions of computational, artificial and biological intelligence; my hope is that a careful discussion of the relationship between systems that exhibit each of these properties will serve to guide rational expectations and development of models that exhibit or mimic “human behavior”.

This article adds to the growing and pretty amusing literature that tries to explain what I meant. I will add my own opinion to the many others that offer an explanation for the current popularity of the term CI.

Our native intelligence captures and encodes our knowledge into our biological neural networks, and communicates them to the external world via linguistic expressions of a natural language. These linguistic expressions are naturally constrained by syntax and semantics of a given natural language and its cultural base of abstractions. Next, an accepted scientific paradigm and its language further restricts these linguistic expressions when propositional and predicate expressions are formulated in order to express either assumed or observed relationships between elements of a given domain of concern. Finally the symbols are represented with numbers in order to enable a computational aparatus to execute those assumed or observed relationships within the uniqueness of a given numerical scale. In this manner, our knowledge of a particular system’s behavior patterns are first expressed in a linguistic form and then transformed into computational expressions through at least two sets of major transformations stated above, i.e., first from language to formulae next from formulae to numbers.

In this context, fuzzy normal-form formulae of linguistic expressions are derived with the construction of “Extended Truth Tables”. Depending on the set of axioms exhibited and/or we are willing to impose on the linguistic expressions of our native intelligence, we might arrive at different computational intelligence expressions for purposeful goal-oriented control of systems. In particular, it is shown that derivation of normal-form formulae for “Fuzzy Middle” and “Fuzzy Contradiction” lead to unique and enriched interpretations in comparison to the classical “Excluded Middle” and “Crisp Contradiction” expressions.

In a series of papers initiated by Resconi et al. [1], interpretations for various uncertainty theories were proposed, including fuzzy set theory, Dempster-Shafer theory, and possible theory, using models of modal logic [1– 2– 3]. There were two main reasons for pursuing research in this direction. The first reason was to offer the standard semantics of modal logic as a unifying framework within which it would be possible to compare and relate uncertainty theories to each other. Since, from time to time, some of the uncertainty theories are questioned regarding their internal adequateness, the second reason was to support them by developing interpretations for them in a relatively well-established area: in our case, modal logic. This paper is a summary of these efforts. To avoid unnecessary repetition of previous material, we will not repeat all the basic definitions and properties; the reader is referred to relevant literature for fuzzy set theory [5], for possibility theory [3], for Dempster-Shafer theory [4], and for modal logic [8]. A more thorough treatment of the material summarised in this paper is covered in the above-mentioned papers [4,5].

The study of fuzzy relational equations is one of the most appealing subjects in fuzzy set theory, both from a mathematical and a systems modelling point of view. The basic fuzzy relational equations are the sup-T equations, with T a t-norm. In this overview paper, we deal with these equations in a general lattice-theoretic framework. We consider t-norms on bounded ordered sets, and in particular on complete lattices. We then solve sup-T equations on distributive, complete lattices of which all elements are either join-irreducible or join-decomposable. Solution sets are represented by means of root systems. Some additional necessary and sufficient solvability conditions are listed. Also systems of sup-T equations are discussed. The theoretical results presented are applied to the real unit interval and to the real unit hypercube. In the latter case, particular attention is paid to pointwise extensions of t-norms defined on the real unit interval and the corresponding residual operators.

Specificity is introduced and shown to be a measure of the information contained in a fuzzy subset. A characterization of this measure is provided as well as a number of manifestations of the measure.

Although a good amount of theory has evolved over the last 30 years about the desirable or “good” properties of membership values for fuzzy sets and relations, relatively little is known about how to define the membership values in an objective way to satisfy those properties. As a result, one often ignores those properties when developing applications of fuzzy sets. We present here some successful designs of membership functions having good properties.

New methods for constructing generalized triangular operators, using a minimum and maximum fuzziness approach are outlined. Based on the entropy of a fuzzy subset, defined by using the equilibrium of the generalized fuzzy complement, the concept of elementary entropy function and its generalizations are introduced. These functions assign a value to each element of a fuzzy subset that characterizes its degree of fuzziness. It is shown that these functions can be used to construct the entropy of a fuzzy subset. Using this mapping, the generalized intersections and unions are defined as mappings, that assign the least and the most fuzzy membership grade to each of the elements of the operators’ domain, respectively. Next further classes of new generalized T-operators are introduced, also defined as minimum and maximum entropy operations. It is shown that they are commutative semigroup operations on [0,1] with identity elements but they are not monotonic. Simulations have been carried out so as to determine the effects of these new operators on the performance of the fuzzy controllers. It is concluded that the performance of the fuzzy controller can be improved by using some sets of generalized T-operations for a class of plants.

A systems modeling is proposed with a unification of fuzzy methodologies. Three knowledge representation schemas are presented together with the corresponding approximate reasoning methods. Unsupervised learning of fuzzy sets and rules is reviewed with recent developments in fuzzy cluster analysis techniques. The resultant fuzzy sets are determined with a Euclidian distance-based similarity view of membership functions. Finally, an intelligent fuzzy system model development is proposed with proper learning in order to adapt to an actual system performance output. In this approach, connectives are not chosen a priori but learned with an iterative training depending on a given data set.

Fuzzy inference systems represent an important part of fuzzy logic. In most practical applications (i.e., control) such systems perform crisp nonlinear mapping, which is specified in the form of fuzzy rules encoding expert or common-sense knowledge about the problem at hand. This paper shows an equivalence between fuzzy system representation and more traditional (mathematical) forms of function parameterization commonly used in statistics and neural nets. This connection between fuzzy and mathematical representations of a function is crucial for understanding advantages and limitations of fuzzy inference systems. In particular, the main advantages are interpretation capability and the ease of encoding a priori knowledge, whereas the main limitation is the lack of learning capabilities. Finally, we outline several major approaches for learning (estimation) of fuzzy rules from the training data.

Decision analysis and decision support is an area in which applications of fuzzy set theory, have been found since the early 1970s. Algorithmic as well as knowledge-based approaches have been suggested. The meaning of the term “decision” has also been defined differently in different areas, as has the meaning of “uncertainty”. This paper will first sketch different meanings of “decision” and “uncertainty” and then focus on algorithmic approaches relevant for decision support systems.

This paper is about so-called neuro-fuzzy systems, which combine methods from neural network theory with fuzzy systems. Such combinations have been considered for several years already. However, the term neuro-fuzzy still lacks proper definition, and still has the flavour of a buzzword to it. In this paper we try to give it a meaning in the context of three applications of fuzzy systems, which are fuzzy control, fuzzy classification, and fuzzy function approximation.

A model and the decision reasoning processes of a two-layer organising supervisory controller for complex systems have been developed. It is based on fuzzy Petri-net algorithms and fuzzy rule production system for decision and command control. This model follows the fundamental idea of the original intelligent controller of G.N. Saridis, but adds on a new generic property. This fuzzy-Petri-net organising controller employs the advantages of both the qualitative modelling potential of L.A. Zadeh’s fuzzy logic and of the discrete-event genesis of K.A. Petri’s networks. Thus it accomplishes full compatibility of mathematical formalisms of the organising and co-ordinating levels of G.N. Saridis’ architecture and greatly reduces the rules needed due to possibility distribution evaluation and Petri-net-supported reasoning in comparison with Stellakis-Valavanis fuzzy solution for the organiser.

This paper deals with a review of the non-linear threshold logic developed in collaboration by D. Dubois, G. Resconi and A. Raymondi. This is a significant extension of the neural threshold logic pioneered by McCulloch and Pitts. The output of their formal neuron is given by the Heaviside function with an argument depending on a linear weighted sum of the inputs and a threshold parameter. All Boolean tables cannot be represented by such a formal neuron. For example, the exclusive OR and the parity problem need hidden neurons to be resolved. A few years ago, Dubois proposed a non-linear fractal neuron to resolve the exclusive OR problem with only one single neuron. Then Dubois and Resconi introduce the non-linear threshold logic, that is to say a Heaviside function with a non-linear sum of the inputs which can represent any Boolean tables with only one neuron where the Dubois’ non-linear neuron model is a Heaviside fixed function. In this framework the supervised learning is direct, that is to say without recursive algorithms for computing the weights and threshold, related to the new foundation of the threshold logic by Resconi and Raymondi. This paper will review the main aspects of the linear and non-linear threshold logic with direct learning and applications in pattern recognition with the software TurboBrain. This constitutes a new tool in the framework of Soft Computing.

A simple conventional neural model has difficulty in solving a lot of problems that are easy for neurobiology. In fact, for example, in the conventional neural model we cannot generate a model by the superposition of its sub-models. Neurobiology suggests a new type of computation where analogue variables are encoded in time or in frequency (rate of spikes). This neurobiology computation, recently described by Hopfield [3], is a particular case of a new computing principle [5]. The first step of this principle was given by Gabor (1954) with his hologram physical process [7] and also with his original intelligent machine. Hoffman [11, 12] gives a Lie transformation group as a model of the neuropsychology of visual perception. We can consider this model as a particular case of new computing. In the new computing principle, a computation in an intelligent machine is capable of perceiving order in a situation previously considered disorder [5] and can learn how to choose among functions the function that best approximate the supervisor’s response [4]. The instrument to perceive order is the Morphogenetic Neuron, whose elements are the morphogenetic field (MF), the morphogenetic reference space (MRS) the morphogenetic sources (MS) and the morphogenetic elementary field (MEF). To learn and perceive order the new principle create sources or MS to generate elementary fields MEF. Linear superposition of MEF gives us the desired field or MF. The ordinary neuron [3] is a special case of MF for crisp weights. For fuzzy weights we can improve the ordinary neuron. The morphogenetic neuron becomes a complex membership function for a fuzzy set. Experiments of primary cortical representation of sound suggest that co-ordination tuning forms an organised topographic map across the cortical surface (stimulus specificity) or MF. Other experiments suggest that primates, birds and insects use local detectors to correlate signals sampled at one location with those sampled after a delay in another locations. The animals reshape the MF to obtain desired results.

This paper is a review of a new theoretical basis for modelling neural Boolean networks by non-linear digital equations. With real numbers, soft Boolean tables can be generated. With integer numbers, these digital equations are Heaviside fixed functions in the framework of the threshold logic. These can represent non-linear neurons which can be split very easily into a set of McCulloch and Pitts formal neurons with hidden neurons. It is demonstrated that any Boolean tables can be very easily represented by such neural networks where the weights are always either an activation weight +1 or an inhibition weight -1, with integer threshold. The parity problem is fully solved by a fractal neural network based on XOR. From a feedback of the hidden neurons to the inputs in a XOR non-linear equations, it is showed that the neurons compete with each other. Moreover, the feedback of the output to the inputs for a XOR non-linear neuron gives rise to fractal chaos. A model of a stack memory can be designed from such a chaos map Binary digits are memorised by folding to a real variable by an anti-chaotic hyperincursive process. The retrieval of these data is computed by an incursive chaotic map from the last value of the variable. Incursion is an extension of recursion for which each iterate is computed in function of variables not only defined in the past and the present time by also in the future. Hyperincursion is an incursion generating multiple iterates at each step. The basic map is the PearlVerhulst one in the zone of fractal chaos. The hyperincursive memory realises a coding of the input binary message under the form similar to the Gray code. This is based on a soft exclusive OR equation mixing binary digits with real numbers.

First three competitive learning models are reviewed: learning vector quantization, fuzzy learning vector quantization, and a deterministic scheme called the dog-rabbit (DR) model. These models can be used with labeled data to generate multiple prototypes for classifier design. Then these three models are compared to three methods that are not based on competitive learning: a clumping method due to C.L. Chang; a new modification of C.L. Chang’s method; and a derivative of the batch fuzzy c-means algorithm due to Yen and C.W. Chang. The six multiple prototype methods are then compared to the sample-mean based nearest prototype classifier using the Iris data. All six multiple prototype methods yield lower error rates than the labeled subsample means classifier (which yields 11 errors with 3 prototypes). The modified Chang’s method is, for the Iris data and processing protocols used in this study, the best of the six schemes in one sense; it finds 11 prototypes that yield a resubstitution error rate of 0. In a different sense, the DR method is best, yielding a classifier that commits only 3 errors with 5 prototypes.

Data analysis has been described as “the search for structure in data”, or as a means of reducing complexity. Most of the traditional methods for data analysis or data mining are dichotomous, i.e. they assume that patterns to be detected are two-valued. Whenever this is not the case the relationship between data or elements on one hand and classes to which these data shall be assigned on the other hand becomes gradual. In these cases fuzzy classification methods become appropriate. After a short introduction into data analysis algorithmic as well as knowledge based approaches of fuzzy data analysis are described. Finally tools and industrial applications will be presented.

In this paper we explain in a tutorial manner the technique of reasoning in probabilistic and possibilistic network structures, which is based on the idea to decompose a multi-dimensional probability or possibility distribution and to draw inferences using only the parts of the decomposition. Since constructing probabilistic and possibilistic networks by hand can be tedious and time-consuming, we also discuss how ta learn probabilistic and possibilistic networks from a data, i.e. how to determine from a database of sample cases an appropriate decomposition of the underlying probability or possibility distribution.

Various image pattern recognition techniques based on fuzzy technology developed by the authors’ group have been surveyed. First fuzzy clustering is applied to the remote sensing images. It is a modified version of the well known FCM. Then a shape recognition algorithm is presented for a robotics assembling line. It is a fuzzy discriminant tree method for real-time use. Finally a fuzzy dynamic image understanding system is presented. It can understand the dynamic images on general roads in Japan, where a fuzzy frame based knowledge representation and a special kind of fuzzy inference engine are introduced.

Visual perception is a difficult task to automate. This process, known as computer vision, has received a considerable amount of attention for the last three or four decades. Even with all of the research and development efforts, relatively few real computer vision systems have been put into routine use - these being primarily in controlled environments. Yet, the potential of general purpose vision systems which can effectively operate in varying scenarios is so great that much research continues to be devoted to the components of a computer vision system. These components include tasks such as noise removal, smoothing, and sharpening of contrast (low-level vision); segmentation of images to isolate objects and regions and description and recognition of the segmented regions (intermediate-level vision); and finally interpretation of the scene (high-level vision). Uncertainty exists in every phase of computer vision. Some of the sources of this uncertainty include: additive and non-additive noise of various sorts and distributions in low-level vision, imprecisions in computations and vagueness in class definitions in intermediate-level vision, and ambiguities in interpretations and ill-posed questions in high-level vision. The use of multiple image sources can aid in making better judgements about scene content, but the use of more than one source of information poses new questions of how the complementary and supplementary information should be combined, how redundant information should be treated and how conflicts should be resolved.

This paper deals with some intelligent control schemes for robotic systems, such as a hierarchical control based on fuzzy, neural network, genetic algorithm, reinforcement learning control, and group behavior control scheme. We also introduce the network robotic system, which is a new trend in robotic systems. The hierarchical control scheme has three levels: learning level, skill level and adaptation level. The learning level manipulates symbols to reason logically for control strategies. The skill level produces control references along with the control strategies and sensory information on environments. The adaptation level controls robots and machines while adapting to their environments which include uncertainties. For these levels and to connect them, artificial intelligence, neural networks, fuzzy logic, and genetic algorithms are applied to the hierarchical control system while integrating and synthesizing themselves. To be intelligent, the hierarchical control system learns various experiences both in a top-down manner and a bottom-up manner. The reinforcement learning is very important for acquisition of the control signal without any previous information of the system or environment. The group behavior control scheme which is one of the artificial life research areas, and the network robot control scheme are also very important for multiple robotic systems. Thus, these control schemes are effective for intelligent robotics.

The paper aims to present well-described examples of VLSI-dedicated architectures for soft computing applications in terms of performances and internal functional structure. In addition, closely connected to the described hardware components, an example of a Neuro-Fuzzy software architecture able to perform a system identification process on available measurements, is presented with some reference applications.

In this study, first recent applications of fuzzy logic in design and control of manufacturing systems are briefly reviewed. Fuzzy logic control is presented as a technique to implement real-time control algorithms that can be embedded in workstations in the framework of intelligent hierarchical control of manufacturing systems. Then two applications of fuzzy logic in design and control of manufacturing systems are presented. The first application is a fuzzy decomposition method for performance evaluation of manufacturing systems. A model of a workcell with rework is used to describe the fuzzy decomposition method. Fuzzy logic is used as a gain scheduler in the search algorithm of the decomposition method. It is observed that fuzzy logic improves the convergence rate of the decomposition method substantially. The second application is a fuzzy flow controller that adjusts the production rate of a failure prone manufacturing system in order to minimize total inventory carrying and backorder costs while satisfying uncertain demand. As an extension of this model, adaptive fuzzy flow rate controller is discussed.

We discuss the major characteristics of two classes of fuzzy-logic techniques for the planning and control of systems operating in highly uncertain environments. These applications are characterized by strong requirements for robust behavior and for reactive response to unexpected circumstances. These requirements demand that the decision/control policies be capable of attaining, to the highest possible degree, a number of purposive and reactive goals. Fuzzy logic is an attractive approach to treat this type of questions because of its ability to combine numerical treatments of decision-making problems, its reliance on artificial-intelligence techniques for the context-dependent activation of control rules, and its conceptual relations to analogical-reasoning methods based on notions of similarity and resemblance.

Techniques in the first class — developed in the context of autonomous robot applications — are based on hierarchical supervisory approaches that divide decision/control responsibilities between low-level controllers — concerned with the attainment of specific goals — and high-level supervisors — deliberating about context-dependent goal attainability. Methods in the second class — developed for applications with stringent real-time requirements — are based on an axiomatic approach to the formal representation of knowledge about the relative importance of goals in various operational contexts.

NASA is not unique in its quest for a product development cycle that is better, faster, and cheaper. Major advances in technical information management will be required to achieve significant and obvious process improvement goals. A vision of order for the associated systems of unstructured and unconnected files and databases is the first step towards organization. This is provided by examining the basic nature of technical information, item relationships, change and knowledge processing demands to be placed on any management system that supports all aspects of data representation and exchange during the product’s full propose, design, develop and deploy life cycle. An infrastructure that partitions product technical information relative to the perspectives of creation time phase, type and the cause of change provides sufficient structure. This enables maximal use of existing CAD/CAE/CAM/… software tool systems and digital library data mining capabilities. Introducing the concept of packaging technical information in a machine interpretable manner, at key life cycle deliverable and product review milestone points, provides the fastener needed for the attachment of the relevant soft computing and intelligent support capabilities discussed at the NATO Advanced Study Institute on Soft Computing and reported elsewhere within this volume. It also provides the basis upon which task automation capabilities can evolve.