Clustering techniques for Fuzzy Cognitive Map design for time series modeling
Introduction
Understanding and modeling complex phenomena in our environment is among the most important problems that we face. There is a need not only for numerical approaches that express knowledge in a form of raw numbers, but also for qualitatively-oriented methods that provide description of information in a human-centered way, coherent with the way how we perceive and learn.
Though giant strides have been made in the recent decades in brain studies, yet there are many unraveled mysteries. Humans do not think in a binary, “raw”, machine-alike way. Instead, we acquire and process knowledge in a form of concepts. Acknowledging it leads to formulation of modeling frameworks that aim at mimicking a concepts-based knowledge representation system. Cognitive Maps, and in particular a family of Cognitive Maps named Fuzzy Cognitive Maps (FCMs), are examples of such approaches.
FCM represents knowledge with a signed directed graph. Nodes in the map correspond to abstract concepts, edges to the relationships between them. In an FCM-based model we also evaluate the levels of concepts activation. By iterative exploration of the map, we observe how concepts' states change in response to various input factors and interactions between them. In other words, we learn structures in data. Such information representation scheme provides an inherent ability to model and forecast phenomena.
The research reported in this paper is built upon the premise that FCMs provide necessary means to represent complex phenomena in an intuitive way, similar to the way in which human beings think and reason. Let us stress here the role of linguistic labels, which we attach to FCM nodes in order to enhance human-centeredness of the model.
We take under investigation the time series modeling procedure, in which map nodes correspond to phenomenon levels. Examples of linguistic labels for such nodes are: “low level of phenomenon”, “moderate”, “high”, etc. We postulate that the number of nodes in the map (i.e. map design) should be learned from data. Having in mind that the number of nodes is the elementary factor determining ease of interpretation we present an experimental study, where we employ cluster validity indexes to evaluate different map designs.
The primary objective of our research is to provide a comprehensive, algorithmically-aided methodology for time series modeling using FCMs. Map design is a crucial step of the modeling procedure that despite its importance is often neglected. The particular aim of this study is to introduce a new FCM design technique based on cluster validity indexes that allows constructing models that represent time series well. This method equips the modeler with a measure for FCM evaluation that can be employed before the training process. So far the modeling methodology that we work with required repeating the training procedure in order to compare several ready models and select the best one. To avoid such inconvenience, we propose a novel data-driven FCM evaluation scheme. We show that it enhances the modeling process, helps constructing good maps and allows avoiding repetitions of the computationally costly training procedure. The contribution of our study is very practical: we present a suite of cluster validity indexes that perform well in the task of FCM nodes evaluation.
We admit that employing cluster validity indexes in a domain that at the first sight seems to be unfamiliar may raise questions. Let us argue that in fact there are elementary similarities between FCM design and clustering. The FCM-based time series modeling method that we develop expresses causal relationships between concepts that correspond to different levels of a phenomenon. Let us give a very simple example: say we have an FCM with three nodes corresponding to low, moderate, and high level of precipitation. The data that we used to construct the map has been gathered on a daily basis. The trained map could give us a prediction in the following form: if today we have, say, low precipitation then tomorrow we will most likely have, say, moderate precipitation. Connections in the FCM (edges in the map) that let us predict daily precipitation have been learned from actual time series data. The original time series, which typically is a very long sequence of numerical values, has been embedded in a particular coordinates system. Properties of the coordinates system, i.e. the modeling space and its dimensionality, play crucial role. Given example was on purpose a trivial case, recalled for illustrative purposes. Detecting regularities in the modeling space leads to extraction of meaningful concepts that generalize numerous data points. In consequence, we may replace inconvenient numerical values with general concepts. It becomes apparent that we do not know what would be the optimal number of concepts to represent a given dataset. The more concepts we have, the more detailed predictions we can make. At the same time, too many concepts cause confusion when we interpret the model. In this perspective, we propose to apply cluster validity indexes to evaluate various FCM designs. In a series of empirical experiments we show that the proposed approach lets us to form maps with superior quality.
The paper is organized as follows. In Section 2 we introduce basic notions related to FCMs. Section 3 presents the methodology of interest for time series modeling. Section 4 addresses the issues of map design and presents our approach. Section 5 presents experiments. Finally, Section 6 covers conclusions.
Section snippets
Knowledge modeling with Fuzzy Cognitive Maps – preliminaries
FCMs appeared in the literature in 1986 in [1]. B. Kosko, the father of FCMs, inspired by the works of a political scientist R. Axelrod [2] proposed a novel soft approach to knowledge modeling that soon gained a substantial popularity.
FCMs are an important class of models expressing hazy sequential linkages among concepts. An example of an FCM with three nodes is in Fig. 1.
An FCM based on c concepts is represented with a weighted directed graph. Concepts, also called nodes, are
Time series modeling with Fuzzy Cognitive Maps
In our research we refer to the approach to time series analysis with FCMs outlined in [3]. A time series is recognized as a scalar sequence:where zi is a real number and M is the number of data points in the scalar time series. The time series is transformed into a two-dimensional space by computing increment values: , where . Now, the time series is a sequence of pairs, where the start index is two, because the change of amplitude cannot be set together with
Design considerations
Extraction of concepts for an FCM is a pivotal design phase. Proper concepts and time series representation induce a good model. The goal of modeling with FCMs is to extract generalized relationships in the data. Hence, the objective is to select abstract concepts that represent the information in aggregated fashion and cover the dataset. It is worth to emphasize that once we have found a proper set of concepts we may start describing the entire dataset (that could be a so-called big data time
Environment
In the study we have applied various cluster validity indexes for FCM design evaluation. First, we elaborate in detail on two case examples. Subsequently, we present less detailed results for further datasets.
Experiments have been executed in R environment. The following external R libraries have been used: pso [26] for executing the Particle Swarm Optimization procedure, e1071 [27] for the fuzzy c-means algorithm, clusterCrit [25] for cluster validity indexes, dbscan [28] for the DBSCAN
Conclusion
The article discusses a comprehensive framework for time series modeling with FCMs. We have built upon an existing approach to time series modeling with FCMs and introduced a method for map design. In particular, we have shown that optimal FCM design, i.e. a collection of concepts that make the map, entails a superior model. The benefits of a well-designed map are twofold. First, it is the easiness of its interpretation and second, a satisfying numerical quality of predictions. We have employed
Acknowledgments
The research is supported by the National Science Center, grant No 2012/07/B/ST6/01501, decision No DEC-2012/07/B/ST6/01501.
Wladyslaw Homenda received the M.Sc. and Ph.D. degrees from Warsaw University of Technology, Warsaw, Poland, and the D.Sc. degree from the System Research Institute of Polish Academy of Sciences, Poland. He is currently an Associate Professor with the Faculty of Mathematics and Information Science, Warsaw University of Technology. He is also with the Faculty of Economics and Informatics in Vilnius (Lithuania) of the University of Bialystok, Poland. His main research interests are in theoretical
References (31)
Fuzzy cognitive maps
Int. J. Man-Mach. Stud.
(1986)- et al.
Multi-step prediction of pulmonary infection with the use of evolutionary fuzzy cognitive maps
Neurocomputing
(2012) - et al.
A complex-valued neuro-fuzzy inference system and its learning mechanism
Neurocomputing
(2014) - et al.
An experimental study of the dynamics of the certainty neuron fuzzy cognitive maps
Neurocomputing
(1999) - et al.
Development of dynamic cognitive networks as complex systems approximators: validation in financial time series
Appl. Soft Comput.
(2005) - et al.
Dynamic optimization of fuzzy cognitive maps for time series forecasting
Knowl.-Based Syst.
(2016) - et al.
Evolutionary learning of fuzzy grey cognitive maps for the forecasting of multivariate, interval-valued time series
Int. J. Approx. Reason.
(2014) - et al.
Design of fuzzy cognitive maps using neural networks for predicting chaotic time series
Neural Netw.
(2010) Structure of decision: the cognitive maps of political elites
Can. J. Political Sci.
(1979)- et al.
Numerical and linguistic prediction of time series
IEEE Trans. Fuzzy Syst.
(2008)
A fuzzy cognitive map approach to differential diagnosis of specific language impairment
Artif. Intell. Med.
Fuzzy cognitive maps learning using particle swarm optimization
J. Intell. Inf. Syst.
Cited by (28)
Comparing fuzzy cognitive maps: Methods and their applications in team communication
2022, International Journal of Industrial ErgonomicsCitation Excerpt :Any number of clustering methods could be applied to FCMs as they generally consist of multiple concepts that may have distinctive meanings but may still form a common theme ripe for clustering (Yoon and Jetter, 2016). Cluster analysis has been used sparingly with FCMs (Homenda and Jastrzebska, 2017), and more often with CMs and social network analysis (Eden, 2004; Eden et al., 1992; Vanwindekens et al., 2014; Özesmi and Özesmi, 2004). Most clustering methods are categorized as node measures because, similar to the presence/absence measure, they focus on binning different nodes based on similarity.
Fuzzy representational structures for trend based analysis of time series clustering and classification
2021, Knowledge-Based SystemsCitation Excerpt :These parameters are also weighted based on their importance in time series sequences [37]. Evaluating the cluster primitives during the training phase to obtain the optimized structures is an innovative research carried out by [38]. Fuzzy cognitive map structures are selected in training phase based on their clustering ability.
Temporal gap statistic: A new internal index to validate time series clustering
2021, Chaos, Solitons and FractalsCitation Excerpt :Despite the experimental study, the authors emphasize that it is not possible to determine the best index to validate time series clustering in the context of biomedical images. As the aforementioned work, there exists several similar papers such as [18–22], and [23]. Their main focus is the development of new clustering algorithms or the adaptation of existing ones.
Analyzing determinants of environmental conduct in small and medium-sized enterprises: A sociotechnical approach
2020, Journal of Cleaner ProductionCitation Excerpt :FCMs were first introduced by Kosko (1986: 65) as “fuzzy-graph structures for representing causal reasoning”. Following a similar approach to human reasoning and decision-making processes, FCMs complement cognitive maps with fuzzy logic, providing the necessary means to represent complex phenomena intuitively (Papageorgiou and Salmeron, 2013; Dodurka et al., 2017; Homenda and Jastrzebska, 2017). The construction of FCMs needs to be supported by a group of experts who can identify which concepts are relevant to the decision problems in question.
Short-term cognitive networks, flexible reasoning and nonsynaptic learning
2019, Neural NetworksCitation Excerpt :This is not the only option though. Previous research (Homenda & Jastrzebska, 2017) has produced successful solutions where the nodes in the FCM are actually information granules, bundling more information than just a bare observation in a time series. Other approaches (Poczeta, Kubuś, & Yastrebov, 2019) apply machine learning to make an appropriate selection of concepts based on data.
Combined approach to forecasting of manufacturing system target indicators in a changing external environment
2019, Procedia Computer Science
Wladyslaw Homenda received the M.Sc. and Ph.D. degrees from Warsaw University of Technology, Warsaw, Poland, and the D.Sc. degree from the System Research Institute of Polish Academy of Sciences, Poland. He is currently an Associate Professor with the Faculty of Mathematics and Information Science, Warsaw University of Technology. He is also with the Faculty of Economics and Informatics in Vilnius (Lithuania) of the University of Bialystok, Poland. His main research interests are in theoretical foundations of computer science, knowledge representation and processing and intelligent computing technologies, specifically in the areas of man-machine communication and human-centric computing, fuzzy modeling and Granular Computing, knowledge discovery and data mining. He currently serves as an Associate Editor of Information Sciences and is a member of several editorial boards of other international journals. More information can be found at http://www.mini.pw.edu.pl/~homenda/
Agnieszka Jastrzebska received her M.Sc. Eng. degree in Computer Engineering from the Rzeszow University of Technology, Poland and Ph.D. degree from the Warsaw University of Technology, Poland. She is research and teaching assistant at the Faculty of Mathematics and Information Science at the Warsaw University of Technology. In the past 4 years she was awarded prestigious scholarships from the Institute of Computer Science of Polish Academy of Sciences, the Systems Research Institute of Polish Academy of Sciences, and the Center for Advanced Studies of Warsaw University of Technology. Her main research interests include machine learning, computational intelligence, and fuzzy modeling.