Applications of the growing self-organizing map

doi:10.1016/S0925-2312(98)00037-X

Neurocomputing

Volume 21, Issues 1–3, 6 November 1998, Pages 91-100

https://doi.org/10.1016/S0925-2312(98)00037-X Get rights and content

Abstract

The growing self-organizing map (GSOM), an extension of Kohonen’s self-organizing map algorithm, adapts not only the position of the map weight vectors in the input space, but also the topology of the map output space grid. This additional feature allows for an unsupervised generation of dimension-reducing projections with optimal neighborhood preservation, even if the effective dimensionality of the input data set is not known. In three case studies involving real-world data sets we show that the GSOM is able to reproducably generate projections with a very good degree of neighborhood preservation. For one of the data sets, an experimentally obtained time series from a nonlinear system, the correct dimensionality $d^{A} ≈3$ of the underlying attractor is known from other methods; here the GSOM leads to maps without space grids which are also three dimensional.

Introduction

The self-organization of topographic maps is a broadly pursued research topic, both in the biological realm (development of maps in sensory and motor areas of the brain 2, 18, 20, 23) and in signal processing applications. Here, neural maps are utilized in the fashion of neighborhood preserving vector quantizers which map data from an (possibly high-dimensional) input space $V ⊆ R^{d^{V}}$ onto neurons in a map output space $A$ . In both fields, Kohonen’s self-organizing map (SOM) algorithm provides the basis for many investigations (for a brief introduction to the SOM, see the accompanying article [15], for an overview over applications of the SOM, see [16]). The neurons are located at positions $r_{k}, k=1,…, N$ , in the map output space $A$ and have associated to them weight vectors $m_{r_{k}}$ which determine a particular position in the map input space $V$ . The mapping $Ψ_{V→A}$ is realized by a winner takes it all rule $Ψ_{V→A} : v ↦ s = argmin r_{k} ∈ A ‖ v − m_{r_{k}} ‖.$ To achieve this projection, SOMs adapt the weight vectors $m_{r_{k}}$ by learning steps which involve the presentation of data points (or stimuli) $v$ , followed by adaptation steps $Δ m_{r_{k}} =εh_{r_{k}s} (v − m_{r_{k}}).$ The neighborhood function $h_{r_{k}s}$ , usually chosen to be of Gaussian shape, $h_{r_{k}s} = exp − ‖ r_{k} − s ‖^{2} 2σ^{2},$ enforces neurons which are close in the map output space to keep their weight vectors closeby in the map input space.

In this way, a neighborhood preserving mapping from input to output space is established, provided the map output space matches the topology of the data manifold in the input space. If this is not the case, as in the famous example of a square input space which is mapped onto a line of neurons as an output space, the SOM-algorithm cannot possibly achieve a neighborhood preserving projection (the issue of defining or measuring neighborhood preservation is tricky, for a more intricate discussion see 4, 13, 21). As the effective dimensionality of a data manifold in the input space is most often (a priori) unknown, straightforward application of the SOM with fixed output space topology does not guarantee an optimal neighborhood preservation. The growing self-organizing map-algorithm (GSOM, [3]) provides a method to get around this problem. In the GSOM, the output space has the shape of a generalized hypercube, parametrized by extensions n₁×n₂×n₃… along the different dimensions. During a growth procedure, these different extensions, as well as the weight vectors $m_{r_{k}}$ , are objects of the adaptation. A result of $n_{j} =1, ∀j>d^{A}$ means that the output space grid is $d^{A}$ -dimensional. In the following sections, we first describe the operation of the GSOM in some detail, and then proceed to three case studies demonstrating the application of the GSOM to the neighborhood preserving projection of real-world data sets.

Section snippets

The growing self-organizing map

Initially, a GSOM starts with just two neurons, i.e. a minimal one-dimensional configuration. After adaptation of the two corresponding weight vectors following the regular SOM procedure, either a third neuron is added to form a line of increased length, or it is decided to open up a further output space dimension, resulting in a 2×2-network. In all further growth steps, the map output space grid is further enlarged by increasing the extension into one of the existing dimensions by one, i.e. $n_{i}$

Results of simulations

The GSOM-procedure has been shown to work well on synthetic data sets [3]. In particular, the influence of noise on the map development of the GSOM was investigated. Thereby, the data were embedded in a high-dimensional data space with noise in the additional data dimensions. It was shown that the GSOM detects the relevant shape of the data up to a noise level of 5%. Because of these considerations, here we only refer to [3] for a further analysis.

However, more challenging tests for such an

Discussion

As pointed out in the introduction, the necessity of prior specification of an output space lattice, and the desire to achieve an optimal preservation of neighborhoods in SOMs are somewhat opposing constraints. One simple solution of the dilemma is to test several sensible map lattice structures, and to choose the best among them using some measure of “how well the map matches the data” (various measures have been proposed along these lines 1, 6, 19, 21, 24). A more elegant solution consists in

Thomas Villmann was born in 1963 in Potsdam, Germany. He studied mathematics and received a Diploma degree in 1990 and a PhD in computer science in 1996, both from the Leipzig University, Germany. Presently, he is with the Klinik for Psychotherapy and Psychosomatic Medicine of the University of Leipzig. His research interests include the theory of artificial neutral networks, in particular neural maps, neural modelling, evolutionary algorithms and hybrid systems. In addition he applies the

References (24)

J.C. Bezdek et al.
An index of topology preservation for feature extraction
Pattern Recognition
(1995)
B. Fritzke
Growing cell structures – a self-organizing network for unsupervised and supervised learning
Neural Networks
(1994)
P. Grassberger et al.
Measuring the strangeness of strange attractors
Physica D
(1983)
H. Heng et al.
Analysing the chaotic motion of a driven pendulum
Chaos, Solit. Fract.
(1992)
Th. Martinetz et al.
Topology representing networks
Neural Networks
(1994)
H.-U. Bauer et al.
Quantifying the neighborhood preservation of self-organizing feature maps
IEEE Trans. Neural Networks
(1992)
H.-U. Bauer
Development of oriented ocular dominance bands as a consequence of areal geometry
Neural Comput.
(1995)
H.-U. Bauer et al.
Growing a hypercubical output space in a self-organizing feature map
IEEE Trans. Neural Networks
(1997)
H.-U. Bauer, M. Herrmann, Th. Villmann, Neural Maps and Topographic Vector Quantization, submitted to Neural Network...
H.-P. Bähr, Th. Vögtle, Digitale Bildverarbeitung: Anwendung in Photogrammetrie, Kartographie und Fernerkundung,...

J.M. Digman

Personality structureemergence of the five-factor model

Ann. Rev. Psychol.

(1990)

B. Fritzke

Growing grida self-organizing network with constant neighborhood range and adaptation strength

Neural Processing Lett.

(1995)

Cited by (39)

A survey on machine learning for recurring concept drifting data streams
2023, Expert Systems with Applications
Citation Excerpt :
For this reason, it has been adapted in different research works (Liu & Ban, 2015; Prudent & Ennaji, 2005; Si, Lin, & Vuong, 2000) that have introduced dynamic (growing) methods for online learning. An example of these is growing self-organising maps (GSOM) (Fritzke, 1996; Villmann & Bauer, 1998). It grows nodes at the edges of the map when the total distance of an example exceeds a threshold, which allows it to track regions that may present dynamic behaviours when the original SOM would stabilise and lose its capacity to re-shape.
The problem of concept drift has gained a lot of attention in recent years. This aspect is key in many domains exhibiting non-stationary as well as cyclic patterns and structural breaks affecting their generative processes. In this survey, we review the relevant literature to deal with regime changes in the behaviour of continuous data streams. The study starts with a general introduction to the field of data stream learning, describing recent works on passive or active mechanisms to adapt or detect concept drifts, frequent challenges in this area, and related performance metrics. Then, different supervised and non-supervised approaches such as online ensembles, meta-learning and model-based clustering that can be used to deal with seasonalities in a data stream are covered. The aim is to point out new research trends and give future research directions on the usage of machine learning techniques for data streams which can help in the event of shifts and recurrences in continuous learning scenarios in near real-time.
Topological structural analysis based on self-adaptive growing neural network for shape feature extraction
2022, Neurocomputing
Citation Excerpt :
For this reason, many researchers improved the algorithm on the basis of SOM, hoping to automatically adjust the network structure according to the inherent properties in the input data [16–18]. GSOM (growing self-organizing map) can dynamically and adaptively grow neurons from the boundary of the network in the process of network training [19]. Growing Grid is a kind of self-growing neural network, which grows by adding a row or a column of neurons, so it maintains a square network topology [20].
Self-organizing neural network has been widely used to extract the topological structure of image shape because it features topological preservation, dynamic adaptation, clustering and dimensionality reduction. However, it is difficult to automatically extract the topology structure with an appropriate number of neurons from the complex and diverse data. In this paper, a novel self-organizing neural network called self-adaptive growing neural network (SAGNN) is proposed, which can generate an appropriate number of neurons autonomously according to the size of input data without setting the total number of neurons in advance. Firstly, Similarity Evaluation Index (SEI) is proposed to evaluate the similarity between the output network and the input space. Then, on the basis of growing neural gas (GNG) network, the SEI as a network growth control condition is introduced into the SAGNN, so that the SAGNN can grow neurons on demand until the expected quantization error is not significantly improved. Experiments involving both artificial and real data sets show that SAGNN can extract the topological structure from the unsupervised data without any prior conditions (including the appropriate number of neurons).
Machine learning for holistic visualization of STEMI registry data
2021, Journal of Biomedical Informatics
Widespread adoption of evidence-based guidelines and treatment pathways in ST-Elevation Myocardial Infarction (STEMI) patients has considerably improved cardiac survival and decreased the risk of recurrent myocardial infarction. However, survival outcomes appear to have plateaued over the last decade. The hope underpinning the current study is to engage data visualization to develop a more holistic understanding of the patient space, supported by principles and techniques borrowed from traditionally disparate disciplines, like cartography and machine learning.
The Minnesota Heart Institute Foundation (MHIF) STEMI database is a large prospective regional STEMI registry consisting of 180 variables of heterogeneous data types on more than 5000 patients spanning 15 years. Initial assessment and preprocessing of the registry database was undertaken, followed by a first proof-of-concept implementation of an analytical workflow that involved machine learning, dimensionality reduction, and data visualization. 38 pre-admission variables were analyzed in an all-encompassing representation of pre-index STEMI event data. We aim to generate a holistic visual representation — a map of the multivariate patient space — by training a high-resolution self-organizing neural network consisting of several thousand neurons. The resulting 2-D lattice arrangement of n-dimensional neuron vectors allowed patients to be represented as point locations in a 2-D display space. Patient attributes were then visually examined and contextualized in the same display space, from demographics to pre-existing conditions, event-specific procedures, and STEMI outcomes. Data visualizations implemented in this study include a small-multiple display of neural component planes, composite visualization of the multivariate patient space, and overlay visualization of non-training attributes.
Our study represents the first known marriage of cartography and machine learning techniques to obtain visualizations of the multivariate space of a regional STEMI registry. Combining cartographic mapping techniques and artificial neural networks permitted the transformation of the STEMI database into novel, two-dimensional visualizations of patient characteristics and outcomes. Notably, these visualizations also drive the discovery of anomalies in the data set, informing corrections applied to detected outliers, thereby further refining the registry for integrity and accuracy. Building on these advances, future efforts will focus on supporting further understanding of risk factors and predictors of outcomes in STEMI patients. More broadly, the thorough visual exploration of display spaces generated through a conjunction of dimensionality reduction with the mature technology base of geographic information systems appears a promising direction for biomedical research.
Online state space generation by a growing self-organizing map and differential learning for reinforcement learning
2020, Applied Soft Computing
Citation Excerpt :
This allows the network to be kept growing until the accuracy is sufficiently good. Villmann and Bauer presented an SOM in which growing is not restricted to rows or columns, and the grid becomes a hypercube to achieve low recognition errors and preserves neighborhoods between the data space and the map output space [21,22]. On the other hand, a growing hierarchical SOM proposed by Dittenbach et al. has a hierarchical architecture composed of independent growing SOMs [23].
In this research, we develop a method integrating a growing self-organizing map and differential learning system for online reinforcement learning which adaptively builds the state structure. In the conventional method, models and information on the environment are required beforehand, whereas the proposed method automatically estimates the state transitions from differentials of input signals and from these builds the state space without reference to prior information on the environment. Also, since it is an online learning method, the proposed method requires less computation and no batch memory. Through numerical experiments, we show that the proposed method has the same performance as the conventional method with the information given and that the learning time is shortened by abstraction of the state space.
A hand shape instruction recognition and learning system using growing SOM with asymmetric neighborhood function
2016, Neurocomputing
Citation Excerpt :
To overcome these problems, different improved SOM variants have been proposed. For example, to deal with the exhaustion of units, Growing SOM (GSOM) is proposed in [22–25], and Transient-SOM (T-SOM) is proposed in our previous works [26,27]. To keep stable learning convergence and use less parameters, Parameterless SOM (PL-SOM) is proposed by Berglund & Sitte in [28] and we adopted its idea to GSOM as a Parameterless Growing SOM (PL-G-SOM) [7–9,29].
For Human–Machine Interaction systems, it is a convenient method to send user׳s instructions to robots, TV sets, and other electronic equipments by showing different shapes of a hand of user. In our previous works, we proposed to use improved Kohonen׳s Self-Organizing Maps (SOMs), i.e., Transient-SOM (T-SOM) and Parameterless Growing SOM (PL-G-SOM) to recognize different patterns of hand shapes given by different bendings of five fingers of a hand. Recently, an asymmetric neighborhood function was proposed and introduced into the conventional SOM to improve the learning performance by Aoki and Aoyagi. In this paper, we propose to employ their asymmetric neighborhood function into Growing SOM (GSOM), which is an improved SOM to deal with additional online learning for input data. Furthermore, the improved GSOM is applied to a hand shape recognition and instruction learning system, and the results of experiments with eight kinds of instructions showed the effectiveness of the proposed system.
Constrained Semi-supervised growing self-organizing map
2015, Neurocomputing
Semi-supervised clustering tries to surpass the limits of unsupervised clustering using extra information contained in occasional labeled data points. However, providing such labeled samples is not always possible or easy in real world applications. A weaker, yet still very useful option is providing constraints on the unlabeled training samples, which is the focus of the Constrained Semi-Supervised (CSS) clustering. On the other hand, online learning has gained considerable amount of interests in real world problems with massive sample size or streaming behavior, as lack of memory and computational resources seriously restrict the application of the offline and batch methods. However, the existing algorithms for online CSS clustering problem either assumed that the entire dataset is available and added constraints incrementally or considered chunks of constrained data points and applied an offline CSS clustering algorithm. Thus, none of them can be categorized as a genuine online CSS clustering algorithm.
In this paper, we propose CS2GS, an online CSS clustering algorithm. CS2GS is constructed by modifying the online learning process of Semi-Supervised Growing Self-Organizing Map, and converting it to an iterative constrained metric learning problem that can be solved using the Bregman׳s iterative projections. The proposed CS2GS is studied via a series of thorough tests using synthetic and real data including selections from UCI datasets and FEP – a recent bilingual corpus used for sentence aligning stage of machine translation. Experimental results show the effectiveness of CS2GS in online CSS clustering, and prove that indeed, the limits of the system accuracy may be pushed higher using unlabeled samples.

View all citing articles on Scopus

Hans-Ulrich Bauer was born in 1961 in Germany. He studied physics and received a Master’s degree from the University of California at San Diego, a Diploma degree from the Technical University of Munich, and a PhD from the University of Frankfurt in 1990. His interest in neural networks began several years ago when he investigated the stability of recurrent networks and their application to speech processing during his graduate work. Currently he is at the Max-Planck-Institute für Strömungsphysik in Göttingen where he applies ideas from nonlinear dynamics to the analysis of neural systems. One of his particular interests involves the self-organization of neural maps. A different line of research is concerned with the modeling and functional interpretation of dynamical neural responses, primarily in the visual system.

¹: This work has been supported by Deutsche Forschungsgemeinschaft, SFB 185 “Nichtlineare Dynamik”, TP E6.

View full text

Applications of the growing self-organizing map1

Abstract

Introduction

Section snippets

The growing self-organizing map

Results of simulations

Discussion

Pattern Recognition

Neural Networks

Physica D

Chaos, Solit. Fract.

Neural Networks

Quantifying the neighborhood preservation of self-organizing feature maps

IEEE Trans. Neural Networks

Development of oriented ocular dominance bands as a consequence of areal geometry

Neural Comput.

Growing a hypercubical output space in a self-organizing feature map

IEEE Trans. Neural Networks

Personality structureemergence of the five-factor model

Ann. Rev. Psychol.

Growing grida self-organizing network with constant neighborhood range and adaptation strength

Neural Processing Lett.

Applications of the growing self-organizing map¹