Adaptive Modelling, Estimation and Fusion from Data

Conventional science and engineering is based on a set of abstract physical or phenomenological principles based on an evolving scientific theory such as Newtonian mechanics, wave theory or thermodynamics, which are then validated by experimental or observational data. However for analytic tractability and ease of comprehension much of this theory is linear and time invariant, yet as the performance requirement and the range of operation of controlled processes increases, nonlinear, nonstationary and stochastic system behaviours need to be adequately represented. The power of current parallel computers allows simulation methods such as computational fluid dynamics, computational electromagnetics, or finite element methods to be integrated with observational data to resolve some of these nonlinear modelling problems. However such approaches are inappropriate to processes that are too complex to have ‘local’ phenomenological representations (such as Navier Stokes, or Maxwell equations), or the underlying causal processes or physical relationships are a priori unknown. If measurement data or information is available in sufficient quantities and richness, then models or relationships between a systems independent and dependent (input-output) variables can be estimated by empirical or data driven paradigms. This is not a new approach, since it is the basis of classical statistical estimation theory whereby the properties of some a priori unknown system output y(t) = f(x(t)) is estimated from available sets of input-output data \( {D_N} = \{ x(t), y(t)\} _{t = 1}^N \), drawn from an unknown probability distribution, which is then utilised to predict future outputs to unseen input data (x(t)) (the principle of generalisation), where the observed data set is the collection of all observed input-output regressors of the unknown process f(·).

In this chapter we provide the basic model mathematical representations that are used throughout the book. Both parametric and nonparametric models require structural representations which reflect the modeller’s prior knowledge of the underlying process, or is selected so as to provide a form that ensures process identification easily from observed data, or is selected with another end process in mind such as control or condition monitoring. Generally in control and fault diagnosis problems, the model representation is in state space form (see Section 2.2) as knowledge of the unknown systems states are required for detection of process faults, or for state feedback control, or for state vector data fusion (see Section 9.3). Fundamental to this book is the representation of nonlinear observable processes by additive basis function expansions for which the basis functions are locally defined (i.e. have compact local support) rather than global basis functions such as general polynomials. In the sequel this representation coupled with linearly adjustable parameters is shown to have many knowledge and computational based advantages such as parameterisation via linear optimisation, easy incorporation of prior knowledge, and model transparency with direct links to fuzzy rule base representations.

Despite the fact that artificial neural networks (ANNs) have been proposed primarily as nonlinear learning systems, considerable insight into the behaviour of these networks can be gained from linear modelling techniques, for which the literature is vast (see for example [8, 137, 181]), and significantly the resultant theory is directly applicable to a powerful and special class of ANNs, i.e. linear-in-the-parameters networks, which forms the basis of this book. An obvious reason to use ANNs is their ability to approximate arbitrarily well any continuous nonlinear function, with the specific network architecture and parameter/weight adjustment algorithm determining how well learning is achieved. Of particular interest in adaptive control and estimation is the ability of algorithms to model, track and control on-line. However, it is infeasible to assume that the input signals excite the whole of the state space (a prerequisite of identification theory), and so it is necessary to consider the effects of a reduced input signal on overall functional approximation for various network architectures and associated learning laws. Here learning must be local, in that adjustable network weights or parameters should only affect the network’s output locally. In ANNs, the vast majority of supervised learning rules are based on the assumption that the nonlinear network can be locally linearised, so that it is natural to develop on-line learning algorithms with provable learning network stability and convergence conditions, and to choose ANNs which have linear-in-the-parameters with local behavioural characteristics.

Fuzzy logic and fuzzy systems have received considerable attention both in scientific and popular media, yet the basic techniques of vagueness go back to at least the 1920s. Zadeh’s seminal paper in 1965 on fuzzy logic introduced much of the terminology that is used conventionally in fuzzy logic today. The considerable success of fuzzy logic products, as in automobiles, cameras, washing machines, rice cookers, etc. has done much to temper much of the scorn poured out by the academic community on the ideas first postulated by Zadeh. The existing fuzzy logic literature, number of international conferences and academic journals, together with a rapidly increasing number and diversity of applications is a testament to the vitality and importance of this subject, despite the continuing debate over its intellectual viability.

Whilst neurofuzzy systems have become an attractive powerful data modelling technique, combining well established learning laws of neural networks and the linguistic transparency of fuzzy logic, they do suffer from the curse of dimensionality. A network with two inputs with seven fuzzy sets per input produces a complete rule set of 49 rules, whereas a system with five inputs requires 16,800 (≃ 7⁵) rules or weights. Clearly conventional fuzzy and neurofuzzy systems are practically limited to low dimensional modelling. This drawback is a direct consequence of employing a lattice based partitioning strategy on the input space. Algorithms that do not use orthogonal split input lattice structures will potentially avoid this problem, however replacing it with another produces a lack of interpretability or transparency: the prime purpose of using fuzzy/neurofuzzy algorithms. However the modelling capabilities of any modelling technique are heavily dependent on their structure which must be determined similarly to its parameters, when little a priori knowledge is available. To solve high dimensional problems and yet retain all the desirable properties of neurofuzzy systems (e.g. linear-in-the-weights, transparency, partition of unity), some form of model complexity reduction must be performed, producing parsimonious models. Hence during model construction the following principles should be employed:

The previously described model construction algorithms of Chapter 5 resolved model complexity by utilising the divide and conquer strategy, by decomposing high dimensional problems into a number of lower dimensional submodels each with variable dependencies whose composite solution yields the original complex problem. Central to the conventional neurofuzzy approach is an orthogonal axis partitioning of the input space, which is the main cause of the curse of dimensionality. Clearly, other decompositions or partitioning are possible including irregular simplexes (see Figure 6.1(a)) and data clustering with centred Gaussians (see Figure 6.1(b)).

We have already demonstrated in previous chapters that data-b ased modelling is a complex and demanding process especially if there is little prior knowledge about the underlying process. In practice there is usually some process structural knowledge that can be exploited in model construction, additionally specific model structures lend themselves to subsequent uses such as a control design and estimation (see Chapter 8). For many processes the derived models are only valid across a limited domain (for empirical data modelling this is usually determined by the data gathering process) , and are often implicitly represented in the model. In this regard the concept of local models is a promising te chnique since they combine conventional system theory approaches (especially for locally linear models — see Chapter 6) with adaptive learning algorithms.

The previous chapter discussed nonlinear state estimation based on neurofuzzy linearisation models, where available measurements are from a single set of measurements. One of the most important state estimation applications is in target tracking, where nonlinearity and uncertainty are inherently severe. The main purpose of a target tracking system is to obtain a target trajectory, which is more accurate than the direct trajectory observation, and to predict the target behaviour in the near future. In many target tracking problems, in order to obtain more accurate and robust tracking performance, a dynamic target is often detected by multiple disparate sensors, each with different measurement dynamics and noise characteristics. For example, in ship collision avoidance guidance and control, as shown in Figure 9.1, each ship has on-board sensors to measure the positions of itself and other ships or obstacles, and there is a vessel traffic control centre that can also measure ship positions and communicate with each ship. For a particular target (ship or obstacle), there are plenty of associated sensor measurements at different levels and with different accuracy and reliability. Various data fusion problems naturally arise such as how can the multisensor measurements be combined to obtain a joint estimate of the target state vector, which is superior to the individual sensor based estimates and how can the target information be shared robustly from the various data sources (or locations)?

The class of models considered so far are generalised linear models that construct nonlinear models by linear combinations of nonlinear basis functions such as B-splines, or Gaussian radial basis in the input or observed variables x. The power of these models is their ability to incorporate prior knowledge by structurally designing the network through choice of the type, number and position of the basis functions. This form of structural regularisation is the basis of many of the construction algorithms introduced in this book. All of these generalised linear model are examples of parametric models in which weights or parameters are identified by using linear optimisation techniques (see Chapter 3). A further class of models can be defined which do not explicitly depend on a set of parameters, the so-called nonparametric models, are applicable to sparse data sets in relatively high dimensional spaces. In nonparametric models, the parameters are not predetermined, but are determined by the training data so that the model capacity reflects complexity contained in the data. Of particular importance in this context is the class of nonparametric models whose output is a linear combination of functions of the observations x, where the linear weighting functions are determined by the characteristics of the kernel functions. The approximation of the approximant is taken over a functional including some measure of the data fit and a functional penalising certain characteristics of the approximant, this ensures a well posed solution.