A novel adaptive control design method for stochastic nonlinear systems using neural network

Fault tolerant control systems with actuator failure compensation have received many interests from the researchers of industrial control field over decades [1‐7]. Serious studies in computer science have been dedicated to address important theoretical and practical questions, raised in adaptive nonlinear control systems, where dynamic surface control (DSC) method served as a novel useful tool for designing adaptive control systems, especially for nonlinear strict-feedback [8], [9], and fractional-order [10] systems.

An important research question, which was not addressed in those studies [11‐21], is effect of stochastic behaviors and Prandtl–Ishlinskii (PI) hysteresis on the system performance. PI or backlash-like hysteresis and dead-zone phenomena are considered as the two important general nonlinearities, seen in the literature. However, a general adaptive control method with the capability of incorporating both stochastic and nonlinear behaviors of the control system, including the joint Prandtl–Ishlinskii hysteresis and dead-zone phenomena, cannot be seen in those studies in an objective way. One of the problems in developing such a generalized method corresponds to stability of the methods at the presence of an unknown nonlinearity.

Dynamic surface method has been employed by several neural network-based methods for nonlinear control systems [9, 12, 21‐24]. However, this is not true for stochastic nonlinear systems, when general nonlinearities such as PI hysteresis and dead zone appear in the actuators. To the best of our knowledge, the presented methods are mainly based on the backstepping method, which makes this method an appropriate baseline study [25]. To a lesser extent, a nonlinear stochastic system was studied, under the condition of actuator dead-zone, which considers either the time-delay [17, 18], or pure-feedback control design method [20]. It is important to note that in most of practical cases, the control systems, i.e., autonomous vehicle systems, nonlinear stochastic conditions are involved [26, 27]. In addition to these conditions, nonlinear behaviors such as dead-zone and hysteresis are typically seen in the actuators [11‐19, 21]. Ignoring such the conditions can lead to serious flaws like internal instability and physical damages. However, recently adaptive dynamic surface control for uncertain nonstrict-feedback systems is investigated in [28, 29].

In this paper, neural network in conjunction with dynamic surface control design is employed to introduce a novel method of adaptive control design for nonlinear stochastic systems with a general class of different actuator nonlinearities, including PI hysteresis and dead-zone. These nonlinearities might be a result of actuator aging, a faulty condition of the actuator, or its intrinsical characteristic. The unknown dynamics of the system are innovatively approximated using a Radial Basis Function (RBF) neural network, where the universal approximation capability of the method makes it possible to approximate a wide range of nonlinear Lipschitz functions. Furthermore, the minimal-learning-parameters algorithm is elaboratively employed to reduce the number of adaptive parameters in an online updating way, which effectively reduces the calculational complexities. In order to show effectiveness of the RBF in both the parameter approximation, and in the nonlinearity compensation of the actuators, a sophistication of the method is also proposed as a baseline method for comparison. In this baseline, compensation of the actuator nonlinearity is performed using an adaptive eliminating term.

The stability analysis of the proposed method along with the baseline are theoretically proved and confirmed by simulation. Performance of the direct method of backstepping is also investigated as another baseline for comparison. It is shown that the proposed controller guarantees the boundedness of all the closed-loop signals, where the tracking error remains in an arbitrary small vicinity of the origin, in terms of the mean quartic value. It is shown that the proposed method exhibits superior performance both in the failure-free condition and in different cases of the actuator nonlinearity, compared to the baselines.

The presented method offers extensive applications in a broad range of the engineering and industrial fields such as flight control [30], autonomous vehicle control systems [31, 32], turbo-machine design [33‐35], piezo-actuators [36] and micro-electro-mechanical-systems (MEMSs) [37], and also in various military applications [38].

The main contributions of the paper are: (1) presenting a novel neural network-based method for designing adaptive controller for nonlinear stochastic system with broad range of the actuator nonlinearity, (2) presenting a sophistication of the method as a baseline for the study, in which nonlinearity of the actuator is directly compensated without using the neural network, (3) analytically proving stability of the mentioned methods in failure free condition and also at the presence of the actuator nonlinearities, i.e., PI hysteresis, and dead-zone, (4) exploring performance of the direct backstepping method, detailed in [12], for a broad range of the actuator nonlinearity, as the second baseline study, (5) comparing the proposed method along with the two baselines using different cases of actuator nonlinearities, and studying privileges and limitations of each of the methods.

The paper is organized as follows. Section 1, presents a literature review on the previously published studies. In Section 3, preliminaries and problem statements are described. In Section 4, the methods along with the theorems are presented, which contains the main contributions of the paper. Simulation examples are presented in Section 5. In Section 6 and 7, discussion and conclusion of the paper are presented, respectively. In addition to the main sections of the paper, there are also five appendices, in which details of the theorem proofs are included accordingly.

Actuator failure can occur in many practical systems, named plants, that may lead to the plant instability and even sometime catastrophic events [1−7, 27, 39‐44]. Systematic design methods for different nonlinear control systems have been studied in the form of the strict-feedback, pure feedback, and block-strict-feedback [45], where various direct methods have been investigated for the purpose of actuator failure compensation [39‐44]. Backstepping design method was proposed as a systematic adaptive controller design, which is still considered as one of the mostly used methods for nonlinear systems. Backstepping-based methods for compensation of the actuator failures such as sliding-mode control [42], and adaptive failure compensation [5, 39, 41, 43, 44, 46‐49] have been proposed for several practical and theoretical systems. Among these methods, the problem of accommodating infinite number of actuator failures/faults in control systems has been investigated in [5]. Backstepping method was theoretically studied to be employed for adaptive control design for the parameter-strict-feedback systems [43], and its capabilities in compensating actuator nonlinearities for a flight control system were investigated [11]. Radial Bases Function (RBF) neural network has been integrated with backstepping method to overcome the problem of uncertain nonlinear systems in pure-feedback form with PI hysteresis [21]. Backstepping controller design method using adaptive neural networks was proposed in conjunction with variable separation and minimal-learning-parameters algorithm technique for stochastic nonlinear single–input–single–output systems in the form of nonstrict-feedback with unknown backlash-like hysteresis [21], strict-feedback [20], and pure-feedback [50], [51].

Although the backstepping design technique has many useful benefits for the designers, it suffers from an inherent problem, called ‘explosion of complexity’, that occurs with increasing system order, due to the continuously differentiation of virtual control signal and system states. Dynamic Surface Control (DSC) method was introduced as another alternative method, which resolves explosion of complexity [8‐10, 23, 32, 36‐38, 52, 53]. It avoids continuous differentiation of virtual control inputs leading to ‘explosion of terms’. DSC has been privileged over backstepping in several studies [9]. Integration of adaptive neural network and DSC was studied in nonlinear strict-feedback systems [12], and also in time-delayed nonlinear systems [15], under failure-free condition of the actuators, as well as a certain form of the PI hysteresis [16]. The effect of actuator dead-zone in nonlinear systems was separately studied for adaptive DSC method [23].

Application of dynamic surface method has been studied in several applied researches, such as controlling pneumatic servo system [32], trajectory tracking control of underactuated surface vehicles [36], suppressing chatter in a micro-milling machine with piezo-actuators [37], controlling micro-electro mechanical gyroscope systems [38], controlling process of continuous heavy cargo airdrop of nonlinear transport aircraft [52], controlling of spacecraft terminal safe approach with actuator saturation [53], and precise position tracking problem of permanent magnet synchronous motors [54].

In many practical systems their parameters, and dynamics, as well as the corresponding disturbances are unknown, but can likely show stochastic and mostly nonlinear characteristics. Details of a complete course of stochastic systems and stochastic differential equation are found in [17], [18].

Adaptive neural networks were employed in conjunction with the dynamic surface technique for nonlinear stochastic systems with either time-delays or dead-zone in the actuators [15]. A certain class of nonlinear systems, but not stochastic, with unknown Prandtl–Ishlinskii hysteresis was studied by X. Zhang et al. and the performance of the design method was investigated [22]. In this study, an adaptive neural DSC controller was constructed to eliminate the effect of unknown actuator hysteresis. The adaptive neural network was utilized in DSC design method to stabilize nonlinear time-delay systems with unknown disturbances [9]. Adaptive neural network control systems have been investigated for specific cases of uncertain nonlinear strict-feedback systems [12], and also a class of time-delay nonlinear systems with PI hysteresis with dynamic uncertainties [16‐18]. Nevertheless, for nonlinear stochastic cases, the adaptive neural network dynamic surface design was studied only under the condition of time delayed and actuator dead-zone [25].

A stochastic nonlinear system with strict-feedback can be defined by its state variable \(x={\left[{x}_{1},{x}_{2},\dots , {x}_{n}\right]}^{T}\in {R}^{n}\):where \(\psi\) is an r-dimensional variable introduced as standard Brownian motion defined on a complete probability space,¹ and \(f_{i} \left( \cdot \right), g_{i} \left( \cdot \right):R^{i} \times R^{ + } \to R\), \(\psi_{i}^{T} : R^{i} \times R^{ + } \to R^{i \times r}\) are unknown smooth functions in \(x_{i} \in R^{i}\) with zero initial conditions [25]. It should be noted that \(u\) in Eq. (1) is the control input that is by itself the output of an actuator, which can be subjected to different nonlinearities such as Prandtl–Ishlinskii (PI) hysteresis, or dead-zone.

Prandtl–Ishlinskii (PI) hysteresis is a nonlinearity defined as follows:where \(u(t)\) is the output of the actuator, \(v(t)\) is the input signal to the actuator, \(p(r)\) is the density function, \({p}_{0}= {\int }_{0}^{R}p(r)dr\) is a constant which depends on the density function \(p(r)\), and \({F}_{r}\left[v\right]\left(t\right)\) is a function, describing the nonlinearity behavior, and named the “play operator” [13]. It should be noted that Eq. (2) decomposes the hysteretic action into two terms, describing the linear reversible part and the nonlinear hysteretic behavior, at its first and the second terms, respectively. This decomposition is crucial since it facilitates utilization of the currently available control techniques for the controller design [15]. An actuator with PI hysteresis is a component with memory, and therefore its value depends on its previous outputs in time. Consequently, for an input \(v\left(t\right)\in {C}_{m}\left[0,{t}_{E}\right]\), where \({C}_{m}\left[0,{t}_{E}\right]\) is the space of piecewise monotone continuous functions, and the play operator is defined by:

withwhere \(0 = t_{0} < t_{1} < \ldots < t_{N} = t_{E}\) is a partition of \(\left[ {0,t_{E} } \right]\) such that the function v is monotone on each of the sub-interval \(\left[ {t_{i} , t_{i + 1} } \right]\), and \(u_{ - 1} \in R\) is the general initial condition [13].

Consider the PI-model expressed by the play operator in (7), the hysteresis output \(u(t)\) can be expressed as [14]:

where

It should be noted that (11) is bounded, and the detailed description of its boundedness is discussed in [16‐20]. Furthermore, in this paper it is assumed that the characteristics of PI hysteresis nonlinearity in the actuator is unknown and should be estimated by the controller.

Actuator dead-zone is another form of the nonlinearity model can be described as follows:where \(u\) is the output of the dead-zone, \(v\) is the input of the dead-zone, \({b}_{l}<0\) and \({b}_{r}>0\) are unknown parameters of the dead-zone, which should be estimated by the control system, named as the start and end of the dead-zone, respectively. The output of the dead-zone is not measurable, and therefore the smooth and bounded first derivative functions \({g}_{l}(v)\) and \({g}_{r}\left(v\right)\) are employed to express the output. In order to achieve a pseudolinear relationship between the input and output of the dead-zone, the following expression is often employed:where detailed description of functions \({K}^{T}\left(t\right),\Phi (t)\), and \(d(v)\) can be found in [16]–[19]. However, \({K}^{T}\left(t\right)\Phi (t)\) is bounded, \(\left|d\left(v\right)\right|\le {p}^{*}\), and \({p}^{*}\) is an unknown positive constant [19].

One way to approximate the unknown dynamic of the actuator nonlinearity is the use of a Radial Basis Function Neural Network (RBF). It provides universal approximating capability, by which any unknown continuous function \(f\left(Z\right):{R}^{n}\to R\) can be approximated as follows:where \(Z\in {\Omega }_{\mathrm{Z}}\subset {R}^{q}\) is the input vector with \(q\) being the neural networks input dimension, \(W={\left[{\fancyscript{w}}_{1},{\fancyscript{w}}_{2},\cdots , {\fancyscript{w}}_{l}\right]}^{T}\in {R}^{l}\), is the weight vector of neural networks with \(l>1\), the neural networks node number, and \(\zeta \left(Z\right)=\left[{\varsigma }_{1}\left(Z\right),{\varsigma }_{2}\left(Z\right),\cdots ,{\varsigma }_{l}\left(Z\right)\right]\) is the basis function vector with \({\varsigma }_{i}(Z)\) being chosen as Gaussian function following the form:

The \({\mu }_{i}=[{\mu }_{i1},\cdots ,{\mu }_{iq}]\) is the center of the respective field and \({\mu }_{i}\) is the width of the Gaussian function [25]. \(\delta (Z)\) is the approximation error and satisfies \(\left|\delta \left(Z\right)\right|\le \varepsilon\), \(\varepsilon >0\). W^*T is the ideal constant weight vector [25] and is defined as:

For simplicity, by using the minimal-learning-parameters algorithm an unknown constant \(\theta\) is introduced as:

We consider a stochastic nonlinear system in strict-feedback form with unknown dynamics where the actuator is subjected to a nonlinearity. The method proposed by the following sequels employs a radial basis neural network to estimate unknown dynamics of the system, and hence to design the adaptive control method.

Performance of the proposed ANNDSC method, along with the two baseline methods, is evaluated and compared in a tracking problem using a 3rd-order benchmark system. Details of the benchmark system are found in [25]. Another alternative benchmark of 2rd-order system can be found in [25], but we used the 3rd-order ones with more complexities to explore performance of the methods and hence provide a better comparison, under a rather complex condition. This benchmark for study considers a stochastic nonlinear system in strict-feedback form:

The simulation study considers a failure-free condition together with two other cases of the actuator nonlinearity, i.e., actuator dead-zone, and the actuator PI hysteresis for the proposed method, ANNDSC, along with the two baseline design methods, named APIC-DSC and backstepping design, respectively. The ANNDSC-based controller for the failure-free, and the two cases of nonlinearity, is designed using Eqs. (13)–(17) as follows:where \({Z}_{1}={S}_{1},\) \({Z}_{2}=[{S}_{1},{S}_{2},\widehat{\theta }]\), \({Z}_{3}=[{S}_{1},{S}_{2},{S}_{3},\widehat{\theta }]\), \(p\left(r\right)={e}^{-0.067{\left(r-1\right)}^{2}}, r\in \left[\mathrm{0,10}\right], v\left(t\right)=\frac{7\mathrm{sin}\left(3t\right)}{1+t}, t\in [\mathrm{0,2}\pi ]\), and \({u}_{-1}=0\) are the input vectors of neural networks. Equation (24) shows the design steps of a control system of order 3, which clearly involves three design steps. At each of the step, firstly the error surface \({S}_{i}\) is calculated by subtracting the desired value \({z}_{i}\) from the actual value of state \({x}_{i}\). Using the error surface, along with the neural network weights, \(\widehat{\theta }\), the virtual control input of the step, \({x}_{i+1}\), is estimated. The desired value is calculated for each step by passing the virtual control input signal through a first-order filter, except for the final step, where the actual control input is directly generated. Subsequently, the neural network weights are found using the adaptive law, by which the unknown dynamics and nonlinearities of the system will be approximated.

The design parameter set for the simulation is \({[{k}_{0},k}_{1},{k}_{2},{k}_{3}]\), [\({a}_{1},{a}_{2},{a}_{3}]\), \({\epsilon }_{2}=0.006,\) and \({\epsilon }_{3}=0.008\), which are obtained empirically considering the transient performance, the limitations on control effort growth, the closed-loop internal stability of the system, and improvement in the tracking error. The simulation runs under the initial condition of \({\left[{x}_{1}\left(0\right),{x}_{2}\left(0\right),{x}_{3}(0)\right]}^{T}={\left[\mathrm{0,0.6,0.4}\right]}^{T}\), \(\widehat{\theta }(0)=0.1\). The three RBF neural networks \({W}_{1}^{T}{\zeta }_{1}({Z}_{1})\), \({W}_{2}^{T}{\zeta }_{2}({Z}_{2})\), and \({W}_{3}^{T}{\zeta }_{3}({Z}_{3})\) are chosen to contain eleven nodes with the centers spaced evenly within the intervals \([-\mathrm{5,5}]\), \(\left[-\mathrm{5,5}\right]\times \left[-\mathrm{5,5}\right]\times [-\mathrm{5,5}]\), and \(\left[-\mathrm{5,5}\right]\times \left[-\mathrm{5,5}\right]\times \left[-\mathrm{5,5}\right]\times [-\mathrm{5,5}]\), respectively.

The APIC-DSC baseline controller for the system in Eq. (23) is designed using Eqs. (13), (14), (19)–(22). This controller is completely similar to Eq. (24), unless the controller input signal:where \({[{k}_{0},k}_{1},{k}_{2},{k}_{3}]\), [\({a}_{1},{a}_{2},{a}_{3}]\) are design parameters, obtained empirically likewise the ANNDSC case. \({p}_{max}=0.2, R=2, { \gamma }_{pr}=1, \sigma =2\), and other parameters are similar to the previous simulation step. The backstepping case is entirely implemented as detailed in [25].

Simulation results of these three cases are depicted in Figs. 3, 4, 5, 6 and 7. The system output is depicted against the reference sinusoidal input, for all the three methods, under the three actuator conditions.

Minimal deflection from the desired form of sinusoidal wave is seen for all the methods and conditions. In order to quantitate the deflection from the desired output, the Integrated Mean Square Error of the actual outputs is calculated with respect to the inputs. Figure 4 demonstrates the IMSE for the all methods and conditions.

Outperformance of ANNDSC is seen for all the case with the minimal IMSE. In the tracking problem, lower IMSE is regarded as an indication of the better performance. For the failure free cases, the relative depression in IMSE of the proposed method is observed to be 25% and 11% as compared to the backstepping and APIC-DSC, respectively. Nevertheless, effectiveness of the method is further highlighted when there is a nonlinearity condition in the actuator. For the PI hysteresis, the proposed method improves the tracking performance by 76% and 38% as reflected by the relative IMSE for the backstepping and APIC-DSC method, respectively. For the dead-zone nonlinearity, this relative outperformance is, however, 32% and 49%, showing a good improvement in the tracking performance. For the dead-zone condition, the APIC-DSC offers the worst IMSE, implying that improvement in the PI hysteresis is served at the expense of impairing the performance for other condition, when direct method is employed. It is also seen that all the methods offer their optimal performance at the absence of the actuator nonlinearity.

In order to investigate internal stability of the control methods, closed-loop signals and states of the three control systems are plotted with different actuator nonlinearities. Figure 5 shows the closed-loop states.

The close-loop signals of the APIC-DSC are considerably higher than the ANNDSC and the backstepping, showing further tendency to internal instability in practical situations, even though the values are bounded. This is confirmed by the control input signal, depicted in Fig. 6.

The ANNDSC method exhibits smaller control effort, compared to the two baselines. The control input signal of ANNDSC shows smoother and low oscillatory waveform, which provides a more reliable functionality in practice. The risk of the internal stability is by far highest for the APIC-DSC, even though the outputs are not far different for all the methods. It is important to note that high amplitude of the control input signal can practically put the system into the risk of actuator saturation. These conditions sometimes make finding a control strategy impractical, despite showing acceptable tracking.

Figure 7 demonstrates adaptive law of the three methods.

As seen in Fig. 7, the adaptive law, \(\widehat{\theta }\), damps quicker for the ANNDSC and APIC-DSC, revealing faster convergence for the neural network-based methods compared to the backstepping one. Figures 8 and 9 show the system outputs and the control inputs, for a case of the joint dead-zone and PI hysteresis nonlinearities, occurring at two different time instances.

All the three methods show good performance in tracking the output. However, the APIC-DSC dramatically increases the control inputs on the occurrence of the dead-zone. This makes the APIC-DSC an inappropriate candidate for the practical situations, where such the large value of the control input put the system into the risk of saturation.

The paper suggested an adaptive control design method for nonlinear stochastic systems with a general class of the actuator nonlinearity. In contrast to the existing techniques relying on the backstepping design method [11]–[21], the proposed method employed dynamic surface control design, along with neural networks through an algorithm of minimal learning parameters, to avoid the “explosion of complexity” and decline the computational efforts. This favorable feature which cannot be seen in the backstepping-based methods will become especially important for the systems with increased order. Such the implementation improves agility of the design method to be suitable for an online application. The paper proved boundedness of all the closed-loop signals and convergence of all the error signals to a small vicinity of the origin at the presence of two different nonlinearities, commonly seen at the actuators, dead-zone and hysteresis, in both analytic and simulation manners.

Although certain nonlinearities have been investigated in recent studies [25], the joint dead-zone and hysteresis were not included in the studies. In many practical applications, actuators can accidentally encounter with any of the dead-zone and hysteresis, due to the aging. It is sometimes critically important to consider such the conditions in the design method.

We introduced a baseline method for nonlinear stochastic system, named APIC-DSC, sophisticated for compensating the actuator hysteresis. In this baseline method, adaptive neural network is not invoked for the compensation. It is analytically proved that the closed-loop signals remain bounded in probability. This method although shows acceptable performance for the failure-free and also for the hysteresis conditions, but dramatically increases the control input at the presence of the dead-zone.

Considering Figs. 5 and 9, the control effort of the ANNDSC is much less than the two other baseline methods. It is possible to improve the tracking at the cost of increasing the control effort. It might, however, lead to actuator saturation or internal instability of the system. It was observed that the control effort is by far lower for ANNDSC than the two baseline methods.

In this study, the proposed method was empirically optimized by jointly considering the tracking performance and the control effort. Among the design parameters, the set of \(\left[ {k_{0} ,k_{1} ,k_{2} ,k_{3} } \right]\) and \(\left[ {a_{1} ,a_{2} ,a_{3} } \right]\) have more effect on the transient and the steady state characteristics of the system where the \({k}_{3},\) and \({a}_{3}\) directly affect the control input of the system. However, the proposed method can be well-integrated with the genetic algorithm for finding an optimal set of the design parameters. This is also true for other metaheuristic methods, or natural-based algorithms, such as ant colony algorithm. For our baseline study of backstepping method, we used the same set of the design parameters described in [25] as an initial set, and followed similar empirical procedure for improving the performance, as was done for ANNDSC and APIC-DSC.

Selecting an appropriate sampling rate plays an important role in efficient performance of any control system. A low sampling rate can lead to system instability, while on the other hand, an excessive sampling rate increases redundant complexities. A recent study proposed an interesting systematic method, named FIRCEP, that can be easily employed for finding an optimal sampling rate [55].

We used MATLAB R2017b for the simulations and analysis. Nowadays, there are various platforms, commercially available for efficient implementation in the practical situations and real plants, such as PLC systems with strong computational power. It is obvious that such the implementations demand a level of the practical considerations.

This paper proposed a novel adaptive design method for nonlinear stochastic control systems using neural network. The proposed method was investigated under joint conditions of the actuator nonlinearities, defined as the dead-zone and the Prandtl–Ishlinskii hysteresis. Stability analysis was analytically studied and confirmed by the simulation results in a tracking problem. Performance of the proposed method was compared to a baseline of widely used method, the backstepping method. It is observed that using the proposed neural network in conjunction with the dynamic surface method, considerably enhances performance of the control design method, and meanwhile decreases the computational complexities as well as the control effort.