A graph search and neural network approach to adaptive nonlinear model predictive control

https://doi.org/10.1016/j.engappai.2016.07.001Get rights and content

Abstract

Systems with a priori unknown and time-varying dynamic behavior pose a significant challenge in the field of Nonlinear Model Predictive Control (NMPC). When both the identification of the nonlinear system and the optimization of control inputs are done robustly and efficiently, NMPC may be applied to control such systems. This paper considers stable systems and presents a novel method for adaptive NMPC, called Adaptive Sampling Based Model Predictive Control (Adaptive SBMPC), that combines a radial basis function neural network identification algorithm with a nonlinear optimization method based on graph search. Unlike other NMPC methods, it does not rely on linearizing the system or gradient based optimization. Instead, it discretizes the input space to the model via pseudo-random sampling and feeds the sampled inputs through the nonlinear model, producing a searchable graph. For this discretization, an optimal path is found using Lifelong Planning A, an efficient graph search method. Adaptive SBMPC is used in simulation to identify and control a simple plant with clearly visualized nonlinear behavior. In these simulations, both fixed and time-varying dynamic systems are considered. Results are compared with an adaptive version of Neural GPC, an existing NMPC algorithm based on Newton–Raphson optimization and a back propagation neural network model. When the cost function exhibits many local minima, Adaptive SBMPC is successful in finding a low-cost solution that appears close globally optimal while Neural GPC converges to a solution that is only locally optimal. This paper presents the method, soundness and completeness theory, and two simulated NMPC examples. The first is a transparent single-input single-output example, and the second considers a more complex power plant combustion process with two inputs and three outputs.

Introduction

Model predictive control (MPC) is widely used in industry (Qin and Badgwell, 1997, Janakiraman et al., 2016, Płaczek, 2014), and although most MPC implementations use linear models, nonlinear models allow for better performance over a wider operating range (Berber and Kravaris, 1998, Grancharova and Johansen, 2012, Zhao et al., 2001, Henson, 1998). Furthermore, adaptive implementations of Nonlinear MPC (NMPC), which assume a stable plant as considered in this paper, provide the additional benefit of enabling the model to be updated as plant dynamics change. Several NMPC techniques have been developed by extending existing linear MPC techniques to handle plants with strong nonlinearities (Qin and Badgwell, 1997, Soloway and Haley, 1996, Bemporad and Morari, 1999). Drawbacks of currently used Newton-type methods include the computational expense of computing first and second derivatives, possible convergence to local minima that are globally suboptimal, and a lack of robustness due to overly fine tuning requirements. An adaptive approach to NMPC, Adaptive Sampling Based Model Predictive Control (Adaptive SBMPC) is first presented in Reese and Collins (2014). Here, we go beyond the previous research by providing more algorithm detail, a theoretical derivation of completeness, a comparison to the adaptive form of an existing NMPC method, and new simulation results.

Adaptive SBMPC is not an extension of a linear MPC technique; instead, it applies an optimization method that does not require gradient computations. The method is based on input sampling (Dunlap et al., 2011b, Rufli and Siegwart, 2010, Reese, 2015, Dokeroglu and Cosar, 2016), which here refers to the pseudorandom or low-correspondence discretization of a continuous set. In input sampling, the space of all valid input vectors is sampled, yielding a set of discrete input vectors that are used to represent the space. Sampling in this sense is not to be confused with the concept of (usually periodic) time sampling in a sampled-data system. The method may be used generally with any input–output model and does not inherently prefer linear or nonlinear models. This paper presents a comparison to an adaptive form of Neural GPC (Soloway and Haley, 1996) using both an transparent and simple example (results Cases 1, 2, and 3) and an application to a real world problem, the regulation of emissions for a simulated coal-burning power plant (results Cases 4 and 5).

The Generalized Predictive Control (GPC) method (Clarke, 1987) was the first to merge adaptive control techniques with MPC. GPC handles plants with changing dynamics by using an adaptive linear model and performs well despite unknown time delays, which is in general an advantage of MPC approaches. One particular disadvantage of GPC over other MPC methods is that there is no guarantee that hard input and output constraints will be satisfied. Although Clarke mentions the potential of modification to handle constraints, neither the original GPC nor any of the nonlinear GPC extensions mentioned below guarantee constraint satisfaction.

When implementing MPC, the model that is used for prediction is obtained in one of the several ways. While some take the model to be specified a priori (Diehl et al., 2006, Hovorka et al., 2004, Karampoorian and Mohseni, 2010), it is often practical to perform system identification and fit a model from observed input–output behavior (Clarke, 1987).

Linear MPC techniques often use a Least-Squares, Gradient Descent, or Newton method to fit a linear model to observed data (Qin and Badgwell, 1997). Nonlinear MPC techniques, which are far less commonly used, often fit a Neural Network, Neuro-Fuzzy, Nonlinear Polynomial, or other Nonlinear State Space model to predict system behavior (Qin and Badgwell, 2003). This paper focuses on techniques using the neural network pattern recognition paradigm, which is useful for representing general nonlinear system behavior. Neural networks achieve this by using computational building blocks called hidden units or neurons. It is possible to capture the behavior of a nonlinear plant by training and updating a neural network to predict the future response of the system based on past observations.

GPC has been extended to nonlinear systems using neural network models, yielding one of the first and most widely used adaptive NMPC algorithms, Neural GPC (Soloway and Haley, 1996). As illustrated in Fig. 1, this method consists of an identification phase, using a Back Propagation Network (BPN), and an MPC optimization phase using Newton's method. The cost function J is minimized, using computed gradient and Hessian values to seek a locally optimal sequence of inputs. Constraint and damping terms are added to the GPC cost function to penalize input constraint violations and avoid potential instability of the Newton's method solver.

Neural GPC enables control of a multiple-input multiple-output (MIMO) plant. However, in prior publications, each implementation fixed the neural network parameters after the learning phase ends. Hence, although the formulation of Neural GPC allows for adaptation, the research in published literature did not perform adaptive control. Neural GPC has been applied experimentally to a single-input single-output (SISO) nonlinear magnetic levitation system using a network with only three computational units in the hidden layer (Haley et al., 1999). For this paper, Neural GPC is implemented for comparison to the proposed algorithm. In this implementation, adaptation continues during the control phase, which allows the algorithm to adapt to a time-varying plant. Another neural-network-based NMPC approach called Explicit Black Box NMPC was recently introduced (Grancharova et al., 2011), but is a SISO result that cannot utilize the adaptive capability of a neural network model. Adaptive Predictive Control performs NMPC using neural networks for both identification and control (Larrea et al., 2010). The authors, however, list high computational cost due to inversion of a large matrix as a drawback of the method. Like the other neural network results, the plant controlled by this method is SISO. The tendency in the literature to present results for SISO plants serves as an indicator that performing NMPC that is practical and computationally efficient for MIMO nonlinear plants is a formidable challenge.

NMPC has also been applied to nonlinear systems which are identified without the use of neural networks. Fuzzy GPC is applied to the SISO, nonlinear, simple inverted pendulum (Cipriano and Saez, 1996), and Wuxi et al. (2012) present an adaptive Fuzzy GPC implementation that controls a nonlinear time-varying SISO plant. In each case, the methods based on nonlinear fuzzy models are described as computationally costly. This is due to the intensive computational effort required to solve the Diophantine equations required in the GPC optimization.

This paper develops an adaptive NMPC approach known as Adaptive SBMPC that has the ability to avoid local minima. This method is applied in simulation to both SISO and MIMO systems including a power plant combustion example. The optimization approach, Sampling Based Model Predictive Optimization (SBMPO) (Dunlap et al., 2008, Dunlap et al., 2010), is a graph search algorithm which discretizes the input space using sampling. It does not require gradient computations and easily handles the changes in model structure that occur as the internal model changes. Determining stability conditions for this method involves determining whether a goal state is present in the graph and showing that the optimization method is complete, i.e., it finds a goal state whenever such a state is reachable by the graph. Assuming a stable plant, system dynamics are learned and represented using a neural network, which, based on the patterns of observed data, can grow or shrink as the control system operates. The approach introduced here is very general and has potential applications in a wide variety of domains, including process control, automotive engine control, power system control, and robot motion planning.

SBMPO, the optimization portion of SBMPC, has been successfully applied to trajectory generation for robot systems with highly nonlinear plant dynamics (Chuy et al., 2012, Ordonez et al., 2013). However, in those applications, the dynamics were well known and modeled analytically. In addition, a receding horizon was not used. In this research SBMPO is used with a receding horizon, hence becoming SBMPC. More importantly, the nonlinear model is identified online and updated while the system is being controlled. Hence, this is the first adaptive form of SBMPC and it is demonstrated that this method is feasible for real time implementation.

To model nonlinear dynamics, we employ a special neural network structure, called the Radial Basis Function (RBF) network, which has been shown to be general enough to represent arbitrary nonlinear functions of multiple variables (Park and Sandberg, 1991, Park and Sandberg, 1993). In this research an RBF network is trained and adapted online using the Minimal Resource Allocation Network (MRAN) learning algorithm (Lu et al., 1997) to produce high-fidelity nonlinear models from limited sequences of training data.

This paper is organized as follows. Section 2 describes the identification and control methodologies used in Adaptive SBMPC. Section 3 explains the local minimum avoidance capability of Adaptive SBMPC, illustrated with a simulation. Section 4 presents simulation results comparing Adaptive SBMPC with Neural GPC for a transparent SISO system, while Section 5 presents MIMO results for the simulation of a more practical combustion control problem. Finally, Section 6 presents conclusions.

Section snippets

Methodology

The Adaptive SBMPC approach to nonlinear MPC (Reese, 2015) consists of identification of an approximate system model during the learning phase followed by simultaneous identification and control during the control phase. As shown in Fig. 2, a neural network is used to model the plant dynamics and SBMPC is used to generate actuation signals to control the plant. Remarks on the neural network structure and computational complexity of the MRAN identification algorithm as well as the details of the

MPC and local minimum avoidance

Model predictive control optimization techniques based on sampling have an inherent advantage over techniques that rely on gradients. Gradient-based methods seek out solutions that are locally minimal, but it is possible that the found local minimum is significantly less optimal than the true global minimum. Sampling-based methods, however, may be employed to explore beyond nearby local minima and find more optimal solutions if they exist. This section contains a transparent illustration of

SISO simulation and results

The next series of results will compare Adaptive SBMPC with an established adaptive NMPC method, Neural GPC, applied to nonlinear plants of increasing complexity. Both Adaptive SBMPC and Neural GPC were used to control the benchmark SISO nonlinear plant (Lu et al., 1998) described by the difference equationy(k)=29β(k)40sin(16u(k1)+8y(k1)β(k)(3+4u(k1)2+4y(k1)2)+15u(k1)+15y(k1),where β(k) is a parameter that may be modified to alter the behavior of the plant. The MPC cost function,J=i=0N1(

Power plant combustion application

We continue the comparison between Adaptive SBMPC and Neural GPC methods with a simulated power plant combustion control example.

Energy policy and the rising cost of fossil fuels has highlighted the need for efficient and reliable methods to control the combustion processes of power plants. Legislation that taxes or sets limits on carbon emissions is becoming more common (Linn et al., 2014, Rao and Rubin, 2002, Q&a, 2014), which has increased the demand for control systems that reliably meet

Conclusions

Adaptive SBMPC, a new approach to adaptive nonlinear model predictive control, is presented and applied in simulation to a modified nonlinear control problems from the literature. The SBMPO algorithm for optimization is shown via mathematical proof to be a complete search algorithm subject to sufficient sampling density and length of the prediction horizon. Unlike approaches that optimize by linearization, convergence to a nearly global minimum is achievable even when numerous local minima

Acknowledgments

This research was sponsored in part by the Army Research Office (ARO) under contract W911NF-13-1-0122 and the Florida Center for Advanced Aero Propulsion (FCAAP).

SBMPO algorithm pseudocode

Algorithm 1

Calculate input sequence U via SBMPO graph search.

Insert root node into the queue with
i0
CE,i0
ND,i0
parentiNULL
xix0
Evaluate heuristic cost
CH,i HEUR_COST_EVAL
Pop the top node from the queue
while(ND,top<N)(i<MAX_NODES)do
 whileCE,top=NULLdo
 Evaluate edge cost
 CE,topCE,parent+EDGE_COST_EVAL
 Re-insert node into queue, sorted by CE+CH
 Pop the top node from the queue
 end while
 Expand by Generating B child nodes
 forj[i+1,i+B)do
 Generate input space sample
 ujHALTON_SEQUENCE_LOOKUP
 Integrate the model
 xj

SBMPO soundness and completeness proof

This proof of soundness and completeness for the algorithm listed in Appendix A is loosely based on the procedure in Karaman, 2010, in which soundness and completeness is derived for a different graph search algorithm. Its culmination, Theorem B.6, guarantees that, subject to sufficient depth and breadth of sampling, SBMPO will find a globally optimal solution.

Assumption B.1

The heuristic function is optimistic.

Assumption B.2

All reachable states have finite edge costs.

Assumption B.3

All edge costs are non-negative.

Lemma B.4

The Algorithm will

List of tuning parameter choices

Table C1, Table C2, Table C3, Table C4 give the tuning parameters used in the simulation for each identification and control method. Unless otherwise noted the same tuning parameters were used for all three cases.

References (58)

  • D.Q. Mayne et al.

    Constrained model predictive controlstability and optimality

    Automatica

    (2000)
  • M. Morari et al.

    Model predictive controlpast, present and future

    Comput. Chem. Eng.

    (1999)
  • B. Płaczek

    A self-organizing system for urban traffic control based on predictive interval microscopic model

    Eng. Appl. Artif. Intell.

    (2014)
  • S.J. Qin et al.

    A survey of industrial model predictive control technology

    Control Eng. Pract.

    (2003)
  • H. Zhao et al.

    A nonlinear industrial model predictive controller using integrated pls and neural net state-space model

    Control Eng. Pract.

    (2001)
  • G. Zhu et al.

    Gbest-guided artificial bee colony algorithm for numerical function optimization

    Appl. Math. Comput.

    (2010)
  • Bemporad, A., Morari, M., 1999. Robust model predictive control: a survey. In: Robustness in Identification and...
  • R. Berber et al.
    (1998)
  • Chuy, O., Collins, E., Dunlap, D., Sharma, A., 2012. Sampling-based direct trajectory generation using the minimum time...
  • Cipriano, A. Saez, D., 1996. Fuzzy generalized predictive control and its application to an inverted pendulum. In:...
  • Clarke, D., 1987. Generalized predictive control: a robust self-tuning algorithm. In: American Control Conference,...
  • Cretnik, J. 1992. Modern Automatic Combustion Control (Ph.D. dissertation). Ljubljana School of Electrical and Computer...
  • Delling, D., Sanders, P., Schultes, D., Wagner, D., 2009. Engineering route planning algorithms. In: Algorithmics of...
  • Dunlap, D.D., Collins, E.G., Caldwell C.V., 2008. Sampling based model predictive control with application to...
  • Dunlap, D.D, Caldwell, C.V., Collins E.G., 2010. Nonlinear model predictive control using sampling and goal-directed...
  • Dunlap, D., Yu, W., Collins, E.G., Caldwell, C.V., 2011. Motion planning for steep hill climbing. In: 2011 IEEE...
  • Dunlap, D.D., Caldwell, C.V., Emmanuel, J., Collins, G., Chuy, O., 2011b. Motion planning for mobile robots via...
  • C. Ericson
    (2005)
  • E. Gallestey et al.

    Model predictive control and the optimization of power plant load while considering lifetime consumption

    IEEE Trans. Power Syst.

    (2002)
  • Cited by (21)

    • Nonlinear model-predictive control with disturbance rejection property using adaptive neural networks

      2017, Journal of the Franklin Institute
      Citation Excerpt :

      Neural networks (NNs) are known as universal approximators using different learning algorithms. Hence, they are widely used to model nonlinear systems [5,7,8]. For MPCs, the NN provides the system model only based on the input–output data and hence, less knowledge of the system is required.

    • Automated process synthesis for optimal flowsheet design of a hybrid membrane cryogenic carbon capture process

      2017, Journal of Cleaner Production
      Citation Excerpt :

      Computational intelligence methods and computer-based design systems have been widely used in the designing and modelling of complex industrial process applications. These techniques attempt to represent the behaviour of a process by computational expression and predict the adapting response rules (Reese and Collins, 2016). In optimization-based process flowsheet synthesis, optimization methods such as mathematical programming (e.g. mixed integer linear or non-linear programming) and evolutionary algorithms (e.g. differential evolution approach) (Angira and Babu, 2006) are used to synthesise a high performance flowsheet by screening large numbers of possible flowsheets as available within a predefined superstructure.

    View all citing articles on Scopus
    View full text