A graph search and neural network approach to adaptive nonlinear model predictive control
Introduction
Model predictive control (MPC) is widely used in industry (Qin and Badgwell, 1997, Janakiraman et al., 2016, Płaczek, 2014), and although most MPC implementations use linear models, nonlinear models allow for better performance over a wider operating range (Berber and Kravaris, 1998, Grancharova and Johansen, 2012, Zhao et al., 2001, Henson, 1998). Furthermore, adaptive implementations of Nonlinear MPC (NMPC), which assume a stable plant as considered in this paper, provide the additional benefit of enabling the model to be updated as plant dynamics change. Several NMPC techniques have been developed by extending existing linear MPC techniques to handle plants with strong nonlinearities (Qin and Badgwell, 1997, Soloway and Haley, 1996, Bemporad and Morari, 1999). Drawbacks of currently used Newton-type methods include the computational expense of computing first and second derivatives, possible convergence to local minima that are globally suboptimal, and a lack of robustness due to overly fine tuning requirements. An adaptive approach to NMPC, Adaptive Sampling Based Model Predictive Control (Adaptive SBMPC) is first presented in Reese and Collins (2014). Here, we go beyond the previous research by providing more algorithm detail, a theoretical derivation of completeness, a comparison to the adaptive form of an existing NMPC method, and new simulation results.
Adaptive SBMPC is not an extension of a linear MPC technique; instead, it applies an optimization method that does not require gradient computations. The method is based on input sampling (Dunlap et al., 2011b, Rufli and Siegwart, 2010, Reese, 2015, Dokeroglu and Cosar, 2016), which here refers to the pseudorandom or low-correspondence discretization of a continuous set. In input sampling, the space of all valid input vectors is sampled, yielding a set of discrete input vectors that are used to represent the space. Sampling in this sense is not to be confused with the concept of (usually periodic) time sampling in a sampled-data system. The method may be used generally with any input–output model and does not inherently prefer linear or nonlinear models. This paper presents a comparison to an adaptive form of Neural GPC (Soloway and Haley, 1996) using both an transparent and simple example (results Cases 1, 2, and 3) and an application to a real world problem, the regulation of emissions for a simulated coal-burning power plant (results Cases 4 and 5).
The Generalized Predictive Control (GPC) method (Clarke, 1987) was the first to merge adaptive control techniques with MPC. GPC handles plants with changing dynamics by using an adaptive linear model and performs well despite unknown time delays, which is in general an advantage of MPC approaches. One particular disadvantage of GPC over other MPC methods is that there is no guarantee that hard input and output constraints will be satisfied. Although Clarke mentions the potential of modification to handle constraints, neither the original GPC nor any of the nonlinear GPC extensions mentioned below guarantee constraint satisfaction.
When implementing MPC, the model that is used for prediction is obtained in one of the several ways. While some take the model to be specified a priori (Diehl et al., 2006, Hovorka et al., 2004, Karampoorian and Mohseni, 2010), it is often practical to perform system identification and fit a model from observed input–output behavior (Clarke, 1987).
Linear MPC techniques often use a Least-Squares, Gradient Descent, or Newton method to fit a linear model to observed data (Qin and Badgwell, 1997). Nonlinear MPC techniques, which are far less commonly used, often fit a Neural Network, Neuro-Fuzzy, Nonlinear Polynomial, or other Nonlinear State Space model to predict system behavior (Qin and Badgwell, 2003). This paper focuses on techniques using the neural network pattern recognition paradigm, which is useful for representing general nonlinear system behavior. Neural networks achieve this by using computational building blocks called hidden units or neurons. It is possible to capture the behavior of a nonlinear plant by training and updating a neural network to predict the future response of the system based on past observations.
GPC has been extended to nonlinear systems using neural network models, yielding one of the first and most widely used adaptive NMPC algorithms, Neural GPC (Soloway and Haley, 1996). As illustrated in Fig. 1, this method consists of an identification phase, using a Back Propagation Network (BPN), and an MPC optimization phase using Newton's method. The cost function J is minimized, using computed gradient and Hessian values to seek a locally optimal sequence of inputs. Constraint and damping terms are added to the GPC cost function to penalize input constraint violations and avoid potential instability of the Newton's method solver.
Neural GPC enables control of a multiple-input multiple-output (MIMO) plant. However, in prior publications, each implementation fixed the neural network parameters after the learning phase ends. Hence, although the formulation of Neural GPC allows for adaptation, the research in published literature did not perform adaptive control. Neural GPC has been applied experimentally to a single-input single-output (SISO) nonlinear magnetic levitation system using a network with only three computational units in the hidden layer (Haley et al., 1999). For this paper, Neural GPC is implemented for comparison to the proposed algorithm. In this implementation, adaptation continues during the control phase, which allows the algorithm to adapt to a time-varying plant. Another neural-network-based NMPC approach called Explicit Black Box NMPC was recently introduced (Grancharova et al., 2011), but is a SISO result that cannot utilize the adaptive capability of a neural network model. Adaptive Predictive Control performs NMPC using neural networks for both identification and control (Larrea et al., 2010). The authors, however, list high computational cost due to inversion of a large matrix as a drawback of the method. Like the other neural network results, the plant controlled by this method is SISO. The tendency in the literature to present results for SISO plants serves as an indicator that performing NMPC that is practical and computationally efficient for MIMO nonlinear plants is a formidable challenge.
NMPC has also been applied to nonlinear systems which are identified without the use of neural networks. Fuzzy GPC is applied to the SISO, nonlinear, simple inverted pendulum (Cipriano and Saez, 1996), and Wuxi et al. (2012) present an adaptive Fuzzy GPC implementation that controls a nonlinear time-varying SISO plant. In each case, the methods based on nonlinear fuzzy models are described as computationally costly. This is due to the intensive computational effort required to solve the Diophantine equations required in the GPC optimization.
This paper develops an adaptive NMPC approach known as Adaptive SBMPC that has the ability to avoid local minima. This method is applied in simulation to both SISO and MIMO systems including a power plant combustion example. The optimization approach, Sampling Based Model Predictive Optimization (SBMPO) (Dunlap et al., 2008, Dunlap et al., 2010), is a graph search algorithm which discretizes the input space using sampling. It does not require gradient computations and easily handles the changes in model structure that occur as the internal model changes. Determining stability conditions for this method involves determining whether a goal state is present in the graph and showing that the optimization method is complete, i.e., it finds a goal state whenever such a state is reachable by the graph. Assuming a stable plant, system dynamics are learned and represented using a neural network, which, based on the patterns of observed data, can grow or shrink as the control system operates. The approach introduced here is very general and has potential applications in a wide variety of domains, including process control, automotive engine control, power system control, and robot motion planning.
SBMPO, the optimization portion of SBMPC, has been successfully applied to trajectory generation for robot systems with highly nonlinear plant dynamics (Chuy et al., 2012, Ordonez et al., 2013). However, in those applications, the dynamics were well known and modeled analytically. In addition, a receding horizon was not used. In this research SBMPO is used with a receding horizon, hence becoming SBMPC. More importantly, the nonlinear model is identified online and updated while the system is being controlled. Hence, this is the first adaptive form of SBMPC and it is demonstrated that this method is feasible for real time implementation.
To model nonlinear dynamics, we employ a special neural network structure, called the Radial Basis Function (RBF) network, which has been shown to be general enough to represent arbitrary nonlinear functions of multiple variables (Park and Sandberg, 1991, Park and Sandberg, 1993). In this research an RBF network is trained and adapted online using the Minimal Resource Allocation Network (MRAN) learning algorithm (Lu et al., 1997) to produce high-fidelity nonlinear models from limited sequences of training data.
This paper is organized as follows. Section 2 describes the identification and control methodologies used in Adaptive SBMPC. Section 3 explains the local minimum avoidance capability of Adaptive SBMPC, illustrated with a simulation. Section 4 presents simulation results comparing Adaptive SBMPC with Neural GPC for a transparent SISO system, while Section 5 presents MIMO results for the simulation of a more practical combustion control problem. Finally, Section 6 presents conclusions.
Section snippets
Methodology
The Adaptive SBMPC approach to nonlinear MPC (Reese, 2015) consists of identification of an approximate system model during the learning phase followed by simultaneous identification and control during the control phase. As shown in Fig. 2, a neural network is used to model the plant dynamics and SBMPC is used to generate actuation signals to control the plant. Remarks on the neural network structure and computational complexity of the MRAN identification algorithm as well as the details of the
MPC and local minimum avoidance
Model predictive control optimization techniques based on sampling have an inherent advantage over techniques that rely on gradients. Gradient-based methods seek out solutions that are locally minimal, but it is possible that the found local minimum is significantly less optimal than the true global minimum. Sampling-based methods, however, may be employed to explore beyond nearby local minima and find more optimal solutions if they exist. This section contains a transparent illustration of
SISO simulation and results
The next series of results will compare Adaptive SBMPC with an established adaptive NMPC method, Neural GPC, applied to nonlinear plants of increasing complexity. Both Adaptive SBMPC and Neural GPC were used to control the benchmark SISO nonlinear plant (Lu et al., 1998) described by the difference equationwhere is a parameter that may be modified to alter the behavior of the plant. The MPC cost function,
Power plant combustion application
We continue the comparison between Adaptive SBMPC and Neural GPC methods with a simulated power plant combustion control example.
Energy policy and the rising cost of fossil fuels has highlighted the need for efficient and reliable methods to control the combustion processes of power plants. Legislation that taxes or sets limits on carbon emissions is becoming more common (Linn et al., 2014, Rao and Rubin, 2002, Q&a, 2014), which has increased the demand for control systems that reliably meet
Conclusions
Adaptive SBMPC, a new approach to adaptive nonlinear model predictive control, is presented and applied in simulation to a modified nonlinear control problems from the literature. The SBMPO algorithm for optimization is shown via mathematical proof to be a complete search algorithm subject to sufficient sampling density and length of the prediction horizon. Unlike approaches that optimize by linearization, convergence to a nearly global minimum is achievable even when numerous local minima
Acknowledgments
This research was sponsored in part by the Army Research Office (ARO) under contract W911NF-13-1-0122 and the Florida Center for Advanced Aero Propulsion (FCAAP).
SBMPO algorithm pseudocode
Algorithm 1 Calculate input sequence U via SBMPO graph search. Insert root node into the queue with Evaluate heuristic cost HEUR_COST_EVAL Pop the top node from the queue whiledo whiledo Evaluate edge cost EDGE_COST_EVAL Re-insert node into queue, sorted by Pop the top node from the queue end while Expand by Generating B child nodes fordo Generate input space sample HALTON_SEQUENCE_LOOKUP Integrate the model
SBMPO soundness and completeness proof
This proof of soundness and completeness for the algorithm listed in Appendix A is loosely based on the procedure in Karaman, 2010, in which soundness and completeness is derived for a different graph search algorithm. Its culmination, Theorem B.6, guarantees that, subject to sufficient depth and breadth of sampling, SBMPO will find a globally optimal solution. Assumption B.1 The heuristic function is optimistic. Assumption B.2 All reachable states have finite edge costs. Assumption B.3 All edge costs are non-negative. Lemma B.4 The Algorithm will
List of tuning parameter choices
Table C1, Table C2, Table C3, Table C4 give the tuning parameters used in the simulation for each identification and control method. Unless otherwise noted the same tuning parameters were used for all three cases.
References (58)
- et al.
Application of a gain scheduling generalized predictive controller to a solar power plant
Control Eng. Pract.
(1994) - et al.
On the optimal Halton sequence
Math. Comput. Simul.
(2005) - et al.(2006)
- et al.
A novel multistart hyper-heuristic algorithm on the grid for the quadratic assignment problem
Eng. Appl. Artif. Intell.
(2016) - et al.
Explicit stochastic predictive control of combustion plants based on gaussian process models
Automatica
(2008) - et al.
Explicit output-feedback nonlinear predictive control based on black-box models
Eng. Appl. Artif. Intell.
(2011) Nonlinear model predictive controlcurrent status and future directions
Comput. Chem. Eng.
(1998)- et al.
An elm based predictive control method for hcci engines
Eng. Appl. Artif. Intell.
(2016) - et al.
Lifelong planning a
Artif. Intell.
(2004) - et al.
The parallel genetic algorithm as function optimizer
Parallel Comput.
(1991)
Constrained model predictive controlstability and optimality
Automatica
Model predictive controlpast, present and future
Comput. Chem. Eng.
A self-organizing system for urban traffic control based on predictive interval microscopic model
Eng. Appl. Artif. Intell.
A survey of industrial model predictive control technology
Control Eng. Pract.
A nonlinear industrial model predictive controller using integrated pls and neural net state-space model
Control Eng. Pract.
Gbest-guided artificial bee colony algorithm for numerical function optimization
Appl. Math. Comput.
Model predictive control and the optimization of power plant load while considering lifetime consumption
IEEE Trans. Power Syst.
Cited by (21)
Nonlinear model-predictive control with disturbance rejection property using adaptive neural networks
2017, Journal of the Franklin InstituteCitation Excerpt :Neural networks (NNs) are known as universal approximators using different learning algorithms. Hence, they are widely used to model nonlinear systems [5,7,8]. For MPCs, the NN provides the system model only based on the input–output data and hence, less knowledge of the system is required.
Automated process synthesis for optimal flowsheet design of a hybrid membrane cryogenic carbon capture process
2017, Journal of Cleaner ProductionCitation Excerpt :Computational intelligence methods and computer-based design systems have been widely used in the designing and modelling of complex industrial process applications. These techniques attempt to represent the behaviour of a process by computational expression and predict the adapting response rules (Reese and Collins, 2016). In optimization-based process flowsheet synthesis, optimization methods such as mathematical programming (e.g. mixed integer linear or non-linear programming) and evolutionary algorithms (e.g. differential evolution approach) (Angira and Babu, 2006) are used to synthesise a high performance flowsheet by screening large numbers of possible flowsheets as available within a predefined superstructure.
Adaptive Model Predictive Control for Linear Systems with Parametric Uncertainties and Input Delay
2023, Chinese Control Conference, CCCAn Advanced Control Strategy for the Evaporation Section of An Integrated First- and Second-Generation Ethanol Sugarcane Biorefinery
2023, Chemical and Biochemical Engineering Quarterly