Rotor resistance and excitation inductance estimation of an induction motor using deep-Q-learning algorithm

Abstract

To estimate the parameters of an induction motor in a data-based manner, this paper proposes a new offline method to estimate rotor resistance and excitation inductance based on the deep-Q-learning approach. In this method, parameter estimation can be facilitated without disturbing the model error or operating state. To achieve this goal, three important elements, namely observation, action, and reward, are appropriately designed. To improve the robustness and accelerate the convergence, a new concept, denoted as Q-sensitivity, is proposed and investigated in detail. The experimental results show that a high-Q-sensitivity design can allow the proposed method to obtain a fast and torque-maximized estimation. Results from the comparative studies confirm the accuracy and robustness of the proposed method.

Introduction

Indirect field-oriented control (IFOC) is one of the most popular strategies used in high-performance induction motor (IM) applications. Since IFOC can exploit an inherent slip relation, it is essentially a feed-forward control method, where accurate values of at least some motor parameters are required to be estimated to obtain robust control performance. For example, the rotor time constant may not be correct or precisely known or the slip cannot be correctly attained due to motor heating, field-weakening, and other variation-induced changes. This would result in detuning of the controller and a loss of correct field orientation (Novotny and Lipo, 1996).

To avoid such situations, it is necessary to introduce estimation methods to equip the vector controller with accurate IM parameter values. So far, several methods have been proposed to estimate the correct motor parameters, including model reference adaptive system (MRAS), extended Kalman filter (EKF), sliding mode observer, and recursive least squares Zerdali and Barut, 2017, Yin et al., 2016a, Yin et al., 2016b, Yin et al., 2016c, Yang et al., 2017, Djadi et al., 2017. This class of methods can be summarized as model-based methods since their performance strongly depends on the accuracy of the approximated IM model. Model-based methods generally suffer from two shortcomings. First, they are very sensitive to noise, where the algorithm becomes very unstable under any noise overlaying the IM model. For example, Shi et al. (2012) proposed an EKF algorithm to estimate the rotor speed and position of permanent magnet synchronous motor. However, this algorithm is difficult to be practically applied owing to its sensitivity to noise. Second, in some situations, these methods may fail due to parameter perturbation. For example, MRAS cannot be used at low frequencies due to a drop in stator resistance voltage (Maiti et al., 2008).

Recent advances in artificial intelligence areas have provided new possibility of estimating motor parameters in a data-based manner. These strategies can utilize a range of data training architectures including artificial neural networks (ANNs), support vector machine (SVM), genetic algorithm, and particle swarm optimization (PSO) Papa and Koroušić-Seljak, 2005, Liu et al., 2008, Woodley et al., 2005, Liu et al., 2017. Data-based methods are generally more robust and accurate since they are independent of the motor models. However, they face a significant challenge. In general, most successful data-based applications require considerable hand-labelled training data, where reliable hand-labelled training data are rather difficult to obtain in practical engineering applications. Karanayil et al. (2007) introduced an ANN to estimate the resistance of a stator and rotor in sensorless applications. However, the dataset cannot be used for training the ANN, and the voltage model of the motor is used instead. In other words, the strategies should be model-based rather than data-based. Bilski (2014) presented an SVM-based strategy; however, the method of obtaining the dataset remained unclear in this method. Sakthivel et al. (2010) investigated an MOPSO strategy to estimate the IM parameters, where the manufacturer data were used as reference instead of the training dataset.

A feasible choice for obtaining the hand-labelled data is to use reinforcement learning, in which the agent can learn the correct knowledge from the environment and generate the label data automatically. Recently, the application of reinforcement learning in industry has become a hot issue due to theoretical breakthroughs. Khan et al. (2011) presented a humanoid robotic arm control strategy based on $Q$-learning and approximate dynamic programming. Yin et al. (2016a) proposed an approximate dynamic programming method to reduce the time delay of affected passengers in a congested metro line. Deisenroth and Rasmussen (2009) presented a Gauss process reinforcement learning algorithm and applied it to inverted pendulum control in hardware. Yin et al. (2016b) develop an integrated train operation algorithm based on $Q$-learning to realize real-time train operations with online adjusting the timetable. Zhang et al. (2017) presented an energy-efficient scheduling for real-time systems based on the deep $Q$-learning (DQL) model. Prashanth and Bhatnagar (2011) proposed a reinforcement learning algorithm along with the function approximation algorithm, which can be applied to traffic signal control.

This paper proposes a data-based method with labelled data obtained in a simple manner. By focusing on this objective, an estimation approach is proposed based on the DQL algorithm. In particular, the training dataset can be generated automatically during the estimation procedure in DQL, where the dataset is trained simultaneously. With a careful design of the algorithm’s architecture, DQL can obtain an unbiased estimation of parameters after a few iterations of the training procedure.

The main contributions of this paper can be summarized as follows.

(1) Although breakthrough has been achieved by DQL in some domains, such as computer vision games, recently (Mnih et al., 2013), no study has applied DQL to the motor control technology. In this study, DQL is used successfully for motor parameter estimation.

(2) To ensure DQL feasibility, the ways of designing the DQL architecture are also discussed, which include the design of observation, action, and reward. Moreover, an indicator, namely $Q$-sensitivity, is introduced to guarantee the robustness and efficiency of the algorithm, which also demonstrates that the algorithm with high $Q$-sensitivity is more stable and can converge at a much faster rate.

(3) Some comparative experiments are designed to illustrate the DQL performance. Through these experiments, DQL is verified to have more accuracy and robustness than some other traditional methods. These experiments demonstrate that DQL can generate a maximum-torque output in any state.

The remainder of this paper is organized as follows. In Section 2, an induction model of the motor with some parameters is established. In Section 3, a new DQL architecture is designed, where the observation, reward, and action are appropriately designed. The experimental and comparative studies are extensively illustrated in Section 4. Finally, the conclusion is drawn in Section 5.