Neural identification of dynamic systems on FPGA with improved PSO learning
Graphical abstract
.
Highlights
► This work introduces hardware implementation of artificial neural networks (ANNs) with learning ability on Field Programmable Gate Array (FPGA) for dynamic system identification. ► The learning phase is accomplished by using the improved particle swarm optimization (PSO). ► The improved PSO is obtained by modifying the velocity update function. Adding an extra term to the velocity update function reduced the possibility of stucking in a local minimum.
Introduction
Artificial neural networks (ANNs) have been widely and successfully used in many fields to solve complex problems. ANNs are capable of successfully modeling non-linear complex correlation between input and output of a system. The most commonly used ANN is the multi layer perceptron (MLP) [1], [2]. ANN is trained using a data set which is compatible with input–output relation of system to be modeled. Training is usually performed using back propagation (BP) algorithm. In recent years evolutionary algorithms are also used for ANN training as an alternative to BP algorithm.
For example population-based particle swarm optimization (PSO) [3], capable of stochastic search, has been successfully utilized in neural network training in addition to its usage in function optimization and fuzzy system control [4], [5], [6], [7].
In recent years FPGAs, capable of parallel processing, are emerging as a predominant embedded system platform for many ANN applications [8], [9], [10], [11], [12], [13]. In the literature numerous of hardware based ANN implementations on FPGAs with offline training [1], [14], [15], [16], [17] and online training [18], [19], [20] are reported. In these implementations different number formats such as fixed-point numbers with different bit width [16], [18], [20], floating-point numbers [1], [14], [17], [18], [19] and integer numbers [15] are utilized. Nonlinear logarithmic sigmoid [14], [15], [16], [17], [18], [19] and nonlinear hyperbolic tangent [1], [16], [20] are employed as activation function. To implement nonlinear activation functions on hardware several approaches have been proposed: look-up table [15], [20], piecewise-linear [1], [16], [18], parabolic [14] and functional [17], [19].
In this work we present FPGA implementation of a system identification module based on ANN with PSO based online training. ANN and its PSO based online training module implemented in single precision floating-point number format and neural cell activation functions are implemented using functional approaches. In functional approach a divider is used in addition to adder and multiplier when compared to piecewise-linear and parabolic approach. Divider usage could be considered as the disadvantage of functional approach, but this approach has several advantages: (i) contrary to look-up table approach, no memory needed (ii) contrary to piecewise-linear approach, no control statements needed. The most important contribution of this work is the modified velocity update function of the PSO. In this paper authors proposed to add an extra term to the function which reduces the possibility of stucking in a local minimum during training. The experimental results indicate that the PSO algorithm utilizing the modified velocity update function converges faster and produces more accurate results.
The rest of the paper is organized as follows. Section 2 provides brief introduction to standard PSO and proposed improved PSO. Section 3 describes hardware implementation of ANN and its PSO based training module. Experimental results and performance of proposed approach in hardware are given in Section 4. The comparison results of the proposed improved PSO with other methods are given in Section 5 and the conclusions are given in Section 6.
Section snippets
Standard and proposed modified PSO
PSO, inspired from social interactions among animals, is a high performance optimizer. PSO performs search using a population of particles. Each particle is directed to the most significant parts of the search space as a result of interactions between particles. The algorithm starts with randomly assigned particles. In every iteration the velocity and the location of particles are updated. Each particle's position is evaluated as a possible solution candidate. In a swarm with N particles, the
PSO based ANN training ON FPGA
In this section, the details of PSO based ANN training on FPGA is described. The block diagram of the proposed architecture with the intention of dynamic system identification is shown in Fig. 1. PSO based ANN training on FPGA has four main stages as given in Fig. 2. Short description of these four stages is as follows.
Experimental results
The details of PSO based ANN training implementation on FPGA (Altera Cyclone II EP2C70F672C6), is given in previous section. The hardware implementation is experimentally tested with four dynamic system identification problems. Learning phase for the dynamic system identification problem is operated as illustrated in Fig. 1. ANN identification model, whose parameters are determined at the end of the learning phase, is tested for the data set unseen in the learning phase. This testing phase is
Comparison of improved PSO with other methods
To give more general idea about the performance of the proposed improved PSO, it has been compared with BP, artificial bee colony (ABC) [29] and conventional PSO [3] algorithms on software environment. Comparisons are carried out for ANN training for the identification examples given in the previous section. Codes for ABC algorithm has been taken from [30] and adopted to the identification examples. ANN architectures are the same as ones given in the previous section for each example. Learning
Conclusion
In this paper, the hardware implementation of ANN with PSO online training for system identification is proposed. Contribution of this work is twofold. First, it introduces the modified velocity update function for PSO (Eq. (6)). Authors proposed to add an extra term (α3λ(n)) to the function which reduces the possibility of stucking in a local minimum. Second, it proposes an effective system identification architecture based on ANN with PSO learning on FPGA. To prove the effectiveness of the
References (30)
- et al.
Multi-step ahead nonlinear identification of Lorenz's chaotic system using radial basis neural network with learning by clustering and particle swarm optimization
Chaos, Solitons & Fractals
(2008) - et al.
PSO-based neural network optimization and its utilization in a boring machine
Journal of Materials Processing Technology
(2006) - et al.
Application of a novel fuzzy classifier to fault detection and isolation of the DAMADICS benchmark problem
Control Engineering Practice
(2006) Optimizations of PID gains by particle swarm optimizations in fuzzy based automatic generation control
Electric Power Systems Research
(2004)- et al.
Dynamic MAC-based architecture of artificial neural networks suitable for hardware implementation on FPGAs
Neurocomputing
(2009) A hardware implementation of artificial neural networks using field programmable gate arrays
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
(2007)- et al.
Training wawelet networks for nonlinear dynamic input–output modeling
Neurocomputing
(1998) - et al.
A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function
Neurocomputing
(2006) - et al.
ANN- and PSO-based synthesis of on-chip spiral inductors for RF ICs
IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems
(2008) - et al.
The particle swarm-explosion, stability and convergence in a multidimensional complex space
IEEE Transactions on Evolutionary Computation
(2002)
A retinomorphic architecture based on discrete-time cellular neural networks using reconfigurable computing
Neurocomputing
FPGA implementation of a neural network for a real time hand tracking system
Design and FPGA-implementation of a neural network
Neural network implementation in hardware using FPGAs
Lecture Notes in Computer Science
Towards an FPGA based reconfigurable computing environment for neural network implementations
Cited by (70)
Software and hardware co-design and implementation of intelligent optimization algorithms
2022, Applied Soft ComputingNew mixed-coding PSO algorithm for a self-adaptive and automatic learning of Mamdani fuzzy rules
2020, Engineering Applications of Artificial IntelligenceArtificial Neural Networks based thermodynamic and economic analysis of a hydrogen production system assisted by geothermal energy on Field Programmable Gate Array
2019, International Journal of Hydrogen EnergyCitation Excerpt :The block diagram in Fig. 4 shows the nonlinear model of a neuron that constitutes the basis for the design of ANN [51]. The activation functions produce the neuron outputs using the sum of the multiplication results of input and weight values with bias value [52]. The nonlinearity of the activation function enables the prediction of any function.
Assessment and control of wind turbine by support vector machines
2018, Sustainable Energy Technologies and AssessmentsA comparison of fitness functions for identifying an LCD Glass-handling robot system
2017, MechatronicsCitation Excerpt :In the previous studies about system identification, the intelligent algorithm was also implemented to identify the system parameters. The modified particle swarm optimization [11] and the improving particle swarm optimization [12] were implemented to identify all parameters of robots by modifying the velocity update function. The optimal system parameters and optimal control parameters were found in [13].
Efficient hardware implementation of radial basis function neural network with customized-precision floating-point operations
2017, Control Engineering Practice