Elsevier

Applied Soft Computing

Volume 12, Issue 9, September 2012, Pages 2707-2718
Applied Soft Computing

Neural identification of dynamic systems on FPGA with improved PSO learning

https://doi.org/10.1016/j.asoc.2012.03.022Get rights and content

Abstract

This work introduces hardware implementation of artificial neural networks (ANNs) with learning ability on field programmable gate array (FPGA) for dynamic system identification. The learning phase is accomplished by using the improved particle swarm optimization (PSO). The improved PSO is obtained by modifying the velocity update function. Adding an extra term to the velocity update function reduced the possibility of stucking in a local minimum. The results indicates that ANN, trained using improved PSO algorithm, converges faster and produces more accurate results with a little extra hardware utilization cost.

Highlights

► This work introduces hardware implementation of artificial neural networks (ANNs) with learning ability on Field Programmable Gate Array (FPGA) for dynamic system identification. ► The learning phase is accomplished by using the improved particle swarm optimization (PSO). ► The improved PSO is obtained by modifying the velocity update function. Adding an extra term to the velocity update function reduced the possibility of stucking in a local minimum.

Introduction

Artificial neural networks (ANNs) have been widely and successfully used in many fields to solve complex problems. ANNs are capable of successfully modeling non-linear complex correlation between input and output of a system. The most commonly used ANN is the multi layer perceptron (MLP) [1], [2]. ANN is trained using a data set which is compatible with input–output relation of system to be modeled. Training is usually performed using back propagation (BP) algorithm. In recent years evolutionary algorithms are also used for ANN training as an alternative to BP algorithm.

For example population-based particle swarm optimization (PSO) [3], capable of stochastic search, has been successfully utilized in neural network training in addition to its usage in function optimization and fuzzy system control [4], [5], [6], [7].

In recent years FPGAs, capable of parallel processing, are emerging as a predominant embedded system platform for many ANN applications [8], [9], [10], [11], [12], [13]. In the literature numerous of hardware based ANN implementations on FPGAs with offline training [1], [14], [15], [16], [17] and online training [18], [19], [20] are reported. In these implementations different number formats such as fixed-point numbers with different bit width [16], [18], [20], floating-point numbers [1], [14], [17], [18], [19] and integer numbers [15] are utilized. Nonlinear logarithmic sigmoid [14], [15], [16], [17], [18], [19] and nonlinear hyperbolic tangent [1], [16], [20] are employed as activation function. To implement nonlinear activation functions on hardware several approaches have been proposed: look-up table [15], [20], piecewise-linear [1], [16], [18], parabolic [14] and functional [17], [19].

In this work we present FPGA implementation of a system identification module based on ANN with PSO based online training. ANN and its PSO based online training module implemented in single precision floating-point number format and neural cell activation functions are implemented using functional approaches. In functional approach a divider is used in addition to adder and multiplier when compared to piecewise-linear and parabolic approach. Divider usage could be considered as the disadvantage of functional approach, but this approach has several advantages: (i) contrary to look-up table approach, no memory needed (ii) contrary to piecewise-linear approach, no control statements needed. The most important contribution of this work is the modified velocity update function of the PSO. In this paper authors proposed to add an extra term to the function which reduces the possibility of stucking in a local minimum during training. The experimental results indicate that the PSO algorithm utilizing the modified velocity update function converges faster and produces more accurate results.

The rest of the paper is organized as follows. Section 2 provides brief introduction to standard PSO and proposed improved PSO. Section 3 describes hardware implementation of ANN and its PSO based training module. Experimental results and performance of proposed approach in hardware are given in Section 4. The comparison results of the proposed improved PSO with other methods are given in Section 5 and the conclusions are given in Section 6.

Section snippets

Standard and proposed modified PSO

PSO, inspired from social interactions among animals, is a high performance optimizer. PSO performs search using a population of particles. Each particle is directed to the most significant parts of the search space as a result of interactions between particles. The algorithm starts with randomly assigned particles. In every iteration the velocity and the location of particles are updated. Each particle's position is evaluated as a possible solution candidate. In a swarm with N particles, the

PSO based ANN training ON FPGA

In this section, the details of PSO based ANN training on FPGA is described. The block diagram of the proposed architecture with the intention of dynamic system identification is shown in Fig. 1. PSO based ANN training on FPGA has four main stages as given in Fig. 2. Short description of these four stages is as follows.

Experimental results

The details of PSO based ANN training implementation on FPGA (Altera Cyclone II EP2C70F672C6), is given in previous section. The hardware implementation is experimentally tested with four dynamic system identification problems. Learning phase for the dynamic system identification problem is operated as illustrated in Fig. 1. ANN identification model, whose parameters are determined at the end of the learning phase, is tested for the data set unseen in the learning phase. This testing phase is

Comparison of improved PSO with other methods

To give more general idea about the performance of the proposed improved PSO, it has been compared with BP, artificial bee colony (ABC) [29] and conventional PSO [3] algorithms on software environment. Comparisons are carried out for ANN training for the identification examples given in the previous section. Codes for ABC algorithm has been taken from [30] and adopted to the identification examples. ANN architectures are the same as ones given in the previous section for each example. Learning

Conclusion

In this paper, the hardware implementation of ANN with PSO online training for system identification is proposed. Contribution of this work is twofold. First, it introduces the modified velocity update function for PSO (Eq. (6)). Authors proposed to add an extra term (α3λ(n)) to the function which reduces the possibility of stucking in a local minimum. Second, it proposes an effective system identification architecture based on ANN with PSO learning on FPGA. To prove the effectiveness of the

References (30)

  • J. Martinez et al.

    A retinomorphic architecture based on discrete-time cellular neural networks using reconfigurable computing

    Neurocomputing

    (2008)
  • M. Krips et al.

    FPGA implementation of a neural network for a real time hand tracking system

  • H. Ossoinig et al.

    Design and FPGA-implementation of a neural network

  • S. Sahin et al.

    Neural network implementation in hardware using FPGAs

    Lecture Notes in Computer Science

    (2006)
  • J. Zhu et al.

    Towards an FPGA based reconfigurable computing environment for neural network implementations

  • Cited by (70)

    • Artificial Neural Networks based thermodynamic and economic analysis of a hydrogen production system assisted by geothermal energy on Field Programmable Gate Array

      2019, International Journal of Hydrogen Energy
      Citation Excerpt :

      The block diagram in Fig. 4 shows the nonlinear model of a neuron that constitutes the basis for the design of ANN [51]. The activation functions produce the neuron outputs using the sum of the multiplication results of input and weight values with bias value [52]. The nonlinearity of the activation function enables the prediction of any function.

    • Assessment and control of wind turbine by support vector machines

      2018, Sustainable Energy Technologies and Assessments
    • A comparison of fitness functions for identifying an LCD Glass-handling robot system

      2017, Mechatronics
      Citation Excerpt :

      In the previous studies about system identification, the intelligent algorithm was also implemented to identify the system parameters. The modified particle swarm optimization [11] and the improving particle swarm optimization [12] were implemented to identify all parameters of robots by modifying the velocity update function. The optimal system parameters and optimal control parameters were found in [13].

    View all citing articles on Scopus
    View full text