Skip to main content
Top

2017 | OriginalPaper | Chapter

Conjugate Gradient Algorithms for Quaternion-Valued Neural Networks

Author : Călin-Adrian Popa

Published in: Recent Advances in Soft Computing

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper introduces conjugate gradient algorithms for training quaternion-valued feedforward neural networks. Because these algorithms had better performance than the gradient descent algorithm in the real- and complex-valued cases, the extension to the quaternion-valued case was a natural idea. The classical variants of the conjugate gradient algorithm are deduced starting from their real-valued variants, and using the framework of the HR calculus. The resulting quaternion-valued training methods are exemplified on time series prediction applications, showing a significant improvement over the quaternion gradient descent algorithm.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Multilayer perceptrons to approximate quaternion valued functions. Neural Netw. 10(2), 335–342 (1997)CrossRef Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Multilayer perceptrons to approximate quaternion valued functions. Neural Netw. 10(2), 335–342 (1997)CrossRef
2.
go back to reference Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Neural Networks in Multidimensional Domains Fundamentals and New Trends in Modelling and Control. Lecture Notes in Control and Information Sciences, vol. 234. Springer, London (1998)CrossRefMATH Arena, P., Fortuna, L., Muscato, G., Xibilia, M.: Neural Networks in Multidimensional Domains Fundamentals and New Trends in Modelling and Control. Lecture Notes in Control and Information Sciences, vol. 234. Springer, London (1998)CrossRefMATH
3.
go back to reference Beale, E.: A derivation of conjugate gradients. In: Lootsma, F.A. (ed.) Numerical Methods for Nonlinear Optimization, pp. 39–43. Academic Press, London (1972) Beale, E.: A derivation of conjugate gradients. In: Lootsma, F.A. (ed.) Numerical Methods for Nonlinear Optimization, pp. 39–43. Academic Press, London (1972)
4.
go back to reference Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press Inc., New York (1995)MATH Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press Inc., New York (1995)MATH
5.
go back to reference Buchholz, S., Le Bihan, N.: Polarized signal classification by complex and quaternionic multi-layer perceptrons. Int. J. Neural Syst. 18(2), 75–85 (2008)CrossRef Buchholz, S., Le Bihan, N.: Polarized signal classification by complex and quaternionic multi-layer perceptrons. Int. J. Neural Syst. 18(2), 75–85 (2008)CrossRef
6.
go back to reference Charalambous, C.: Conjugate gradient algorithm for efficient training of artificial neural networks. IEE Proc. G Circuits Devices Syst. 139(3), 301–310 (1992)CrossRef Charalambous, C.: Conjugate gradient algorithm for efficient training of artificial neural networks. IEE Proc. G Circuits Devices Syst. 139(3), 301–310 (1992)CrossRef
7.
go back to reference Ujang, C.B., Took, C., Mandic, D.: Split quaternion nonlinear adaptive filtering. Neural Netw. 23(3), 426–434 (2010)CrossRef Ujang, C.B., Took, C., Mandic, D.: Split quaternion nonlinear adaptive filtering. Neural Netw. 23(3), 426–434 (2010)CrossRef
8.
go back to reference Ujang, C.B., Took, C., Mandic, D.: Quaternion-valued nonlinear adaptive filtering. IEEE Trans. Neural Netw. 22(8), 1193–1206 (2011)CrossRef Ujang, C.B., Took, C., Mandic, D.: Quaternion-valued nonlinear adaptive filtering. IEEE Trans. Neural Netw. 22(8), 1193–1206 (2011)CrossRef
9.
go back to reference Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)MathSciNetCrossRefMATH Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)MathSciNetCrossRefMATH
10.
go back to reference Isokawa, T., Kusakabe, T., Matsui, N., Peper, F.: Quaternion neural network and its application. In: Palade, V., Howlett, R.J., Jain, L. (eds.) KES 2003. LNCS (LNAI), vol. 2774, pp. 318–324. Springer, Heidelberg (2003). doi:10.1007/978-3-540-45226-3_44 CrossRef Isokawa, T., Kusakabe, T., Matsui, N., Peper, F.: Quaternion neural network and its application. In: Palade, V., Howlett, R.J., Jain, L. (eds.) KES 2003. LNCS (LNAI), vol. 2774, pp. 318–324. Springer, Heidelberg (2003). doi:10.​1007/​978-3-540-45226-3_​44 CrossRef
11.
go back to reference Jahanchahi, C., Took, C., Mandic, D.: On HR calculus, quaternion valued stochastic gradient, and adaptive three dimensional wind forecasting. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–5. IEEE, July 2010 Jahanchahi, C., Took, C., Mandic, D.: On HR calculus, quaternion valued stochastic gradient, and adaptive three dimensional wind forecasting. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–5. IEEE, July 2010
12.
go back to reference Johansson, E., Dowla, F., Goodman, D.: Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method. Int. J. Neural Syst. 2(4), 291–301 (1991)CrossRef Johansson, E., Dowla, F., Goodman, D.: Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method. Int. J. Neural Syst. 2(4), 291–301 (1991)CrossRef
13.
go back to reference Kusamichi, H., Isokawa, T., Matsui, N., Ogawa, Y., Maeda, K.: A new scheme for color night vision by quaternion neural network. In: International Conference on Autonomous Robots and Agents, pp. 101–106, December 2004 Kusamichi, H., Isokawa, T., Matsui, N., Ogawa, Y., Maeda, K.: A new scheme for color night vision by quaternion neural network. In: International Conference on Autonomous Robots and Agents, pp. 101–106, December 2004
14.
go back to reference Luenberger, D., Ye, Y.: Linear and Nonlinear Programming. International Series in Operations Research & Management Science, vol. 116. Springer, Heidelberg (2008)MATH Luenberger, D., Ye, Y.: Linear and Nonlinear Programming. International Series in Operations Research & Management Science, vol. 116. Springer, Heidelberg (2008)MATH
15.
go back to reference Mandic, D., Chambers, J.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. Wiley, New York (2001)CrossRef Mandic, D., Chambers, J.: Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. Wiley, New York (2001)CrossRef
16.
go back to reference Polak, E., Ribiere, G.: Note sur la convergence de méthodes de directions conjuguées. Revue Française d’Informatique et de Recherche Opérationnelle 3(16), 35–43 (1969)CrossRefMATH Polak, E., Ribiere, G.: Note sur la convergence de méthodes de directions conjuguées. Revue Française d’Informatique et de Recherche Opérationnelle 3(16), 35–43 (1969)CrossRefMATH
17.
20.
go back to reference Took, C., Mandic, D.: The quaternion lms algorithm for adaptive filtering of hypercomplex processes. IEEE Trans. Sig. Process. 57(4), 1316–1327 (2009)MathSciNetCrossRef Took, C., Mandic, D.: The quaternion lms algorithm for adaptive filtering of hypercomplex processes. IEEE Trans. Sig. Process. 57(4), 1316–1327 (2009)MathSciNetCrossRef
21.
go back to reference Took, C., Mandic, D.: Quaternion-valued stochastic gradient-based adaptive IIR filtering. IEEE Trans. Sig. Process. 58(7), 3895–3901 (2010)MathSciNetCrossRef Took, C., Mandic, D.: Quaternion-valued stochastic gradient-based adaptive IIR filtering. IEEE Trans. Sig. Process. 58(7), 3895–3901 (2010)MathSciNetCrossRef
22.
go back to reference Took, C., Mandic, D.: A quaternion widely linear adaptive filter. IEEE Trans. Sig. Process. 58(8), 4427–4431 (2010)MathSciNetCrossRef Took, C., Mandic, D.: A quaternion widely linear adaptive filter. IEEE Trans. Sig. Process. 58(8), 4427–4431 (2010)MathSciNetCrossRef
23.
go back to reference Took, C., Mandic, D., Aihara, K.: Quaternion-valued short term forecasting of wind profile. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, July 2010 Took, C., Mandic, D., Aihara, K.: Quaternion-valued short term forecasting of wind profile. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, July 2010
24.
go back to reference Took, C., Strbac, G., Aihara, K., Mandic, D.: Quaternion-valued short-term joint forecasting of three-dimensional wind and atmospheric parameters. Renewable Energy 36(6), 1754–1760 (2011)CrossRef Took, C., Strbac, G., Aihara, K., Mandic, D.: Quaternion-valued short-term joint forecasting of three-dimensional wind and atmospheric parameters. Renewable Energy 36(6), 1754–1760 (2011)CrossRef
25.
go back to reference Xia, Y., Jahanchahi, C., Mandic, D.: Quaternion-valued echo state networks. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 663–673 (2015)MathSciNetCrossRef Xia, Y., Jahanchahi, C., Mandic, D.: Quaternion-valued echo state networks. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 663–673 (2015)MathSciNetCrossRef
26.
go back to reference Xu, D., Xia, Y., Mandic, D.: Optimization in quaternion dynamic systems: gradient, Hessian, and learning algorithms. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 249–261 (2016)MathSciNetCrossRef Xu, D., Xia, Y., Mandic, D.: Optimization in quaternion dynamic systems: gradient, Hessian, and learning algorithms. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 249–261 (2016)MathSciNetCrossRef
Metadata
Title
Conjugate Gradient Algorithms for Quaternion-Valued Neural Networks
Author
Călin-Adrian Popa
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-58088-3_17

Premium Partner