Top

Cognitive Computation

Published in:

06-03-2020

SOAR Improved Artificial Neural Network for Multistep Decision-making Tasks

Published in: Cognitive Computation | Issue 3/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Recently, artificial neural networks (ANNs) have been applied to various robot-related research areas due to their powerful spatial feature abstraction and temporal information prediction abilities. Decision-making has also played a fundamental role in the research area of robotics. How to improve ANNs with the characteristics of decision-making is a challenging research issue. ANNs are connectionist models, which means they are naturally weak in long-term planning, logical reasoning, and multistep decision-making. Considering that a small refinement of the inner network structures of ANNs will usually lead to exponentially growing data costs, an additional planning module seems necessary for the further improvement of ANNs, especially for small data learning. In this paper, we propose a state operator and result (SOAR) improved ANN (SANN) model, which takes advantage of both the long-term cognitive planning ability of SOAR and the powerful feature detection ability of ANNs. It mimics the cognitive mechanism of the human brain to improve the traditional ANN with an additional logical planning module. In addition, a data fusion module is constructed to combine the probability vector obtained by SOAR planning and the original data feature array. A data fusion module is constructed to convert the information from the logical sequences in SOAR to the probabilistic vector in ANNs. The proposed architecture is validated in two types of robot multistep decision-making experiments for a grasping task: a multiblock simulated experiment and a multicup experiment in a real scenario. The experimental results show the efficiency and high accuracy of our proposed architecture. The integration of SOAR and ANN is a good compromise between logical planning with small data and probabilistic classification with big data. It also has strong potential for more complicated tasks that require robust classification, long-term planning, and fast learning. Some potential applications include recognition of grasping order in multiobject environment and cooperative grasping of multiagents.

previous article Dense-CaptionNet: a Sentence Generation Architecture for Fine-grained Description of Image Semantics

next article A Deep Multi-task Model for Dialogue Act Classification, Intent Detection and Slot Filling

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

https://github.com/thomasaimondy/SoarImprovedANN

https://github.com/thomasaimondy/patchlinemod

Kotseruba I, Tsotsos JK. 40 years of cognitive architectures: core cognitive abilities and practical applications. Artif Intell Rev;40:1–78.

Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C, Qin Y. An integrated theory of the mind. Psychol Rev 2004;111(4):1036.CrossRef

Anderson JR. Human symbol manipulation within an integrated cognitive architecture. Cogn Sci 2005;29(3): 313–341.CrossRef

Laird JE, Newell A, Rosenbloom PS. Soar: an architecture for general intelligence. Artif Intell 1987;33 (1):1–64.CrossRef

Laird JE. 2012. The Soar cognitive architecture. MIT Press, Cambridge.

French RM. Catastrophic forgetting in connectionist networks. Trends Cogn Sci 1999;3(4):128–135.CrossRef

Eliasmith C, Trujillo O. The use and abuse of large-scale brain models. Curr Opinion Neurobiol 2014;25: 1–6.CrossRef

Hawkins J, Ahmad S. Why neurons have thousands of synapses, a theory of sequence memory in neocortex. Front Neural Circ 2016;10:23.

Sun R, Peterson T, Merrill E. A hybrid architecture for situated learning of reactive sequential decision making. Appl Intell 1999;11(1):109–127.CrossRef

10.

O’Reilly RC, Wyatte D, Herd S, Mingus B, Jilk DJ. Recurrent processing during object recognition. Front Psychol 2013;4:124.CrossRef

11.

Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015;61:85–117.CrossRef

12.

He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of the IEEE international conference on computer vision; 2015. p. 1026–1034.

13.

Wang Z, Wang X, Wang G. Learning fine-grained features via a cnn tree for large-scale classification. Neurocomputing 2018;275:1231–1240.CrossRef

14.

Dahl GE, Yu D, Li D, Acero A. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 2012;20(1):30–42.CrossRef

15.

Redmon J, Divvala S, Girshick R, Farhadi A. You only once: look Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 779–788.

16.

Maturana D, Scherer S. Voxnet: a 3d convolutional neural network for real-time object recognition. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2015. p. 922–928.

17.

Oh J, Guo X, Lee H, Lewis RL, Singh S. Action-conditional video prediction using deep networks in atari games. In: Advances in neural information processing systems; 2015. p. 2863–2871.

18.

Weisz G, Budzianowski P, Su P-H, Gasic M. Sample efficient deep reinforcement learning for dialogue systems with large action spaces. IEEE/ACM Trans Audio Speech Lang Process (TASLP) 2018;26(11):2083–2097.CrossRef

19.

Zen H, Sak H. 2015. Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2015. P. 4470–4474.

20.

Finn C, Levine S. 2017. Deep visual foresight for planning robot motion. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2017. p. 2786–2793.

21.

Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves Ax, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature 2015;518(7540):529.CrossRef

22.

Ge L, Ren Z, Li Y, Xue Z, Wang Y, Cai J, Yuan J. 3d hand shape and pose estimation from a single rgb image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. p. 10833–10842.

23.

Dong J, Jiang W, Huang Q, Bao H, Zhou X. Fast and robust multi-person 3d pose estimation from multiple views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. p. 7792–7801.

24.

Huajun Z, Jin Z, Rui W, Tan M. Multi-objective reinforcement learning algorithm and its application in drive system. In 2008 34th Annual Conference of IEEE Industrial Electronics. IEEE; 2008. p. 274–279.

25.

Hester T, Vecerik M, Pietquin O, Lanctot M, Piot B, Horgan D, Quan J, Sendonaris A, Osband I, et al. Deep q-learning from demonstrations. In: Thirty-Second AAAI Conference on Artificial Intelligence; 2018.

26.

Ellefsen KO, Torresen J. Self-adapting goals allow transfer of predictive models to new tasks; 2019.

27.

Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M. Mastering the game of go with deep neural networks and tree search. Nature 2016;529(7587):484–489.CrossRef

28.

Yang Y, Yi L, Fermuller C, Aloimonos Y. Robot learning manipulation action plans by “watching” unconstrained videos from the world wide web. In: Twenty-ninth Aaai Conference on Artificial Intelligence; 2015.

29.

Volodymyr M, Koray K, David S, Rusu AA, Joel Vx, Bellemare MG, Alex G, Martin R, Fidjeland AK, Georg O. Human-level control through deep reinforcement learning. Nature 2015;518(7540): 529.CrossRef

30.

Zhang H, Lan X, Zhou X, Tian Z, Zhang Y, Zheng N. 2018. Visual manipulation relationship network for autonomous robotics. In: IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids). IEEE; 2018. p. 118–125.

31.

Zeng A, Song S, Lee J, Rodriguez A, Funkhouser T. 2019. Tossingbot: learning to throw arbitrary objects with residual physics.

32.

Chen H-Z, Tian G-H, Liu G-L. A selective attention guided initiative semantic cognition algorithm for service robot. Int J Autom Comput 2018;15(5):559–569.CrossRef

33.

Van Dang C, Pham TX, Gil K-J, Shin Y-B, Kim J-W, et al. Implementation of a refusable human-robot interaction task with humanoid robot by connecting soar and ros. J Korea Robot Soc 2017;12(1):55–64.CrossRef

34.

Puigbo J-Y, Pumarola A, Angulo C, Tellez R. Using a cognitive architecture for general purpose service robot control. Connect Sci 2015;27(2):105–117.CrossRef

35.

Zheng J, Cai F, Chen W, Feng C, Chen H. Hierarchical neural representation for document classification. Cogn Comput 2019;11(2):317–327.CrossRef

36.

Zhou K, Wei R, Xu Z, Zhang Q, Lu H, Zhang G. 2019. An air combat decision learning system based on a brain-like cognitive mechanism. Cognitive Computation.

37.

Liu P, Qin X. A new decision-making method based on interval-valued linguistic intuitionistic fuzzy information. Cogn Comput 2019;11(1):125–144.CrossRef

38.

Doumanoglou A, Kouskouridas R, Malassiotis S, Kim T-K. Recovering 6d object pose and predicting next-best-view in the crowd. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. p. 3583–3592.

39.

Hodan T, Michel F, Brachmann E, Kehl W, GlentBuch A, Kraft D, Drost B, Vidal J, Ihrke S, Zabulis X, et al. Bop: benchmark for 6d object pose estimation. In: Proceedings of the European Conference on Computer Vision (ECCV); 2018. p. 19–34.

40.

Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, Navab N. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian conference on computer vision. Springer; 2012. p. 548–562.

41.

Hinterstoisser S, Holzer S, Cagniart C, Ilic S, Konolige K, Navab N, Lepetit V. Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: IEEE International Conference on Computer Vision; 2012.

42.

Van Der Maaten L. Accelerating t-sne using tree-based algorithms. J Mach Learn Res 2014;15(1):3221–3245.MathSciNetMATH

43.

van der Maaten L, Hinton G. Visualizing data using t-sne. J Mach Learn Res 2008;9:2579–2605.MATH

Title: SOAR Improved Artificial Neural Network for Multistep Decision-making Tasks
Publication date: 06-03-2020
Published in: Cognitive Computation / Issue 3/2021
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI: https://doi.org/10.1007/s12559-020-09716-6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2021

Discriminative Dictionary Design for Action Classification in Still Images and Videos

A Hybrid CNN-LSTM Model for Psychopathic Class Detection from Tweeter Users

Relationship Identification Between Conversational Agents Using Emotion Analysis

When Old Meets New: Emotion Recognition from Speech Signals

Possibility Degree and Power Aggregation Operators of Single-Valued Trapezoidal Neutrosophic Numbers and Applications to Multi-Criteria Group Decision-Making

A Deep Multi-task Model for Dialogue Act Classification, Intent Detection and Slot Filling

Premium Partner