Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 4/2020

16.05.2019 | Original Article

Cross-modal learning for material perception using deep extreme learning machine

verfasst von: Wendong Zheng, Huaping Liu, Bowen Wang, Fuchun Sun

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 4/2020

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The material property of an object’s surface is critical for the tasks of robotic manipulation or interaction with its surrounding environment. Tactile sensing can provide rich information about the material characteristics of an object’s surface. Hence, it is important to convey and interpret tactile information of material properties to the users during interaction. In this paper, we propose a visual-tactile cross-modal retrieval framework to convey tactile information of surface material for perceptual estimation. In particular, we use tactile information of a new unknown surface material to retrieve perceptually similar surface from an available surface visual sample set. For the proposed framework, we develop a deep cross-modal correlation learning method, which incorporates the high-level nonlinear representation of deep extreme learning machine and class-paired correlation learning of cluster canonical correlation analysis. Experimental results on the publicly available dataset validate the effectiveness of the proposed framework and the method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Andrew G, Arora R, Bilmes J, Livescu K (2013) Deep canonical correlation analysis. In: International conference on machine learning, pp. 1247–1255 Andrew G, Arora R, Bilmes J, Livescu K (2013) Deep canonical correlation analysis. In: International conference on machine learning, pp. 1247–1255
2.
Zurück zum Zitat Arun KS, Govindan VK (2016) A context-aware semantic modeling framework for efficient image retrieval. Int J Mach Learn Cybern 8(4):1–27 Arun KS, Govindan VK (2016) A context-aware semantic modeling framework for efficient image retrieval. Int J Mach Learn Cybern 8(4):1–27
3.
Zurück zum Zitat Baumgartner E, Wiebel CB, Gegenfurtner KR (2013) Visual and haptic representations of material properties. Multisens Res 26(5):429–455CrossRef Baumgartner E, Wiebel CB, Gegenfurtner KR (2013) Visual and haptic representations of material properties. Multisens Res 26(5):429–455CrossRef
4.
Zurück zum Zitat Burka A, Hu S, Helgeson S, Krishnan S, Gao Y, Hendricks LA, Darrell T, Kuchenbecker KJ (2017) Proton: A visuo-haptic data acquisition system for robotic learning of surface properties. In: IEEE international conference on multisensor fusion and integration for intelligent systems, pp. 58–65 Burka A, Hu S, Helgeson S, Krishnan S, Gao Y, Hendricks LA, Darrell T, Kuchenbecker KJ (2017) Proton: A visuo-haptic data acquisition system for robotic learning of surface properties. In: IEEE international conference on multisensor fusion and integration for intelligent systems, pp. 58–65
5.
Zurück zum Zitat Cao J, Lin Z (2015) Extreme learning machines on high dimensional and large data applications: A survey. Math Problems Eng 2015(3):1–13MATH Cao J, Lin Z (2015) Extreme learning machines on high dimensional and large data applications: A survey. Math Problems Eng 2015(3):1–13MATH
8.
Zurück zum Zitat Feng F, Li R, Wang X (2015) Deep correspondence restricted boltzmann machine for cross-modal retrieval. Neurocomputing 154(C):50–60CrossRef Feng F, Li R, Wang X (2015) Deep correspondence restricted boltzmann machine for cross-modal retrieval. Neurocomputing 154(C):50–60CrossRef
9.
Zurück zum Zitat Fukumizu K, Bach F, Gretton A (2007) Statistical consistency of kernel canonical correlation analysis. J Mach Learn Res 8(2007):361–383MathSciNetMATH Fukumizu K, Bach F, Gretton A (2007) Statistical consistency of kernel canonical correlation analysis. J Mach Learn Res 8(2007):361–383MathSciNetMATH
10.
Zurück zum Zitat Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664CrossRef Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664CrossRef
11.
Zurück zum Zitat Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122CrossRef Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122CrossRef
12.
Zurück zum Zitat Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef
13.
Zurück zum Zitat Kerr E, McGinnity T, Coleman S (2018) Material recognition using tactile sensing. Expert Syst Appl 94:94–111CrossRef Kerr E, McGinnity T, Coleman S (2018) Material recognition using tactile sensing. Expert Syst Appl 94:94–111CrossRef
14.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
15.
Zurück zum Zitat Kroemer O, Lampert CH, Peters J (2011) Learning dynamic tactile sensing with robust vision-based training. IEEE Trans Robot 27(3):545–557CrossRef Kroemer O, Lampert CH, Peters J (2011) Learning dynamic tactile sensing with robust vision-based training. IEEE Trans Robot 27(3):545–557CrossRef
16.
Zurück zum Zitat Kuchenbecker KJ, McMahan W, Landin N, Romano JM (2010) Dimensional reduction of high-frequencey accelerations for haptic rendering, In: International conference on human haptic sensing and touch enabled computer applications, pp 79–86 Kuchenbecker KJ, McMahan W, Landin N, Romano JM (2010) Dimensional reduction of high-frequencey accelerations for haptic rendering, In: International conference on human haptic sensing and touch enabled computer applications, pp 79–86
17.
Zurück zum Zitat Kursun O, Alpaydin E, Favorov OV (2011) Canonical correlation analysis using within-class coupling. Pattern Recognit Lett 32(2):134–144CrossRef Kursun O, Alpaydin E, Favorov OV (2011) Canonical correlation analysis using within-class coupling. Pattern Recognit Lett 32(2):134–144CrossRef
18.
Zurück zum Zitat Liu C, Sun F, Zhang B (2018) Brain-inspired multimodal learning based on neural networks. Brain Sci Adv 4(1):61–72CrossRef Liu C, Sun F, Zhang B (2018) Brain-inspired multimodal learning based on neural networks. Brain Sci Adv 4(1):61–72CrossRef
19.
Zurück zum Zitat Liu H, Sun F (2018) Material identification using tactile perception: a semantics-regularized dictionary learning method. IEEE/ASME Trans Mechatron 23(3):1050–1058CrossRef Liu H, Sun F (2018) Material identification using tactile perception: a semantics-regularized dictionary learning method. IEEE/ASME Trans Mechatron 23(3):1050–1058CrossRef
20.
Zurück zum Zitat Liu H, Sun F, Fang B (2017) Seeing by touching: cross-modal matching for tactile and vision measurements. In: Advanced robotics and mechatronics (ICARM), 2017 2nd international conference on, pp. 257–263. IEEE Liu H, Sun F, Fang B (2017) Seeing by touching: cross-modal matching for tactile and vision measurements. In: Advanced robotics and mechatronics (ICARM), 2017 2nd international conference on, pp. 257–263. IEEE
21.
Zurück zum Zitat Liu H, Sun F, Fang B, Lu S (2018) Multimodal measurements fusion for surface material categorization. IEEE Trans Instrum Meas 67(2):246–256CrossRef Liu H, Sun F, Fang B, Lu S (2018) Multimodal measurements fusion for surface material categorization. IEEE Trans Instrum Meas 67(2):246–256CrossRef
22.
Zurück zum Zitat Liu H, Yu Y, Sun F, Gu J (2017) Visual-tactile fusion for object recognition. IEEE Trans Autom Sci Eng 14(2):996–1008CrossRef Liu H, Yu Y, Sun F, Gu J (2017) Visual-tactile fusion for object recognition. IEEE Trans Autom Sci Eng 14(2):996–1008CrossRef
23.
Zurück zum Zitat Luo S, Bimbo J, Dahiya R, Liu H (2017) Robotic tactile perception of object properties: a review. Mechatronics 48:54–67CrossRef Luo S, Bimbo J, Dahiya R, Liu H (2017) Robotic tactile perception of object properties: a review. Mechatronics 48:54–67CrossRef
24.
Zurück zum Zitat Rasiwasia N, Mahajan D, Mahadevan V, Aggarwal G (2014) Cluster canonical correlation analysis. In: Artificial intelligence and statistics, pp 823–831 Rasiwasia N, Mahajan D, Mahadevan V, Aggarwal G (2014) Cluster canonical correlation analysis. In: Artificial intelligence and statistics, pp 823–831
25.
Zurück zum Zitat Rasiwasia N, Pereira JC, Coviello E, Doyle G, Lanckriet GRG, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: International conference on multimedia. ACM, pp 251–260 Rasiwasia N, Pereira JC, Coviello E, Doyle G, Lanckriet GRG, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: International conference on multimedia. ACM, pp 251–260
26.
Zurück zum Zitat Sinapov J, Sukhoy V, Sahai R, Stoytchev A (2011) Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans Robot 27(3):488–497CrossRef Sinapov J, Sukhoy V, Sahai R, Stoytchev A (2011) Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans Robot 27(3):488–497CrossRef
27.
Zurück zum Zitat Strese M, Lee JY, Schuwerk C, Han Q, Kim HG, Steinbach E (2014) A haptic texture database for tool-mediated texture recognition and classification. In: Haptic, audio and visual environments and games (HAVE), 2014 IEEE international symposium on. IEEE, pp 118–123 Strese M, Lee JY, Schuwerk C, Han Q, Kim HG, Steinbach E (2014) A haptic texture database for tool-mediated texture recognition and classification. In: Haptic, audio and visual environments and games (HAVE), 2014 IEEE international symposium on. IEEE, pp 118–123
28.
Zurück zum Zitat Strese M, Schuwerk C, Iepure A, Steinbach E (2017) Multimodal feature-based surface material classification. IEEE Trans Haptics 10(2):226–239CrossRef Strese M, Schuwerk C, Iepure A, Steinbach E (2017) Multimodal feature-based surface material classification. IEEE Trans Haptics 10(2):226–239CrossRef
29.
Zurück zum Zitat Strese M, Schuwerk C, Steinbach E (2015) Surface classification using acceleration signals recorded during human freehand movement. In: World Haptics conference (WHC), 2015 IEEE. IEEE, pp 214–219 Strese M, Schuwerk C, Steinbach E (2015) Surface classification using acceleration signals recorded during human freehand movement. In: World Haptics conference (WHC), 2015 IEEE. IEEE, pp 214–219
30.
Zurück zum Zitat Ukil S, Ghosh S, Obaidullah SM, Santosh K, Roy K, Das N (2018) Deep learning for word-level handwritten indic script identification. arXiv preprint arXiv:1801.01627 Ukil S, Ghosh S, Obaidullah SM, Santosh K, Roy K, Das N (2018) Deep learning for word-level handwritten indic script identification. arXiv preprint arXiv:​1801.​01627
32.
Zurück zum Zitat Vicente A, Liu J, Yang GZ (2015) Surface classification based on vibration on omni-wheel mobile base. In: Intelligent robots and systems (IROS), 2015 IEEE/RSJ international conference on. IEEE, pp 916–921 Vicente A, Liu J, Yang GZ (2015) Surface classification based on vibration on omni-wheel mobile base. In: Intelligent robots and systems (IROS), 2015 IEEE/RSJ international conference on. IEEE, pp 916–921
33.
Zurück zum Zitat Wang W, Arora R, Livescu K, Bilmes J (2016) On deep multi-view representation learning: objectives and optimization. arXiv preprint arXiv:1602.01024 Wang W, Arora R, Livescu K, Bilmes J (2016) On deep multi-view representation learning: objectives and optimization. arXiv preprint arXiv:​1602.​01024
34.
Zurück zum Zitat Wang X, Cao W (2018) Non-iterative approaches in training feed-forward neural networks and their applications. Soft Comput 18(12):1–4MATH Wang X, Cao W (2018) Non-iterative approaches in training feed-forward neural networks and their applications. Soft Comput 18(12):1–4MATH
35.
Zurück zum Zitat Wang X, Chen A, Feng H (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef Wang X, Chen A, Feng H (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef
36.
Zurück zum Zitat Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRef Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRef
37.
Zurück zum Zitat Zhang J, Ding S, Zhang N, Shi Z (2016) Incremental extreme learning machine based on deep feature embedded. Int J Mach Learn Cybern 7(1):111–120CrossRef Zhang J, Ding S, Zhang N, Shi Z (2016) Incremental extreme learning machine based on deep feature embedded. Int J Mach Learn Cybern 7(1):111–120CrossRef
Metadaten
Titel
Cross-modal learning for material perception using deep extreme learning machine
verfasst von
Wendong Zheng
Huaping Liu
Bowen Wang
Fuchun Sun
Publikationsdatum
16.05.2019
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 4/2020
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-019-00962-1

Weitere Artikel der Ausgabe 4/2020

International Journal of Machine Learning and Cybernetics 4/2020 Zur Ausgabe

Neuer Inhalt