Skip to main content
Top
Published in: Cognitive Computation 5/2023

20-05-2023 | Original Article

Haptic Recognition of Texture Surfaces Using Semi-Supervised Feature Learning Based on Sparse Representation

Authors: Zhiyu Shao, Jiatong Bao, Jingwei Li, Hongru Tang

Published in: Cognitive Computation | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Haptic cognitive models are used to map the physical stimuli of texture surfaces to subjective haptic cognition, providing robotic systems with intelligent haptic cognition to perform dexterous manipulations in a manner that is similar to that of humans. Nevertheless, there is still the question of how to extract features that are stable and reflect the biological perceptual characteristics as the inputs of the models. To address this issue, a semi-supervised sparse representation method is developed to predict subjective haptic cognitive intensity in different haptic perceptual dimensions of texture surfaces. We conduct standardized interaction and perception experiments on textures that are part of common objects in daily life. Effective data cues sifting, perceptual filtering, and semi-supervised feature extraction steps are conducted in the process of sparse representation to ensure that the source data and features are complete and effective. The results indicate that the haptic cognitive model using the proposed method performs well in fitting and predicting perceptual intensity in the perceptual dimensions of “hardness,” “roughness,” and “slipperiness” for texture surfaces. Compared with previous methods, such as models using multilayer regression and hand-crafted features, the use of standardized interaction, cue sifting, perceptual filtering, and semi-supervised feature extraction could greatly improve the accuracy by improving the completeness of collected data, the effectiveness of features, and simulations of some physiological cognitive mechanisms. The improved method can be implemented to improve the performance of the haptic cognitive model for texture surfaces, and can also inspire research on intelligent cognition and haptic rendering systems.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Johansson RS, Flanagan JR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci. 2009;10(5):345–59.CrossRef Johansson RS, Flanagan JR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci. 2009;10(5):345–59.CrossRef
2.
go back to reference Leon JC, Dupeux T, Chardonnet JR, Perret J. Dexterous grasping tasks generated with an add-on end effector of a haptic feedback system. J Comput Inf Sci Eng. 2016;16(3). Leon JC, Dupeux T, Chardonnet JR, Perret J. Dexterous grasping tasks generated with an add-on end effector of a haptic feedback system. J Comput Inf Sci Eng. 2016;16(3).
3.
go back to reference Feng K, Xu Q, Tam LM. Design and development of a dexterous bilateral robotic microinjection system based on haptic feedback. IEEE Trans Autom Sci Eng. 2022;1–11. Feng K, Xu Q, Tam LM. Design and development of a dexterous bilateral robotic microinjection system based on haptic feedback. IEEE Trans Autom Sci Eng. 2022;1–11.
4.
go back to reference Seminara L, Gastaldo P, Watt SJ, Valyear KF, Zuher F, Mastrogiovanni F. Active haptic perception in robots: A review. Front Neurorobot. 2016;13, 53. Seminara L, Gastaldo P, Watt SJ, Valyear KF, Zuher F, Mastrogiovanni F. Active haptic perception in robots: A review. Front Neurorobot. 2016;13, 53.
5.
go back to reference Pestell N, Lloyd J, Rossiter J, Lepora NF. Dual-modal tactile perception and exploration. IEEE Robot Autom Lett. 2018;3(2):1033–40.CrossRef Pestell N, Lloyd J, Rossiter J, Lepora NF. Dual-modal tactile perception and exploration. IEEE Robot Autom Lett. 2018;3(2):1033–40.CrossRef
6.
go back to reference Shi Z-W, Ren Z-Y, Wang W-S, Xiao H, Zeng Y-H, Zhu L-Q. Bioinspired tactile perception platform with information encryption function. Chin Phys B. 2022;31(9): 098506. Shi Z-W, Ren Z-Y, Wang W-S, Xiao H, Zeng Y-H, Zhu L-Q. Bioinspired tactile perception platform with information encryption function. Chin Phys B. 2022;31(9): 098506.
7.
go back to reference Pacchierotti C. Cutaneous haptic feedback in robotic teleoperation. Springer; 2015. Pacchierotti C. Cutaneous haptic feedback in robotic teleoperation. Springer; 2015.
8.
go back to reference Medellin-Castillo HI, Zaragoza-Siqueiros J, Govea-Valladares EH, de la Garza-Camargo H, Lim T, Ritchie JM. Haptic-enabled virtual training in orthognathic surgery. Virtual Reality. 2021;25(1):53–67.CrossRef Medellin-Castillo HI, Zaragoza-Siqueiros J, Govea-Valladares EH, de la Garza-Camargo H, Lim T, Ritchie JM. Haptic-enabled virtual training in orthognathic surgery. Virtual Reality. 2021;25(1):53–67.CrossRef
9.
go back to reference Pinzon D, Byrns S, Zheng B. Prevailing trends in haptic feedback simulation for minimally invasive surgery. Surg Innov. 2016;23(4):415–21.CrossRef Pinzon D, Byrns S, Zheng B. Prevailing trends in haptic feedback simulation for minimally invasive surgery. Surg Innov. 2016;23(4):415–21.CrossRef
10.
go back to reference Wang D, Guo Y, Liu S, Zhang Y, Xiao J. Haptic display for virtual reality: progress and challenges. Virtual Reality and Intelligent Hardware. 2019;1(2). Wang D, Guo Y, Liu S, Zhang Y, Xiao J. Haptic display for virtual reality: progress and challenges. Virtual Reality and Intelligent Hardware. 2019;1(2).
11.
go back to reference Chen Y, Qiu W, Wang X, Zhang M. Tactile rendering of fabric textures based on texture recognition. In: 2019 IEEE THE 2nd International Conference on Micro/Nano Sensors for AI, Healthcare, and Robotics (NSENS), pp. 87–91. IEEE; 2019. Chen Y, Qiu W, Wang X, Zhang M. Tactile rendering of fabric textures based on texture recognition. In: 2019 IEEE THE 2nd International Conference on Micro/Nano Sensors for AI, Healthcare, and Robotics (NSENS), pp. 87–91. IEEE; 2019.
12.
go back to reference Grunwald M. Human haptic perception: basics and applications. Springer; 2008 Grunwald M. Human haptic perception: basics and applications. Springer; 2008
13.
go back to reference Chu V, Mcmahon I, Riano L, Mcdonald CG, He Q, Pereztejada JM, Arrigo M, Darrell T, Kuchenbecker KJ. Robotic learning of haptic adjectives through physical interaction. Robot Auton Syst. 2015;63:279–92.CrossRef Chu V, Mcmahon I, Riano L, Mcdonald CG, He Q, Pereztejada JM, Arrigo M, Darrell T, Kuchenbecker KJ. Robotic learning of haptic adjectives through physical interaction. Robot Auton Syst. 2015;63:279–92.CrossRef
15.
go back to reference Liu H, Sun F. Tactile adjective understanding using structured output-associated dictionary learning. In: Robotic tactile perception and understanding, pp. 97–116. Springer; 2018. Liu H, Sun F. Tactile adjective understanding using structured output-associated dictionary learning. In: Robotic tactile perception and understanding, pp. 97–116. Springer; 2018.
16.
go back to reference Richardson BA, Kuchenbecker KJ. Learning to predict perceptual distributions of haptic adjectives. Front Neurorobot. 2020;13. Richardson BA, Kuchenbecker KJ. Learning to predict perceptual distributions of haptic adjectives. Front Neurorobot. 2020;13.
18.
go back to reference Shao Z, Wu J, Ouyang Q, He C, Cao Z. Multi-layered perceptual model for haptic perception of compliance. Electronics. 2019;8(12):1497.CrossRef Shao Z, Wu J, Ouyang Q, He C, Cao Z. Multi-layered perceptual model for haptic perception of compliance. Electronics. 2019;8(12):1497.CrossRef
19.
go back to reference Tymms C, Gardner EP, Zorin D. A quantitative perceptual model for tactile roughness. ACM Trans Graph. 2018. Tymms C, Gardner EP, Zorin D. A quantitative perceptual model for tactile roughness. ACM Trans Graph. 2018.
20.
go back to reference Shao Z, Juan WU, Ouyang Q. Analysis of relevant quality metrics and physical parameters in softness perception and assessment system. IEICE Trans Inf Syst. 2019;10:2013–24.CrossRef Shao Z, Juan WU, Ouyang Q. Analysis of relevant quality metrics and physical parameters in softness perception and assessment system. IEICE Trans Inf Syst. 2019;10:2013–24.CrossRef
22.
go back to reference Ji H, Li S, Wang J, Ruan Z. Improving teleoperation through human-aware haptic feedback: a distinguishable and interpretable physical interaction based on the contact state. IEEE Trans Hum Mach Syst. 2022. Ji H, Li S, Wang J, Ruan Z. Improving teleoperation through human-aware haptic feedback: a distinguishable and interpretable physical interaction based on the contact state. IEEE Trans Hum Mach Syst. 2022.
23.
go back to reference Lederman SJ, Klatzky RL. Extracting object properties through haptic exploration. Acta Physiol (Oxf). 1993;84(1):29–40. Lederman SJ, Klatzky RL. Extracting object properties through haptic exploration. Acta Physiol (Oxf). 1993;84(1):29–40.
24.
go back to reference Klatzky R, Reed CL. Haptic exploration. In: Scholarpedia of Touch, pp. 177–183. Springer; 2016. Klatzky R, Reed CL. Haptic exploration. In: Scholarpedia of Touch, pp. 177–183. Springer; 2016.
26.
go back to reference Hauser SC, Gerling GJ. Force-rate cues reduce object deformation necessary to discriminate compliances harder than the skin. IEEE Trans Haptics. 2018;11(2):232–40.CrossRef Hauser SC, Gerling GJ. Force-rate cues reduce object deformation necessary to discriminate compliances harder than the skin. IEEE Trans Haptics. 2018;11(2):232–40.CrossRef
28.
go back to reference Cavdan M, Doerschner K, Drewing K. The many dimensions underlying perceived softness: how exploratory procedures are influenced by material and the perceptual task. In: 2019 IEEE World Haptics Conference (WHC), pp. 437–442. IEEE; 2019. Cavdan M, Doerschner K, Drewing K. The many dimensions underlying perceived softness: how exploratory procedures are influenced by material and the perceptual task. In: 2019 IEEE World Haptics Conference (WHC), pp. 437–442. IEEE; 2019.
29.
go back to reference Jin L, Gao S, Li Z, Tang J. Hand-crafted features or machine learnt features? Together they improve RGB-D object recognition. In: 2014 IEEE International Symposium on Multimedia, pp. 311–319. IEEE; 2014. Jin L, Gao S, Li Z, Tang J. Hand-crafted features or machine learnt features? Together they improve RGB-D object recognition. In: 2014 IEEE International Symposium on Multimedia, pp. 311–319. IEEE; 2014.
30.
go back to reference Zeng G, Zhou J, Jia X, Xie W, Shen L. Hand-crafted feature guided deep learning for facial expression recognition. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 423–430. IEEE; 2018. Zeng G, Zhou J, Jia X, Xie W, Shen L. Hand-crafted feature guided deep learning for facial expression recognition. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 423–430. IEEE; 2018.
33.
go back to reference Fishel JA, Loeb GE. Bayesian exploration for intelligent identification of textures. Front Neurorobot. 2012;6:4–4.CrossRef Fishel JA, Loeb GE. Bayesian exploration for intelligent identification of textures. Front Neurorobot. 2012;6:4–4.CrossRef
34.
go back to reference Sinapov J, Sukhoy V, Sahai R, Stoytchev A. Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans Rob. 2011;27(3):488–97.CrossRef Sinapov J, Sukhoy V, Sahai R, Stoytchev A. Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans Rob. 2011;27(3):488–97.CrossRef
35.
go back to reference Joolee JB, Uddin MA, Jeon S. Deep multi-model fusion network based real object tactile understanding from haptic data. Appl Intell. 2022;1–16. Joolee JB, Uddin MA, Jeon S. Deep multi-model fusion network based real object tactile understanding from haptic data. Appl Intell. 2022;1–16.
36.
go back to reference Shimoe H, Matsumura K, Noma H, Sohgawa M, Okuyama M. Development of artificial haptic model for human tactile sense using machine learning. In: 2017 IEEE SENSORS, pp. 1–3. IEEE; 2017. Shimoe H, Matsumura K, Noma H, Sohgawa M, Okuyama M. Development of artificial haptic model for human tactile sense using machine learning. In: 2017 IEEE SENSORS, pp. 1–3. IEEE; 2017.
37.
go back to reference Lederman SJ, Klatzky RL. Hand movements: a window into haptic object recognition. Cogn Psychol. 1987;19(3):342–68.CrossRef Lederman SJ, Klatzky RL. Hand movements: a window into haptic object recognition. Cogn Psychol. 1987;19(3):342–68.CrossRef
38.
go back to reference Tiest WMB, Kappers AML. Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta Physiol (Oxf). 2006;121(1):1–20. Tiest WMB, Kappers AML. Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta Physiol (Oxf). 2006;121(1):1–20.
39.
go back to reference Okamoto S, Nagano H, Yamada Y. Psychophysical dimensions of tactile perception of textures. IEEE Trans Haptics. 2013;6(1):81–93.CrossRef Okamoto S, Nagano H, Yamada Y. Psychophysical dimensions of tactile perception of textures. IEEE Trans Haptics. 2013;6(1):81–93.CrossRef
41.
go back to reference Jalali A, Farsi H. A new steganography algorithm based on video sparse representation. Multimed Tools Appl. 2020;79(3):1821–46.CrossRef Jalali A, Farsi H. A new steganography algorithm based on video sparse representation. Multimed Tools Appl. 2020;79(3):1821–46.CrossRef
42.
go back to reference Unnikrishnan P, Govindan V, Kumar SM. Enhanced sparse representation classifier for text classification. Expert Syst Appl. 2019;129:260–72.CrossRef Unnikrishnan P, Govindan V, Kumar SM. Enhanced sparse representation classifier for text classification. Expert Syst Appl. 2019;129:260–72.CrossRef
43.
go back to reference Arbib MA. The handbook of brain theory and neural networks, 2nd edn. MIT Press, Cambridge, MA, USA; 2002. Arbib MA. The handbook of brain theory and neural networks, 2nd edn. MIT Press, Cambridge, MA, USA; 2002.
45.
go back to reference Ye X, Choi B, Choi HR, Kang S. Pen-type sensor for surface texture perception. Robot and Human Interactive Communication. 2007;642–647. Ye X, Choi B, Choi HR, Kang S. Pen-type sensor for surface texture perception. Robot and Human Interactive Communication. 2007;642–647.
46.
go back to reference Lawrence D, Pao LY, Dougherty AM, Salada MA, Pavlou Y. Rate-hardness: a new performance metric for haptic interfaces 2000;16(4):357–371. Lawrence D, Pao LY, Dougherty AM, Salada MA, Pavlou Y. Rate-hardness: a new performance metric for haptic interfaces 2000;16(4):357–371.
47.
go back to reference Rosenberg, L.B., Adelstein, B.D.: Perceptual decomposition of virtual haptic surfaces. 1993;46–53. Rosenberg, L.B., Adelstein, B.D.: Perceptual decomposition of virtual haptic surfaces. 1993;46–53.
48.
go back to reference Shirado, H., Maeno, T.: Modeling of human texture perception for tactile displays and sensors 2005;629–630 Shirado, H., Maeno, T.: Modeling of human texture perception for tactile displays and sensors 2005;629–630
49.
go back to reference Klocker A, Oddo CM, Camboni D, Penta M, Thonnard J. Physical factors influencing pleasant touch during passive fingertip stimulation. PLoS ONE. 2014;9(7):1–10.CrossRef Klocker A, Oddo CM, Camboni D, Penta M, Thonnard J. Physical factors influencing pleasant touch during passive fingertip stimulation. PLoS ONE. 2014;9(7):1–10.CrossRef
50.
go back to reference Okamoto S, Nagano H, Kidoma K, Yamada Y. Specification of individuality in causal relationships among texture-related attributes, emotions, and preferences. Int J Affect Eng. 2016;15(1):11–9.CrossRef Okamoto S, Nagano H, Kidoma K, Yamada Y. Specification of individuality in causal relationships among texture-related attributes, emotions, and preferences. Int J Affect Eng. 2016;15(1):11–9.CrossRef
51.
go back to reference Hashim IHM, Kumamoto S, Takemura K, Maeno T, Okuda S, Mori Y. Tactile evaluation feedback system for multi-layered structure inspired by human tactile perception mechanism. Sensors. 2017;17(11):2601.CrossRef Hashim IHM, Kumamoto S, Takemura K, Maeno T, Okuda S, Mori Y. Tactile evaluation feedback system for multi-layered structure inspired by human tactile perception mechanism. Sensors. 2017;17(11):2601.CrossRef
52.
go back to reference Zhang T, Zhang X. Injection of traditional hand-crafted features into modern CNN-based models for SAR ship classification: what, why, where, and how. Remote Sens. 2021;13(11):2091.CrossRef Zhang T, Zhang X. Injection of traditional hand-crafted features into modern CNN-based models for SAR ship classification: what, why, where, and how. Remote Sens. 2021;13(11):2091.CrossRef
53.
go back to reference Saba T, Mohamed AS, El-Affendi M, Amin J, Sharif M. Brain tumor detection using fusion of hand crafted and deep learning features. Cogn Syst Res. 2020;59:221–30.CrossRef Saba T, Mohamed AS, El-Affendi M, Amin J, Sharif M. Brain tumor detection using fusion of hand crafted and deep learning features. Cogn Syst Res. 2020;59:221–30.CrossRef
54.
go back to reference Khan MA, Sharif M, Akram T, Raza M, Saba T, Rehman A. Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Appl Soft Comput. 2020;87: 105986. Khan MA, Sharif M, Akram T, Raza M, Saba T, Rehman A. Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Appl Soft Comput. 2020;87: 105986.
55.
go back to reference Su R, Liu T, Sun C, Jin Q, Jennane R, Wei L. Fusing convolutional neural network features with hand-crafted features for osteoporosis diagnoses. Neurocomputing. 2020;385:300–9.CrossRef Su R, Liu T, Sun C, Jin Q, Jennane R, Wei L. Fusing convolutional neural network features with hand-crafted features for osteoporosis diagnoses. Neurocomputing. 2020;385:300–9.CrossRef
Metadata
Title
Haptic Recognition of Texture Surfaces Using Semi-Supervised Feature Learning Based on Sparse Representation
Authors
Zhiyu Shao
Jiatong Bao
Jingwei Li
Hongru Tang
Publication date
20-05-2023
Publisher
Springer US
Published in
Cognitive Computation / Issue 5/2023
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI
https://doi.org/10.1007/s12559-023-10141-8

Other articles of this Issue 5/2023

Cognitive Computation 5/2023 Go to the issue

Premium Partner