Advanced
search

To read this content please select one of the options below:

Multi-modal haptic image recognition based on deep learning

Dong Han (State Key Laboratory of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, Nanjing, China)
Hong Nie (State Key Laboratory of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, Nanjing, China)
Jinbao Chen (State Key Laboratory of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, Nanjing, China)
Meng Chen (Aerospace System Engineering Shanghai, Shanghai, China)
Zhen Deng (Department of Informatics, Institute of Technical Aspects of Multimodal Systems, University of Hamburg, Hamburg, Germany)
Jianwei Zhang (Department of Informatics, Institute of Technical Aspects of Multimodal Systems, University of Hamburg, Hamburg, Germany)

Sensor Review

ISSN: 0260-2288

Article publication date: 8 February 2018

Issue publication date: 3 July 2018

494

Abstract

Purpose

This paper aims to improve the diversity and richness of haptic perception by recognizing multi-modal haptic images.

Design/methodology/approach

First, the multi-modal haptic data collected by BioTac sensors from different objects are pre-processed, and then combined into haptic images. Second, a multi-class and multi-label deep learning model is designed, which can simultaneously learn four haptic features (hardness, thermal conductivity, roughness and texture) from the haptic images, and recognize objects based on these features. The haptic images with different dimensions and modalities are provided for testing the recognition performance of this model.

Findings

The results imply that multi-modal data fusion has a better performance than single-modal data on tactile understanding, and the haptic images with larger dimension are conducive to more accurate haptic measurement.

Practical implications

The proposed method has important potential application in unknown environment perception, dexterous grasping manipulation and other intelligent robotics domains.

Originality/value

This paper proposes a new deep learning model for extracting multiple haptic features and recognizing objects from multi-modal haptic images.

Keywords

Acknowledgements

The authors would like to thank the anonymous reviewers for their critical and constructive review of the manuscript. This study was supported by Funding of Jiangsu Innovation Program for Graduate Edu- cation (no. KYLX16_0388 ), the Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China (no. 51675264 ) and Open Foundation of Shanghai Key Laboratory of Space- craft Mechanism.

Citation

Han, D., Nie, H., Chen, J., Chen, M., Deng, Z. and Zhang, J. (2018), "Multi-modal haptic image recognition based on deep learning", Sensor Review, Vol. 38 No. 4, pp. 486-493. https://doi.org/10.1108/SR-08-2017-0160

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited

Related articles

Manage cookies

We’re listening — tell us what you think

Join us on our journey