Top

Journal of Intelligent Manufacturing

Published in:

02-12-2021

Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor

Authors: Huitaek Yun, Hanjun Kim, Young Hun Jeong, Martin B. G. Jun

Published in: Journal of Intelligent Manufacturing | Issue 3/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Sound and vibration analysis are prominent tools for machine health diagnosis. Especially, neural network (NN) strategies have focused on finding complex and nonlinear relationships between the sensor signal and the machine status to detect machine faults. However, it is difficult to collect enough amount of fault data as much as normal status data for training general NN models. To resolve the issue, this paper proposes the autoencoder-based anomaly detection framework for industrial robot arms using an internal sound sensor. The autoencoder uses signals in the normal state of the robots for training the model. It reconstructs the input signals as output, and anomalous states are found from high reconstruction error. Two stethoscopes were attached to the surface of the robot joint as sensors, and the sounds were recorded by USB microphone attached to the outlet of the stethoscopes. Features were extracted from STFT spectrogram images of the gathered sound, then used to train and test an autoencoder model. The reconstruction errors of the autoencoder were compared to distinguish the abnormal status from normal one. The experimental results suggest that the stethoscopes prevent the interference of noise, and the collected sound signals can be utilized for detecting machine anomalies.

previous article Texture surface defect detection of plastic relays with an enhanced feature pyramid network

next article Reconfigurability improvement in Industry 4.0: a hybrid genetic algorithm-based heuristic approach for a co-generation of setup and process plans in a reconfigurable environment

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Al-Ghamd, A. M., & Mba, D. (2006). A comparative experimental study on the use of acoustic emission and vibration analysis for bearing defect identification and estimation of defect size. Mechanical Systems and Signal Processing, 20(7), 1537–1571. https://doi.org/10.1016/j.ymssp.2004.10.013CrossRef

Bibaeva, V. (2018). Using metaheuristics for hyper-parameter optimization of convolutional neural networks. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP) (pp. 1–6). https://doi.org/10.1109/MLSP.2018.8516989

Bittencourt, A. C., & Gunnarsson, S. (2012). Static friction in a robot joint—modeling and identification of load and temperature effects. Journal of Dynamic Systems, Measurement, and Control. https://doi.org/10.1115/1.4006589CrossRef

Chamberlain, D., Mofor, J., Fletcher, R., & Kodgule, R. (2015). Mobile stethoscope and signal processing algorithms for pulmonary screening and diagnostics. In 2015 IEEE Global Humanitarian Technology Conference (GHTC) (pp. 385–392). https://doi.org/10.1109/GHTC.2015.7344001

Chebil, J., Noel, G., Mesbah, M., & Deriche, M. (2009). Wavelet decomposition for the detection and diagnosis of faults in rolling element bearings. Jordan Journal of Mechanical and Industrial Engineering, 3(4), 260–267.

Cong, F., Chen, J., Dong, G., & Pecht, M. (2013). Vibration model of rolling element bearings in a rotor-bearing system for fault diagnosis. Journal of Sound and Vibration, 332(8), 2081–2097. https://doi.org/10.1016/j.jsv.2012.11.029CrossRef

Dohnal, F., & Sekhar, A. S. (2014). Current signature analysis for unbalance fault detection in a rotor supported by active magnetic bearings. International Journal of Condition Monitoring, 4(1), 2–8. https://doi.org/10.1784/204764214813883315CrossRef

Eren, L., Ince, T., & Kiranyaz, S. (2019). A generic intelligent bearing fault diagnosis system using compact adaptive 1D CNN classifier. Journal of Signal Processing Systems, 91(2), 179–189. https://doi.org/10.1007/s11265-018-1378-3CrossRef

Géron, A. (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. O’Reilly Media.

Gribonval, R. (2008). Linear time‐frequency analysis I: Fourier‐type representations. Time‐Frequency Analysis: Concepts and Methods, 61–91.

Guo, L., Li, N., Jia, F., Lei, Y., & Lin, J. (2017). A recurrent neural network based health indicator for remaining useful life prediction of bearings. Neurocomputing, 240, 98–109. https://doi.org/10.1016/j.neucom.2017.02.045CrossRef

Gupta, C. N., Palaniappan, R., Swaminathan, S., & Krishnan, S. M. (2007). Neural network classification of homomorphic segmented heart sounds. Applied Soft Computing, 7(1), 286–297. https://doi.org/10.1016/j.asoc.2005.06.006CrossRef

Hamidi, M., Ghassemian, H., & Imani, M. (2018). Classification of heart sound signal using curve fitting and fractal dimension. Biomedical Signal Processing and Control, 39, 351–359. https://doi.org/10.1016/j.bspc.2017.08.002CrossRef

Heng, A., Zhang, S., Tan, A. C. C., & Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, 23(3), 724–739. https://doi.org/10.1016/j.ymssp.2008.06.009CrossRef

Henriquez, P., Alonso, J. B., Ferrer, M. A., & Travieso, C. M. (2014). Review of automatic fault diagnosis systems using audio and vibration signals. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 44(5), 642–652. https://doi.org/10.1109/TSMCC.2013.2257752CrossRef

Immovilli, F., Bellini, A., Rubini, R., & Tassoni, C. (2010). Diagnosis of bearing faults in induction machines by vibration or current signals: A critical comparison. IEEE Transactions on Industry Applications, 46(4), 1350–1359. https://doi.org/10.1109/TIA.2010.2049623CrossRef

Jaber, A. A., & Bicker, R. (2016). Fault diagnosis of industrial robot gears based on discrete wavelet transform and artificial neural network. Insight—Non-Destructive Testing and Condition Monitoring, 58(4), 179–186. https://doi.org/10.1784/insi.2016.58.4.179CrossRef

Janssens, O., Slavkovikj, V., Vervisch, B., Stockman, K., Loccufier, M., Verstockt, S., et al. (2016). Convolutional neural network based fault detection for rotating machinery. Journal of Sound and Vibration, 377, 331–345. https://doi.org/10.1016/j.jsv.2016.05.027CrossRef

Jardine, A. K. S., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 20(7), 1483–1510. https://doi.org/10.1016/j.ymssp.2005.09.012CrossRef

Khan, S., & Yairi, T. (2018). A review on the application of deep learning in system health management. Mechanical Systems and Signal Processing, 107, 241–265. https://doi.org/10.1016/j.ymssp.2017.11.024CrossRef

Kingma, D. P., & Ba, J. (2017). Adam: a method for stochastic optimization. arXiv:1412.6980 [cs]. http://arxiv.org/abs/1412.6980. Accessed 31 May 2020

Kramer, M. A. (1991). Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37(2), 233–243. https://doi.org/10.1002/aic.690370209CrossRef

Liu, H., Zhou, J., Zheng, Y., Jiang, W., & Zhang, Y. (2018). Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders. ISA Transactions, 77, 167–178. https://doi.org/10.1016/j.isatra.2018.04.005CrossRef

Liu, Y., & Zhang, Y. (2015). Toward welding robot with human knowledge: A remotely-controlled approach. IEEE Transactions on Automation Science and Engineering, 12(2), 769–774. https://doi.org/10.1109/TASE.2014.2359006CrossRef

Maglogiannis, I., Loukis, E., Zafiropoulos, E., & Stasis, A. (2009). Support vectors machine-based identification of heart valve diseases using heart sounds. Computer Methods and Programs in Biomedicine, 95(1), 47–61. https://doi.org/10.1016/j.cmpb.2009.01.003CrossRef

Martin, H. R., & Honarvar, F. (1995). Application of statistical moments to bearing failure detection. Applied Acoustics, 44(1), 67–77. https://doi.org/10.1016/0003-682X(94)P4420-BCrossRef

Meng, Q., Sen, D., Wang, S., & Hayes, L. (2008). Impulse response measurement with sine sweeps and amplitude modulation schemes. In 2008 2nd International Conference on Signal Processing and Communication Systems (pp. 1–5). https://doi.org/10.1109/ICSPCS.2008.4813749

Mikolajczyk, T. (2012a). Manufacturing using robot. Advanced Materials Research, 463–464, 1643–1646. https://doi.org/10.4028/www.scientific.net/AMR.463-464.1643CrossRef

Mikolajczyk, T. (2012b). System to surface control in robot machining. Advanced Materials Research, 463–464, 708–711. https://doi.org/10.4028/www.scientific.net/AMR.463-464.708CrossRef

Mikolajczyk, T. (2013). Indication of machining area with the robot’s camera using. Applied Mechanics and Materials, 282, 146–151. https://doi.org/10.4028/www.scientific.net/AMM.282.146CrossRef

Mikolajczyk, T. (2015). Control system for industrial robot equipped with tool for advanced task in manufacturing. Applied Mechanics and Materials, 783, 105–113. https://doi.org/10.4028/www.scientific.net/AMM.783.105CrossRef

Oh, D. Y., & Yun, I. D. (2018). Residual error based anomaly detection using auto-encoder in SMD machine sound. Sensors, 18(5), 1308. https://doi.org/10.3390/s18051308CrossRef

Pan, Y., Chen, J., & Guo, L. (2009). Robust bearing performance degradation assessment method based on improved wavelet packet–support vector data description. Mechanical Systems and Signal Processing, 23(3), 669–681. https://doi.org/10.1016/j.ymssp.2008.05.011CrossRef

Park, Y., & Yun, I. D. (2018). Fast adaptive RNN encoder–decoder for anomaly detection in SMD assembly machine. Sensors, 18(10), 3573. https://doi.org/10.3390/s18103573CrossRef

Peng, Z. K., Tse, P. W., & Chu, F. L. (2005). A comparison study of improved Hilbert–Huang transform and wavelet transform: Application to fault diagnosis for rolling bearing. Mechanical Systems and Signal Processing, 19(5), 974–988. https://doi.org/10.1016/j.ymssp.2004.01.006CrossRef

Qi, Y., Wang, Y., Zheng, X., & Wu, Z. (2014). Robust feature learning by stacked autoencoder with maximum correntropy criterion. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6716–6720). https://doi.org/10.1109/ICASSP.2014.6854900

Rai, V. K., & Mohanty, A. R. (2007). Bearing fault diagnosis using FFT of intrinsic mode functions in Hilbert–Huang transform. Mechanical Systems and Signal Processing, 21(6), 2607–2615. https://doi.org/10.1016/j.ymssp.2006.12.004CrossRef

Safizadeh, M. S., & Latifi, S. K. (2014). Using multi-sensor data fusion for vibration fault diagnosis of rolling element bearings by accelerometer and load cell. Information Fusion, 18, 1–8. https://doi.org/10.1016/j.inffus.2013.10.002CrossRef

Shaffer, D., Ragai, I., Danesh-Yazdi, A., & Loker, D. (2018). Investigation of the feasibility of using microphone arrays in monitoring machining conditions. Manufacturing Letters, 15, 132–134. https://doi.org/10.1016/j.mfglet.2017.12.008CrossRef

Shao, H., Jiang, H., Zhao, H., & Wang, F. (2017). A novel deep autoencoder feature learning method for rotating machinery fault diagnosis. Mechanical Systems and Signal Processing, 95, 187–204. https://doi.org/10.1016/j.ymssp.2017.03.034CrossRef

Stojanovic, V., Nedic, N., Prsic, D., Dubonjic, L., & Djordjevic, V. (2016). Application of Cuckoo search algorithm to constrained control problem of a parallel robot platform. The International Journal of Advanced Manufacturing Technology, 87(9), 2497–2507. https://doi.org/10.1007/s00170-016-8627-zCrossRef

Sun, Y., Xue, B., Zhang, M., & Yen, G. G. (2018). An experimental study on hyper-parameter optimization for stacked auto-encoders. In 2018 IEEE Congress on Evolutionary Computation (CEC) (pp. 1–8). https://doi.org/10.1109/CEC.2018.8477921

Tao, H., Wang, P., Chen, Y., Stojanovic, V., & Yang, H. (2020). An unsupervised fault diagnosis method for rolling bearing using STFT and generative neural networks. Journal of the Franklin Institute, 357(11), 7286–7307. https://doi.org/10.1016/j.jfranklin.2020.04.024CrossRef

Tao, S., Zhang, T., Yang, J., Wang, X., & Lu, W. (2015). Bearing fault diagnosis method based on stacked autoencoder and softmax regression. In 2015 34th Chinese Control Conference (CCC) (pp. 6331–6335). https://doi.org/10.1109/ChiCC.2015.7260634

Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning (pp. 1096–1103). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/1390156.1390294

Wang, D., Tse, P. W., & Tsui, K. L. (2013). An enhanced Kurtogram method for fault diagnosis of rolling element bearings. Mechanical Systems and Signal Processing, 35(1), 176–199. https://doi.org/10.1016/j.ymssp.2012.10.003CrossRef

Yun, H., Kim, H., Kim, E., & Jun, M. B. G. (2020). Development of internal sound sensor using stethoscope and its applications for machine monitoring. Procedia Manufacturing, 48, 1072–1078. https://doi.org/10.1016/j.promfg.2020.05.147CrossRef

Zhang, B., Li, W., Hao, J., Li, X.-L., & Zhang, M. (2018). Adversarial adaptive 1-D convolutional neural networks for bearing fault diagnosis under varying working condition. arXiv:1805.00778 [cs, eess]. http://arxiv.org/abs/1805.00778.

Zhou, C., & Paffenroth, R. C. (2017). Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 665–674). Halifax, NS, Canada: Association for Computing Machinery. https://doi.org/10.1145/3097983.3098052

Zhu, D., Feng, X., Xu, X., Yang, Z., Li, W., Yan, S., & Ding, H. (2020). Robotic grinding of complex components: A step towards efficient and intelligent machining–challenges, solutions, and applications. Robotics and Computer-Integrated Manufacturing, 65, 101908. https://doi.org/10.1016/j.rcim.2019.101908CrossRef

Title: Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor
Authors: Huitaek Yun
Hanjun Kim
Young Hun Jeong
Martin B. G. Jun
Publication date: 02-12-2021
Publisher: Springer US
Published in: Journal of Intelligent Manufacturing / Issue 3/2023
Print ISSN: 0956-5515
Electronic ISSN: 1572-8145
DOI: https://doi.org/10.1007/s10845-021-01862-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 3/2023

Autonomous VOs management based on industry 4.0: a systematic literature review

Recognizing defects in stainless steel welds based on multi-domain feature expression and self-optimization

Data-driven models for predictions of geometric characteristics of bead fabricated by selective laser melting

Real-time laser keyhole welding penetration state monitoring based on adaptive fusion images using convolutional neural networks

Simulation and deep reinforcement learning for adaptive dispatching in semiconductor manufacturing systems

Reconfigurability improvement in Industry 4.0: a hybrid genetic algorithm-based heuristic approach for a co-generation of setup and process plans in a reconfigurable environment

Premium Partners