Top

Journal of Intelligent Manufacturing

Published in:

03-05-2023

Visual analytics for digital twins: a conceptual framework and case study

Authors: Hangbin Zheng, Tianyuan Liu, Jiayu Liu, Jinsong Bao

Published in: Journal of Intelligent Manufacturing | Issue 4/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The new generation of intelligent manufacturing systems requires a deep integration of human-cyber-physical spaces. Visual analytics plays a critical role in effectively navigating humans through twin data, enabling them to make better decisions and discover new knowledge, contributing to the evolution of digital twins. However, due to the lack of study of visual analytics components and their inter-relations in the context of the digital twin, the visual analytic process of digital twin systems lacks guidance to fully integrate human domain experience and cognitive abilities into the intelligent decision-making process of digital twin systems. Therefore, this paper proposes a conceptual visual analytics framework for digital twins (DTVA). Moreover, from the perspective of data and models in digital twins, the works concerning key factors of visual analytics, including digital-twin visual representation (DTV), analytical reasoning (DTA), and human–machine interface (DTI), are analyzed from data flow, task flow, and visual flow. At last, a case study of DTVA-based anomaly detection, which focuses on the abnormality detection of crane operation status, is presented. This work can support digital twin visual analytics, put forward ideas for the direction of visual analytics, and provide a promising direction for digital twin visualization services.

previous article Genetic algorithm optimization based on manufacturing prediction for an efficient tolerance allocation approach

next article A deep learning framework for defect prediction based on thermographic in-situ monitoring in laser powder bed fusion

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Abreu, F. H., Soares, A., Paulovich, F. V., & Matwin, S. (2021). A trajectory scoring tool for local anomaly detection in maritime traffic using visual analytics. ISPRS International Journal of Geo-Information, 10(6), 412–433. https://doi.org/10.3390/ijgi10060412CrossRef

Card, M. (1999). Readings in information visualization: Using vision to think. Morgan Kaufmann.

Choi, S. H., Park, K.-B., Roh, D. H., Lee, J. Y., Mohammed, M., Ghasemi, Y., & Jeong, H. (2022). An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation. Robotics and Computer-Integrated Manufacturing, 73, 102258–102277. https://doi.org/10.1016/j.rcim.2021.102258CrossRef

Decisionflow. (2014). Visual analytics for high-dimensional temporal event sequence data. IEEE transactions on visualization and computer graphics, 20(12), 1783–1792. https://doi.org/10.1109/tvcg.2014.2346682

Fan, Y., Yang, J., Chen, J., Hu, P., Wang, X., Xu, J., & Zhou, B. (2021). A digital-twin visualized architecture for flexible manufacturing system. Journal of Manufacturing Systems, 60, 176–201. https://doi.org/10.1016/j.jmsy.2021.05.010CrossRef

Gao, Z., Li, J., Dong, M., Yang, R., & Liu, L. (2022). Human-system interaction based on eye tracking for a virtual workshop. Sustainability, 14(11), 6841–6859. https://doi.org/10.3390/su14116841CrossRef

Grieves, M. (2014). Digital twin: Manufacturing excellence through virtual factory replication. White Paper, 1(2014), 1–7. https://doi.org/10.5281/zenodo.1493930CrossRef

Haber, R. B., & McNabb, D. A. (1990). Visualization idioms: A conceptual model for scientific visualization systems. Visualization in Scientific Computing, 74, 74–93.

Han, Z., Li, Y., Yang, M., Yuan, Q., Ba, L., & Xu, E. (2020). Digital twin-driven 3d visualization monitoring and traceability system for general parts in continuous casting machine. Journal of Advanced Mechanical Design, Systems, and Manufacturing, 14(7), 1–15. https://doi.org/10.1299/jamdsm.2020jamdsm0100CrossRef

He, Y., Guo, J., & Zheng, X. (2018). From surveillance to digital twin: Challenges and recent advances of signal processing for industrial internet of things. IEEE Signal Processing Magazine, 35(5), 120–129. https://doi.org/10.1109/msp.2018.2842228CrossRef

Hu, F., Hao, Q., Sun, Q., Cao, X., Ma, R., Zhang, T., & Lu, J. (2016). Cyberphysical system with virtual reality for intelligent motion recognition and training. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(2), 347–363. https://doi.org/10.1109/tsmc.2016.2560127CrossRef

Jiang, J., Li, H., Mao, Z., Liu, F., Zhang, J., Jiang, Z., & Li, H. (2022). A digital twin auxiliary approach based on adaptive sparse attention network for diesel engine fault diagnosis. Scientific Reports, 12(1), 1–18. https://doi.org/10.1038/s41598-021-04545-5CrossRef

Keim, D., & Zhang, L. (2011). Solving problems with visual analytics: Challenges and applications. Proceedings of the 11th international conference on knowledge management and knowledge technologies (pp. 1–4).

Keim, D. A., Mansmann, F., Schneidewind, J., Thomas, J., & Ziegler, H. (2008). Visual analytics: Scope and challenges. Lecture Notes in Computer Science, 4404, 76–90. https://doi.org/10.1007/978-3-540-71080-6_6CrossRef

Lei, Z., Zhou, H., Hu, W., Liu, G.-P., Guan, S., & Feng, X. (2021). Toward a web-based digital twin thermal power plant. IEEE Transactions on Industrial Informatics, 18(3), 1716–1725. https://doi.org/10.1109/tii.2021.3086149CrossRef

Li, L., Ma, H., Wei, M., Zhang, X., Chen, Q., Xin, Y., et al. (2022). Thermal power plant turbine rotor digital twin automation construction and monitoring system. Mathematical Problems in Engineering, 2022, 1–11. https://doi.org/10.1155/2022/8527281CrossRef

Liu, D., Du, Y., Chai, W., Lu, C., & Cong, M. (2022a). Digital twin and data-driven quality prediction of complex die-casting manufacturing. IEEE Transactions on Industrial Informatics, 18(11), 8119–8128. https://doi.org/10.1109/TII.2022.3168309CrossRef

Liu, J., Zheng, H., Jiang, Y., Liu, T., & Bao, J. (2022b). Visual analytics approach for crane anomaly detection based on digital twin. International conference on cooperative design, visualization and engineering (pp. 1–12).

Liu, M., Li, X., Li, J., Liu, Y., Zhou, B., & Bao, J. (2022c). A knowledge graph-based data representation approach for iiot-enabled cognitive manufacturing. Advanced Engineering Informatics, 51, 101515–101527. https://doi.org/10.1016/j.aei.2021.101515CrossRef

Liu, S., Bao, J., & Zheng, P. (2023). A review of digital twin-driven machining: From digitization to intellectualization. Journal of Manufacturing Systems, 67, 361–378. https://doi.org/10.1016/j.jmsy.2023.02.010CrossRef

Liu, S., Lu, S., Li, J., Sun, X., Lu, Y., & Bao, J. (2021). Machining process-oriented monitoring method based on digital twin via augmented reality. The International Journal of Advanced Manufacturing Technology, 113(11), 3491–3508. https://doi.org/10.1007/s00170-021-06838-5CrossRef

Liu, S., Lu, Y., Li, J., Shen, X., Sun, X., & Bao, J. (2023). A blockchain-based interactive approach between digital twin-based manufacturing systems. Computers & Industrial Engineering, 175, 108827–108841. https://doi.org/10.1016/j.cie.2022.108827CrossRef

Liu, S., Lu, Y., Shen, X., & Bao, J. (2023). A digital thread-driven distributed collaboration mechanism between digital twin manufacturing units. Journal of Manufacturing Systems, 68, 145–159. https://doi.org/10.1016/j.jmsy.2023.02.014CrossRef

Lu, Y., Liu, C., Kevin, I., Wang, K., Huang, H., & Xu, X. (2020). Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robotics and Computer-Integrated Manufacturing, 61, 101837–101851. https://doi.org/10.1016/j.rcim.2019.101837CrossRef

Maciel, F., Lourenço, A., Carvalho, P., & Melo, P. (2017). Visual and interactive concerns for vr applications: A case study. International conference of design, user experience, and usability (pp. 510–523).

Riveiro, M.J. (2011). Visual analytics for maritime anomaly detection (Unpublished doctoral dissertation). Örebro universitet.

Tao, F., Liu, W., Zhang, M., Hu, T., Qi, Q., Zhang, H., & Huang, Z. (2019). Fivedimension digital twin model and its ten applications. Computer Integrated Manufacturing Systems, 25(1), 1–18. https://doi.org/10.1016/b978-0-12-817630-6.00003-5CrossRef

Thomas, J.J. (2005). Illuminating the path:[the research and development agenda for visual analytics]. IEEE Computer Society.

Tong, X., Liu, Q., Pi, S., & Xiao, Y. (2020). Real-time machining data application and service based on IMT digital twin. Journal of Intelligent Manufacturing, 31(5), 1113–1132. https://doi.org/10.1007/s10845-019-01500-0CrossRef

Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021). Deep learning for time series forecasting: A survey. Big Data, 9(1), 3–21. https://doi.org/10.1089/big.2020.0159CrossRef

Venna, S., Gottumukkala, R., & Raghavan, V. (2016). Visual analytic decisionmaking environments for large-scale time-evolving graphs. Handbook of statistics (Vol. 35, pp. 81–115). Elsevier.

Victor, V. S., Schmeißer, A., Leitte, H., & Gramsch, S. (2022). Visual parameter space analysis for optimizing the quality of industrial nonwovens. IEEE Computer Graphics and Applications, 42(2), 56–67. https://doi.org/10.1109/MCG.2022.3155867CrossRef

Wu, M., Su, W., Chen, L., Liu, Z., Cao, W., & Hirota, K. (2019). Weight-adapted convolution neural network for facial expression recognition in human-robot interaction. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(3), 1473–1484. https://doi.org/10.1007/978-3-030-61577-2_5CrossRef

Xiong, J., Hsiang, E.-L., He, Z., Zhan, T., & Wu, S.-T. (2021). Augmented reality and virtual reality displays: Emerging technologies and future perspectives. Light: Science & Applications, 10(1), 216–246. https://doi.org/10.1038/s41377-021-00658-8CrossRef

Xu, K., Wang, Y., Yang, L., Wang, Y., Qiao, B., Qin, S., & Qu, H. (2019). Clouddet: Interactive visual analysis of anomalous performances in cloud computing systems. IEEE Transactions on Visualization and Computer Graphics, 26(1), 1107–1117. https://doi.org/10.1109/TVCG.2019.2934613CrossRef

Xu, P., Mei, H., Ren, L., & Chen, W. (2016). Vidx: Visual diagnostics of assembly line performance in smart factories. IEEE Transactions on Visualization and Computer Graphics, 23(1), 291–300. https://doi.org/10.1109/TVCG.2016.2598664CrossRef

Yi, J. S., Ah Kang, Y., Stasko, J., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1224–1231. https://doi.org/10.1109/TVCG.2007.70515CrossRef

Zhan, P., Wang, S., Wang, J., Qu, L., Wang, K., Hu, Y., & Li, X. (2021). Temporal anomaly detection on iiot-enabled manufacturing. Journal of Intelligent Manufacturing, 32(6), 1669–1678. https://doi.org/10.1007/s10845-021-01768-1CrossRef

Zhang, T., Ma, Y., Xu, D., Liu, C., & Ding, J. (2020). ihppvis: Interactive visual analysis of industrial data in heavy plate production. IFAC-PapersOnLine, 53(2), 12050–12055. https://doi.org/10.1016/j.ifacol.2020.12.745CrossRef

Zhang, Y., Zhang, Y., Xia, W., Wu, B., & Zhang, D. (2021). Monitoring and visual analysis of processing data based on time-position mapping. China Mechanical Engineering, 32(20), 2449–2457. https://doi.org/10.3969/j.issn.1004-132X.2021.20.007CrossRef

Zhou, J., Zhou, Y., Wang, B., & Zang, J. (2019). Human-cyber-physical systems (hcpss) in the context of new-generation intelligent manufacturing. Engineering, 5(4), 624–636. https://doi.org/10.1016/j.eng.2019.07.015CrossRef

Zhou, X., Xu, X., Liang, W., Zeng, Z., Shimizu, S., Yang, L. T., & Jin, Q. (2022). Intelligent small object detection for digital twin in smart manufacturing with industrial cyber-physical systems. IEEE Transactions on Industrial Informatics, 18(2), 1377–1386. https://doi.org/10.1109/TII.2021.3061419CrossRef

Zhu, Z., Xi, X., Xu, X., & Cai, Y. (2021). Digital twin-driven machining process for thinwalled part manufacturing. Journal of Manufacturing Systems, 59, 453–466. https://doi.org/10.1016/j.jmsy.2021.03.015CrossRef

Židek, K., Pitel’, J., Adámek, M., Lazorík, P., & Hošovský, A. (2020). Digital twin of experimental smart manufacturing assembly system for industry 4.0 concept. Sustainability, 12(9), 3658–3674. https://doi.org/10.3390/su12093658CrossRef

Title: Visual analytics for digital twins: a conceptual framework and case study
Authors: Hangbin Zheng
Tianyuan Liu
Jiayu Liu
Jinsong Bao
Publication date: 03-05-2023
Publisher: Springer US
Published in: Journal of Intelligent Manufacturing / Issue 4/2024
Print ISSN: 0956-5515
Electronic ISSN: 1572-8145
DOI: https://doi.org/10.1007/s10845-023-02135-y

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 4/2024

A novel 2.5D machining feature recognition method based on ray blanking algorithm

A deep learning framework for defect prediction based on thermographic in-situ monitoring in laser powder bed fusion

A fast spatio-temporal temperature predictor for vacuum assisted resin infusion molding process based on deep machine learning modeling

Data-driven indirect punch wear monitoring in sheet-metal stamping processes

Process parameter optimization for reproducible fabrication of layer porosity quality of 3D-printed tissue scaffold

Developing an explainable hybrid deep learning model in digital transformation: an empirical study

Premium Partners