nach oben

Erschienen in:

25.03.2022 | Original Article

Control of adaptive running platform based on machine vision technologies and neural networks

verfasst von: Artem D. Obukhov, Mikhail N. Krasnyanskiy, Denis L. Dedov, Victoria V. Vostrikova, Daniil V. Teselkin, Ekaterina O. Surkova

Erschienen in: Neural Computing and Applications | Ausgabe 15/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The paper considers the problem of selection of the optimal method for controlling an adaptive running platform for movement organization in virtual reality. The analysis of existing approaches for the control of such systems is carried out, a list of existing methods is formed, and new functions are developed. In order to solve the problem of user positioning within the framework of an adaptive running platform, two approaches, based on virtual reality trackers and using a machine vision, are considered and implemented. The problem of the study includes the choice of the optimal method of controlling an adaptive running platform to ensure maximum user comfort when walking on it. The study methodology includes the creation of a simplified model of human interaction with a running platform, the formalization of the assessment of the quality of movement, various methods of human positioning and platform control functions, followed by an experimental part on their comparison and the search for an optimal approach to running platform control. It was proved that machine vision technologies and neural networks allow positioning a person with sufficient accuracy, and they are deprived of the disadvantages of trackers. Moreover, comparative studies of six different control functions were carried out with the tracker-based positioning method and with the use of machine vision technologies. The most universal is the nonlinear one, a detailed zonal and proportional–differential functions are recommended for the tracker-based positioning method, linear and proportional–differential functions are recommended for positioning based on machine vision. The scientific novelty of the study consists in the formalization and comparison of various control methods of adaptive running platforms (based on linear and nonlinear functions, proportional differential law and neural networks), methods of positioning a person on them (using cameras and trackers), which will expand the area of knowledge about the optimal control functions of this class of devices. The practical significance of the research lies in a comprehensive description of the solution of the problems of organizing the control of adaptive running platforms and positioning a person by various methods.

Vorheriger Artikel A hybrid sigma-pi neural network for combined intuitionistic fuzzy time series prediction model

Nächster Artikel Toward multi-category garments virtual try-on method by coarse to fine TPS deformation

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Silva EC, Sirqueira CA, Pinto FG (2021) The immersion in virtual reality of control room activity. In: Black NL, PatrickNeumann W, Noy I (eds) Congress of the international ergonomics association. Springer, Cham, pp 82–88. https://doi.org/10.1007/978-3-030-74608-7_11CrossRef

Demirel D, Hamam A, Scott C, Karaman B, Toker O, Pena L (2021) Towards a new chemistry learning platform with virtual reality and haptics. In: International conference on human-computer interaction. Springer, Cham, pp 253–267. https://doi.org/10.1007/978-3-030-77943-6_16

Li W, Xiao JX (2021) User experience in digital museums: a case study of the palace museum in Beijing. In: International conference on human-computer interaction. Springer, Cham, pp 436–448. https://doi.org/10.1007/978-3-030-77411-0_28

Khasanova GF, Shageeva FT (2019) Poster: variable scenarios of the VR use in training specialists for chemical industry. In: International conference on interactive collaborative learning. Springer, Cham, pp 808–813. https://doi.org/10.1007/978-3-030-40271-6_79

Mitrut O, Moldoveanu A, Petrescu L, Petrescu C, Moldoveanu F (2021) A review of virtual therapists in anxiety and phobias alleviating applications. In: International conference on human–computer interaction. Springer, Cham, pp 71–79. https://doi.org/10.1007/978-3-030-77599-5_6

Ekechukwu END, Nzeakuba IC, Dada OO, Nwankwo KO, Olowoyo P, Utti VA, Owolabi MO (2021) Virtual reality, a neuroergonomic and neurorehabilitation tool for promoting neuroplasticity in stroke survivors: a systematic review with meta-analysis. In: Congress of the international ergonomics association. Springer, Cham, pp 495–508. https://doi.org/10.1007/978-3-030-74614-8_64

Homayounpour M, Butter D, Vasta S, Merryweather A (2021). Validation of an inverse kinematic VR manikin in seated tasks: application in ergonomics training. In: Congress of the international ergonomics association. Springer, Cham, pp 367–373. https://doi.org/10.1007/978-3-030-74614-8_45

Jiménez MC, James AM, Maureira MAG, Kniestedt I (2017) Dreadful virtualities: a comparative case study of player responses to a horror game in virtual reality and flat screen. In: International conference on advances in computer entertainment. Springer, Cham, pp 239–260. https://doi.org/10.1007/978-3-319-76270-8_17

Lee J (2020) Problem-based gaming via an augmented reality mobile game and a printed game in foreign language education. Educ Inf Technol 27:1–29. https://doi.org/10.1007/s10639-020-10391-1CrossRef

10.

Echtler F, Sturm F, Kindermann K, Klinker G, Stilla J, Trilk J, Najafi H (2004) The intelligent welding gun: augmented reality for experimental vehicle construction. In: Ong SK, Nee AYC (eds) Virtual and augmented reality applications in manufacturing. Springer, London, pp 333–360. https://doi.org/10.1007/978-1-4471-3873-0_17CrossRef

11.

Inoue Y (2012) Virtual reality learning environments. In: Seel NM (ed) Encyclopedia of the sciences of learning. Springer, Boston. https://doi.org/10.1007/978-1-4419-1428-6_651CrossRef

12.

Dhaya R (2020) Improved image processing techniques for user immersion problem alleviation in virtual reality environments. J Innov Image Process 2(02):77–84. https://doi.org/10.36548/jiip.2020.2.002CrossRef

13.

Berna-Moya JL, Martinez-Plasencia D (2019) Exploring the effects of replicating shape, weight and recoil effects on VR shooting controllers. In: IFIP conference on human–computer interaction. Springer, Cham, pp 763–782. https://doi.org/10.1007/978-3-030-29381-9_45

14.

Jia D, Hermans A, Leibe B (2020) DR-SPAAM: a spatial-attention and auto-regressive model for person detection in 2D range data. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 10270–10277. https://doi.org/10.1109/IROS45743.2020.9341689

15.

Hsieh MC, Lee JJ (2018) Preliminary study of VR and AR applications in medical and healthcare education. J Nurs Health Stud 3(1):12. https://doi.org/10.21767/2574-2825.100030CrossRef

16.

Nilsson NC, Serafin S, Steinicke F, Nordahl R (2018) Natural walking in virtual reality: a review. Comput Entertain 16(2):1–22. https://doi.org/10.1145/3180658CrossRef

17.

Habgood MJ, Moore D, Wilson D, Alapont S (2018) Rapid, continuous movement between nodes as an accessible virtual reality locomotion technique. In: 2018 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 371–378. https://doi.org/10.1109/VR.2018.8446130

18.

Krompiec P, Park K (2019) Enhanced player interaction using motion controllers for first-person shooting games in virtual reality. IEEE Access 7:124548–124557. https://doi.org/10.1109/ACCESS.2019.2937937CrossRef

19.

Saredakis D, Szpak A, Birckhead B, Keage HA, Rizzo A, Loetscher T (2020) Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front Hum Neurosci 14:96. https://doi.org/10.1089/g4h.2017.0190CrossRef

20.

Farmani Y, Teather RJ (2018) Viewpoint snapping to reduce cybersickness in virtual reality. In: Proceedings of the 44th graphics interface conference, pp 168–175. https://doi.org/10.20380/GI2018.23

21.

Yildirim C (2020) Don’t make me sick: investigating the incidence of cybersickness in commercial virtual reality headsets. Virtual Real 24(2):231–239. https://doi.org/10.1007/s10055-019-00401-0MathSciNetCrossRef

22.

Han DT, Suhail M, Ragan ED (2018) Evaluating remapped physical reach for hand interactions with passive haptics in virtual reality. IEEE Trans Vis Comput Graph 24(4):1467–1476. https://doi.org/10.1109/TVCG.2018.2794659CrossRef

23.

Winter C, Kern F, Gall D et al (2021) Immersive virtual reality during gait rehabilitation increases walking speed and motivation: a usability evaluation with healthy participants and patients with multiple sclerosis and stroke. J Neuroeng Rehabil 18:68. https://doi.org/10.1186/s12984-021-00848-wCrossRef

24.

Andersen K, Gaab SJ, Sattarvand J, Harris FC (2020) METS VR: mining evacuation training simulator in virtual reality for underground mines. In: 17th international conference on information technology—new generations (ITNG 2020). Springer, Cham, pp 325–332. https://doi.org/10.1007/978-3-030-43020-7_43

25.

Bouguila L, Evequoz F, Courant M, Hirsbrunner B (2004) Walking-pad: a step-in-place locomotion interface for virtual environments. In: Proceedings of the 6th international conference on multimodal interfaces, pp 77–81. https://doi.org/10.1145/1027933.1027948

26.

Solanki D, Kumar S, Raj P, Lahiri U (2019) Body weight support assisted virtual reality based treadmill walk with gait characterization. In: 2019 10th international conference on computing, communication and networking technologies (ICCCNT). IEEE, pp 1–7. https://doi.org/10.1109/ICCCNT45670.2019.8944611

27.

Lidia-Cristina B, Horatiu R, Nicu-George B (2021) Design of improved mechatronic system for Virtusphere. In: 2021 international conference on applied and theoretical electricity (ICATE). IEEE, pp 1–4. https://doi.org/10.1109/ICATE49685.2021.9464938

28.

Bozgeyikli E (2018) Locomotion in virtual reality video games. In: Lee N (ed) Encyclopedia of computer graphics and games. Springer, Cham. https://doi.org/10.1007/978-3-319-08234-9_186-1CrossRef

29.

Kemeny A, Chardonnet JR, Colombet F (2020) Visualization and motion systems. In: Getting rid of cybersickness. Springer, Cham. https://doi.org/10.1007/978-3-030-59342-1_3

30.

Warren LE, Bowman DA (2017) User experience with semi-natural locomotion techniques in virtual reality: the case of the Virtuix Omni. In: Proceedings of the 5th symposium on spatial user interaction, pp 163–163. https://doi.org/10.1145/3131277.3134359

31.

Obukhov A, Siukhin A, Dedov D (2020) The model of the automatic control system for a treadmill based on neural networks. In: 2020 international multi-conference on industrial engineering and modern technologies (FarEastCon). IEEE, pp 1–5. https://doi.org/10.1109/FarEastCon50210.2020.9271589

32.

Mittal R, Karre SA, Reddy YR (2021) Designing limitless path in virtual reality environment. In: Chen JYC, Fragomeni G (eds) Virtual, augmented and mixed reality. HCII 2021. Lecture notes in computer science, vol 12770. Springer, Cham. https://doi.org/10.1007/978-3-030-77599-5_7CrossRef

33.

De Regt A, Barnes SJ, Plangger K (2020) The virtual reality value chain. Bus Horiz 63(6):737–748. https://doi.org/10.1016/j.bushor.2020.08.002CrossRef

34.

Auralius M, Yoon JW (2011) An automatic speed control system of a treadmill with ultrasonic sensors. J Inst Control Robot Syst 17(5):505–511. https://doi.org/10.5302/J.ICROS.2011.17.5.505CrossRef

35.

Qi Y, Soh CB, Gunawan E, Low KS (2014) Ambulatory measurement of three-dimensional foot displacement during treadmill walking using wearable wireless ultrasonic sensor network. IEEE J Biomed Health Inform 19(2):446–452. https://doi.org/10.1109/JBHI.2014.2316998CrossRef

36.

Gembalczyk G, Duda S, Kciuk S, Gąsiorek D, Mężyk A (2019) Mechatronic treadmill for gait reeducation with control algorithm of treadmill speed adaptation. Proc Inst Mech Eng C J Mech Eng Sci 233(7):2239–2247. https://doi.org/10.1177/0954406218790279CrossRef

37.

Wu M, George Hornby T, Landry JM, Roth H, Schmit BD (2011) A cable-driven locomotor training system for restoration of gait in human SCI. Gait Posture 33(2):256–260. https://doi.org/10.1016/j.gaitpost.2010.11.016CrossRef

38.

Caserman P, Garcia-Agundez A, Konrad R, Göbel S, Steinmetz R (2019) Real-time body tracking in virtual reality using a Vive tracker. Virtual Real 23(2):155–168. https://doi.org/10.1007/s10055-018-0374-zCrossRef

39.

Yoon J, Asl H, Pyo S (2018) Intelligent control scheme to facilitate abrupt stopping on self-adjustable treadmills. In: IEEE international conference on robotics and automation, pp 1639–1644. https://doi.org/10.1109/ICRA.2018.8462897

40.

Kamkar S, Ghezloo F, Moghaddam HA, Borji A, Lashgari R (2020) Multiple-target tracking in human and machine vision. PLoS Comput Biol. https://doi.org/10.1371/journal.pcbi.1007698CrossRef

41.

Barris S, Button C (2008) A review of vision-based motion analysis in sport. Sports Med 38:1025–1043. https://doi.org/10.2165/00007256-200838120-00006CrossRef

42.

Rosdi MHAB, Ghani ASA (2022) investigation on accuracy of sensors in sensor fusion for object detection of autonomous vehicle based on 2D lidar and ultrasonic sensor. In: Recent trends in mechatronics towards industry 4.0. Springer, Singapore, pp 761–770.https://doi.org/10.1007/978-981-33-4597-3_68

43.

Gauglitz G (2005) Direct optical sensors: principles and selected applications. Anal Bioanal Chem 381:141–155. https://doi.org/10.1007/s00216-004-2895-4CrossRef

44.

Ikbal MS, Ramadoss V, Zoppi M (2020) Dynamic pose tracking performance evaluation of HTC vive virtual reality system. IEEE Access 9:3798–3815. https://doi.org/10.1109/ACCESS.2020.3047698CrossRef

45.

Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recogn 36(3):585–601. https://doi.org/10.1016/S0031-3203(02)00100-0CrossRef

46.

Tsutsui H, Miura J, Shirai Y (2001) Optical flow-based person tracking by multiple cameras. In: Conference documentation international conference on multisensor fusion and integration for intelligent systems. MFI 2001 (Cat. No. 01TH8590). IEEE, pp 91–96. https://doi.org/10.1109/MFI.2001.1013514.

47.

Wang X, Garg S, Tran SN, Bai Q, Alty J (2021) Hand tremor detection in videos with cluttered background using neural network based approaches. Health Inf Sci Syst 9(1):1–14. https://doi.org/10.1007/s13755-021-00159-3CrossRef

48.

Wu Y, Lin Q, Yang M, Liu J, Tian J, Kapil D, Vanderbloemen L (2022) A computer vision-based yoga pose grading approach using contrastive skeleton feature representations. Healthcare 10(1):36. https://doi.org/10.3390/healthcare10010036CrossRef

49.

Cao Z, Simon T, Wei SE, Sheikh Y (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7291–7299. https://doi.org/10.1109/CVPR.2017.143

50.

Pyo S, Lee H, Yoon J (2021) Development of a novel omnidirectional treadmill-based locomotion interface device with running capability. Appl Sci 11(9):4223CrossRef

51.

Souman JL, Giordano PR, Frissen I, Luca AD, Ernst MO (2010) Making virtual walking real: perceptual evaluation of a new treadmill control algorithm. ACM Trans Appl Percept 7(2):1–14CrossRef

52.

Wang W, Yang K, Zhu Y, Mu H (2020) Speed adaptation and acceleration ripple suppression of treadmill user system using a virtual force moment balance model. Trans Inst Meas Control 42(2):322–329. https://doi.org/10.1177/0142331219868924CrossRef

53.

Bowtell MV, Tan H, Wilson AM (2009) The consistency of maximum running speed measurements in humans using a feedback-controlled treadmill, and a comparison with maximum attainable speed during overground locomotion. J Biomech 42(15):2569–2574. https://doi.org/10.1016/j.jbiomech.2009.07.024CrossRef

54.

Zhou Y, Chen S (2020) Fuzzy control for treadmill permanent magnet synchronous motor speed system. In: 2020 5th international conference on advanced robotics and mechatronics (ICARM). IEEE, pp 646–651. https://doi.org/10.1109/ICARM49381.2020.9195339

55.

Weng K, Turk B, Dolores L, Nguyen TN, Celler B, Su S, Nguyen HT (2010) Fast tracking of a given heart rate profile in treadmill exercise. In: 2010 annual international conference of the IEEE engineering in medicine and biology. IEEE, pp 2569–2572. https://doi.org/10.1109/IEMBS.2010.5626650

56.

Kim J, Chang M, Jeon D (2018) The AI supervisor for the effective treadmill training system of rehabilitation and exercise. In: International conference on NeuroRehabilitation. Springer, Cham, pp 195–199. https://doi.org/10.1007/978-3-030-01845-0_39

57.

Li L, Luo M (2020) Research on an improved single neuron PI control strategy. In: 2020 international conference on computer network, electronic and automation (ICCNEA). IEEE, pp 409–413. : https://doi.org/10.1109/ICCNEA50255.2020.00090

58.

Manju A, Valarmathie P (2021) Video analytics for semantic substance extraction using OpenCV in python. J Ambient Intell Humaniz Comput 12(3):4057–4066. https://doi.org/10.1007/s12652-020-01780-yCrossRef

59.

Brdjanin A, Dardagan N, Dzigal D, Akagic A (2020) Single object trackers in OpenCV: a benchmark. In: 2020 international conference on INnovations in Intelligent SysTems and Applications (INISTA). IEEE, pp 1–6

60.

Hartley R, Li H (2020) Stereovision. In: Ang M, Khatib O, Siciliano B (eds) Encyclopedia of robotics. Springer, Berlin. https://doi.org/10.1007/978-3-642-41610-1_100-1CrossRef

61.

Ahmadyan A, Hou T, Wei J, Zhang L, Ablavatski A, Grundmann M (2020) Instant 3D object tracking with applications in augmented reality. arXiv preprint arXiv:2006.13194.

62.

Kim S-h, Hwang Y (2021) A survey on deep learning based methods and datasets for monocular 3D object detection. Electronics 10(4):517. https://doi.org/10.3390/electronics10040517CrossRef

63.

Ahmadyan A, Zhang L, Ablavatski A, Wei J, Grundmann M (2021) Objectron: a large scale dataset of object-centric videos in the wild with pose annotations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7822–7831. https://doi.org/10.1109/CVPR46437.2021.00773

64.

Obukhov AD, Krasnyanskiy MN (2021) Neural network method for automatic data generation in adaptive information systems. Neural Comput Appl. https://doi.org/10.1007/S00521-021-06169-2CrossRef

Titel: Control of adaptive running platform based on machine vision technologies and neural networks
verfasst von: Artem D. Obukhov
Mikhail N. Krasnyanskiy
Denis L. Dedov
Victoria V. Vostrikova
Daniil V. Teselkin
Ekaterina O. Surkova
Publikationsdatum: 25.03.2022
Verlag: Springer London
Erschienen in: Neural Computing and Applications / Ausgabe 15/2022
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-022-07166-9

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 15/2022

Research on the effect of government media and users’ emotional experience based on LSTM deep neural network

Human motion tracking and 3D motion track detection technology based on visual information features and machine learning

Enterprise management heterogeneity and enterprise investment behavior based on intelligent scheduling system

Position/force evaluation-based assist-as-needed control strategy design for upper limb rehabilitation exoskeleton

Road rage detection algorithm based on fatigue driving and facial feature point location

A deep learning analysis for the effect of individual player performances on match results

Premium Partner