Top

Published in:

2025 | OriginalPaper | Chapter

MarkAR: Exploring the Benefits of Combining Microgestures and Mid-Air Marks to Trigger Commands in Augmented Reality

Authors : Charles Bailly, Lucas Pometti, Julien Castet

Published in: Virtual Reality and Mixed Reality

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Immersive technologies like Augmented Reality (AR) have a promising potential for group activities. Collaborative AR systems allow co-located and/or distant users to create, visualize and manipulate together a large variety of virtual content in order to take actions and decisions. However, contrary to traditional 2D interfaces, there is currently no standard interface or generalized interaction technique for AR. This paper explores an approach for triggering commands linked to AR content. We propose MarkAR, a system combining virtual thumbnail representations of AR content (called vignettes), a microgesture and 3D mid-air mark system inspired by marking menus. Experimental results from a qualitative study highlight that MarkAR offers a great usability, subtle interactions and an easy way to trigger both discrete and continuous commands in AR.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter A Tangible Interface for Creating Virtual Cutaways in Mixed Reality

next chapter Visual Search in People with Macular Degeneration: A Virtual Reality Eye-Tracking Study

https://www.microsoft.com/fr-fr/hololens.

https://www.apple.com/apple-vision-pro/.

Bonilla Fominaya, A.M., et al.: MoonBuddy: a voice-based augmented reality user interface that supports astronauts during extravehicular activities. In: Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST 2022 Adjunct, pp. 1–4. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3526114.3558690. https://dl.acm.org/doi/10.1145/3526114.3558690

Broll, W., et al.: ARTHUR: a collaborative augmented environment for architectural design and urban planning. JVRB - J. Virtual Real. Broadcasting 1(2004)(1) (2004). https://doi.org/10.20385/1860-2037/1.2004.1. http://www.jvrb.org/past-issues/1.2004/34

Chaffangeon Caillet, A., Goguey, A., Nigay, L.: \(\upmu \)Glyph: a microgesture notation. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, pp. 1–28. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3544548.3580693. https://dl.acm.org/doi/10.1145/3544548.3580693

Chan, E., Seyed, T., Stuerzlinger, W., Yang, X.D., Maurer, F.: User elicitation on single-hand microgestures. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 2016, pp. 3403–3414. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2858036.2858589

Constantinides, M., Quercia, D.: The future of hybrid meetings. In: Proceedings of the 1st Annual Meeting of the Symposium on Human-Computer Interaction for Work, CHIWORK 2022, pp. 1–6. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3533406.3533415. https://dl.acm.org/doi/10.1145/3533406.3533415

Ens, B., et al.: Grand challenges in immersive analytics. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama Japan, pp. 1–17. ACM (2021). https://doi.org/10.1145/3411764.3446866. https://dl.acm.org/doi/10.1145/3411764.3446866

Ens, B., Quigley, A., Yeo, H.S., Irani, P., Piumsomboon, T., Billinghurst, M.: Counterpoint: exploring mixed-scale gesture interaction for AR applications. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA 2018, pp. 1–6. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3170427.3188513

Francese, R., Passero, I., Zarraonandia, T.: An augmented reality application to gather participant feedback during a meeting. In: De Marco, M., Te’eni, D., Albano, V., Za, S. (eds.) Information Systems: Crossroads for Organization, Management, Accounting and Engineering, pp. 173–180. Physica-Verlag HD, Heidelberg (2012). https://doi.org/10.1007/978-3-7908-2789-7_20CrossRef

Gronier, G., Baudet, A.: Psychometric evaluation of the F-SUS: creation and validation of the French version of the system usability scale. Int. J. Hum.-Comput. Interact. 37(16), 1571–1582 (2021). https://doi.org/10.1080/10447318.2021.1898828CrossRef

10.

Hansberger, J.T., et al.: Dispelling the gorilla arm syndrome: the viability of prolonged gesture interactions. In: Lackey, S., Chen, J. (eds.) VAMR 2017. LNCS, vol. 10280, pp. 505–520. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57987-0_41CrossRef

11.

Huynh, D.N.T., Raveendran, K., Xu, Y., Spreen, K., MacIntyre, B.: Art of defense: a collaborative handheld augmented reality board game. In: Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games, Sandbox 2009, pp. 135–142. Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1581073.1581095. https://dl.acm.org/doi/10.1145/1581073.1581095

12.

Kim, S., Lee, G., Huang, W., Kim, H., Woo, W., Billinghurst, M.: Evaluating the combination of visual communication cues for HMD-based mixed reality remote collaboration. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 1–13. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300403. https://dl.acm.org/doi/10.1145/3290605.3300403

13.

Kulshreshth, A., LaViola, J.J.: Exploring the usefulness of finger-based 3D gesture menu selection. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Ontario, Canada, pp. 1093–1102. ACM (2014). https://doi.org/10.1145/2556288.2557122. https://dl.acm.org/doi/10.1145/2556288.2557122

14.

Lambert, V., Chaffangeon Caillet, A., Goguey, A., Malacria, S., Nigay, L.: Studying the visual representation of microgestures. Proc. ACM Hum.-Comput. Interact. 7(MHCI), 1–36 (2023). https://doi.org/10.1145/3604272. https://dl.acm.org/doi/10.1145/3604272

15.

Li, G., Rempel, D., Liu, Y., Song, W., Adamson, C.H.: Design of 3D microgestures for commands in virtual reality or augmented reality. Appl. Sci. 11(14), 6375 (2021). https://doi.org/10.3390/app11146375CrossRef

16.

Lim, Z.H., Kristensson, P.O.: An evaluation of discrete and continuous mid-air loop and marking menu selection in optical see-through HMDs. In: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, Taipei Taiwan, pp. 1–10. ACM (2019). https://doi.org/10.1145/3338286.3340127. https://dl.acm.org/doi/10.1145/3338286.3340127

17.

Liu, M., Nancel, M., Vogel, D.: Gunslinger: subtle arms-down mid-air interaction. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, NC, USA, pp. 63–71. ACM (2015). https://doi.org/10.1145/2807442.2807489. https://dl.acm.org/doi/10.1145/2807442.2807489

18.

Lu, F., Xu, Y.: Exploring spatial UI transition mechanisms with head-worn augmented reality. In: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, pp. 1–16. ACM (2022). https://doi.org/10.1145/3491102.3517723. https://dl.acm.org/doi/10.1145/3491102.3517723

19.

Lukosch, S., Billinghurst, M., Alem, L., Kiyokawa, K.: Collaboration in augmented reality. Comput. Support. Coop. Work (CSCW) 24(6), 515–525 (2015). https://doi.org/10.1007/s10606-015-9239-0CrossRef

20.

Patterson, R., Winterbottom, M.D., Pierce, B.J.: Perceptual issues in the use of head-mounted visual displays. Hum. Factors J. Hum. Factors Ergon. Soc. 48(3), 555–573 (2006). https://doi.org/10.1518/001872006778606877CrossRef

21.

Pfeuffer, K., et al.: PalmGazer: unimanual eye-hand menus in augmented reality. In: Proceedings of the 2023 ACM Symposium on Spatial User Interaction, SUI 2023, pp. 1–12. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3607822.3614523. https://dl.acm.org/doi/10.1145/3607822.3614523

22.

Piumsomboon, T., et al.: Mini-me: an adaptive avatar for mixed reality remote collaboration. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, pp. 1–13. ACM (2018). https://doi.org/10.1145/3173574.3173620. https://dl.acm.org/doi/10.1145/3173574.3173620

23.

Pourmemar, M., Poullis, C.: Visualizing and interacting with hierarchical menus in immersive augmented reality. In: Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, VRCAI 2019, pp. 1–9. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3359997.3365693. https://dl.acm.org/doi/10.1145/3359997.3365693

24.

Radu, I., Joy, T., Bowman, Y., Bott, I., Schneider, B.: A survey of needs and features for augmented reality collaborations in collocated spaces. Proc. ACM Hum.-Comput. Interact. 5(CSCW1), 1–21 (2021). https://doi.org/10.1145/3449243CrossRef

25.

Rajaram, S., Chen, C., Roesner, F., Nebeling, M.: Eliciting security & privacy-informed sharing techniques for multi-user augmented reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, pp. 1–17. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3544548.3581089. https://dl.acm.org/doi/10.1145/3544548.3581089

26.

Regenbrecht, H., Wagner, M., Baratoff, G.: MagicMeeting: a collaborative tangible augmented reality system. Virtual Reality 6(3), 151–166 (2002). https://doi.org/10.1007/s100550200016CrossRef

27.

Ren, G., O’Neill, E.: 3D marking menu selection with freehand gestures. In: 2012 IEEE Symposium on 3D User Interfaces (3DUI), pp. 61–68 (2012). https://doi.org/10.1109/3DUI.2012.6184185. https://ieeexplore.ieee.org/document/6184185

28.

Schön, D., et al.: Tailor twist: assessing rotational mid-air interactions for augmented reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, pp. 1–14. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3544548.3581461. https://dl.acm.org/doi/10.1145/3544548.3581461

29.

Sharma, A., Roo, J.S., Steimle, J.: Grasping microgestures: eliciting single-hand microgestures for handheld objects. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 1–13. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290605.3300632

30.

Speicher, M., Hall, B.D., Nebeling, M.: What is mixed reality? In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, pp. 1–15. ACM (2019). https://doi.org/10.1145/3290605.3300767. https://dl.acm.org/doi/10.1145/3290605.3300767

31.

White, S., Feng, D., Feiner, S.: Interaction and presentation techniques for shake menus in tangible augmented reality. In: Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2009, pp. 39–48. IEEE Computer Society, USA (2009). https://doi.org/10.1109/ISMAR.2009.5336500

32.

Xu, Z., et al.: TipText: eyes-free text entry on a fingertip keyboard. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, pp. 883–899. ACM (2019). https://doi.org/10.1145/3332165.3347865

33.

Zhang, L., Agrawal, A., Oney, S., Guo, A.: VRGit: a version control system for collaborative content creation in virtual reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, pp. 1–14. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3544548.3581136. https://dl.acm.org/doi/10.1145/3544548.3581136

34.

Zhao, S., Balakrishnan, R.: Simple vs. compound mark hierarchical marking menus. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, UIST 2004, pp. 33–42. Association for Computing Machinery, New York (2004). https://doi.org/10.1145/1029632.1029639. https://dl.acm.org/doi/10.1145/1029632.1029639

35.

Zhu, Y., Li, N.: Virtual and augmented reality technologies for emergency management in the built environments: a state-of-the-art review. J. Saf. Sci. Resilience 2(1), 1–10 (2021). https://doi.org/10.1016/j.jnlssr.2020.11.004. https://www.sciencedirect.com/science/article/pii/S266644962030030X

Title: MarkAR: Exploring the Benefits of Combining Microgestures and Mid-Air Marks to Trigger Commands in Augmented Reality
Authors: Charles Bailly
Lucas Pometti
Julien Castet
Publisher: Springer Nature Switzerland
Book: Virtual Reality and Mixed Reality
Print ISBN: 978-3-031-78592-4

Electronic ISBN: 978-3-031-78593-1

Copyright Year: 2025
DOI: https://doi.org/10.1007/978-3-031-78593-1_11

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"