skip to main content
10.1145/3411764.3445193acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Best Paper

How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

Published:07 May 2021Publication History

ABSTRACT

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefits over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate findings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them. We also identify research directions for improving evaluation methods and offer ideas for how to make studies more ecologically valid and rigorous.

Skip Supplemental Material Section

Supplemental Material

References

  1. Ferran Argelaguet and Carlos Andujar. 2013. A survey of 3D object selection techniques for virtual environments. Computers & Graphics 37, 3 (2013), 121–136.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ferran Argelaguet, Carlos Andujar, and Ramon Trueba. 2008. Overcoming eye-hand visibility mismatch in 3D pointing selection. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2008), 43–46. https://doi.org/10.1145/1450579.1450588Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Oscar Ariza, Gerd Bruder, Nicholas Katzakis, and Frank Steinicke. 2018. Analysis of Proximity-Based Multimodal Feedback for 3D Selection in Immersive Virtual Environments. 25th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2018 - Proceedings (2018), 327–334. https://doi.org/10.1109/VR.2018.8446317Google ScholarGoogle ScholarCross RefCross Ref
  4. Marc Baloup, Thomas Pietrzak, and Géry Casiez. 2019. RayCursor: A 3D Pointing Facilitation Technique based on Raycasting. (2019), 1–12. https://doi.org/10.1145/3290605.3300331Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Louise Barkhuus and Jennifer A Rode. 2007. From mice to men-24 years of evaluation in CHI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vol. 10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Anil Ufuk Batmaz and Wolfgang Stuerzlinger. 2019. Effects of 3d rotational jitter and selection methods on 3d pointing tasks. 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings (2019), 1687–1692. https://doi.org/10.1109/VR.2019.8798038Google ScholarGoogle ScholarCross RefCross Ref
  7. Benjamin B Bederson, Benjamin B Bederson, Ben Shneiderman, 2003. The craft of information visualization: readings and reflections. Morgan Kaufmann.Google ScholarGoogle Scholar
  8. Bobby Bodenheimer, Sarah Creem-Regehr, Jeanine Stefanucci, Elena Shemetova, and William B. Thompson. 2017. Prism aftereffects for throwing with a self-avatar in an immersive virtual environment. Proceedings - IEEE Virtual Reality 0 (2017), 141–147. https://doi.org/10.1109/VR.2017.7892241Google ScholarGoogle ScholarCross RefCross Ref
  9. Doug Bowman, Ernst Kruijff, Joseph J LaViola Jr, and Ivan P Poupyrev. 2004. 3D user interfaces: Theory and Practice(1 ed.). Addison-Wesley Professional.Google ScholarGoogle Scholar
  10. Doug A Bowman, Donald B Johnson, and Larry F Hodges. 2001. Testbed evaluation of virtual environment interaction techniques. Presence: Teleoperators & Virtual Environments 10, 1(2001), 75–95.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Doug A Bowman, Ernst Kruijff, Joseph J LaViola Jr, and Ivan Poupyrev. 2001. An introduction to 3-D user interface design. Presence: Teleoperators & Virtual Environments 10, 1(2001), 96–108.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. David Brickler, Matias Volonte, Jeffrey W. Bertrand, Andrew T. Duchowski, and Sabarish V. Babu. 2019. Effects of stereoscopic viewing and haptic feedback, sensory-motor congruence and calibration on near-field fine motor perception-action coordination in virtual reality. 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings (2019), 28–37. https://doi.org/10.1109/VR.2019.8797744Google ScholarGoogle ScholarCross RefCross Ref
  13. John Brooke. 1996. SUS: A ’Quick and Dirty’ Usability Scale. Usability evaluation in industry(1996), 189. https://doi.org/10.1201/9781498710411-35Google ScholarGoogle ScholarCross RefCross Ref
  14. Michael D Byrne, Bonnie E John, Neil S Wehrle, and David C Crow. 1999. The tangled web we wove: A taskonomy of WWW use. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 544–551.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Kelly Caine. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 981–992. https://doi.org/10.1145/2858036.2858498Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Stuart K Card, Thomas P Moran, and Allen Newell. 1983. The psychology of human-computer interaction. 1983.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Emmanuelle Chapoulie, Maud Marchal, Evanthia Dimara, Maria Roussou, Jean Christophe Lombardo, and George Drettakis. 2014. Evaluation of direct manipulation using finger tracking for complex tasks in an immersive cube. Virtual Reality 18, 3 (2014), 203–217. https://doi.org/10.1007/s10055-014-0246-0Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Olivier Chapuis, Renaud Blanch, and Michel Beaudouin-Lafon. 2007. Fitts’ law in the wild: A field study of aimed movements. (2007).Google ScholarGoogle Scholar
  19. Andy Cockburn and Bruce J. McKenzie. 2001. What do web users do? An empirical analysis of web use. Int. J. Hum. Comput. Stud. 54, 6 (2001), 903–922. https://doi.org/10.1006/ijhc.2001.0459Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Nguyen-Thong Dang. 2007. A survey and classification of 3D pointing techniques. In 2007 IEEE international conference on research, innovation and vision for the future. IEEE, 71–80.Google ScholarGoogle ScholarCross RefCross Ref
  21. Henrique G. Debarba, Sami Perrin, Bruno Herbelin, and Ronan Boulic. 2015. Embodied interaction using non-planar projections in immersive virtual reality. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 13-15-November-2015 (2015), 125–128. https://doi.org/10.1145/2821592.2821603Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Sarah A Douglas, Arthur E Kirkpatrick, and I Scott MacKenzie. 1999. Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 215–222.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Gregory W. Edwards, Woodrow Barfield, and Maury A. Nussbaum. 2004. The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment. Virtual Reality 7, 2 (2004), 112–119. https://doi.org/10.1007/s10055-004-0120-6Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement.Journal of Experimental Psychology 47, 6 (1954), 381–391. https://doi.org/10.1037/h0055392Google ScholarGoogle ScholarCross RefCross Ref
  25. Cédric Fleury, Thierry Duval, Valérie Gouranton, and Anthony Steed. 2012. Evaluation of remote collaborative manipulation for scientific data analysis. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2012), 129–136. https://doi.org/10.1145/2407336.2407361Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Scott Frees and G. Drew Kessler. 2005. Precise and rapid interaction through scaled manipulation in immersive virtual environments. Proceedings - IEEE Virtual Reality 2005 (2005), 99–106. https://doi.org/10.1109/vr.2005.60Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Eg Su Goh, Mohd Shahrizal Sunar, and Ajune Wanis Ismail. 2019. 3D object manipulation techniques in handheld mobile augmented reality interface: A review. IEEE Access 7(2019), 40581–40601.Google ScholarGoogle ScholarCross RefCross Ref
  28. Mar Gonzalez-Franco and Tabitha C. Peck. 2018. Avatar Embodiment. Towards a Standardized Questionnaire. Frontiers in Robotics and AI 5 (2018), 74. https://doi.org/10.3389/frobt.2018.00074Google ScholarGoogle ScholarCross RefCross Ref
  29. Jan Gugenheimer, David Dobbelstein, Christian Winkler, Gabriel Haas, and Enrico Rukzio. 2016. FaceTouch: Enabling Touch Interaction in Display Fixed UIs for Mobile Virtual Reality. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 49–60. https://doi.org/10.1145/2984511.2984576Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Chris Hand. 1997. A survey of 3D interaction techniques. In Computer graphics forum, Vol. 16. Wiley Online Library, 269–281.Google ScholarGoogle Scholar
  31. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 – 183. https://doi.org/10.1016/S0166-4115(08)62386-9Google ScholarGoogle ScholarCross RefCross Ref
  32. Rorik Henrikson, Tovi Grossman, Sean Trowbridge, Daniel Wigdor, and Hrvoje Benko. 2020. Head-Coupled Kinematic Template Matching: A Prediction Model for Ray Pointing in VR. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kasper Hornbæk. 2006. Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies 64 (02 2006), 79–102. https://doi.org/10.1016/j.ijhcs.2005.06.002Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kasper Hornbæk, Søren S Sander, Javier Andrés Bargas-Avila, and Jakob Grue Simonsen. 2014. Is once enough? On the extent and content of replications in human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3523–3532.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jacek Jankowski and Martin Hachet. 2015. Advances in interaction with 3D environments. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 152–190.Google ScholarGoogle Scholar
  36. Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3Google ScholarGoogle ScholarCross RefCross Ref
  37. Panayiotis Koutsabasis and Panagiotis Vogiatzidakis. 2019. Empirical research in mid-air interaction: A systematic review. International Journal of Human–Computer Interaction 35, 18(2019), 1747–1768.Google ScholarGoogle Scholar
  38. Joseph J LaViola Jr, Ernst Kruijff, Ryan P McMahan, Doug Bowman, and Ivan P Poupyrev. 2017. 3D user interfaces: theory and practice(2 ed.). Addison-Wesley Professional.Google ScholarGoogle Scholar
  39. Jaeyeon Lee, Mike Sinclair, Mar Gonzalez-Franco, Eyal Ofek, and Christian Holz. 2019. TORC: A virtual reality controller for in-hand high-dexterity finger interaction. Conference on Human Factors in Computing Systems - Proceedings (2019), 1–13. https://doi.org/10.1145/3290605.3300301Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Sangyoon Lee, Jinseok Seo, Gerard Jounghyun Kim, and Chan-Mo Park. 2003. Evaluation of pointing techniques for ray casting selection in virtual environments. In Third international conference on virtual reality and its application in industry, Vol. 4756. International Society for Optics and Photonics, 38–44.Google ScholarGoogle ScholarCross RefCross Ref
  41. Nianlong Li, Teng Han, Feng Tian, Jin Huang, Minghui Sun, Pourang Irani, and Jason Alexander. 2020. Get a Grip: Evaluating Grip Gestures for VR Input using a Lightweight Pen. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Klemen Lilija, Henning Pohl, and Kasper Hornbæk. 2020. Who Put That There? Temporal Navigation of Spatial Recordings by Direct Manipulation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–11.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. R. W. Lindeman, J. L. Sibert, and J. N. Templeman. 2001. The effect of 3D widget representation and simulated surface constraints on interaction in virtual environments. Proceedings - Virtual Reality Annual International Symposium (2001), 141–148. https://doi.org/10.1109/vr.2001.913780Google ScholarGoogle ScholarCross RefCross Ref
  44. Mayra Donaji Barrera Machuca and Wolfgang Stuerzlinger. 2019. The effect of stereo display deficiencies on virtual hand pointing. In Conference on Human Factors in Computing Systems - Proceedings. Association for Computing Machinery. https://doi.org/10.1145/3290605.3300437Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. I. Scott MacKenzie. 1992. Fitts’ Law as a Research and Design Tool in Human-Computer Interaction. Human–Computer Interaction 7, 1 (1992), 91–139. https://doi.org/10.1207/s15327051hci0701_3Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. I Scott MacKenzie and R William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI’03 extended abstracts on Human factors in computing systems. 754–755.Google ScholarGoogle Scholar
  47. Diako Mardanbegi, Benedikt Mayer, Ken Pfeuffer, Shahram Jalaliniya, Hans Gellersen, and Alexander Perzl. 2019. EyeSeeThrough: Unifying tool selection and application in virtual environments. 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings (2019), 474–483. https://doi.org/10.1109/VR.2019.8797988Google ScholarGoogle ScholarCross RefCross Ref
  48. Alejandro Martin-Gomez, Ulrich Eck, and Nassir Navab. 2019. Visualization techniques for precise alignment in VR: A comparative study. 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings (2019), 735–741. https://doi.org/10.1109/VR.2019.8798135Google ScholarGoogle ScholarCross RefCross Ref
  49. Antonella Maselli and Mel Slater. 2013. The building blocks of the full body ownership illusion. Frontiers in human neuroscience 7 (2013), 83.Google ScholarGoogle Scholar
  50. Sven Mayer, Valentin Schwind, Robin Schweigert, and Niels Henze. 2018. The effect of offset correction and cursor on mid-air Pointing in real and virtual environments. Conference on Human Factors in Computing Systems - Proceedings 2018-April (2018), 1–13. https://doi.org/10.1145/3173574.3174227Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Joseph E McGrath. 1995. Methodology matters: Doing research in the behavioral and social sciences. In Readings in Human–Computer Interaction. Elsevier, 152–169.Google ScholarGoogle Scholar
  52. Daniel Mendes, Fabio Marco Caputo, Andrea Giachetti, Alfredo Ferreira, and J Jorge. 2019. A survey on 3D virtual object manipulation: From the desktop to immersive virtual environments. In Computer graphics forum, Vol. 38. Wiley Online Library, 21–45.Google ScholarGoogle Scholar
  53. Daniel Mendes, Filipe Relvas, Alfredo Ferreira, and Joaquim Jorge. 2016. The Benefits of DOF Separation in Mid-air 3D Object Manipulation. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 02-04-November-2016 (2016), 261–268. https://doi.org/10.1145/2993369.2993396Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Daniel Mendes, Maurício Sousa, Rodrigo Lorena, Alfredo Ferreira, and Joaquim Jorge. 2017. Using custom transformation axes for mid-air manipulation of 3D virtual objects. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST Part F131944(2017). https://doi.org/10.1145/3139131.3139157Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Mathias Moehring and Bernd Froehlich. 2011. Effective manipulation of virtual objects within arm’s reach. Proceedings - IEEE Virtual Reality(2011), 131–138. https://doi.org/10.1109/VR.2011.5759451Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. David Moher, Alessandro Liberati, Jennifer Tetzlaff, Douglas G Altman, Prisma Group, 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med 6, 7 (2009), e1000097.Google ScholarGoogle Scholar
  57. Roberto A. Montano Murillo, Sriram Subramanian, and Diego Martinez Plasencia. 2017. Erg-O: Ergonomic optimization of immersive virtual environments. UIST 2017 - Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology(2017), 759–771. https://doi.org/10.1145/3126594.3126605Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Aske Mottelson and Kasper Hornbæk. 2017. Virtual reality studies outside the laboratory. In Proceedings of the 23rd acm symposium on virtual reality software and technology. 1–10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Yun Suen Pai, Tilman Dingler, and Kai Kunze. 2019. Assessing hands-free interactions for VR using eye gaze and electromyography. Virtual Reality 23, 2 (2019), 119–131. https://doi.org/10.1007/s10055-018-0371-2Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Soonchan Park, Seokyeol Kim, and Jinah Park. 2012. Select Ahead: Efficient object selection technique using tendency of recent cursor movements. APCHI’12 - Proceedings of the 2012 Asia Pacific Conference on Computer-Human Interaction (2012), 51–58. https://doi.org/10.1145/2350046.2350060Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Vijay M. Pawar and Anthony Steed. 2009. Evaluating the influence of haptic Force-Feedback on 3d selection tasks using natural egocentric gestures. Proceedings - IEEE Virtual Reality(2009), 11–18. https://doi.org/10.1109/VR.2009.4810992Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Vijay M. Pawar and Anthony Steed. 2009. Profiling the behaviour of 3D selection tasks on movement time when using natural haptic pointing gestures. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2009), 79–82. https://doi.org/10.1145/1643928.1643947Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Alex Peer and Kevin Ponto. 2019. Mitigating Incorrect Perception of Distance in Virtual Reality through Personalized Rendering Manipulation. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 244–250.Google ScholarGoogle ScholarCross RefCross Ref
  64. Etienne Peillard, Thomas Thebaud, Jean-Marie Norrnand, Ferran Argelaguet, Guillaume Moreau, and Anatole Lécuyer. 2019. Virtual Objects Look Farther on the Sides: The Anisotropy of Distance Perception in Virtual Reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 227–236.Google ScholarGoogle Scholar
  65. Duc Minh Pham and Wolfgang Stuerzlinger. 2019. Is the pen mightier than the controller? A comparison of input devices for selection in virtual and augmented reality. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2019). https://doi.org/10.1145/3359996.3364264Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Ivan Poupyrev, Suzanne Weghorst, Mark Billinghurst, and Tadao Ichikawa. 1997. A framework and testbed for studying manipulation techniques for immersive VR. In Proceedings of the ACM symposium on Virtual reality software and technology. 21–28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Yuan Yuan Qian and Robert J Teather. 2017. The eyes don’t have it: an empirical comparison of head-based and eye-based selection in virtual reality. In Proceedings of the 5th Symposium on Spatial User Interaction. 91–98.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Ludwig Sidenmark, Christopher Clarke, Xuesong Zhang, Jenny Phu, and Hans Gellersen. 2020. Outline Pursuits: Gaze-assisted Selection of Occluded Objects in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Ludwig Sidenmark and Hans Gellersen. 2019. Eye & Head: Synergetic eye and head movement for gaze pointing and selection. UIST 2019 - Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(2019), 1161–1174. https://doi.org/10.1145/3332165.3347921Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of Presence in Immersive Virtual Environments. Teleoperators and Virtual Environments - Presence 3 (01 1994).Google ScholarGoogle Scholar
  71. R. William Soukoreff and I. Scott MacKenzie. 2004. Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts’ law research in HCI. International journal of human-computer studies 61, 6 (2004), 751–789.Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based text entry In Virtual Reality. In Conference on Human Factors in Computing Systems - Proceedings, Vol. 2018-April. https://doi.org/10.1145/3173574.3174221Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Frank Steinicke and Gerd Bruder. 2014. A self-experimentation report about long-term use of fully-immersive technology. In Proceedings of the 2nd ACM symposium on Spatial user interaction. 66–69.Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Rasmus Stenholt. 2012. Efficient selection of multiple objects on a large scale. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2012), 105–112. https://doi.org/10.1145/2407336.2407357Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Wolfgang Stuerzlinger and Robert J Teather. 2014. Considerations for targets in 3D pointing experiments. Proceedings of HCI Korea(2014), 162–168.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Youssef S Tanagho, Gerald L Andriole, Alethea G Paradis, Kerry M Madison, Gurdarshan S Sandhu, J Esteban Varela, and Brian M Benway. 2012. 2D versus 3D visualization: impact on laparoscopic proficiency using the fundamentals of laparoscopic surgery skill set. Journal of Laparoendoscopic & Advanced Surgical Techniques 22, 9(2012), 865–870.Google ScholarGoogle ScholarCross RefCross Ref
  77. Robert J Teather and Wolfgang Stuerzlinger. 2011. Pointing at 3D targets in a stereo head-tracked virtual environment. In 2011 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 87–94.Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Tanh Quang Tran, Hyun Ju Shin, Wolfgang Stuerzlinger, and Jung Hyun Han. 2017. Effects of virtual arm representations on interaction in virtual environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST Part F131944(2017). https://doi.org/10.1145/3139131.3139149Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Huawei Tu, Susu Huang, Jiabin Yuan, Xiangshi Ren, and Feng Tian. 2019. Crossing-based selection with virtual reality head-mounted displays. In Conference on Human Factors in Computing Systems - Proceedings(CHI ’19). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3290605.3300848Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Manuel Veit, Antonio Capobianco, and Dominique Bechmann. 2009. Influence of degrees of freedom’s manipulation on performances during orientation tasks in virtual reality environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 1, 212 (2009), 51–58. https://doi.org/10.1145/1643928.1643942Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Keith Vertanen and Per Ola Kristensson. 2011. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. 295–298.Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Jia Wang and Robert W. Lindeman. 2015. Object impersonation: Towards effective interaction in tablet- and HMD-based hybrid virtual environments. 2015 IEEE Virtual Reality Conference, VR 2015 - Proceedings (2015), 111–118. https://doi.org/10.1109/VR.2015.7223332Google ScholarGoogle ScholarCross RefCross Ref
  83. Alan Traviss Welford. 1968. Fundamentals of skill.(1968).Google ScholarGoogle Scholar
  84. Johann Wentzel, Greg d’Eon, and Daniel Vogel. 2020. Improving Virtual Reality Ergonomics Through Reach-Bounded Non-Linear Input Amplification. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Steve Whittaker, Loren Terveen, and Bonnie A Nardi. 2000. Let’s stop pushing the envelope and start addressing it: a reference task agenda for HCI. Human–Computer Interaction 15, 2-3 (2000), 75–106.Google ScholarGoogle Scholar
  86. Curtis Wilkes and Doug A. Bowman. 2008. Advantages of velocity-based scaling for distant 3D manipulation. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2008), 23–29. https://doi.org/10.1145/1450579.1450585Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Chadwick A. Wingrave, Ryan Tintner, Bruce N. Walker, Doug A. Bowman, and Larry F. Hodges. 2005. Exploring individual differences in raybased selection: Strategies and traits. Proceedings - IEEE Virtual Reality 2005 (2005), 163–170. https://doi.org/10.1109/vr.2005.35Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7, 3(1998), 225–240. https://doi.org/10.1162/105474698565686Google ScholarGoogle ScholarDigital LibraryDigital Library
  89. Jackie Yang, Hiroshi Horii, Alexander Thayer, and Rafael Ballagas. 2018. VR Grabbers: Ungrounded haptic retargeting for precision grabbing tools. In UIST 2018 - Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, Inc, 889–899. https://doi.org/10.1145/3242587.3242643Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. Zhong Yang, Yi Li, Yang Zheng, Weidong Chen, and Xiaoxiang Zheng. 2012. An interaction system using mixed hand gestures. In Proceedings of the 10th asia pacific conference on Computer human interaction. 125–132.Google ScholarGoogle ScholarDigital LibraryDigital Library
  91. Chun Yu, Yizheng Gu, Zhican Yang, Xin Yi, Hengliang Luo, and Yuanchun Shi. 2017. Tap, Dwell or Gesture? Exploring Head-Based Text Entry Techniques for HMDs. (2017), 4479–4488. https://doi.org/10.1145/3025453.3025964Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Shumin Zhai and Paul Milgram. 1993. Human performance evaluation of manipulation schemes in virtual environments. In Proceedings of IEEE Virtual Reality Annual International Symposium. IEEE, 155–161.Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. David J. Zielinski, Hrishikesh M. Rao, Mark A. Sommer, and Regis Kopper. 2015. Exploring the effects of image persistence in low frame rate virtual environments. 2015 IEEE Virtual Reality Conference, VR 2015 - Proceedings (2015), 19–26. https://doi.org/10.1109/VR.2015.7223319Google ScholarGoogle ScholarCross RefCross Ref
  94. Tim Zindulka, Myroslav Bachynskyi, and Jörg Müller. 2020. Performance and Experience of Throwing in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–8.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in
          • Published in

            cover image ACM Conferences
            CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
            May 2021
            10862 pages
            ISBN:9781450380966
            DOI:10.1145/3411764

            Copyright © 2021 ACM

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 7 May 2021

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed limited

            Acceptance Rates

            Overall Acceptance Rate6,199of26,314submissions,24%

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format