Yuhang Zhao
Cynthia L. Bennett
Hrvoje Benko
Edward Cutrell
Meredith Ringel Morris
ABSTRACT
Traditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world. Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller. Our study showed that Canetroller was a promising tool that enabled visually impaired participants to navigate different virtual spaces. We discuss potential applications supported by Canetroller ranging from entertainment to mobility training.
Supplemental Material
Available for Download
zip
pn1758.sbv.zip (1.1 KB)
The auxiliary material includes the captions for the video preview.
- Rebecca Adams, Paul Finn, Elisabeth Moes, Kathleen Flannery, and Albert "Skip" Rizzo. 2009. Distractibility in Attention/Deficit/ Hyperactivity Disorder (ADHD): The Virtual Reality Classroom. Child Neuropsychology 15, 2: 120--135.Google Scholar
- B. Ando, S. Baglio, V. Marletta, and A. Valastro. 2015. A Haptic Solution to Assist Visually Impaired in Mobility Tasks. IEEE Transactions on HumanMachine Systems 45, 5: 641--646.Google ScholarCross Ref
- Bruno Andò, Salvatore Baglio, Cristian Orazio Lombardo, Vincenzo Marletta, E. A. Pergolizzi, Antonio Pistorio, and Angelo Emanuele Valastro. 2015. An Electronic Cane with a Haptic Interface for Mobility Tasks.. Springer, Cham, 189--200.Google Scholar
- Bruno Andò, Salvatore Baglio, Vincenzo Marletta, and Angelo Valastro. 2017. A Tilt Compensated Haptic Cane for Obstacle Detection.. Springer, Cham, 141-- 151.Google Scholar
- Bruno Andò, Salvatore Baglio, and Nicola Pitrone. 2008. A Contactless Haptic Cane for Blind People. In 12th IMEKO TC1 & TC7 Joint Symposium on Man Science & Measurement, 147--152.Google Scholar
- Joseph Bates. 1992. Virtual Reality, Art, and Entertainment. Presence: Teleoperators and Virtual Environments 1, 1: 133--138.Google ScholarDigital Library
- J Bell, S Bolanowski, and M H Holmes. 1994. The structure and function of Pacinian corpuscles: a review. Progress in neurobiology 42, 1: 79--128. Retrieved September 5, 2017 from http://www.ncbi.nlm.nih.gov/pubmed/7480788Google Scholar
- J. Borenstein and I. Ulrich. 1997. The GuideCane-a computerized travel aid for the active guidance of blind pedestrians. In Proceedings of International Conference on Robotics and Automation, 1283--1288.Google Scholar
- Chetz Colwell, Helen Petrie, Andrew Hardwick, and Martlesham Heath. 1998. Use of a haptic device by blind and sighted people?: perception of virtual textures and objects. Improving the Quality of Life for the European Citizen?: Technology for Inclusive Design and Equality, JULY: 1--8.Google Scholar
- Chetz Colwell, Helen Petrie, Diana Kornbrot, Andrew Hardwick, and Stephen Furner. 1998. Haptic virtual reality for blind computer users. In Proceedings of the third international ACM conference on Assistive technologies - Assets '98, 92--99. Google ScholarDigital Library
- J. L. González-Mora, A. Rodríguez-Hernández, L. F. Rodríguez-Ramos, L. Díaz-Saco, and N. Sosa. 1999. Development of a new space perception system for blind people, based on the creation of a virtual acoustic space.. Springer, Berlin, Heidelberg, 321--330.Google Scholar
- José Luis González-Mora, Antonio Francisco Rodríguez-Hernández, Enrique Burunat, F. Martin, and M A Castellano. 2006. Seeing the world by hearing: Virtual Acoustic Space (VAS) a new space perception system for blind people. In 2nd International Conference on Information & Communication Technologies, 837--842.Google ScholarCross Ref
- E W Hill, J. J. Rieser, M. Hill, J Halpin, and R Halpin. 1993. How persons with visual impairments explore novel spaces? A study of strategies used by exceptionally good and exceptionally poor performers. J Vis Impair Blind 87: 295--301. Retrieved August 31, 2017 from http://psycnet.apa.org/record/1994--18302001Google ScholarCross Ref
- D. Jack, R. Boian, A.S. Merians, M. Tremaine, G.C. Burdea, S.V. Adamovich, M. Recce, and H. Poizner. 2001. Virtual reality-enhanced stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 9, 3: 308--318.Google ScholarCross Ref
- W. H. Jacobson. 1993. The Art and Science of Teaching Orientation and Mobility to Persons with Visual Impairments. AFB Press.Google Scholar
- G Jansson, H Petrie, C Colwell, and D Kornbrot. 1999. Haptic virtual environments for blind people: Exploratory experiments with two devices. The International Journal of Virtual Reality 3, 4: 8--17. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/348e/45107167a03250 51e60c883c153572a127e4.pdfGoogle ScholarCross Ref
- Katerina Kalyvioti and Tassos A. Mikropoulos. 2014. Virtual Environments and Dyslexia: A Literature Review. Procedia Computer Science 27: 138--147.Google ScholarCross Ref
- Yoshihiro Kawai and Fumiaki Tomita. Evaluation of Interactive Tactile Display System. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/c8da/db1e1caa0b4e31 7db3bb8f22c6a45a52f8c6.pdfGoogle Scholar
- Dae Shik Kim, Robert Wall Emerson, and Amy Curtis. 2009. Drop-off Detection with the Long Cane: Effects of Different Cane Techniques on Performance. Journal of visual impairment & blindness 103, 9: 519--530. Retrieved September 2, 2017 from http://www.ncbi.nlm.nih.gov/pubmed/21209791Google ScholarCross Ref
- Henry König, Jochen Schneider, and Thomas Strothotte. Haptic Exploration of Virtual Buildings Using Non-Realistic Haptic Rendering. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/5f54/198fc98e01be1b 73eb1d31b4ef6b95070792.pdfGoogle Scholar
- Michael Kotlyar, Christopher Donahue, Paul Thuras, Matt G. Kushner, Natalie O'Gorman, Erin A Smith, and David E. Adson. 2008. Physiological response to a speech stressor presented in a virtual reality environment. Psychophysiology 45, 6: 1034--1037.Google ScholarCross Ref
- O Lahav and D Mioduser. Multisensory virtual environment for supporting blind persons' acquisition of spatial cognitive mapping, orientation, and mobility skills. Retrieved August 31, 2017 from http://playpen.icomtek.csir.co.za/~acdc/assistive devices/Artabilitation2008/archive/2002/papers/2002_28.pdfGoogle Scholar
- Orly Lahav and David Mioduser. 2004. Exploration of Unknown Spaces by People Who Are Blind Using a Multi-sensory Virtual Environment. Journal of Special Education Technology 19, 3: 15--23. Retrieved August 31, 2017 fromGoogle ScholarCross Ref
- Orly Lahav and David Mioduser. 2008. Construction of cognitive maps of unknown spaces using a multisensory virtual environment for people who are blind. Computers in Human Behavior 24, 3: 1139--1155. Google ScholarDigital Library
- Mark L. Latash, EA Atree, BM Brooks, DA Johnson, EA Attree, A Bellner, M Rydmark, H Poizner, L Hughey, CL Richards, Y Bisson, A Rovetta, S Rushton, C Selis, and J Wann. 1998. Virtual reality: A fascinating tool for motor rehabilitation (to be used with caution). Disability and Rehabilitation 20, 3: 104-- 105.Google ScholarCross Ref
- A. Lecuyer, P. Mobuchon, C. Megard, J. Perret, C. Andriot, and J.-P. Colinot. HOMERE: a multimodal system for visually impaired people to explore virtual environments. In IEEE Virtual Reality, 2003. Proceedings., 251--258. Google ScholarDigital Library
- Fabrizio Leo, Elena Cocchi, and Luca Brayda. 2017. The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25, 7: 861--872.Google ScholarCross Ref
- Shachar Maidenbaum, Shelly Levy-Tzedek, Daniel Robert Chebat, and Amir Amedi. 2013. Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the "EyeCane": Feasibility study. PLoS ONE 8, 8: e72555.Google ScholarCross Ref
- Willie Martin, Kory Dancer, Kevin Rock, Christopher Zeleney, and Kumar Yelamarthi. 2009. The Smart Cane?: An Electrical Engineering Design Project. 1--9. Retrieved August 31, 2017 from http://people.cst.cmich.edu/yelam1k/CASE/Publication s_files/Yelamarthi_ASEE_NCS_2009.pdfGoogle Scholar
- M. P. Menikdiwela, K.M.I.S. Dharmasena, and A.M. Harsha S. Abeykoon. 2013. Haptic based walking stick for visually impaired people. In 2013 International conference on Circuits, Controls and Communications (CCUBE), 1--6.Google ScholarCross Ref
- Lotfi B Merabet and Jaime Sanchez. 2009. AudioBased Navigation Using Virtual Environments: Combining Technology and Neuroscience. AER Journal: Research and Practice in Visual Impairment and Blindness 2: 128--137.Google Scholar
- Zahira Merchant, Ernest T. Goetz, Lauren Cifuentes, Wendy Keeney-Kennicutt, and Trina J. Davis. 2014. Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education 70: 29--40.Google ScholarCross Ref
- Alma S Merians, David Jack, Rares Boian, Marilyn Tremaine, Grigore C Burdea, Sergei V Adamovich, Michael Recce, and Howard Poizner. 2002. Virtual Reality--Augmented Rehabilitation for Patients Following Stroke. Physical Therapy 37, 1: 975--987.Google Scholar
- Thomas D. Parsons, Todd Bowerly, J. Galen Buckwalter, and Albert A. Rizzo. 2007. A Controlled Clinical Comparison of Attention Performance in Children with ADHD in a Virtual Reality Classroom Compared to Standard Neuropsychological Methods. Child Neuropsychology 13, 4: 363--381.Google ScholarCross Ref
- Maurizio de Pascale, Sara Mulatto, and Domenico Prattichizzo. 2008. Bringing Haptics to Second Life for Visually Impaired People.. Springer, Berlin, Heidelberg, 896--905. Google ScholarDigital Library
- Lorenzo Picinali, Amandine Afonso, Michel Denis, and Brian F.G. Katz. 2014. Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge. International Journal of Human Computer Studies 72, 4: 393--407. Google ScholarDigital Library
- S. Reardon. 2011. Playing by Ear. Science 333, 6051: 1816--1818.Google Scholar
- Irvin Rock and Jack Victor. 1964. Vision and Touch: An Experimentally Created Conflict between the Two Senses. Science 143, 3606. Retrieved August 28, 2017 from http://science.sciencemag.org/content/143/3606/594Google ScholarCross Ref
- Joseph M. Romano and Katherine J. Kuchenbecker. 2012. Creating Realistic Virtual Textures from Contact Acceleration Data. IEEE Transactions on Haptics 5, 2: 109--119. Google ScholarDigital Library
- J H Sánchez and M A Sáenz. 2006. Assisting the mobilization through subway networks by users with visual disabilities. Virtual Reality & Assoc. Tech. Retrieved August 30, 2017 from www.dcc.uchile.cl/~jsanchezGoogle Scholar
- JAIME SÁNCHEZ and MAURICIO LUMBRERAS. 1999. Virtual Environment Interaction Through 3D Audio by Blind Children. CyberPsychology & Behavior 2, 2: 101--111.Google ScholarCross Ref
- David W. Schloerb, Orly Lahav, Joseph G. Desloge, and Mandayam A. Srinivasan. 2010. BlindAid: Virtual environment system for self-reliant trip planning and orientation and mobility training. In 2010 IEEE Haptics Symposium, 363--370. Google ScholarDigital Library
- Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10, 3: 266--281. Google ScholarDigital Library
- Yoshikazu Seki and Tetsuji Sato. 2011. A training system of orientation and mobility for blind people using acoustic virtual reality. IEEE Transactions on Neural Systems and Rehabilitation Engineering 19, 1: 95--104.Google ScholarCross Ref
- SK Semwal. 2001. MoVE: Mobiltiy training in haptic virtual environment. piglet. uccs. edu/~ semwal/NSF2001PS. pdf: 1--18.Google Scholar
- Vaibhav Singh, Rohan Paul, Dheeraj Mehra, Anuraag Gupta, Vasu Dev Sharma, Saumya Jain, Chinmay Agarwal, Ankush Garg, Sandeep Singh Gujral, M Balakrishnan, and K Paul. 2010. "Smart" Cane for the Visually Impaired: Design and Controlled Field Testing of an Affordable Obstacle Detection System. TRANSED 2010: 12th International Conference on Mobility and Transport for Elderly and Disabled Persons 53, 9: 1689--1699.Google Scholar
- Lore Thaler, Galen M. Reich, Xinyu Zhang, Dinghe Wang, Graeme E. Smith, Zeng Tao, Raja Syamsul Azmir Bin. Raja Abdullah, Mikhail Cherniakov, Christopher J. Baker, Daniel Kish, and Michail Antoniou. 2017. Mouth-clicks used by blind expert human echolocators -- signal description and model based signal synthesis. PLOS Computational Biology 13, 8: e1005670.Google ScholarCross Ref
- Virgil Tiponut, Zoltan Haraszy, Daniel Ianchis, and Ioan Lie. 2008. Acoustic virtual reality performing man-machine interfacing of the blind. In Proceedings of the 12th WSEAS international conference on Systems, 345--349. Google ScholarDigital Library
- M A Torres-Gil, O Casanova-Gonzalez, and José Luis González-Mora. 2010. Applications of virtual reality for visually impaired people. WSEAS Transactions on Computers 9, 2: 184--193. Retrieved from http://dl.acm.org/citation.cfm?id=1852403.1852412 Google ScholarDigital Library
- Shari Trewin, Mark Laff, Vicki Hanson, and Anna Cavender. 2009. Exploring Visual and Motor Accessibility in Navigating a Virtual World. ACM Transactions on Accessible Computing 2, 2: 1--35. Google ScholarDigital Library
- D. Tzovaras, G. Nikolakis, G. Fergadis, S. Malasiotis, and M. Stavrakis. 2002. Design and implementation of virtual environments training of the visually impaire. Proceedings of the fifth international ACM conference on Assistive technologies - Assets '02 12, 2: 41. Google ScholarDigital Library
- Dimitrios Tzovaras, Konstantinos Moustakas, Georgios Nikolakis, and Michael G. Strintzis. 2009. Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Personal and Ubiquitous Computing 13, 1: 51--58. Google ScholarDigital Library
- Iwan Ulrich and Johann Borenstein. 2001. The GuideCane - Applying Mobile Robot Technologies to Assist the Visually Impaired. IEEE Transactions on Systems, Man, and Cybernetics, -Part A: Systems and Humans 31, 2: 131--136. Retrieved August 31, 2017 from http://www.cs.unc.edu/~welch/class/mobility/papers/G uideCane.pdf Google ScholarDigital Library
- Maria Virvou and George Katsionis. 2008. On the usability and likeability of virtual reality games for education: The case of VR-ENGAGE. Computers and Education 50, 1: 154--178. Google ScholarDigital Library
- M Helmy Abd Wahab, Aa Talib, and Ha Kadir. 2011. Smart Cane: Assistive Cane for Visually-impaired People. IJCSI International Journal of Computer Science Issues 8, 4: 21--27. https://doi.org/1694-0814Google Scholar
- Steven Wall and Stephen Brewster. 2006. Feeling What You Hear?: Tactile Feedback for Navigation of Audio Graphs. Conference on Human Factors in Computing Systems, April: 1123--1132. Google ScholarDigital Library
- Gareth R. White, Geraldine Fitzpatrick, and Graham McAllister. 2008. Toward accessible 3D virtual environments for the blind and visually impaired. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts DIMEA '08, 134. Google ScholarDigital Library
- Michele A Williams, Caroline Galbraith, Shaun K Kane, and Amy Hurst. "Just Let the Cane Hit It": How the Blind and Sighted See Navigation Differently. In Proceedings of the 16th interal ACM SIGACCESS conference on Computers & Accessibility - ASSETS'14. 271--224. Google ScholarDigital Library
- Michele a Williams, Amy Hurst, and Shaun K Kane. 2013. "Pray Before You Step out": Describing Personal and Situational Blind Navigation Behaviors. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility: 28:1---28:8. Google ScholarDigital Library
- Limin Zeng, Mei Miao, and Gerhard Weber. 2012. Interactive Audio-haptic Map Explorer on a Tactile Display. Interacting with Computers 27, 4: 413--429.Google ScholarCross Ref
- M. Zyda. 2005. From Visual Simulation to Virtual Reality to Games. Computer 38, 9: 25--32. Google ScholarDigital Library
- How Virtual Reality Facilitates Social Connection | Facebook IQ. Retrieved August 28, 2017 from https://www.facebook.com/iq/articles/how-virtualreality-facilitates-social-connectionGoogle Scholar
- Phantom Force Feedback. Retrieved August 31, 2017 from http://www.cgl.ucsf.edu/chimera/1.2470/docs/Contribu tedSoftware/phantom/phantom.htmlGoogle Scholar
- Novint - Products. Retrieved August 31, 2017 from http://www.novint.com/index.php/productsGoogle Scholar
- Geomagic Touch (formerly Geomagic Phantom Omni) Overview. Retrieved August 31, 2017 from http://www.geomagic.com/en/products/phantomomni/overviewGoogle Scholar
- Long Cane Techniques, Study Guide: APH Step-byStep. Retrieved September 2, 2017 from http://tech.aph.org/sbs/04_sbs_lc_study.htmlGoogle Scholar
- igroup presence questionnaire (IPQ) Database | igroup.org -- project consortium. Retrieved September 11, 2017 from http://www.igroup.org/pq/ipq/data.phpGoogle Scholar
Index Terms
- Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation
Recommendations
Demonstration of Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing SystemsTraditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller [6], a haptic cane controller that simulates white cane interactions, enabling people with visual ...
Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsCurrent Virtual Reality (VR) technologies focus on rendering visuospatial effects, and thus are inaccessible for blind or low vision users. We examine the use of a novel white cane controller that enables navigation without vision of large virtual ...
Hands-On: Using Gestures to Control Descriptions of a Virtual Environment for People with Visual Impairments
UIST '22 Adjunct: Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and TechnologyVirtual reality (VR) uses three main senses to relay information: sight, sound, and touch. People with visual impairments (PVI) rely primarily on auditory and haptic feedback to receive information in VR. While researchers have explored several ...
Comments