Atsushi Matsubayashi
ABSTRACT
In this study, we prototype and examine a system that allows a user to manipulate a 3D virtual object with multiple fingers without wearing any device. An autostereoscopic display produces a 3D image and a depth sensor measures the movement of the fingers. When a user touches a virtual object, haptic feedback is provided by ultrasound phased arrays. By estimating the cross section of the finger in contact with the virtual object and by creating a force pattern around it, it is possible for the user to recognize the position of the surface relative to the finger. To evaluate our system, we conducted two experiments to show that the proposed feedback method is effective in recognizing the object surface and thereby enables the user to grasp the object quickly without seeing it.
Supplemental Material
- 2018. Bullet Real-Time Physics Simulation. https://pybullet.orgGoogle Scholar
- 2018. Intel Corporation. https://www.intel.com/Google Scholar
- Gerald Bianchi, Benjamin Knoerlein, Gabor Szekely, and Matthias Harders. 2006. High precision augmented reality haptics. In Proceedings of EuroHaptics, Vol. 6. 169--178.Google Scholar
- Alex Butler, Otmar Hilliges, Shahram Izadi, Steve Hodges, David Molyneaux, David Kim, and Danny Kong. 2011. Vermeer: Direct Interaction with a 360? Viewable 3D Display. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. ACM, 569--576. Google ScholarDigital Library
- Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 505--514. Google ScholarDigital Library
- L. R. Gavrilov. 2008. The possibility of generating focal regions of complex configurations in application to the problems of stimulation of human receptor structures by focused ultrasound. Acoustical Physics 54, 2 (2008), 269--278.Google ScholarCross Ref
- Radim Hal-sr and Jan Flusser. 1998. Numerically stable direct least squares fitting of ellipses. In Proceedings of 6th International Conference in Central Europe on Computer Graphics and Visualization, Vol. 98. Citeseer, 125--132.Google Scholar
- Seungju Han and Joonah Park. 2014. Holo-haptics: Haptic interaction with a see-through 3d display. In Proceedings of the IEEE International Conference on Consumer Electronics. IEEE, 512--513.Google ScholarCross Ref
- William S. Harwin and Nicholas Melder. 2002. Improved haptic rendering for multi-finger manipulation using friction cone based god-objects. In Proceedings of the Eurohaptics conference. Citeseer, 82--85.Google Scholar
- Keisuke Hasegawa and Hiroyuki Shinoda. 2013. A method for distribution control of aerial ultrasound radiation pressure for remote vibrotactile display. In SICE Annual Conference, 2013 Proceedings of. IEEE, 223--228.Google Scholar
- Otmar Hilliges, David Kim, Shahram Izadi, Malte Weiss, and Andrew Wilson. 2012. HoloDesk: Direct 3D Interactions with a Situated Seethrough Display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2421--2430. Google ScholarDigital Library
- Seki Inoue, Yasutoshi Makino, and Hiroyuki Shinoda. 2015. Active touch perception produced by airborne ultrasonic haptic hologram. In Proceedings of the World Haptics Conference. IEEE, 362--367.Google ScholarCross Ref
- Seki Inoue, Yasutoshi Makino, and Hiroyuki Shinoda. 2016. MidAir Ultrasonic Pressure Control on Skin by Adaptive Focusing. In Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 68--77. Google ScholarDigital Library
- Seki Inoue, Yasutoshi Makino, and Hiroyuki Shinoda. 2016. Scalable Architecture for Airborne Ultrasound Tactile Display. In Proceedings of the International AsiaHaptics conference. Springer, 99--103.Google Scholar
- Takayuki Iwamoto, Mari Tatezono, and Hiroyuki Shinoda. 2008. Noncontact method for producing tactile sensation using airborne ultrasound. In Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 504--513. Google ScholarDigital Library
- Hideki Kakeya, Ken Okada, and Hayato Takahashi. 2018. {Papers} Time-Division Quadruplexing Parallax Barrier with Subpixel-Based Slit Control. ITE Transactions on Media Technology and Applications 6, 3 (2018), 237--246.Google ScholarCross Ref
- Nam-Woo Kim, Dong-Hak Shin, Dong-Jin Kim, Byung-Gook Lee, and Eun-Soo Kim. 2009. Vision-Based 3D Fingertip Interface for Spatial Interaction in 3D Integral Imaging System. In Proceedings of the International Conference on Complex, Intelligent and Software Intensive Systems. 1006--1011.Google ScholarCross Ref
- Georgios Korres and Mohamad Eid. 2016. Haptogram: ultrasonic point-cloud tactile stimulation. IEEE Access 4 (2016), 7758--7769.Google ScholarCross Ref
- P. J. J. Lamore, H. Muijser, and C. J. Keemink. 1986. Envelope detection of amplitude-modulated high-frequency sinusoidal signals by skin mechanoreceptors. The Journal of the Acoustical Society of America 79, 4 (1986), 1082--1085.Google ScholarCross Ref
- Benjamin Long, Sue Ann Seah, Tom Carter, and Sriram Subramanian. 2014--11. Rendering Volumetric Haptic Shapes in Mid-air Using Ultrasound. ACM Trans. Graph. 33, 6 (2014--11), 181:1--181:10. Google ScholarDigital Library
- Yasutoshi Makino, Yoshikazu Furuyama, Seki Inoue, and Hiroyuki Shinoda. 2016. HaptoClone (Haptic-Optical Clone) for mutual teleenvironment by real-time 3D image transfer with midair force Feedback. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1980--1990. Google ScholarDigital Library
- Yasuaki Monnai, Keisuke Hasegawa, Masahiro Fujiwara, Kazuma Yoshino, Seki Inoue, and Hiroyuki Shinoda. 2014. HaptoMime: Midair Haptic Interaction with a Floating Virtual Screen. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology. ACM, 663--667. Google ScholarDigital Library
- Sylvain Paris and Frdo Durand. 2006. A fast approximation of the bilateral filter using a signal processing approach. In Proceedings of the European conference on computer vision. Springer, 568--580. Google ScholarDigital Library
- Radu Bogdan Rusu and Steve Cousins. 2011. 3d is here: Point cloud library (pcl). In Proceedings of the IEEE International Conference on Robotics and automation. IEEE, 1--4.Google ScholarCross Ref
- Antti Sand, Ismo Rakkolainen, Poika Isokoski, Jari Kangas, Roope Raisamo, and Karri Palovuori. 2015. Head-mounted Display with Midair Tactile Feedback. In Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology. ACM, 51--58. Google ScholarDigital Library
- Rajinder Sodhi, Ivan Poupyrev, Matthew Glisson, and Ali Israr. 2013. AIREAL: interactive tactile experiences in free air. ACM Transactions on Graphics 32, 4 (2013), 134. Google ScholarDigital Library
- Ryoko Takahashi, Keisuke Hasegawa, and Hiroyuki Shinoda. 2018. Lateral Modulation of Midair Ultrasound Focus for Intensified Vibrotactile Stimuli. In Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 276--288.Google ScholarCross Ref
- A. Talvas, M. Marchal, and A. LÃlcuyer. 2013. The god-finger method for improving 3D interaction with virtual objects through simulation of contact area. In Proceedings of 2013 IEEE Symposium on 3D User Interfaces. 111--114.Google Scholar
- C. Ting, T. Jen, C. Chen, H. D. Shieh, and Y. Huang. 2016. 3D Air-Touch User Interface With High Touch Accuracy on Stereoscopic Displays. Journal of Display Technology 12, 5 (2016), 429--434.Google ScholarCross Ref
- Guo-Zhen Wang, Yi-Pai Huang, Tian-Sheuan Chang, and Tsu-Han Chen. 2014. Bare finger 3D air-touch system using an embedded optical sensor array for mobile displays. Journal of Display Technology 10, 1 (2014), 13--18.Google ScholarCross Ref
- Takumi Yoshida, Keitaro Shimizu, Tadatoshi Kurogi, Sho Kamuro, Kouta Minamizawa, Hideaki Nii, and Susumu Tachi. 2011. RePro3D: full-parallax 3D display with haptic feedback using retro-reflective projection technology. In Proceedings of the IEEE International Symposium on VR Innovation. IEEE, 49--54.Google ScholarCross Ref
- Qu Zhang and Hideki Kakeya. 2014. A High Quality Autostereoscopy System Based on Time-Division Quadplexing Parallax Barrier. E97-C, 11 (2014), 1074--1080.Google Scholar
- Craig B Zilles and J Kenneth Salisbury. 1995. A constraint-based god-object method for haptic display. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 3. IEEE, 146--151. Google ScholarDigital Library
Index Terms
- Direct Finger Manipulation of 3D Object Image with Ultrasound Haptic Feedback
Recommendations
Tactile Feedback of Object Slip Facilitates Virtual Object Manipulation
Recent advances in myoelectric prosthetic technology have enabled more complex movements and interactions with objects, but the lack of natural haptic feedback makes object manipulation difficult to perform. Our research effort aims to develop haptic ...
Direct touch haptic display using immersive illusion with interactive virtual finger
EuroHaptics'12: Proceedings of the 2012 international conference on Haptics: perception, devices, mobility, and communication - Volume Part IIn attempt to improve haptic interaction and manipulation technique in surface computers, we propose an interactive haptic feedback display that utilizes ‘direct touch' on a surface computer without using a mediation device. The proposed haptic display ...
Can Haptic Feedback on One Virtual Object Increase the Presence of Another Virtual Object?
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and TechnologyThis paper investigated whether increased presence from experiencing haptic feedback on one virtual object can transfer to another virtual object. Two similar studies were run in different environments: an immersive virtual environment and a mixed ...
Comments