Sean Follmer
Hiroshi Ishii
ABSTRACT
We introduce a novel input device, deForm, that supports 2.5D touch gestures, tangible tools, and arbitrary objects through real-time structured light scanning of a malleable surface of interaction. DeForm captures high-resolution surface deformations and 2D grey-scale textures of a gel surface through a three-phase structured light 3D scanner. This technique can be combined with IR projection to allow for invisible capture, providing the opportunity for co-located visual feedback on the deformable surface. We describe methods for tracking fingers, whole hand gestures, and arbitrary tangible tools. We outline a method for physically encoding fiducial marker information in the height map of tangible tools. In addition, we describe a novel method for distinguishing between human touch and tangible tools, through capacitive sensing on top of the input surface. Finally we motivate our device through a number of sample applications.
Supplemental Material
- Xbox.com | Kinect. 2010. http://www.xbox.com/en-US/kinect.Google Scholar
- ofxStructured Light. http://code.google.com/p/structured-light/.Google Scholar
- Cassinelli, A. and Ishikawa, M. Khronos projector. ACM SIGGRAPH 2005 Emerging Technologies, ACM Press (2005), 10. Google ScholarDigital Library
- Chan, L.W., Wu, H.T., Kao, H.S., Lin, H.R., Chen, M.Y,Hsu, Jane, Hung, Y.P. Enabling beyond-surface interactions for interactive surface with an invisible projection. Proc. UIST 2010, ACM Press (2010), 263--272. Google ScholarDigital Library
- Han, J.Y. Low-cost multi-touch sensing through frustrated total internal reflection. Proc. UIST 2005, ACM Press (2005), 115. Google ScholarDigital Library
- Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Garcia-Mendoza, A., and Butz, A. Interactions in the Air : Adding Further Depth to Interactive Tabletops. Proc. UIST 2009, ACM Press (2009), 139--148. Google ScholarDigital Library
- Hilliges, O., Kim, D., and Izadi, S. Creating malleable interactive surfaces using liquid displacement sensing. Proc. Tabletop 2008, IEEE Press (2008), 157--160.Google ScholarCross Ref
- Hook, J., Taylor, S., Butler, A., Villar, N., and Izadi, S. A reconfigurable ferromagnetic input device. Proc. UIST 2009, ACM Press (2009), 51. Google ScholarDigital Library
- Ishii, H. and Ullmer, B. Tangible bits. Proc. CHI 1997, ACM Press (1997), 234--241. Google ScholarDigital Library
- Izadi, S., Hodges, S., Taylor, S., et al. Going beyond the display. Proc. UIST 2008, ACM Press (2008), 269. Google ScholarDigital Library
- Jansen, Y., Karrer, T., and Borchers, J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. Proc. ITS 2010, ACM Press (2010), 11--14. Google ScholarDigital Library
- Johnson, M.K. and Adelson, E.H. Retrographic sensing for the measurement of surface texture and shape. Proc. IEEE CVPR 2009, IEEE Press (2009), 1070--1077.Google ScholarCross Ref
- Kaltenbrunner, M. and Bencina, R. reacTIVision. Proc. TEI 2007, ACM Press (2007), 69. Google ScholarDigital Library
- Lanman, D. and Taubin, G. Build your own 3D scanner. ACM SIGGRAPH 2009 Courses, ACM Press (2009), 1--94. Google ScholarDigital Library
- Large, M.J., Large, T., and Travis, A.R.L. Parallel Optics in Waveguide Displays: A Flat Panel Autostereoscopic Display. Journal of Display Technology 6, 10 (2010), 431--437.Google ScholarCross Ref
- Lecuyer, A., Coquillart, S., Kheddar, A., Richard, P., and Coiffet, P. Pseudo-haptic feedback: can isometric input devices simulate force feedback? Proc. IEEE Virtual Reality 2000, IEEE Comput. Soc, 83--90. Google ScholarDigital Library
- Lei, S. and Zhang, S. Flexible 3-D shape measurement using projector defocusing. Optics Letters 34, 20 (2009), 3080.Google ScholarCross Ref
- Malik, S. and Laszlo, J. Visual touchpad. Proc. ICMI 2004, ACM Press (2004), 289. Google ScholarDigital Library
- Massie, T.H. and Salisbury, J.K. The PHANTOM Haptic Interface: A Device for Probing Virtual Objects. Proceedings of the ASME Winter Annual Meeting Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, (1994), 295--300.Google Scholar
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. Efficiently combining positions and normals for precise 3D geometry. ACM Transactions on Graphics 24, 3 (2005), 536. Google ScholarDigital Library
- Otsuki, M., Sugihara, K., Kimura, A., Shibata, F., and Tamura, H. MAI painting brush: an interactive device that realizes the feeling of real painting. Proc. UIST 2010, ACM Press (2010), 97--100. Google ScholarDigital Library
- Overholt, D. The MATRIX: a novel controller for musical expression. Proc. NIME 2001, AMC (2001), 1--4. Google ScholarDigital Library
- Piper, B., Ratti, C., and Ishii, H. Illuminating clay. Proc. CHI 2002, ACM Press (2002), 355. Google ScholarDigital Library
- Rosenberg, I. and Perlin, K. The UnMousePad. Proc. SIGGRAPH 2009, ACM Press (2009), 1. Google ScholarDigital Library
- Sato, T., Mamiya, H., Tokui, T., Koike, H., and Fukuchi, K. PhotoelasticTouch: transparent rubbery interface using a LCD and photoelasticity. ACM SIGGRAPH 2009 Emerging Technologies, ACM (2009), 1--1. Google ScholarDigital Library
- Sheng, J., Balakrishnan, R., and Singh, K. An interface for virtual 3D sculpting via physical proxy. Computer graphics and interactive techniques in Australasia and South East Asia, (2006), 213. Google ScholarDigital Library
- Sile O'Modhrain. Playing by Feel: Incorporating Haptic Feedback into Computer-Based musical Instruments. 2000. https://ccrma.stanford.edu/~sile/thesis.html. Google ScholarDigital Library
- Sinclair, M. The haptic lens. Ext. Abstracts ACM SIGGRAPH '97, ACM Press (1997), 179. Google ScholarDigital Library
- Smith, J.D., Graham, T.C.N., Holman, D., and Borchers, J. Low-Cost Malleable Surfaces with Multi-Touch Pressure Sensitivity. TABLETOP 2007, IEEE (2007), 205--208.Google Scholar
- Tom White. Introducing Liquid Haptics in High Bandwidth Human Computer Interfaces. 1998. http://dspace.mit.edu/handle/1721.1/62938.Google Scholar
- Valino Koh, J.T.K., Karunanayaka, K., Sepulveda, J., Tharakan, M.J., Krishnan, M., and Cheok, A.D. Liquid interface. Proc. ACE 2010, ACM Press (2010), 45. Google ScholarDigital Library
- Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J., Raymaekers, C., and Van Reeth, F. IntuPaint: Bridging the gap between physical and digital painting. TABLETOP 2008, IEEE (2008), 65--72.Google ScholarCross Ref
- Viciana-Abad, R., Lecuona, A.R., and Poyade, M. The Influence of Passive Haptic Feedback and Difference Interaction Metaphors on Presence and Task Performance. Presence: Teleoperators and Virtual Environments 19, 3 (2010), 197--212. Google ScholarDigital Library
- Vlack, K., Mizota, T., Kawakami, N., Kamiyama, K., Kajimoto, H., and Tachi, S. Gelforce: a vision-based traction field computer interface. Ext. Abstracts CHI 2005, ACM Press (2005), 1154--1155. Google ScholarDigital Library
- Vogt, F., Chen, T., Hoskinson, R., and Fels, S. A malleable surface touch interface. ACM SIGGRAPH 2004 Sketches, ACM Press (2004), 36. Google ScholarDigital Library
- Weiss, M., Wagner, J., Jansen, Y., et al. SLAP widgets. Proc. CHI 2009, ACM Press (2009), 481. Google ScholarDigital Library
- Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., and Kirk, D. Bringing physics to the surface. Proc. UIST 2008, ACM Press (2008), 67. Google ScholarDigital Library
- Wilson, A.D. TouchLight. Proc. ICMI 2004, ACM Press (2004), 69. Google ScholarDigital Library
- Zhang, S. and Yau, S.-T. Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector. Applied Optics 46, 1 (2007), 36.Google ScholarCross Ref
- Zhang, S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Optics and Lasers in Engineering 48, 2 (2010), 149--158..Google ScholarCross Ref
Index Terms
- deForm: an interactive malleable surface for capturing 2.5D arbitrary objects, tools and touch
Recommendations
Non-Rigid HCI: A Review of Deformable Interfaces and Input
DIS '19: Proceedings of the 2019 on Designing Interactive Systems ConferenceDeformable interfaces are emerging in HCI and prototypes show potential for non-rigid interactions. Previous reviews looked at deformation as a material property of shape-changing interfaces and concentrated on output. As such,deformable input was under-...
Sinkpad: a malleable mouse pad consisted of an elastic material
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsThe computer mouse has been used for more than 40 years; users, however, can only perform simple actions. To solve this problem, we present "Sinkpad", an augmented mouse pad that has a malleable surface consisted of an elastic material. Sinkpad augments ...
Deformation Techniques for Shape Changing Interfaces
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsSince the introduction of shape changing interfaces, research in the field has contributed a number of taxonomies to classify shape changing interfaces according to different characteristics, including shape, interaction mapping, material, actuation, and ...
Comments