Mohamed Khamis
Daniel Buschek
Tobias Thieron
Florian Alt
Andreas Bulling
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The parallax effect describes the displacement between the perceived and detected touch locations on a touch-enabled surface. Parallax is a key usability challenge for interactive displays, particularly for those that require thick layers of glass between the screen and the touch surface to protect them from vandalism. To address this challenge, we present EyePACT, a method that compensates for input error caused by parallax on public displays. Our method uses a display-mounted depth camera to detect the user's 3D eye position in front of the display and the detected touch location to predict the perceived touch location on the surface. We evaluate our method in two user studies in terms of parallax correction performance as well as multi-user support. Our evaluations demonstrate that EyePACT (1) significantly improves accuracy even with varying gap distances between the touch surface and the display, (2) adapts to different levels of parallax by resulting in significantly larger corrections with larger gap distances, and (3) maintains a significantly large distance between two users' fingers when interacting with the same object. These findings are promising for the development of future parallax-free interactive displays.
Supplemental Material
Available for Download
zip
khamis.zip (13.4 KB)
Supplemental movie, appendix, image and software files for, EyePACT: Eye-Based Parallax Correction on Touch-Enabled Interactive Displays
- Christopher Ackad, Martin Tomitsch, and Judy Kay. 2016. Skeletons and Silhouettes: Comparing User Representations at a Gesture-based Large Display. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 2343--2347. DOI:http://dx.doi.org/10.1145/2858036.2858427 Google Scholar
Digital Library
- Florian Alt, Andreas Bulling, Lukas Mecke, and Daniel Buschek. 2016. Attention, Please!: Comparing Features for Measuring Audience Attention Towards Pervasive Displays. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS ‘16). ACM, New York, NY, USA, 823--828. DOI:http://dx.doi.org/10.1145/2901790.2901897 Google ScholarDigital Library
- N.R. Andrysco. 2015. Gaze-assisted touchscreen inputs. (May 7 2015). https://www.google.de/patents/WO2015065478A1?cl=en WO Patent App. PCT/US2013/068,125.Google Scholar
- Carmelo Ardito, Paolo Buono, Maria Francesca Costabile, and Giuseppe Desolda. 2015. Interaction with Large Displays: A Survey. ACM Comput. Surv. 47, 3, Article 46 (Feb. 2015), 38 pages. DOI:http://dx.doi.org/10.1145/2682623 Google ScholarDigital Library
- Leo B. Baldwin. 2014. Correcting for parallax in electronic displays. (Nov. 11 2014). https://www.google.de/patents/US8884928 US Patent 8,884,928. Filed January 26, 2012.Google Scholar
- S. Bathiche, J.R. Cheatham, P.H. Dietz, M.G. Dyor, P.A. Eckhoff, A. Gupta, K.P. Hinckley, R.A. Hyde, M.Y. Ishikawa, J.T. Kare, and others. 2015. Systems and methods for parallax compensation. (June 2 2015). https://www.google.com/patents/US9047002 US Patent 9,047,002. Filed March 15, 2013.Google Scholar
- Daniel Buschek and Florian Alt. 2015. TouchML: A Machine Learning Toolkit for Modelling Spatial Touch Targeting Behaviour. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI ‘15). ACM, New York, NY, USA, 110--114. DOI:http://dx.doi.org/10.1145/2678025.2701381 Google ScholarDigital Library
- Daniel Buschek, Simon Rogers, and Roderick Murray-Smith. 2013. User-specific Touch Models in a Cross-device Context. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘13). ACM, New York, NY, USA, 382--391. DOI:http://dx.doi.org/10.1145/2493190.2493206 Google ScholarDigital Library
- T. F. Cootes, G. J. Edwards, and C. J. Taylor. 1998. Active appearance models. Springer Berlin Heidelberg, Berlin, Heidelberg, 484--498. DOI:http://dx.doi.org/10.1007/BFb0054760 Google ScholarCross Ref
- T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. 1995. Active Shape Models 8 Their Training and Application. Comput. Vis. Image Underst. 61, 1 (Jan. 1995), 38--59. DOI:http://dx.doi.org/10.1006/cviu.1995.1004 Google ScholarDigital Library
- Nigel Davies, Sarah Clinch, and Florian Alt. 2014. Pervasive Displays: Understanding the Future of Digital Signage (1st ed.). Morgan 8 Claypool Publishers.Google ScholarDigital Library
- Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39, 2 (01 May 2007), 175--191. DOI:http://dx.doi.org/10.3758/BF03193146 Google ScholarCross Ref
- Miriam Greis, Florian Alt, Niels Henze, and Nemanja Memarovic. 2014. I Can Wait a Minute: Uncovering the Optimal Delay Time for Pre-moderated User-generated Content on Public Displays. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 1435--1438. DOI:http://dx.doi.org/10.1145/2556288.2557186 Google ScholarDigital Library
- Double Glazing Info. 2017. Webpage. (2017). http://www.double-glazing-info.com/Choosing-your-windows/Types-of-glass/Security-glass Retrieved February 11, 2017.Google Scholar
- InterelectroniX. 2017. Webpage. (2017). https://www.interelectronix.com/en/products/touch-screen/custom/laminated-glass.html Retrieved February 11, 2017.Google Scholar
- Mikkel R. Jakobsen and Kasper Hornbæk. 2016. Negotiating for Space?: Collaborative Work Using a Wall Display with Mouse and Touch Input. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 2050--2061. DOI:http://dx.doi.org/10.1145/2858036.2858158 Google ScholarDigital Library
- Mikkel R. Jakobsen, Yvonne Jansen, Sebastian Boring, and Kasper Hornbæk. 2015. Should I Stay or Should I Go? Selecting Between Touch and Mid-Air Gestures for Large-Display Interaction. Springer International Publishing, Cham, 455--473. DOI:http://dx.doi.org/10.1007/978-3-319-22698-9_31 Google ScholarDigital Library
- Van Lier Jan. 2015. Touchscreen device with parallax error compensation. (Aug. 6 2015). https://www.google.com/patents/US20150220207 US Patent App. 14/426,105. Filed September 25, 2013.Google Scholar
- Mohamed Khamis, Florian Alt, and Andreas Bulling. 2015. A Field Study on Spontaneous Gaze-based Interaction with a Public Display Using Pursuits. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (UbiComp/ISWC‘15 Adjunct). ACM, New York, NY, USA, 863--872. DOI:http://dx.doi.org/10.1145/2800835.2804335 Google ScholarDigital Library
- Mohamed Khamis, Andreas Bulling, and Florian Alt. 2015. Tackling Challenges of Interactive Public Displays Using Gaze. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (UbiComp/ISWC‘15 Adjunct). ACM, New York, NY, USA, 763--766. DOI:http://dx.doi.org/10.1145/2800835.2807951 Google ScholarDigital Library
- Mohamed Khamis, Ozan Saltuk, Alina Hang, Katharina Stolz, Andreas Bulling, and Florian Alt. 2016. TextPursuits: Using Text for Pursuits-Based Interaction and Calibration on Public Displays. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘16). ACM, New York, NY, USA, 12. DOI:http://dx.doi.org/10.1145/2971648.2971679 Google ScholarDigital Library
- Azam Khan, George Fitzmaurice, Don Almeida, Nicolas Burtnyk, and Gordon Kurtenbach. 2004. A Remote Control Interface for Large Displays. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST ‘04). ACM, New York, NY, USA, 127--136. DOI:http://dx.doi.org/10.1145/1029632.1029655 Google ScholarDigital Library
- Douglas Lanman, Matthew Hirsch, Yunhee Kim, and Ramesh Raskar. 2010. Content-adaptive Parallax Barriers: Optimizing Dual-layer 3D Displays Using Low-rank Light Field Factorization. ACM Trans. Graph. 29, 6, Article 163 (Dec. 2010), 10 pages. DOI:http://dx.doi.org/10.1145/1882261.1866164 Google ScholarDigital Library
- David Lee, KyoungHee Son, Joon Hyub Lee, and Seok-Hyung Bae. 2012. PhantomPen: Virtualization of Pen Head for Digital Drawing Free from Pen Occlusion; Visual Parallax. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST ‘12). ACM, New York, NY, USA, 331--340. DOI:http://dx.doi.org/10.1145/2380116.2380159 Google ScholarDigital Library
- Joon Hyub Lee and Seok-Hyung Bae. 2013. Binocular Cursor: Enabling Selection on Transparent Displays Troubled by Binocular Parallax. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 3169--3172. DOI:http://dx.doi.org/10.1145/2470654.2466433 Google ScholarDigital Library
- David Lindlbauer, Klemen Lilija, Robert Walter, and Jörg Müller. 2016. Influence of Display Transparency on Background Awareness and Task Performance. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 1705--1716. DOI:http://dx.doi.org/10.1145/2858036.2858453 Google ScholarDigital Library
- Päivi Majaranta and Andreas Bulling. 2014. Eye Tracking and Eye-Based Human-Computer Interaction. In Advances in Physiological Computing. Springer, 39--65. Google ScholarCross Ref
- Bastian Migge and Andreas Kunz. 2010. User Model for Predictive Calibration Control on Interactive Screens. In Cyberworlds (CW), 2010 International Conference on. 32--37. DOI:http://dx.doi.org/10.1109/CW.2010.18 Google ScholarDigital Library
- Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘12). ACM, New York, NY, USA, 297--306. DOI:http://dx.doi.org/10.1145/2207676.2207718 Google ScholarDigital Library
- Pilkington Optilam. 2017. Webpage. (2017). http://www.pilkington.com/products/bp/bybenefit/safety-security/optilam/ Retrieved February 11, 2017.Google Scholar
- Richard L. Potter, Linda J. Weldon, and Ben Shneiderman. 1988. Improving the Accuracy of Touch Screens: An Experimental Evaluation of Three Strategies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘88). ACM, New York, NY, USA, 27--32. DOI:http://dx.doi.org/10.1145/57167.57171 Google ScholarDigital Library
- Brian Rogers and Maureen Graham. 1979. Motion Parallax as an Independent Cue for Depth Perception. Perception 8, 2 (1979), 125--134. DOI:http://dx.doi.org/10.1068/p080125 arXiv:http://dx.doi.org/10.1068/p080125 PMID: 471676. Google ScholarCross Ref
- Constantin Schmidt, Jörg Müller, and Gilles Bailly. 2013. Screenfinity: Extending the Perception Area of Content on Very Large Public Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 1719--1728. DOI:http://dx.doi.org/10.1145/2470654.2466227 Google ScholarDigital Library
- Andrew Sears, Catherine Plaisant, and Ben Shneiderman. 1992. Advances in Human-computer Interaction (Vol. 3). Ablex Publishing Corp., Norwood, NJ, USA, Chapter A New Era for High Precision Touchscreens, 1--33. http://dl.acm.org/citation.cfm?id=159433.159436Google Scholar
- Brian A. Smith, Qi Yin, Steven K. Feiner, and Shree K. Nayar. 2013. Gaze Locking: Passive Eye Contact Detection for Human-object Interaction. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ‘13). ACM, New York, NY, USA, 271--280. DOI:http://dx.doi.org/10.1145/2501988.2501994 Google ScholarDigital Library
- Nikolai Smolyanskiy, Christian Huitema, Lin Liang, and Sean Eron Anderson. 2014. Real-time 3D Face Tracking Based on Active Appearance Model Constrained by Depth Data. Image Vision Comput. 32, 11 (Nov. 2014), 860--869. DOI:http://dx.doi.org/10.1016/j.imavis.2014.08.005 Google ScholarDigital Library
- Yusuke Sugano, Xucong Zhang, and Andreas Bulling. 2016. AggreGaze: Collective Estimation of Audience Attention on Public Displays. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ‘16). ACM, New York, NY, USA, 821--831. DOI:http://dx.doi.org/10.1145/2984511.2984536 Google ScholarDigital Library
- Fabian Timm and Erhardt Barth. 2011. Accurate Eye Centre Localisation by Means of Gradients.. In VISAPP, Leonid Mestetskiy and JosÃ1' Braz (Eds.). SciTePress, 125--130. http://www.inb.uni-luebeck.de/publikationen/pdfs/TiBa11b.pdfGoogle Scholar
- Dimitar Valkov, Frank Steinicke, Gerd Bruder, and Klaus Hinrichs. 2011. 2D Touching of 3D Stereoscopic Objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11). ACM, New York, NY, USA, 1353--1362. DOI:http://dx.doi.org/10.1145/1978942.1979142 Google ScholarDigital Library
- L.J. Van. 2014. Touchscreen device with parallax error compensation. (April 10 2014). https://www.google.de/patents/WO2014053369A1?cl=en WO Patent App. PCT/EP2013/069,895.Google Scholar
- Klen Čopič Pucihar, Paul Coulton, and Jason Alexander. 2013. Creating a Stereoscopic Magic-lens to Improve Depth Perception in Handheld Augmented Reality. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘13). ACM, New York, NY, USA, 448--451. DOI:http://dx.doi.org/10.1145/2493190.2494660 Google ScholarDigital Library
- Mélodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous Interaction with Displays Based on Smooth Pursuit Eye Movement and Moving Targets. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘13). ACM, New York, NY, USA, 439--448. DOI:http://dx.doi.org/10.1145/2493432.2493477 Google ScholarDigital Library
- Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Vol. 1. I--511--I--518 vol.1. DOI:http://dx.doi.org/10.1109/CVPR.2001.990517 Google ScholarCross Ref
- Robert Walter, Gilles Bailly, and Jörg Müller. 2013. StrikeAPose: Revealing Mid-air Gestures on Public Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 841--850. DOI:http://dx.doi.org/10.1145/2470654.2470774 Google ScholarDigital Library
- Daryl Weir, Daniel Buschek, and Simon Rogers. 2013. Sparse selection of training data for touch correction systems. In 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ‘13, Munich, Germany, August 27--30, 2013. 404--407. DOI:http://dx.doi.org/10.1145/2493190.2493241 Google ScholarDigital Library
- Daryl Weir, Simon Rogers, Roderick Murray-Smith, and Markus Löchtefeld. 2012. A user-specific machine learning approach for improving touch accuracy on mobile devices. In The 25th Annual ACM Symposium on User Interface Software and Technology, UIST ‘12, Cambridge, MA, USA, October 7--10, 2012. 465--476. DOI:http://dx.doi.org/10.1145/2380116.2380175 Google ScholarDigital Library
- Yanxia Zhang, Andreas Bulling, and Hans Gellersen. 2013. SideWays: A Gaze Interface for Spontaneous Interaction with Situated Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 851--860. DOI:http://dx.doi.org/10.1145/2470654.2470775 Google ScholarDigital Library
- Zhengyou Zhang. 2012. Microsoft Kinect Sensor and Its Effect. IEEE MultiMedia 19, 2 (Feb 2012), 4--10. DOI:http://dx.doi.org/10.1109/MMUL.2012.24 Google ScholarDigital Library
- Zytronic Zybrid. 2017. Webpage. (2017). http://zytronic.co.uk/support/product-info/zybrid/ Retrieved February 11, 2017.Google Scholar
Index Terms
- EyePACT: Eye-Based Parallax Correction on Touch-Enabled Interactive Displays
Recommendations
Physically Guiding Touch Screen Interaction with Public Displays
PerDis '15: Proceedings of the 4th International Symposium on Pervasive DisplaysThis paper presents a novel approach on guiding people how to interact with touch screen based public displays. In our approach, we apply an additional layer of transparent plastic employing different shapes of laser cut holes that is placed on top of ...
Tackling challenges of interactive public displays using gaze
UbiComp/ISWC'15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable ComputersFalling hardware prices led to a widespread use of public displays. Common interaction techniques for such displays currently include touch, mid-air, or smartphone-based interaction. While these techniques are well understood from a technical ...
CapWidgets: tangile widgets versus multi-touch controls on mobile devices
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing SystemsWe present CapWidgets, passive tangible controls for capacitive touch screens. CapWidgets bring back physical controls to off-the-shelf multi-touch surfaces as found in mobile phones and tablet computers. While the user touches the widget, the surface ...
Comments