ABSTRACT
We present a novel optimization-based algorithm for the design and fabrication of customized, deformable input devices, capable of continuously sensing their deformation. We propose to embed piezoresistive sensing elements into flexible 3D printed objects. These sensing elements are then utilized to recover rich and natural user interactions at runtime. Designing such objects is a challenging and hard problem if attempted manually for all but the simplest geometries and deformations. Our method simultaneously optimizes the internal routing of the sensing elements and computes a mapping from low-level sensor readings to user-specified outputs in order to minimize reconstruction error. We demonstrate the power and flexibility of the approach by designing and fabricating a set of flexible input devices. Our results indicate that the optimization-based design greatly outperforms manual routings in terms of reconstruction accuracy and thus interaction fidelity.
Supplemental Material
- Gilles Bailly and Antti Oulasvirta. 2014. Toward Optimal Menu Design. interactions 21, 4 (July 2014), 40-45. DOI: http://dx.doi.org/10.1145/2617814 Google ScholarDigital Library
- Gilles Bailly, Antti Oulasvirta, Timo Kotzing, and Sabrina Hoppe. 2013. MenuOptimizer: Interactive Optimization of Menu Systems. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 331-342. DOI: http://dx.doi.org/10.1145/2501988.2502024 Google ScholarDigital Library
- Ravin Balakrishnan, George Fitzmaurice, Gordon Kurtenbach, and Karan Singh. 1999. Exploring Interactive Curve and Surface Manipulation Using a Bend and Twist Sensitive Input Strip. In Proceedings of the 1999 Symposium on Interactive 3D Graphics (I3D '99). ACM, NY, NY, USA, 111-118. DOI: http://dx.doi.org/10.1145/300523.300536 Google ScholarDigital Library
- Bernd Bickel, Moritz Bacher, Miguel A. Otaduy, Hyunho Richard Lee, Hanspeter Pfister, Markus Gross, and Wojciech Matusik. 2010. Design and Fabrication of Materials with Desired Deformation Behavior. ACM Trans. on Graphics (Proc. SIGGRAPH) 29, 3 (2010). Google ScholarDigital Library
- Eric Brockmeyer, Ivan Poupyrev, and Scott Hudson. 2013. PAPILLON: designing curved display surfaces with printed optics. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, NY, NY, USA, 457-462. DOI: http://dx.doi.org/10.1145/2501988.2502027 Google ScholarDigital Library
- Lee A. Danisch, Kevin Englehart, and Andrew Trivett. 1999. Spatially continuous six-degrees-of-freedom position and orientation sensor. (1999). DOI: http://dx.doi.org/10.1117/12.339112Google Scholar
- Nan-Wei Gong, Jürgen Steimle, Simon Olberding, Steve Hodges, Nicholas Edward Gillian, Yoshihiro Kawahara, and Joseph A. Paradiso. 2014. PrintSense: a versatile sensing technique to support multimodal flexible surface interaction. In Proceedings of ACM conference on Human factors in computing systems - ACM CHI. ACM Press, NY, NY, USA, 1407-1410. DOI: http://dx.doi.org/10.1145/2556288.2557173 Google ScholarDigital Library
- Fenner Drives Inc. Accessed: 2015-08-07. Fenner Drives Inc. NinjaFlex filament. http://www.ninjaflex3d.com. (Accessed: 2015-08-07).Google Scholar
- MakerBot Industries. Accessed: 2015-08-05. MakerBot Replicator 2X. https://eu.makerbot.com/shop/de/3d-drucker/replicator-2x. (Accessed: 2015-08-05).Google Scholar
- Alec Jacobson, Daniele Panozzo, Oliver Glauser, Cedric Pradalier, Otmar Hilliges, and Olga Sorkine-Hornung. 2014. Tangible and Modular Input Device for Character Articulation. ACM Trans. Graph. 33, 4, Article 82 (July 2014), 12 pages. DOI: http://dx.doi.org/10.1145/2601097.2601112 Google ScholarDigital Library
- Amit Joe Lopes, Eric MacDonald, and Ryan B. Wicker. 2012. Integrating stereolithography and direct print technologies for 3D structural electronics fabrication. Rapid Prototyping Journal 18, 2 (March 2012), 129-143. DOI: http://dx.doi.org/10.1108/13552541211212113Google Scholar
- Wolfgang Kabsch. 1976. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography 32, 5 (1976), 922-923.Google ScholarCross Ref
- Andreas Karrenbauer and Antti Oulasvirta. 2014. Improvements to Keyboard Optimization with Integer Programming. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, NY, NY, USA, 621-626. DOI: http://dx.doi.org/10.1145/2642918.2647382 Google ScholarDigital Library
- T. Kato, A. Yamamoto, and T. Higuchi. 2003. Shape recognition using piezoelectric thin films. In Industrial Technology, 2003 IEEE International Conference on, Vol. 1. 112-116 Vol.1. DOI: http://dx.doi.org/10.1109/ICIT.2003.1290252Google ScholarCross Ref
- S. J. Leigh, R. J. Bradley, C. P. Purssell, D. R. Billson, and D. A. Hutchins. 2012. A Simple, Low-Cost Conductive Composite Material for 3D Printing of Electronic Sensors. PLoS ONE 7, 11 (2012). DOI: http://dx.doi.org/doi: 10.1371/journal.pone.0049365Google Scholar
- M Melnykowycz, B Koll, D Scharf, and F. Clemens. 2014. Comparison of Piezoresistive Monofilament Polymer Sensors. Sensors 14, 1 (2014), 1278-1294. DOI: http://dx.doi.org/10.3390/s140101278Google ScholarCross Ref
- Joseph T Muth, Daniel M Vogt, Ryan L Truby, Yigit Menguc¸, David B Kolesky, Robert J Wood, and Jennifer A Lewis. 2014. Embedded 3D Printing of Strain Sensors within Highly Stretchable Elastomers. Advanced Materials 26, 36 (2014), 6307-6312.Google ScholarCross Ref
- Simon Olberding, Michael Wessely, and Jurgen Steimle. 2014. PrintScreen: Fabricating Highly Customizable Thin-film Touch-Displays. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, NY, NY, USA, 281-290. DOI: http://dx.doi.org/10.1145/2642918.2647413 Google ScholarDigital Library
- Huaishu Peng, Jennifer Mankoff, Scott E. Hudson, and James McCann. 2015. A Layered Fabric 3D Printer for Soft Interactive Objects. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 1789-1798. DOI: http://dx.doi.org/10.1145/2702123.2702327 Google ScholarDigital Library
- Thiago Pereira, Szymon Rusinkiewicz, and Wojciech Matusik. 2014. Computational Light Routing: 3D Printed Optical Fibers For Sensing and Display. ACM Transactions on Graphics 33, 3 (June 2014), 1-13. DOI: http://dx.doi.org/10.1145/2602140 Google ScholarDigital Library
- 3D Prima. Accessed: 2015-08-03. 3D Prima Conductive ABS. http://www.3dprima.com/en/filaments-for-3d-printers/abs-175mm/3d-prima-conductive-abs-175mm-1-kg-spool-black.html. (Accessed: 2015-08-03).Google Scholar
- Recreus. Accessed: 2015-08-07. Recreus Filaflex filament. http://recreus.com/en/4filaflex-filaments. (Accessed: 2015-08-07).Google Scholar
- Christian Rendl, Patrick Greindl, Michael Haller, Martin Zirkl, Barbara Stadlober, and Paul Hartmann. 2012. PyzoFlex: printed piezoelectric pressure sensing foil. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12. ACM Press, NY, NY, USA, 509. DOI: http://dx.doi.org/10.1145/2380116.2380180 Google ScholarDigital Library
- Christian Rendl, David Kim, Sean Fanello, Patrick Parzer, Christoph Rhemann, Jonathan Taylor, Martin Zirkl, Gregor Scheipl, Thomas Rothl¨ander, Michael Haller, and Shahram Izadi. 2014. FlexSense: A Transparent Self-sensing Deformable Surface. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). 129-138. DOI: http://dx.doi.org/10.1145/2642918.2647405 Google ScholarDigital Library
- Valkyrie Savage, Andrew Head, Björn Hartmann, Dan B. Goldman, Gautham Mysore, and Wilmot Li. 2015. Lamello: Passive Acoustic Sensing for Tangible Input Components. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, NY, NY, USA, 1277-1280. DOI: http://dx.doi.org/10.1145/2702123.2702207 Google ScholarDigital Library
- Valkyrie Savage, Ryan Schmidt, Tovi Grossman, George Fitzmaurice, and Bjorn Hartmann. 2014. A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). 3-12. DOI: http://dx.doi.org/10.1145/2642918.2647374 Google ScholarDigital Library
- Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. 2004. Gummi: A Bendable Computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, NY, NY, USA, 263-270. DOI: http://dx.doi.org/10.1145/985692.985726 Google ScholarDigital Library
- Melina Skouras, Bernhard Thomaszewski, Stelian Coros, Bernd Bickel, and Markus Gross. 2013. Computational Design of Actuated Deformable Characters. ACM Trans. Graph. 32, 4, Article 82 (July 2013), 10 pages. DOI: http://dx.doi.org/10.1145/2461912.2461979 Google ScholarDigital Library
- Ronit Slyper, Ivan Poupyrev, and Jessica Hodgins. 2011. Sensing through structure: designing soft silicone sensors. In Proceedings of the ?fth international conference on Tangible, embedded, and embodied interaction. ACM, 213-220. Google ScholarDigital Library
- Brian A. Smith, Xiaojun Bi, and Shumin Zhai. 2015. Optimizing Touchscreen Keyboards for Gesture Typing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 3365-3374. DOI: http://dx.doi.org/10.1145/2702123.2702357 Google ScholarDigital Library
- Olga Sorkine and Marc Alexa. 2007. As-rigid-as-possible surface modeling. In Symposium on Geometry processing, Vol. 4. Google ScholarDigital Library
- Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta. 2015. Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 3643-3652. DOI: http://dx.doi.org/10.1145/2702123.2702136 Google ScholarDigital Library
- Jurgen Steimle, Andreas Jordt, and Pattie Maes. 2013. Flexpad: Highly Flexible Bending Interactions for Projected Handheld Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 237-246. DOI: http://dx.doi.org/10.1145/2470654.2470688 Google ScholarDigital Library
- Stratasys. Accessed: 2015-09--25. Objet Connex 3D Printers. http://www.stratasys.com/3d-printers/design-series/connex-systems. (Accessed: 2015-09--25).Google Scholar
- Creative Tools. Accessed: 2015-08-03. Palmiga conductive TPU. http://www.creativetools.se/pi-etpu-95--250-carbon-black-1kg-se. (Accessed: 2015-08-03).Google Scholar
- Marynel Vazquez, Eric Brockmeyer, Ruta Desai, Chris Harrison, and Scott E. Hudson. 2015. 3D Printing Pneumatic Device Controls with Variable Activation Force Capabilities. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 1295-1304. DOI: http://dx.doi.org/10.1145/2702123.2702569 Google ScholarDigital Library
- Karl Willis, Eric Brockmeyer, Scott Hudson, and Ivan Poupyrev. 2012. Printed optics: 3D Printing of Embedded Optical Elements for Interactive Devices. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12. ACM Press, NY, NY, USA, 589-598. DOI: http://dx.doi.org/10.1145/2380116.2380190 Google ScholarDigital Library
- Martin Zirkl, Anurak Sawatdee, Uta Helbig, Markus Krause, Gregor Scheipl, Elke Kraker, Peter Andersson Ersman, David Nilsson, Duncan Platt, Peter Bodö, Siegfried Bauer, Gerhard Domann, and Barbara Stadlober. 2011. An all-printed ferroelectric active matrix sensor network based on only five functional materials forming a touchless control interface. Advanced materials (Deerfield Beach, Fla.) 23, 18 (May 2011), 2069-74. DOI: http://dx.doi.org/10.1002/adma.201100054Google Scholar
Index Terms
- DefSense: Computational Design of Customized Deformable Input Devices
Recommendations
Flexibles: Deformation-Aware 3D-Printed Tangibles for Capacitive Touchscreens
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsWe introduce Flexibles: 3D-printed flexible tangibles that are deformation-aware and operate on capacitive touchscreens. Flexibles add expressive deformation input to interaction with on-screen tangibles. Based on different types of deformation mapping, ...
Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology3D printing is widely used to physically prototype the look and feel of 3D objects. Interaction possibilities of these prototypes, however, are often limited to mechanical parts or post-assembled electronics. In this paper, we present Capricate, a ...
MetaSense: Integrating Sensing Capabilities into Mechanical Metamaterial
UIST '21: The 34th Annual ACM Symposium on User Interface Software and TechnologyIn this paper, we present a method to integrate sensing capabilities into 3D printable metamaterial structures comprised of cells, which enables the creation of monolithic input devices for HCI. We accomplish this by converting select opposing cell ...
Comments