Skip to main content

Über dieses Buch

This book addresses Assistive Augmentation, highlighting the design and development of assistive technologies, user interfaces, and interactions that seamlessly integrate with a user’s mind, body, and behavior, providing an enhanced perception.

Our senses are the dominant channel we use to perceive the world around us. Whether they have impairments or not, people often find themselves at the limits of their sensorial capabilities. Some seek assistive or enhancing devices that enable them to carry out specific tasks or even transform them into a “superhuman” with capabilities well beyond the ordinary. The overarching topic of this book revolves around the design and development of technologies and interfaces that provide enhanced physical, sensorial and cognitive capabilities: “Assistive Augmentation”.

The Assistive Augmentation community convened at an interdisciplinary workshop at the 2014 International Conference on Human Factors in Computing Systems (CHI) in Toronto, Canada. The community is comprised of researchers and practitioners who work at the junction of human–computer interaction, assistive technology and human augmentation.

This edited volume, which represents the first tangible outcome of the workshop, presents stimulating discussions on the challenges of Assistive Augmentation as examined through case studies. These studies focus on two main areas: (1) Augmented Sensors and Feedback Modalities, and (2) Design for Assistive Augmentation.




Our senses are the dominant channel for perceiving the world around us. With impairments and lack thereof, people find themselves at the edge of sensorial capability. Some excel and use their impairment as a gift. Prominent examples are Evelyn Glennie, a hearing-impaired percussionist, and Ben Underwood, whose eyes were removed at when he was 5 years, taught himself echolocation. Some seek assistive or enhancing devices which enable a “disabled” user to carry out a task or even turn the user into a “superhuman” with capabilities well beyond the ordinary. The overarching topic of this volume is centered on the design and development of assistive technology, user interfaces and interactions that seamlessly integrate with a user’s mind, body and behavior in this very way–providing enhanced physical, sensorial and cognitive capabilities. We call this “Assistive Augmentation”.
Jochen Huber, Roy Shilkrot, Pattie Maes, Suranga Nanayakkara

Sensory Enhancement and Substitution


Augmented Sensors

Key elements of the emerging field of Assistive Augmentation are both the substitution and enhancement of senses–means towards “augmented sensors”. We use the term “augmented sensors” to introduce the following subsections of this part of the volume that focus on enhancing a particular sensory channel, remapping information from one sensory modality to another and creating new sensing modalities. We do so by describing our joint vision of such technology developed at the Augmented Human Lab (Singapore University of Technology and Design). We sketch out research thrusts and enablers, highlight application domains and speculate about the future of augmented sensors.
Suranga Nanayakkara, Jochen Huber, Priyashri Sridhar

Scaffolding the Music Listening and Music Making Experience for the Deaf

Music is an important part of our daily life. We listen to the radio, enjoy concerts or make music.
Benjamin Petry, Jochen Huber, Suranga Nanayakkara

Teach Me How! Interactive Assembly Instructions Using Demonstration and In-Situ Projection

When ordering a product the options to personalize or customize the items have increased over the last years. This flexibility has lead to an increasing number of variants in manufactured products. As storing produced items is expensive, companies tend to produce their products lean, i.e. in smaller lot sizes just when they are needed. This lean manufacturing creates more flexible production environments. In our work, we investigate how human workers can be assisted to work in such demanding environments. Therefore, Augmented Reality systems can be used to provide work instructions. First, in this chapter we provide a comprehensive overview about Augmented Reality approaches to support workers directly at the workplace and introduce an assistive system for providing in-situ instructions. Through three user studies, we evaluate the general impact of in-situ instructions, evaluate three instruction creation strategies, and finally evaluate the created instructions using a real product assembly task.
Markus Funk, Lars Lischke, Sven Mayer, Alireza Sahami Shirazi, Albrecht Schmidt

Augmented Narrative: Assisting the Reader with Sound

Augmented Narrative is literature that assists readers by augmenting the narrative text with sound effects. The assistance comes when the reader is not being engaged into the storyline, using smart glasses as a biofeedback that measure levels of engagement through the temperature of the nose. The concept emerges from literary theory, translated into cognitive and computational terms, and put into a framework of cognitive ecology to design literature that provides with an embodied experience of the storyworld.
Susana Sanchez Perez, Naohito Okude, Kai Kunze

Design for Assistive Augmentation


Design for Assistive Augmentation—Mind, Might and Magic

In this preface I argue that the Design for Assistive Augmentation should take 3M’s into consideration: Mind, to observe before acting, to be thoughtful, and open-minded; Might, to consider the capacity, and competency of people and the technology; and, Magic, to have technology wonderfully blended in everyday life activities. I discuss the idea of building computational tools as a way of working, the idea of Creating Unique Technology for Everyone, and the asipiration to create new types of physical and computational dumbbells to help people to build muscles, running shoes to help people run faster, and skis to enable people to have new kind of experience enabled by the tools and to augment human capabilities.
Ellen Yi-Luen Do

Designing Augmented, Domestic Environments to Support Ageing in Place

[S]tereotypes are useful for camouflaging the social arrangements which we impose upon the aged members of our society.
Jeannette Durick, Linda Leung

Sensory Conversation: An Interactive Environment to Augment Social Communication in Autistic Children

For autistic people, sensory interactions throughout daily life are augmented by profound and impactful experiences, not consciously considered by many in the mainstream, or neurotypical population.
Scott Andrew Brown, Petra Gemeinboeck

FingerReader: A Finger-Worn Assistive Augmentation

The FingerReader is a finger-augmenting device equipped with a camera that assists in pointing and touching tasks. While the FingerReader was initially designed as an assistive device for sightless reading of printed text, it expanded to support other activities, such as reading music. The FingerReader is shown to be an intuitive interface for accessing pointable visual material, through user studies and quantitative assessments. This article discusses the origins, design rationale, iterations, applications, evaluations, and an encompassing overview of the past 4 years of work on the FingerReader.
Roy Shilkrot, Jochen Huber, Roger Boldu, Pattie Maes, Suranga Nanayakkara


Weitere Informationen

Premium Partner