Ambient Intelligence in Everyday Life

The human face is used to identify other people, to regulate the conversation by gazing or nodding, to interpret what has been said by lip reading, and to communicate and understand social signals, including affective states and intentions, on the basis of the shown facial expression. Machine understanding of human facial signals could revolutionize user-adaptive social interfaces, the integral part of ambient intelligence technologies. Nonetheless, development of a face-based ambient interface that detects and interprets human facial signals is rather difficult. This article summarizes our efforts in achieving this goal, enumerates the scientific and engineering issues that arise in meeting this challenge and outlines recommendations for accomplishing this objective.

Empathic computing is an emergent paradigm that enables a system to understand human states and feelings and to share this intimate information. The new paradigm is made possible by the convergence of affordable sensors, embedded processors and wireless ad-hoc networks. The power law for multi-resolution channels and mobile-stationary sensor webs is introduced to resolve the information avalanche problems. As empathic computing is sensor-rich computing, particular models such as semantic differential expressions and inverse physics are discussed. A case study of a wearable sensor network for detection of a falling event is presented. It is found that the location of the wearable sensor is sensitive to the results. From the machine learning algorithm, the accuracy reaches up to 90% from 21 simulated trials. Empathic computing is not limited to healthcare. It can also be applied to solve other everyday-life problems such as management of emails and stress.

The eWatch is a wearable sensing, notification, and computing platform built into a wrist watch form factor making it highly available, instantly viewable, ideally located for sensors, and unobtrusive to users. Bluetooth communication provides a wireless link to a cellular phone or stationary computer. eWatch senses light, motion, audio, and temperature and provides visual, audio, and tactile notification. The system provides ample processing capabilities with multiple day battery life enabling realistic user studies. This paper provides the motivation for developing a wearable computing platform, a description of the power aware hardware and software architectures, and results showing how online nearest neighbor classification can identify and recognize a set of frequently visited locations. We then design an activity recognition and monitoring system that identifies the user’s activity in realtime using multiple sensors. We compare multiple time domain feature sets and sampling rates, and analyze the tradeoff between recognition accuracy and computational complexity. The classification accuracy on different body positions used for wearing electronic devices was evaluated.

Ambient Intelligence [1] aims to improve the quality of people’s life by making everyday activities more convenient and enjoyable with digital media. Technically, Ambient Intelligence refers to the presence of a digital environment that is sensitive, adaptive, and responsive to the presence of people. Electronic devices are embedded in furniture, clothing or other parts of the environment; the technology recedes into the background of our everyday lives until only the function (i.e., the user interface) remains visible to people. At the same time, the human moves into the center of attention, in control of the devices around him. These work in concert to support the performance of everyday activities in an intelligent manner.

Hyper-reality describes distributed computing interfaces that weave existing environments with additional channels of sensory feedback to enhance everyday activities without confusing users. To be intuitive and non-intrusive these interfaces use illusionary pre-attentive content that is co-located with the objects and surfaces of a space and synchronous with a user’s actions. Interfaces for an office, a laboratory, a kitchen and a public hallway are presented along with user studies suggesting that augmenting sensory feedback has the potential to simplify tasks and make them safer, while expanding the potential for interaction with everyday environments.

We present Ambient Pre-Communication for always-on tele-communication systems. It allows the Caller to feel more comfortable in initiating communication, as comfortable as in the real world. To start communication in the real world, determining the appropriate voice volume is essential. However, existing tele-communication systems fail to support the Caller in controlling voice volume because the systems don’t provide feedback of the Caller’s voice at the Receiver’s site. Our proposal allows existing tele-communication systems to provide visual feedback. The key idea is to superimpose a ripple-peak-meter, which represents the sound field at the Receiver’s site, on the Receiver’s video signal. The Caller can control his/her voice volume using the ripple-peak-meter. We describe a prototype system and experiments in which we observed the Caller’s response to the ripple-peak-meter display. The proposed method allows the Caller to easily grasp and control the volume of his/her voice at the Receiver’s site and encourages him/her in commencing tele-communication.

Recently, due to the remarkable advancement of technology, the ubiquitous computing environment is becoming a reality. People can directly obtain information anytime from ubiquitous computer. However, conventional computing style with a keyboard and a mouse is not suitable for everyday use. We proposed and developed a Web browser called the AmbientBrowser system that supports people in their daily acquisition of knowledge. It continuously searches Web pages using both system-defined and user-defined keywords, and displays sensors detect users’ and environmental conditions and control the system’s behavior such as knowledge selection or a style of presentation. Thus, the user can encounter a wide variety of knowledge without active operations. It monitors the context of the environment, such as lighting conditions and temperature. In addition, it displays Web pages incrementally in proportion to the context. This paper describes the implementation of the AmbientBrowser system and discusses its effects.

Query by Tapping is an emerging paradigm for content-based music retrieval, which we have explored through our web-based music search system. Based on the results obtained from our system we argue that searching for music by tapping the rhythm of a song’s melody is intuitive and effective. In this paper we describe two novel algorithms to analyze tapping input. We present results indicating good accuracy rates among a broad spectrum of both trained and untrained users. Query by tapping has an important potential as a form of human-computer communication. We indicate how our algorithms to analyze tapping might be used in other areas such as music education and user authentication.

The classical approach to improve human-machine interaction is to make machines seem more like us. One very common way of doing this is to try to make them able to use Human Natural Languages. The trouble is that current speech understanding techniques do not work well in uncontrolled and noisy environments, such as the ones we live and work in. Nor do these attempts mean that the machines use our languages in the way we do: they typically don’t speak much like we do, and we mostly have to speak to them in special unnatural ways for them to be able to understand. Rather than require people to adapt how they speak to machines, so that the machines can understand them, we present a simple artificial language, based upon musical notes, that can be learned and whistled easily by most people, and so used for simple communication with robots and other kinds of machines that we use in our everyday environments.

We summarize our research results on an innovative approach to making smart meeting rooms accessible to hands-free users. Specifically, we developed an autodirective system to acquire speech in a noisy room using a microphone array, and to identify the speech from a privileged speaker among others in real time. We successfully established that a commercial speaker-dependent speech recognition product could recognize beamformed speech acquired using our autodirective algorithm. We used the NIST Smart Flow System and the Mk-III microphone array developed by the National Institute of Standards and Technology to conduct our experiments.

The latest mobile devices are offering more multimedia features, better communication capabilities (Bluetooth, Wi-Fi, GPRS/UMTS) and are more easily programmable than ever before. So far, those devices have been used mainly for communication, entertainment, and as electronic assistants. On the other hand, Ambient Intelligence (AmI) is emerging as a new research discipline merging the fields of Ubiquitous Computing and Communications, Context Awareness and Intelligent User Interfaces. The ultimate goal of AmI is to surround our working and living environments with context-aware, cooperative and invisible devices that will assist and help us in our everyday activities. Current mobile devices, which accompany us anywhere and at anytime, are the most convenient tools to help us benefit from AmI-enhanced environments. In other words, mobile devices are the best candidates to intermediate between us and our surroundings. In consequence, this paper proposes a middleware which aims to make this vision reality following a two-fold objective: (1) to simplify the creation and deployment of physical spaces hosting smart objects and (2) to transform mobile devices into universal remote controllers of those objects.

The elderly population in the world is increasing rapidly and consequently so is demand for new technologies that allow them to live independently. Facilitating the control of household appliances and the home environment through various devices that encompass multimodal and ambient interfaces seems a way to achieve this. In this paper, we lay out the theoretical principles relating to the accommodation of technology for use in the home among older people, followed by a report supporting these principles based on experiments we have carried out. Three modalities of output – audio, visual and multimodal – were tested using two different devices – palmtop and laptop – as realistic prototypes of household appliance controllers. Through experimental design, the applicability of using icons and musical earcons as a medium to transmit information to the user and its suitability to the home was investigated. The use of musical earcons allowed the potential for an ambient interface to be compared with a traditional visual interface for older people. Results showed participants performed markedly better using the multimodal and visual interfaces than with the audio interface. In addition, both groups performed better using the palmtop as compared to the laptop.

People with disabilities in general, and wheelchair users in particular, are one of the groups of people that may benefit more from Ambient Intelligent (AmI) Systems, enhancing their autonomy and quality of life. However, current wheelchairs are usually not equipped with devices capable of accessing services in AmI environments. In this paper, we describe how an electric wheelchair is equipped with an UPnP based module that allows the integration in AmI systems.

Scientific progress is exponentially increasing, and a typical example is the progress in the area of computational biology. Here, problems pertaining to biology and biochemistry are being solved by way of analogy through the application of computational theories from physics, mathematics, statistical mechanics, material science and computer science. More recently, theories from language processing have been applied to the mapping of protein sequences to their structure, dynamics and function under the Biological Language Modeling project. Scientists from diverse computational and linguistics backgrounds collaborate with experimental biologists and have made significant scientific contributions. The essential component of this collaborative discovery is the web server of the biological language modeling toolkit that enables the computational and non-computational scientists to interface and collaborate with each other. The web server acts as the computational laboratory to which researchers from a variety of scientific disciplines and geographical locations come to characterize specific attributes pertaining to their protein or groups of proteins of interest using the available tools. They then combine the results with their domain expertise to arrive at conclusions. The web server is also useful for education of students entering into the research field in computational biology in general. In this paper, we describe this web server and the results that were arrived at through local and global collaboration and education.