HCI International 2018 – Posters' Extended Abstracts

Learning Management systems (LMSs) are becoming integral parts of the teaching and learning process in higher learning institutions. As they are supposed to be used by students who are diverse in terms of ability/disability, gender, learning style, experience, and other factors, they must be designed to be accessible and learnable to all to the extent possible. Several studies have looked into accessibility and usability of LMSs employing different techniques. This study aims to contribute to the existing body of knowledge by providing the user’s perspective. The Oslo Metropolitan University is in the process of replacing the LMS Fronter with Canvas. In this study, eighteen students at who have access to the two LMSs have been contacted to evaluate the learnability and accessibility of both LMSs though task-based interviews and justify whether Canvas was the better choice. The findings show that there are aspects where one LMS is better than the other. The paper thus concludes by providing pointers that could be important to ensure accessibility and learnability of LMSs during their implementation.

Being a leading cause of death and serious long-term disability across the world, stroke and cerebrovascular diseases became a major burden on health and social care. However, research has shown that early therapy intervention with stroke patients has the potential for significant improvements in terms of cognitive and motor abilities. Integration of technology into rehabilitation such as the robot-assisted therapy, virtual reality applications, and telecare systems have changed the way rehabilitation programs are being delivered and it overcame some of the limitations and challenges that come with conventional therapy programs. Understanding the target audience and their behavior is of crucial importance to be considered in an early design phase especially with this kind of systems that have tedious nature and involve multiple groups of users (patients, therapists, and caregivers). In this study, we aim to adopt a participatory design approach that engages users and stakeholders in developing stroke rehabilitation technologies aligned with behavioral archetypes that are modeled around their behavioral perspectives. We believe that using archetypes in user research gives us a better view of behavior in interaction design of rehabilitation systems and provide developers with a model to validate interactive elements in stroke rehabilitation systems and user flows at a macro level. In this paper, a user study was designed to be conducted to validate behavioral archetypes of these groups through interviews and focus group sessions with different physicians, therapists and caregivers in rehabilitation centers.

This article describes the way in which emotions become a crucial component that complements the work of HCI in design and validation of applications created in virtual reality (VR) environments, applicable to the recovery processes of psychological disorders of being’s humans, such as PTSD.

In this study, a personalized recommendation system for efficient integrated cognitive rehabilitation training based on bigdata was developed. The system consists of 5 main phases (collection, storage, processing, analyzing, visualization). First, in the pre-processing process before the collection phase, resulting scores from multiple cognitive rehabilitation contents and patients’ personal information are saved in database. In the collection/storage phases, the patient information saved in the database is saved in bigdata platform. In the processing phase, the data are processed/refined in a necessary form to be utilized in the analysis and statistical processing program, R. Lastly, in the analysis/visualization phases, personalized contents of integrated cognitive rehabilitation training are recommended to patients using the K-Means method of the unsupervised learning algorithms and spiral model through patients’ personal information, MMSE results, cognitive rehabilitation contents results based on the processed/refined data. Patients can utilize the personalized recommendation system for integrated cognitive rehabilitation training based on bigdata to implement cognitive function evaluation and personalized training at home.

This paper proposes the development of a Virtual Environment that facilitates the process of reeducation of the motor and sensory functions of a patient with hemiparesis, through the simulation of physiotherapy exercises. The methodology used for the development of the work was as follows: a characterization of the variables that affect patients with hemiparesis was realized, analyzing the factors that cause this condition, for the determination of the bases of the Virtual Environment; later a model of Virtual Reality Environment was designed that allows the interaction with the patient for its later implementation using a video game engine that leads us to obtain a prototype of applicability in the rehabilitation of patients. Finally, the results are validated with experts in physiotherapy using system tests to verify the rehabilitation of patients with hemiparesis, reviewing the acceptance criteria of the prototype.

The visual impaired people requires being as much independent as possible to navigate indoor environments. The mobility of this kind of people is difficult and several times requires a companion. Therefore there are several methods for object detection and recognition based on computer vision techniques. This work proposes the use of these methods in the context of accessibility for people with vision problems.

Despite active research in input device development, the visually impaired community still find difficulty in interacting with computers. Braille and other conventional methods have limitations while inputting data to the computer due to their vision loss. Gesture-based interaction can offer them a new vista of computer interaction. However, one needs to consider their performance and preference towards hand gesture. So, a gesture elicitation study is done with 25 visually impaired users. A quantitative rating analysis is performed with them, and an optimal set of gestures is obtained. Further, a dactylology is proposed using which visually impaired users can interact with computers. In this work, we present an insight on the gesture selection method and reveal some key facts about the optimal gestures.

There are many devices and applications dealing with text vocalization and some of them are easily accessible and implemented in mobile devices. These apparatuses are used to enable the access of visually impaired people to texts (without figures), in native or foreign language. However, there are too few devices that meet the deformity correction (typical of books), and picture improvements (enhancing the performance of the vocalization) requirements and low cost, really matters if the target group is found in the public Brazilian schools. From a domestic point of view, there is a gap in the products that fulfill these requirements. This work presents the development of a device that contributes to the autonomy of visually impaired people during their books and common documents reading. The equipment implements an Optical Character Recognition (OCR) process converting the textual content into speech, without the use of a computer, performing the task in an autonomous mode.

Patients with hemiplegia with upper limb impairment have difficulties in performing coordinated bimanual movement of hands and activities of daily living (ADL) due to reduced performance in functional movements. This study developed a serious game for upper limb rehabilitation training using a balance handle specifically designed for patients with hemiplegia. Most current serious games designed for healthy people constitute difficulties for hemiplegia patients to use. For this reason, the simple game interface adapted to accommodate patient disabilities and the content tailored to the needs of these patients are warranted to provide gaming environments according to each patient’s condition. Therefore, this study designed the tailored content that offers an environment appropriate for each patient by evaluating range of motion (inclination of a balance handle) of hemiplegia patients, movement speed and others. This game content is designed based on extension, flexion and movement assessment guidelines for the Fugl-Meyer Assessment (FMA), and measures muscle activities of triceps, biceps and deltoid muscles which demonstrate changes in upper limb motor function while performing the content. The experiment results revealed that triceps, biceps and deltoid muscles were activated. This content is expected to be utilized for rehabilitation purposes in hemiplegia patients with upper limb impairment.

Communication is accounted as one of the main abilities related to humankind development, as it allowed people to pass on acquired knowledge. However, conventional communication favors those with great communication skills, i.e. reading, writing or typing skills summed up to oral language; thus, part of the population is excluded from conventional means of communication. Using assistive technologies such as Augmentative and Alternative Communication (AAC), IVA prototype was developed, enabling communication between patients with motor or cognitive difficulties and health professionals. In addition to their basic needs such as hygiene or food, patients were able to express their wishes, for example to choose which type of food they preferred or which activity they wanted to perform. During the validation and testing of the prototype, healthcare professionals considered that learning LIBRAS would be valuable for patients and other professionals. Thus, the prototype was expanded so that it includes LIBRAS resources, allowing patients and professionals in education and health to use a form of communication through gestures and facial expressions. This new prototype was called LIVA. LIVA assumes the use of communication boards, in which the user selects by click what they want. When selecting an option to express patient’s desire, the information is presented in synthesized voice and the sentence is written in Portuguese; in addition, a pre-recorded video with a LIBRAS expert is presented, showing the same information in LIBRAS. Thus, user can understand what the sequence of gestures and expressions represents. The objective of LIVA is that when viewing the word, phrase or expression in LIBRAS the user reproduces the same signals to express themselves and this may allow greater autonomy for individuals with communication difficulties.

Cognitive impairment that can be caused by brain disorders or aging affects quality of life by interfering with abilities to perform activities of daily living (ADL). For this reason, this study developed serious games for the purpose of rehabilitation and proposes an integrated management service model that can recommend the difficulty level of games and manage changes in serious results according to user’s cognitive function ability during rehabilitation. The areas of cognitive training are designed in consideration of game elements in relation to attention, concentration, memory, orientation, executive function and ADL. Based on the analysis results, we established a database for recommendation of game level in the next training session. Attention is further subdivided into selective, sustained and divided attention. Memory is subcategorized into topological memory, associative memory, verbal memory, immediate memory, visual short-term memory, working memory and recall memory. Executive function is classified into inhibitory control, cognitive flexibility and working memory. ADL includes physical ADL and instrumental ADL. This model comprises a total of 19 serious games for cognitive rehabilitation and each game consists of 3 levels of increasing difficulty. Game interface is created with a user-centered design which focuses on visualization for elderly-friendly use. Our integrated management service model for cognitive rehabilitation can provide training on cognitive function in different areas as a single program and recommend the difficulty level of games depending on player’s ability.

This project introduces a simple device as a passive social medium with the purpose to create an inclusive smart city for all its citizens. Often while designing a smart city, people facing physical limitations aren’t perceived as the target users. Systems and designs lack empathy due to the lack of humanistic data.To remedy this, our idea of an integrated helping system, “We Care”, connects people in public areas to people with physical limitations while also collecting data for the government. This system strengthens community collaboration and human to human communication through the use of digital devices.Our proposed solution includes: Wearable band for disabled people provided by the government. Physical device that receives nearby stress signals in the installed shops. Mobile application that notify nearby people for the given stress signal and also emergency contacts in a crisis Database that stores the location of the signal.This solution aims to drive efficient policy-making and programs especially by the government while fostering empathy amongst the public. It also boosts independence and mobility amongst the people with limitations.This project will establish a direct and more humanistic way of collecting data. It is an opportunity to collect insightful data while creating empathy and social inclusion. With the proposed system, we enable mobility with greater independence amongst people with limitations. For the shop, it improves their economy with increase in potential clients. This becomes an alternative to easy accessibility without infrastructure remodeling and investment. Moreover, this device in the shops can be replicated in different domains such as transportation modes, public buildings etc.

Many programs around the nation are starting or have already implemented digitally mediated courses. Since this form of learning is somewhat new, there is still room for improvement in many areas of this digital learning system. Based on student responses, there could be “better features to better enable interaction and communication.” Also, there has not been much improvement on assisting or helping students with any sort of disabilities. This literature culminates in the research questions: “Does the types of webpage created influence the number of accessibility errors on a webpage?” and “Does the individual instructor who created the content for the webpage influence the number of accessibility errors?” To properly understand and answer these research questions, it was necessary to take both a qualitative and quantitative approach to the research questions. Qualitatively, scales and numeric classifications were applied to qualitative aspects such as page type and teaching experience. Quantitatively, regression models were complied to predicted which independent variables are impactful for total accessibility errors. Additionally ANOVA tests and post hoc tests were utilized to understand if there was significance within the independent variables presented. All p values for all three of the experiments were found to be highly significant. Additionally the post hoc tests denoted that there are significant differences between particular types of web pages and the instructors who produce the content for those pages.

A user has hard time doing touch typing with a software keyboard. As these senses highly contribute to allow a user to recognize the layout of the keyboard, a user often feels disoriented to cause more mistypes once the user loses the senses with a software keyboard. We think that an appropriate key is typed in touch typing with a hardware keyboard. A user approximates layout of the keyboard through feeling of the finger tips, and relative distances between fingers. But hardware keyboard has a problem. This problem is a user can’t approximate layout of the keyboard. A current software keyboard provides no straight forward solution. One more fact to be discussed here is that the size of the keyboard matters for the better touch typing. This paper proposes “flexible keyboard for everyone” to tackle this problem. “Flexible keyboard for everyone” is a tailored software keyboard for every single user. “Flexible keyboard for everyone” learns a user’s habit with two major strategies and tries to help the user the better touch typing. This method allocates wider area for a weak key to pick up unintentional mistypes and increase the rate of appropriate types of the weak key to accomplish the better touch typing.

This paper is about creating a sign language dictionary for use in medical, which shows three-dimensional animation to find expressions in sign language. The dictionary is aimed at standardizing and spreading sign language expressions for use in medical, one of the fields that require sign language interpretation. Many medical terms have no corresponding or standardized expressions in sign language. Therefore, it is hard to correctly communicate disease names and more. We have created a sign language dictionary that has 3D animation so users can learn sign language expressions for use in medical. The dictionary mainly includes terms required for an examination/visit at a hospital. For 1,112 Japanese words, a total of 1,272 sign language expressions are included in the dictionary. All of the sign language expressions can be represented as 3D animation.

Some empirical studies showed that children with autism spectrum disorder (ASD) may have similar mathematical competences compared to their typically developing (TD) peers, but deficits in social communication or interaction skills. To date, a number of commercial applications are available to children with ASD for practicing mathematical and cognitive skills, yet few of them intended to promote social interaction and equipped with analytical tools for evaluating their cognitive and social development. Our current project aims to fill this void by developing an educational tabletop application to strengthen the capabilities of mathematics and cognition for children with autism. It provides several practices on counting, recognition, and measurements and could be played either alone or together with other children. It could also serve as edutainment tools to lessens the burden of special education teachers from repeatedly teaching mathematical and cognitive knowledge.

The challenge of aging has become increasingly serious in today’s China. Among the elderly population, stroke is a disease with characteristics of high incidence, high morbidity and high mortality. Based on the understanding and analysis of aging users and stroke rehabilitation status in China, this paper discusses the way to improve the interaction between the users and the product, so to provide the elderly patients with more convenient, relaxed and interesting experience, as well as a positive and optimistic atmosphere through focusing on their emotional experience and the real needs. To achieve these, the incentive mechanism is introduced. The aim is to help the elderly restore activities of daily living and return to society soon through a more scientific and humane way. Moreover, this paper attempts to explore the interaction design approach and application measures based on the incentive theory, and provide reference for the design and development in this field.

As a new type of pension model, community home care has developed rapidly in China in recent years. But at present, the domestic community home care service system is not perfect. There are many problems in the community home care service system, such as the lag of market development, the imbalance of demand and supply, the backward consumption level and the low quality of the service personnel etc. This paper studies the layout and function allocation of community home service centers in Nanjing, and compares two home care service centers in different communities. By analyzing the layout situation and the key factors affecting its functional configuration in two cases, this paper suggests that the community home care service center should integrate their resources and complement the functions with other centers in order to meet the needs of different elderly people, can not only improve the quality of their life, but also can improve the utilization of resources.

ABLE is a gesture-based interactive platform that transforms physical therapy into game play or art creation – for example, virtual painting or digital music creation. ABLE targets older adults with dementia and fragility, employing art and gaming to encourage playful, physical interactions with family members, peers and care providers. The project aims to forge synergy between physical interaction and creative engagement to produce a range of positive effects; the platform aims to reduce boredom, agitation and social isolation while enhancing physiological, affective and cognitive health. Our interdisciplinary team of medical and health scientists, computer scientists, humanities scholars and artists together contribute the aptitudes required to develop ABLE with attention to the specific needs of these users, to design wearable biometric sensors for data capture, and to develop the app in a consumer-friendly interface appropriate for independent use in residences and homes. We are also developing ABLE with physical therapists, to create a menu of scalable physical therapy exercises designed to enhance strength, balance, and agility for variable populations with frailty and dementia presenting with low to severe impairments. As well, we are co-developing the platform with a range of participants (hospital patients, supported housing residents and home residents) to ensure that the experiences are pleasurable and encourage sustained use of over time. Offering a host of physiological, affective and social engagement benefits, ABLE aims to assist older adults, as they age, to stay mobile, active, and engaged with community and the people they love.

Due to the intensified aging of Chinese population, the ratio of elderly people in China’s total population has been increasingly in each year, Elderly community, which has developed into a relatively mature way of old-age care in developed nations such as European countries and the US, is still a relatively new topic in China. Therefore, How to build an entertainment space may be a problem.The maker space in university library is a “destruction and rebuilding” of the original maker space, it reflects more of the auxiliary role in innovation process, while elderly community is the type of elderly living space integrating elderly care and living into one. The 3D printer and digital creation technology in maker space are a type of technology that quickly transforms thoughts into physical products. While enriching the entertainment activities of elderly community, it is necessary to build certain technological method to enable elderly people to, through simply operations, transform their thoughts into physical products. Such unique maker-space-oriented community space design, and its provided methods for elderly entertainment, or its comprehensive keynote of helping people to exercise their physics and mind, is similar to that of maker culture. The embedded-type maker service that is promoted in maker space of university library is to provide corresponding assistance and help by analyzing readers’ innovative thoughts, by integrating such service model into some service models in entertainment space, it could help elderly people to complete their needed creation in the process of entertainment.So, building the entertainment space in elderly community into an unique maker space model like that that in university libraries and strengthening the external world’s physical and psychological stimulation to them could effectively help slowing down elderly people’s intelligent degeneration and organ aging rate. This thesis implements detailed survey on elderly community, finds its existing problems and studies the related entertainment methods, meanwhile, detailed analysis is made on the related characteristics of the maker space of library, and such type of innovation method represented by maker space is integrated with the entertainment way of elderly community, and the entertainment space of elderly community is finally created to make arrangement and allocation of its different activities.

Purpose: Taking the elderly as the research object, this paper puts an emphasis on the inner needs of the elderly in an important position, and prevents the designer from subjectively replacing the user’s expectations with his or her own ideas. Method: This paper proposes two kinds of research methods: interactive experience design method and implicit knowledge explicit design method. It analyzes the user experience process of senior social software, and studies the user expectations and design elements of senior social software use. Conclusion: The deeper exploration and summary of the needs of the elderly will have a certain reference value for the development of social software for the elderly.

This study examines two traditional technology acceptance models and their applications in Aging Service Technology (AST). Technology Acceptance Model (TAM) and United Theory of Acceptance and Use of Technology (UTAUT) have been widely used in diverse technology adoption research. However, these models excessively depend on self-reported subjective measures, attitudes and perceptions of users, and their solutions lack practical contributions. In addition, both models lack unique contextual barriers needed for understanding the AST adoption process. To complement their limitations, a new approach that includes not only additional contextual constructs but also objective measures such as behavioral data or physiological measures is suggested. The new approach will contribute to the development of practical solutions to achieve Aging in Place.

Based on the user-centered research method and the principle of human-computer interaction, and considering the characteristics change and influence factors of the visual mechanism of the elderly group, the information structure and visual perception elements of the vehicle information interface were designed and analyzed in this paper. The color, visual interface model, the icon and characters of the interface were tracked by eye, and then the dialog interface, graphical user interface and touch operation of the on-board information interface were analyzed to get the visual characteristics model and related design strategies of the elderly group. Based on the interactive design model and strategy of vehicle information interface from the research, this paper chose a representative vehicle information interface and carried out the depth research bu combining with the old user’s actual use demand, and used the eye movement tracking technology and the GOMS model principle to evaluate. According to the design model and strategy, the design and practice were carried out from the perspective of the research. The old people can be provided for a more comfortable and friendly user interface and user experience to ease their oppression in operation, in this case, the old people are given more academic attention and humanistic care, which will provide theoretical guidance and technology support for the follow-up related research and practical work.

Objective: This article proceeds from “sense of loneliness of old people”, and takes “Nanjing Happy Valley Gerocomium” as an example to verify the positive role played by the games in improving the sense of loneliness, thus providing future design of the game for the old with an effective theoretical support and design ideas. Methods: At first, the study sorts out the concepts of RPG games and classifies them by content through document research, then analyzes the current situation and development trends of RPG games, and introduces development and current situation of games for the elderly. Next, the study observes and records daily activities of elderly people in “Nanjing Happy Valley Gerocomium” through document research and selects typical respondents for one-on-one interview to identify the needs of the elderly for game design according to their physiological and psychological characteristics. Conclusion: With research methods including empirical analysis and in-depth interview, this article finds out the core reasons why the elderly have a sense of loneliness, proposes design principles of the games suitable for old people for the this problem, and explores the positive role played by game in improving sense of loneliness, thus providing future design of the game for the old with an effective theoretical support and design ideas.

Background: Establishing a comprehensive community-based elderly care service system has become a major strategy to address aging issues in China. E-health applications such as a self-monitoring health device could potentially solve many of the challenges faced by elderly care. In this study, we adapted an evaluation framework to investigate the effect of an integrated care model.Methods/Design: This study mainly evaluates the model’s effect on the users’ self-rated health, hypertension control and life-style change. We recruit 180 elderly participants (over 60 years old) from three communities in Beijing, Hefei and Lanzhou. A comprehensive questionnaire will be used in the baseline survey. A pair of participants with similar statues will be randomly assigned to intervention groups and control groups. The intervention group will receive a home-based self-monitoring device that records daily blood pressure data. Primary care physicians will contact them bi-weekly to give feedbacks on their blood pressure control and other care need. Additionally, participants’ family members will be involved in the process to improve participants’ adherence. After six months of intervention, we will conduct another questionnaire survey that elicits participants’ responses on self-rated health, life-style, satisfaction and cost-effectiveness of the device. Feedbacks from the primary care physicians, family and study participants will be collected as well through face-to-face interviews.Discussion: Our design attempts to capture the effectiveness of integrated E-health care model from different angles. If this model can improve elderly health management, it will in turn provide a feasible and effective solution for establishing the eHealth-based community collaborative elder care nationally wide.

Increasingly more elderly make use of social media in the Netherlands, being at the forefront compared to other European countries. We analyze the language use of Dutch (older) adults in Twitter in order to get insights in their behavior. Previous studies have identified several linguistic features that change as life progresses including pronouns. We don’t find this change and we claim that this might be due to the fact that the users tend to adapt to the language of the platform. This is not the case for the topics addressed since we find differences in the use of hashtags, while the elderly use hashtags mainly in relation to leisure and politics, the younger users employ them mainly in relation to their working life.

This study investigated how supporting search goal refreshing in working memory could cope with older adults’ lower fluid abilities and helped them search for information with a search engine. 18 older and 30 young adults searched for information to four complex search problems with a classic search interface or with a search interface that displayed at all time their ultimate query. Results showed that the support tool improved older adults search strategies when navigating (i.e. reduced the amount of content webpages visited multiple times and the exploitation of the queries produced) and when reformulating (i.e. reduced the time required to reformulate their query).

Mild cognitive impairment (MCI) is considered as the intermediate stage between the cognitive changes of normal aging and dementia, which is marked by decline in memory but preserved ability to perform activities of daily living compared with that of healthy individuals of the same age. These patients need to delay the progression to dementia through early diagnosis and use the standardized Mini-Mental State Examination (MMSE) as an assessment tool.This study realized the content into a convenient mobile application for assessment of cognitive abilities based on MMSE for Dementia Screening (MMSE-DS) modified as a Korean version, and verified the consistency of a modified version compared to MMSE-DS and user satisfaction. Most content items consist of multiple-choice questions with four answer choices to allow convenient assess to the content via mobile devices. Since it is challenging to identically express questionnaire items assessing patient’s active performance as a content, items are represented as following: First, in an item on folding a given paper in half in response to examiner’s command, a subject is asked to draw a line on a yellow rectangle at the location where paper can be folded in half. Second, in an item on exact copying of two intersecting pentagon figures, a subject is asked to find a pentagon figure same as the pentagon shape in the given example. As a result, the Intraclass Correlation Coefficient (ICC) was 0.938, showing a significantly high correlation.

A study was conducted to evaluate the cross-cultural differences of the elderly preferences on touch-based interface design. Hofstede’s theory of cultural dimensions [3] was used as the theoretical framework to guide this study. In this study, different age groups of Korean and Indonesian elderly were recruited to do the usability testing of a touch-based interface design. A prototype was designed with the culture-related features to compare their preferences. Value Survey Module (VSM) 2013 was used to collect their cultural dimensions scores. Usability rating questionnaires were used to measure their satisfaction scores toward the prototype. The interface features rating questionnaire was distributed to collect their preferences toward the touch-based interface design. The objectives of this study were: 1. To identify the differences in cultural dimensions scores, 2. To identify the differences in usability testing scores based on Nielsen’s evaluation criteria, 3. To identify the differences in touch-based interface design preferences of the elderly based on different cultures. Significant differences are found in the cultural dimensions scores, as well as the touch-based interface features ratings. Significant differences are also found in the usability testing which included four criteria: efficiency, learnability, errors, and satisfaction. The results of this study can be used to improve the design of the touch-based interface for the elderly based on their cultural differences.

This paper presents the results of an evaluation of the Augmented REality Sandtable (ARES) as a training tool during the Chemical, Biological, Radiological, and Nuclear (CBRN) Captain’s Career Course (C3) Table Top Exercise (TTX). Two teams, one that used ARES and one that did not, were compared across a series of course assessments, knowledge acquisition tests, and self-reported questionnaires. The ARES team used the system to develop various map overlays, evaluate their proposed strategies while integrating feedback from the CBRN plume-transport and dispersion simulations, and brief their results to the course instructor for evaluation. Results reveal an overall positive perception of ARES in terms of supporting the development of course outputs. Recommendations for future iterations of the system were gathered from the ARES team following the TTX.

This poster describes a transdisciplinary research concerning the development of an interactive binaural soundscape that responds to user 3D displacement. Soundscapes can be described, according to R. Murray Schaffer (1977) in his book “The Tuning of The World” as an immersive environment which is defined by the combination of all acoustic resources, natural and artificial within a given area as modified by environment. These soundscapes might be generated through the recording of an “natural” specific environment using ambient microphones or it might be completely synthesized using pre-recorded acoustic resources in order to simulate an specific soundscape. The development of artificial soundscapes opens up a whole new range of possibilities such as the development of virtual environments that are able to create an immersive context that allows the user to feel inside the virtual environment. When we create a soundscape that is able to respond to user displacement in a 3D environment like a room, for example, can navigate in a virtual 3D sound environment. Trough the use of a system of cameras we are able to track the user position in a 3D environment and adjust accordingly the previous created soundscape using binaural synthesis. The present project uses OpenCV and Pure Data in order to develop this interactive environment that might be used to develop games for the deaf and to create soundscapes to Virtual Reality environments.

Mixed reality systems immerse users into environments where reality is bridged with virtual worlds. The proliferation of augmented reality compatible devices constitutes a useful means to overcome application limitations. The research presented in this paper focuses on the enhancement of mixed reality environments using mobile applications by altering the virtual parts of mixed reality environments, enriching application functionality, promoting social interaction and facilitating user-generated storytelling authoring and narration. The presented ongoing work builds upon a green screen mixed reality application which can be used in combination with one or more augmented reality application instances, showcasing the benefits of employing mobile augmented reality applications to complement MR systems.

The gaming industry always faces the continual challenges of striking balance between real-time computing performance and graphics presentation quality. This is particularly apparent for a game with vast wildlands scenes which demands for a large number of tree models. From the visual perception perspective, this paper investigated whether people can distinguish trees with uniqueness of various features (relative orientation, density of branches, leaf color) using the same tree model in a virtual environment. During the controlled experiment, quantitative research methods were used, which measured the statistics of tree variations. The results suggested that our methods successfully contributed a strategy in saving graphic resources without user consciousness and maintaining high graphics performance.

Various augmented reality systems have been developed and used in diverse areas recently. However, it is still difficult a user to create augmented reality contents since most authoring systems require 3D objects to create contents. Most users are not familiar with the tools that are used to create 3D objects. As a result, most users only create augmented reality contents with provided 3D objects and it limits the diversity of the contents. In this paper we proposed a picture book-based augmented reality authoring system. A user could create diverse augmented reality contents with 2D objects extracted from captured or provided images instead of 3D objects with the proposed system.

With increasing task and system complexity, it becomes necessary to support workers, e.g. performing repair tasks, from a remote location. Current approaches utilize images or a video stream combined with annotations and speech to allow collaboration with remote users. We propose a technique that gives the remote supporter the ability to see a high fidelity point cloud of a real world object in Virtual Reality (VR). The VR user can indicate points of interest via a laser pointer. The local worker sees these indications on top of the real object with an Augmented Reality (AR) headset. A preliminary user study shows that the proposed method is faster and less error-prone regarding the comprehension of the object and the communication between the users. In addition to that, the system has a higher usability. This work shows that even non-virtual, collaborative tasks can be supported by new forms of user interaction using different technologies like VR and AR.

Virtual Reality systems offer great possibilities to analyze and interact with data. However, they still lack a commonly accepted, efficient text input technique that allows users to record their findings. To provide users with an efficient technique for text input, a real keyboard and the user’s hands are transferred into the virtual world. This allows real haptic feedback of the device and, as a user study shows, results in fast and accurate text writing. The proposed approach shows that a real-world ability can be transmitted directly into the virtual world without much loss.

Virtual Reality applications transfer users into immersive virtual environments, which can be shared with other remote or local users. Yet, multiple local users may collide, since the HMD obscures the view on the other party. To overcome this issue, a user avatar allows to estimate the location of others and to avoid collisions. However, different avatar representations can be used. We compare four avatars in a 28 participant user study. The comparison shows that the different avatars affect the number of collisions and the overall feeling of safety. As a result, local VR collaboration applications should represent the body of the user or highlight his or her location to increase user safety.

There still lacks a collaborative framework that integrates existing and new virtual reality systems supporting remote collaboration for data visualization. The collaborative framework currently available might limit the collaborators for collaboration because it is not developed to support interaction with diverse 3D applications while using heterogeneous virtual reality (VR) systems remotely in a synchronized way. We will present our collaborative framework that supports remote collaboration across heterogeneous VR systems to interact with multiple 3D applications. This system is independent of any VR systems and can be used to interact collaboratively in real-time on a shared task. Additionally, we conducted a pilot study to gain perspective on the usability of the system for collaborative work across heterogeneous VR systems. We will present the results on the usability of the system, and the results obtained from the users are motivating and encouraging. Our system has the potential to increase task effectiveness and workflow because it enables shared and real-time interaction while remotely collaborating with diverse 3D applications across heterogeneous VR systems.

In this paper, we present a novel grasp and release method for direct manipulation in virtual reality. We develop a new grasp method in complementary to the proposed release method. Data gloves and smart sensors may achieve high accuracy and speed but users are prone to fatigue after a long use and devices can be expensive. Therefore, we focus our study to develop an algorithm to reduce fatigue and costs of the virtual system. We track the real hands using a depth camera and determine grasp and release states using virtual rays. Without wearing any devices, users are able to grasp and release a virtual object quickly and precisely. Just as eagles use their talons to catch prey, finite virtual rays from the finger tips are used to determine the grasp and release states. We present that our method improves on grasp time, release time and release translational error based on grasp and release tasks. In contrast to the natural interaction metaphor, measured grasp time, release time and the release translational error for our method do not depend on the object types and sizes.

Since the revival of the arcade game market in Japan, a variety of sensory rhythm games have appeared. Among them, KONAMI’s ‘Drummania’, which provides a controller similar to a real drum, became a popular game that continued after its release in 1999. And ‘Drummania’ became the standard for later drum games. Conventional drum games provide players with information about their performances in GUI form through display. At this time, since the line of sight of the player is fixed on the game screen, recognition of each part of the drum is not intuitive. In this research, we propose more intuitive drum contents by applying mixed reality and virtual reality technology to solve this problem. In order to evaluate the intuitiveness of the proposed system, it is applied to the drum learning contents to verify the effectiveness.

Augmented Reality and Mixed Reality technologies make us feel the real world and the virtual world are mixed with each other through visual matching. However, existing mixed reality contents using these technologies have a limitation that interact only with virtual objects and cannot interact with real objects. If a dynamic real object need to be influenced by a virtual object, or vice versa, illusion of Mixed Reality can be broken by the incongruity of the interaction. If these limitations are improved, real objects, which were always static, can operate dynamically and interact with virtual objects, so becomes difficult to distinguish between real objects and virtual objects. In this paper, we propose extended interactions in a mixed reality environment that can give illusion which is difficult to distinguish between real objects and virtual objects. And to confirm this, we implement authoring environment based Hololens.

The exciting thing about augmented reality (AR) as an emerging technology is that there is not yet a common design language, nor a set of standard functionalities or patterns. Users and designers do not have much experience with AR interfaces. When designing AR projects for customers, this is a huge challenge. It is our conviction that prototypes play a crucial role in the design process of AR experiences by capturing the key interactions of AR and delivering early user experiences situated in the relevant context. With ExProtoVAR, we present a lightweight tool to create interactive virtual prototypes of AR applications.

Facing one’s fears could be a way forward for people whose anxieties debilitate them. Exposure-based treatment is an evidence-based approach to reducing pathological anxiety. As a part of treatment, therapists who practice exposure therapy create environments to provide opportunities for patients to experience (and ultimately, overcome) the anxiety that arises for patients in feared situations. More than 50% of anxiety patients, however, never receive treatment. We developed an Android application that uses augmented reality (AR) for providing opportunities for exposure to spiders, a relatively common presentation of specific phobia. With this app, users can engage with exposure therapy without visits to a therapist. In the next phase, we will conduct a user study to determine whether mobile applications equipped with augmented reality technologies can be used to reduce the anxiety response to a feared stimulus (i.e., a tarantula), and whether the app is considered easy to use and acceptable. In this poster, we will provide details about the mobile app and describe the study setup.

Concussions are a common type of traumatic brain injury that can have severe consequences. They are often induced by sports injuries, car accidents, and even falls. The high acceleration in these situations exert rotational forces on the brain that result in biomechanical injury and a malfunctioning metabolism. We propose a method to diagnose and check the progression of a concussion using virtual reality (VR) in conjunction with electromyography (EMG) and centre of mass (CoM) trajectory. The study will primarily focus on concussions sustained by varsity hockey players, with the hopes of expanding to athletes of other sports in the future.

This paper describes characteristics of walking sensation created by a vestibular display (a motion seat). An active input was introduced to a passive presentation of a walking stimulus. The participant triggered one step motion repeatedly by a game-controller button to introduce agency of motion. First, the magnitude of the seat motion was optimized to increase the walking sensation. Then, passive and partially active seat motion was evaluated. As a result, it was shown that added activity increased the walking sensation .

In the field of optical illusion, reverse perspective is used to draw a scene that is opposite to the actual perspective. When the viewing position is changed, a farther object seems to be always coming toward and following the viewer instead of going away. Therefore, we considered whether the reverse perspective can be applied to a dynamic representation of computer animation using multiple combined monitors. In this research, we arranged three monitors in the shape of a corner cube and tried to determine whether the viewer can recognize the concave corner of the cube as the convex corner through the reverse perspective illusion. Furthermore, we developed a virtual environment that enabled us to simulate the reverse perspective illusion by changing the position, angle, and shape of the screen using a head-mounted display and controllers.

In order for machines to understand or express emotion to users, the specific emotions must be formally defined and the software coded to how those emotions are to be expressed. This is particularly important if devices or computer-based tools are utilized in clinical settings, which may be stressful for patients and where emotions can dominate their decision making. We have reported our development and feasibility results of an ontology, Visualized Emotion Ontology (VEO), that links abstract visualizations that express specific emotions. Here, we used VEO with the VEO-Engine, a software API package that interfaces with the VEO. The VEO-Engine was developed in Java using Apache Jena and OWL-API. The software package was tested on a Raspberry Pi machine with a small touchscreen display that linked each visualization to an emotion. The VEO-Engine stores input parameters of emotional situations and valences to reason and interpret users’ emotions using the ontology-based reasoner. With this software, devices can interfaced wirelessly, so smart devices with visual displays can interact with the ontology. By means of the VEO-Engine, we show the portability and usability of the VEO in human-computer interaction.

The ever-connected world created by smartphones has led to initiatives like a ‘digital detox’, in which smartphone users consciously disconnect from email, social media and internet in general for a certain period of time. Since research based on subjective self-reports indicates that extensive smartphone usage and stress are often related, we checked whether a digital detox is effectively associated with a decrease in stress in the short-term and whether this could be measured with objective markers of both smartphone usage and physiological stress. More particularly, we monitored participants for two consecutive weeks: one week of normal smartphone usage and one week of digital detox. We asked them to continuously wear a state-of-the-art wristband device, measuring physiological stress based on skin conductance (SC). In addition, we developed an app called ‘mobileDNA’ to capture detailed information on which apps participants use throughout the day and how much time they spend on them. Although this was a pilot study with a rather low sample size, we found decreased levels of stress during a digital detox week. This finding provides evidence that a digital detox can be an interesting coping mechanism for people experiencing problematic smartphone usage and that further and more extensive research with our methodology has a lot of potential in the future.

In this paper, we proposed EREGE system, EREGE system considers as a face analysis package including face detection, eye detection, eye tracking, emotion recognition, and gaze estimation. EREGE system consists of two parts; facial emotion recognition that recognizes seven emotions such as neutral, happiness, sadness, anger, disgust, fear, and surprise. In the emotion recognition part, we have implemented an Active Shape Model (ASM) tracker which tracks 116 facial landmarks via webcam input. The tracked landmark points are used to extract face expression features. A support Vector machine (SVM) based classifier is implemented which gives rise to robust our system by recognizing seven emotions. The second part of EREGE system is the eye gaze estimation that starts by creating the head model followed by presenting both Active Shape Model (ASM) and Pose from Orthography and Scaling with Iterations (POSIT) algorithms for head tracking and position estimation.

There is still considerable disagreement on key aspects of affective computing - including even how affect itself is conceptualized. Using a multi-modal student dataset collected while students were watching instructional videos and answering questions on a learning platform, we investigated the two key paradigms of how affect is represented through a comparative approach: (1) Affect as a set of discrete states and (2) Affect as a combination of a two-dimensional space of attributes. We specifically examined a set of discrete learning-related affects (Satisfied, Confused, and Bored) that are hypothesized to map to specific locations within the Valence-Arousal dimensions of Circumplex Model of Emotion. For each of the key paradigms, we had five human experts label student affect on the dataset. We investigated two major research questions using their labels: (1) Whether the hypothesized mappings between discrete affects and Valence-Arousal are valid and (2) whether affect labeling is more reliable with discrete affect or Valence-Arousal. Contrary to the expected, the results show that discrete labels did not directly map to Valence-Arousal quadrants in Circumplex Model of Emotion. This indicates that the experts perceived and labeled these two relatively differently. On the other side, the inter-rater agreement results show that the experts moderately agreed with each other within both paradigms. These results imply that researchers and practitioners should consider how affect information would operationally be used in an intelligent system when choosing from the two key paradigms of affect.

There is a growing concern about students’ psychological health at school with mounting pressure as students approach middle school or college age. In light of the fact that the current means of psychological intervention are basically manual intervention, this research explored the possibility of using technology to assist psychological intervention. An emotion management system is thus implemented to provide facial emotion cognitive service and video diary function. In addition to the advantage of keeping a normal diary, students benefit by regulating their emotion when they are recording videos. Teachers can keep track of students’ emotion states so that they can provide corresponding psychological supports when students showed signs of mental disorders. We performed a small-scale pilot study at a local school with satisfying results.

Gamification is widely accepted in mobile health applications as one way to enhance user experience. Moreover, linking gamification with insights from behavior change theory offers a promising approach to ensure user’s adherence and long-term behavior change. Gamification is, however, hardly found in current stress management applications. To close this gap in research, we present Stress-Mentor, a stress management app that realizes established behavior change techniques within an extensive gamification framework.The main gamification elements of Stress-Mentor are an avatar, an agent, the accomplishment of regular tasks, experience points, virtual currency, and badges. These features are activated consecutively to keep the user interested. Each gamification element is linked with several behavior change techniques. The regular tasks teach a broad range of proven stress management techniques.Stress-Mentor’s usage duration is limited by design to ensure the user’s autonomy. After the three month usage period, users should be able to apply the stress management methods in their daily life without the app. The presented gamification concept can be easily adapted for other applications to support mental and physical health.

Perception of social cues is a fundamental communicative skill that can be hampered by hearing and cognitive disorders. Understanding slang and sarcastic intent is often difficult in verbal communication, particularly for individuals who struggle with the perception of social cues. Misinterpretation of slang terms can cause discomfort or social isolation. Sarcasm is particularly difficult to recognize due to its inherently ambiguous and context-dependent nature. We have identified two problems of particular interest in social assistive technologies – slang word sentiment assessment and sarcasm detection. We propose combining a slang sentiment analysis model with a speech emotion analysis model to create an assistive tool, Chat-Box, which will detect social cues such as sarcasm, slang, and sentiment.

In recent years, ideas for more enjoyable playing through the use of emotional design concepts like affectivity and pleasurability factors are attracting attention. In this study, we focused on the effect of blinking and moving light patterns on users’ enjoyment of skateboards. An evaluation experiment was conducted to clarify users’ impressions of these light patterns using the following five evaluation items: (1) level of feeling of wanting to move your body, (2) level of feeling of wanting to get closer, (3) level of feeling of being in danger, (4) level of perception of beauty, and (5) level of preference. From the experiment, we found that there are some characteristics in impression of patterns of blinking and moving light and there is a common impression of the lighting pattern in ‘level of feeling of wanting to move your body’ and ‘level of feeling of being in danger’. Moreover, it was clarified that there are some light patterns that users preferred and felt were more beautiful than others.Based on these results, a glowing skateboard, named “Glowboard”, was proposed using an Arduino system and a non-contact rotation speed sensor. “Glowboard” shows various blinking and moving patterns with different colors of light depending on the rotation speed of the skateboard’s tire. From an evaluation experiment with “Glowboard”, we found that using the various lighting patterns is effective not only developing performance through riding, but also for evaluating skaters’ skill in skateboarding competition. The result is illustrates that “Glowboard” is effective for increasing users’ enjoyment. In the future, it is necessary to clarify the impression and influences of different colors of lighting on enjoyment. “Glowboard” will be developed using a wider variety of light patterns based on the results of further study.

Since the pressure of people’s daily life and work, more and more people tend to be nervous and have negative emotions such as anxiety. In order to properly intervene in people’s emotion, introduces a method of using electroencephalogram (EEG) to evaluate the intervention effect of color and sound cross modal on emotional state. The proposed method consists of three phases: emotion elicitation, emotion intervention, and EEG monitoring. The film clips are used to induce four target emotional states: joy, fear, sadness and relaxation, and a group of music clips and ambient lights were used as intervention factors. According to the two-dimensional model of emotion is described and take the valancing as the adjusting index, the ratio of the average power spectral of the left hemisphere to the right hemisphere are calculated to represent the valence change, thus evaluating the intervention effect. The results show that synesthetic combination of music and light has a more obvious effect on negative emotion intervention, the high-pitched and fast-paced music is in particular. And also Our findings highlighting the existence of cross-modal correspondence between sounds and colors in psychology.

The diversity of information interaction is constantly expanding, a variety of information interaction and cultural exchanges have changed people’s behavior patterns. This paper based on the pressure sensitivity of the 3D touch technology, we made a real-time measurement of the pressure, time interval and other physical quantities of users when they press the screen to enter the information, and obtained the users’real-time emotion information combined with semantic recognition, then added it to the text by way of color, font and so on, which can solve the problem of semantic misunderstanding caused by the single expression of the information in text sequence communication, so that communication is more smooth and diversified. Therefore, This paper explores how to overcome the difference of cultural exchange by using the multiple ways of emotional interaction, aiming to build a more effective and effective way of information interaction mode.

Job stress has been regarded as an important issue in the workplace. Hoboubi et al. (2017) proposed that job satisfaction and job stress are important factors that affect workforce productivity. Employees’ job stress not only influences their mind but also leads to poor performance at work. This research has been initiated with the assumption that the stress employees experience is similar to the stress they experience in the workplace. In addition, similar types of stress occur at the workplace and may be caused from meeting the same persons, same contents meetings, same location or same timings. We designed a web based calendar system called “My Stress Calendar” which provides various functions to subjectively record stressful events, relevant memos, and emotions. To evaluate it, we performed a within-subject research. Twenty subjects who are employees were recruited and evaluated for their perceived stress levels using a modified perceived stress scale, before and after using the system (Cohen et al. 1983). The results indicated that there was a significant difference in subjects’ perceived stress score between the experiments. It turned out that recording stressful events is helpful in preventing stressful events and leads the subjects to stable and positive emotions.

The present study was designed to analyze psychological effects of separating users from their smartphones regarding anxiety and “Fear of Missing Out” (FoMO). In a laboratory experiment, participants (n = 85) went through a seven-minute waiting session. They sat alone in a room - either with or without their phone. Results revealed that participants who waited without their phones reported significantly higher levels of anxiety, afterwards. Further, partial mediation was revealed with FoMO significantly mediating the impact of phone separation on anxiety. The findings suggest a psychological relevance of smartphones which reaches beyond the mere functionalities and applications. It is the entity of the phone itself which seems to be psychologically relevant offering a promising approach for analyzing psychological aspects of humans interacting with technological devices.

We have investigated the organization simulator as a means to prevent workplace depression. The simulator and its communication model show the behaviors of human relationships based on the idea of ‘Message Theory’. In this report, we investigate the effects of the message sending/receiving characteristics, and position of the person in the case of an organization where 9 persons belong. The conditions of simulation are (1) average persons, (2) a positive receiving person, (3) a positive sending person, (4) positive/negative person in the center, (5) positive/negative person in the marginal. As a result, the personal and organizational statuses (positions of the persons, mean distance, and mean mental health index) are varied according to the initial condition. Moreover, usage environment of the simulator based on the communication model were discussed.