HCI International 2020 – Late Breaking Posters

Currently, there is a major lack of funding for research regarding Down syndrome (DS) cases compared with other conditions. As a result, it is clear that in today’s world, the development expectations of individuals with DS have been underestimated. A common problem in this environment occurs because many children with DS have dexterity issues that make manipulating small objects and grasping a pen particularly difficult. In an attempt to provide DS children with the ability to write longer with no discomfort, our proposed solution is a novel attachable pen grip that quantifies forces applied by the human hand while grasping the pen to write. Initially, data from a comprehensive questionnaire was collected on grip styles, writing speed, and how many words are written. Several other research methodologies were used, such as surveys, observations and interviews. Participants included experts, specialists, individuals with DS and families of children with DS. In executing this work, we gained a better understanding of the difficulties the child is facing and we were able to refine the design to best accommodate their needs. The market gap and limited writing tools for children with DS motivated this study to create the suitable product to reduce cramping, improve handwriting practice and to fulfil the physical and emotional need for children with DS. The overall potential benefit of the envisioned product is to improve the quality of life for these children and ease the minds of their caregivers.

Crowdsourcing has the potential to become a preferred tool to rate the accessibility of the built environment and appeal preferences of users who are persons with disabilities (PwDs). Nevertheless, some reliability issues still exist, partially due to the subjectivity of ratings of accessibility features of places that might vary from one PwD to another or their caregivers. In this paper, we present a descriptive overview of existing crowdsourcing applications and the accessibility features that are included in such platforms as well as the diffusion, and popularity of these platforms. We also present several use cases and scenarios of use for these platforms via user populations.

Wheelchair tray tables offer a convenient way for wheelchair users to carry out daily tasks such as eating, reading, and using mobile devices. However, most tray tables are made to serve the majority of wheelchair users and are inaccessible to some with a limited range of motion. In our work, we address this issue by exploring the ergonomic problems and possible solutions. In this paper, we describe the human factors in the design and development of powered wheelchair tray tables. The process of humanistic co-design relies on the direct involvement of the targeted demographic in the design process. This ensures the outcome is centered around the specific needs of the individual. Our approach employs user research studies (e.g., interviews, questionnaires, and actively working with a wheelchair using co-designer) as a means towards gleaning valuable insight into the needs of wheelchair users. In these studies, we sought to explore their experiences with using tables made for wheelchairs. We also collected data about whether the tray tables required external assistance to stow and use, and the problems they faced using existing solutions. We then highlighted the various specific needs presented by the co-designers and questionnaire respondents. These needs are embodied into scenario-based personas in which they may find themselves in need of a table for use with their wheelchairs. Deriving these personas from our survey results provides an effective method of keeping the insight gathered present throughout the design process. Implications for design are discussed.

The Brain-Computer Interfaces (BCI) based on Electroencephalography (EEG), allow that through the processing of impulses or electrical signals generated by the human brain, people who have some type of severe motor disability or suffer from neurological conditions or neurodegenerative diseases, can establish communication with electronic devices. This paper proposes the development of an expert system that generates the control sequences for a neuroprosthesis that will be used in the rehabilitation of patients who cannot control their own muscles through neuronal pathways. This proposal is based on the EGG record during the operation of a BCI under the rare event paradigm and the presence or not of the P300 wave of the Event-Related Potential (ERP). Feature extraction and classification will be implemented on a mobile device using Python as a platform. The processing of the EEG records will allow obtaining the information so that an Expert System implemented in the mobile device, is responsible for determining the control sequences that will be executed by a neuroprosthesis. The tests will be performed by controlling a neuroprosthesis developed by the Instituto Nacional de Rehabilitación in México, which aims to stimulate the movement of a person’s upper limb.

Identity aspects such as gender, race, culture, and socio-economic status should be considered when technologies are designed with and for persons [9]. HCI research in ageing populations can benefit from considering the complexity of identity, use and context of older persons. This short paper draws attention to gender as an aspect of identity in working with ageing populations. It presents results from an interview study with elderly women who live alone in Germany. This study is part of a larger research project dedicated to innovating smart home technologies with and for elderly women who live alone. The research project has comprised of an “exploratory getting-to know each other” session, use experience of a smart speaker and interview studies. This paper presents insights from semi-structured interviews with 7 elderly women who live alone. These insights are also presented as design considerations for smart home technology for elderly women living alone. The study is the basis for further work with the elderly women, which are co-creation sessions to design smart home devices and develop prototypes.

Functional communication is indispensable for child development at all times but during this COVID-19, non-verbal children become more anxious about social distancing and self-quarantine due to sudden aberration on daily designed practices and professional support. These verbally challenged children require the support of Augmentative and Alternative Communication (AAC) for intercommunication. Therefore, during COVID-19, assistance must be provided remotely to these users by a AAC team involving caregivers, teachers, Speech Language Therapist (SLT) to ensure collaborative learning and development of non-verbal child communication skills. However, most of the advanced AAC, such as Speech Generating Devices (SGD), Picture Exchange Communication System (PECS) based mobile applications (Android & iOS) are designed considering the scenario of developed countries and less accessible in developing countries. Therefore, in this study, we are focusing on representing feasible short term strategies, prospective challenges and as long term strategy, a cloud based framework entitled as “Bolte Chai+”, which is an intelligent integrated collaborative learning platform for non-verbal children, parents, caregivers, teachers and SLT. The intelligent analytics within the platform monitors child overall progress by tracking child activity in mobile application and conversely support parents and AAC team to concentrate on individual child ubiquitous abilities. We believe, the proposed framework and strategies will empower non-verbal children and assist researchers, policy makers to acknowledge a definitive solution to implement AAC as communication support in developing countries during COVID-19 pandemic.

Augmentative and Alternative Communication (AAC) technology research not only enhance communication but also communal skills in verbally challenged children with Autism Spectrum Disorder (ASD). However, the research on AAC technology is mainly concentrated in developed countries and less explored in developing countries. In this study, we utilized evaluation method to analyze the prospects, existing practices and future possibilities, benefits and barriers of AAC research in developing countries. It is found that Speech Generating Devices (SGD) are mostly preferred by children and in future artificial intelligence (AI) based mobile application will augment communication skills among verbally challenged children. We conclude with general recommendation on succeeding research, collaborative approach and implementation with funding opportunities for substantial growth of AAC technology research in developing countries. This study will facilitate directions for initiating AAC research in developing countries and will accommodate researchers, developers and stakeholders to acknowledge opportunities, barriers, probable and current state of AAC research.

Visually impaired persons recognize their surrounding with a white cane or a guide dog while walking. This skill called “Orientation and Mobility” is difficult to learn. The training of the “Orientation and Mobility Skills” is performed at the school for visually impaired person. However, the evaluation of this skill is limited to subjective evaluation by teacher. We have proposed that quantitative evaluation of the “Orientation and Mobility Skills” is required. In this paper, we tried to execute the quantitative evaluation of the “Orientation and Mobility Skills” using brain activity measurements. In this experiment, brain activity was measured when subjects are walking in the corridor alone or with guide helper. Experimental subjects were sighted person who was blocked visual information during walking. The blood flow of prefrontal cortex was increased as the movement distance of the subject increased when subjects walk alone. From this result, it can be considered that the feeling of fear and the attention relayed to “Orientation and Mobility Skills” could be measured quantitatively by measuring human brain activities.

Various technology transfer models have been introduced in the literature, and those models have the potential to facilitate the design, development, evaluation, and dissemination of emerging technology. Yet, those models are less likely to be suitable to mobile health (mHealth) technology consumers who have disabilities due to lack of user-centered approach to technology transfer. To address the gap, this paper introduced an innovative framework for mHealth technology transfer to those with disabilities. The framework consists of technology concept, technology engineering, technology embedding, and ongoing participatory design. The framework is expected to contribute to usability, accessibility, and safety of consumer mHealth technology for users with disabilities, ultimately leading to enhancement of health-related quality of life and equity.

The most common problem faced by elderly is loneliness, especially when they live far away from familiy members. Furthermore, decreasing physical ability and mobility that comes with aging can also limit elderly’s ability to socialize with others. Fortunately, modern technology offers a solution to this problem of disconnectedness. Nowadays, people are communicating through social media despite the geographical distance between them. However, due to a technological leap, the elderly in Indonesia are often left behind in adapting and using technology. In this study, a social media application interface was designed based on the results of direct interviews with the elderly. The interface is more user-friedly, with features that can be properly used by the elderly. The effectiveness of this interface and its features is supported by the test result of each feature. The contact feature (task 1) scores 80%, the call log history feature (task 2) scores 100%, the video calling feature (task 3) scores 100%, the community feature (task 4) scores 90%, the entertainment feature (task 5) scores 90%, the voice calling feature (task 6) scores 100% and the notes feature (task 7) scores 77%. The results of this study can be used as a reference for software developers in designing application interfaces for the elderly, especially those who experience technological leap.

With the exponential growth in the technological era, the need arose to bring the users a way to use Braille as a communication interface with computers and smartphones. In order to achieve this, electro-mechanical devices were created, called Braille displays, allowing users to make use of Braille on their own devices. However, access to this kind of devices is difficult, because the embedded technology makes it expensive. In this context, this work aims to create an integrated solution of hardware and software, based on the concept of one Braille cell using only open source components. The proposed system was evaluated by blind volunteers with different Braille knowledge and computer experience.

Frailty is accompanied by limitations in activities of daily living (ADL). These are associated with reduced quality of life, institutionalization and higher health care costs. Long-term monitoring ADL could allow creating effective interventions and thus reduce the occurrence of adverse health outcomes. The main objective of this study was to evaluate if ADL task performance can be assessed by activity measurements based on IMUs, and whether these measures can differentiate individual’s frailty. ADL data was obtained from seventeen elderly who performed two ADL tasks - tea making task (TEA) and gardening task (GARDEN). Acceleration data of the dominant hand was collected using an activity sensor. Participants were split up in two groups, FRAIL (n = 6; Fried score ≥ 2) and CONTROL (n = 11; Fried score ≤ 1) retrospectively. Collected data were used to determine trial duration (TD), relative activity (RA), peak standard deviation (STD), peaks per second (PPS), peaks ratio (RATIO), weighted sum of acceleration per second (SUM), signal to noise ratio (S2N) and mean peak acceleration (MPA). STD, RATIO, SUM and MPA showed good reliability over both tasks. Four of the calculated parameters (RA, PPS, RATIO, SUM) revealed significant results differentiating between FRAIL and CONTROL (effect sizes 1.30–1.77). Multiple linear regression showed that only STD correlated with the Fried score. In summary, the results demonstrate that ADL task performance can be assessed by IMU-based activity measures and further allows drawing conclusions on the frailty status of elderly, although the predictability of the exact Fried score was limited.

Brain computer interfaces (BCI) have been successful in enabling control of external effectors, such as a computer cursor or robotic arm by people with severe motor impairment. While current usage relies heavily on the users’ ability to reliably utilize visual feedback from the interface, a visual-based BCI can be difficult or even unfeasible to use by people whose paralysis is accompanied by limitations of eye movements or vision. Here we present a novel auditory paradigm, and present pilot data from a person with tetraplegia performing a BCI task with auditory cues.Inspired by previous studies, we utilize spatial auditory signals - specifically, head-response transfer functions (HRTF) - to provide task-relevant auditory feedback in real-time via headphones to the BCI user. To test the feasibility and reliability of HRTF-based feedback signals, we performed a sound source localization task in lab members and in one BCI participant with tetraplegia using intracortical recordings. Subjects were first tested on multiple azimuths of sound source, instructed to rely on the auditory cues to identify the cued spatial auditory direction, and then to navigate to the cued targets. Initial psychophysics testing suggests cursor navigation is possible in an auditory-reliant manner using the HRTF filters updated in real-time.The results of this study demonstrate the development and implementation of a novel auditory-reliant intracortical BCI that provides real-time auditory feedback for effector control that has high potential for BCI usage in individuals with severe motor paralysis and sustained visual impairments.

Investigating how the brain integrates multimodal information is critical for quantifying the effects of age on performance for tasks that rely on visual and auditory stimuli (e.g., driving or flying an aircraft). We report on how concurrent performance on a visuospatial task and a passive paired-stimulus auditory electroencephalography (EEG) paradigm were impacted by age. Outcome measures included response times and accuracy for a match-to-sample visuospatial task and event-related potentials (ERPs) derived from a 128-channel dense array EEG system. Older participants were less accurate and responded slower to the visuospatial task than younger participants, particularly in a high-workload condition. ERPs associated with cortical language processing areas showed that older participants displayed less sensory gating than the younger group for P50 and N100 ERP components. In contrast, pronounced sensory gating was found in the older participant group in the frontal cortex, which was driven by disproportionately larger N100 responses for the first stimulus. The present findings further the understanding of age-related neural changes and support the notion that a neural evaluation will one day reliably classify risk states for complex cognitive functions in older adults.

Numerous image processing methods have been proposed to help low vision people, often relied on contrast enhancement algorithms. Their assessment is usually performed by tests on low vision subjects, which are expensive and time consuming. This paper presents a low vision observer model, fully customizable to fit various impaired visual performances, which may be used for early algorithm assessment, and avoiding unnecessary human tests. This model is fitted to visual performances of a subject with degenerative retinal disease, and applied to images processed by two edge enhancement algorithms, allowing to explain their performances in terms of blur reduction and color saturation improvement.

As a typical example of the electronic products with multi- and high- functions, integrated- function microwave oven is said to bring a much more convenient and brand-new lifestyle; however, in the meanwhile, it also compels their users to face many problems in using due to its multifunction. In this study, we conducted a usability test with healthy older adults to determine how well the elderly users can use the microwave oven, and what kinds of processes they are taking as their problem-solving. Based on the results of the usability test, we summarized several types of difficulties-in-use and tried to provide the design solutions based on product semantics, especially focusing on the process of transition between exploration and reliance.

Intuitive second screen experiences involving smartphone or tablet use are currently not possible with existing VR systems. Whenever users are immersed in a virtual reality world, interactive features and content are usually presented as pop-up windows and floating-text notifications, immediately breaking user immersion, serving as a reminder they are navigating a computer-generated environment. In this paper we discuss how to introduce second screen experiences in VR by orchestrating interplay between using handheld devices while using a VR headset. This entails allowing the users to interface with the touchscreen display despite the opaque, view-obstructing visor. We propose achieving this through a “mobile second-screen interreality system” (MSIS), referring a virtual 3D smart device representation inside the VR world, which is tightly coupled with its real-world counterpart, allowing users to manipulate, interface with, and look at the actual device screen, as if the head-mounted display were not present. This can be achieved via advanced screen-sharing technology, and pixel-perfect mapping between the real device and its virtual “double”, ensuring basic functions (such as typing on an on-screen keyboard and/or tapping on application icons) can be carried out effortlessly. Advanced hand and finger-tracking can be used to represent the user’s virtual hands’ and fingers’ movements on the screen, further enhancing virtual presence, and cementing the links established between the human brain and virtual reality space.

Virtual Reality (VR) is a technology that allows the creation of immersive virtual environments. Testing VR applications is challenging due to the almost unlimited possibilites of spacial exploration. This work presents Youkai, a framework for unit testing of Android-based VR applications. Our preliminary results show that our method can find objects on the screen, change camera position and it supports 6 DoF scenarios.

This study aims to explore the effective types of signage in augmented reality (AR) environments. Although AR, a state-of-the-art technology, has been utilized in diverse contexts, its application in real-world situations is limited. This study examines the effectiveness of AR signage in mixed-use shopping malls in terms of wayfinding, which is a continuous problem. First, the time it took to make a decision was measured for three categories of signage: existing signage, AR-textual signage, and AR-graphic signage. Thirty directional sign images from the three categories of signage were used in the experiment. The sign images of the current sign system at the mixed-use mall were used as the control condition. Second, subjects were asked in a questionnaire to evaluate each AR-enhanced sign type in 5 aspects: uniformity, continuity, legibility, clarity, and aesthetics. The result found that AR-graphic signage is the most effective type in terms of decision-making time and aesthetics. However, it was found that for legibility and clarity of signage, textual elements should be adjusted in AR signage environments. Thus, this research suggests the use of mixed graphic and textual signage in AR environments.

Virtual reality (VR) is being used more and more as a way to easily visualize and share ideas, as well as a step in product designing. The initial study presented in this paper is part of a project for getting the general public involved in the design of new busses for Northern Jutland in Denmark, by using VR visualization. An important part of VR visualizations is the correct understanding of scale. Studies show that the perception of scale in VR undergoes compression compared to the real world. In this paper we test how additional visual cues, transitional environments and tactile augmentation in a VR environment can help with the perception of scale. We show that familiar visual cues can help with the perception of scale, but do not remove the compression of perception. We can further mitigate the problem by introducing transitional environments and tactile augmentation, but transitional environments provide a better perception of scale than tactile augmentation.

This research is study involves around a VR based cricket game. It uses the technology of VR to simulate the environment of a player playing cricket in the stadium. The research tries to state that how this type of game creates a more immersive and interactive experience compared to regular 2D/3D video games played in traditional display environments.

The design and manipulation of chemical systems involves understanding the form or morphology of chemical structures. An understand of the form of chemical structure includes an understanding of the components of chemical structure, the functions of the forms and sub-components, and changes in the structure of chemical systems during interaction, maturation, or chemical processes. Viewed from a computer graphic viewpoint these chemical processes can be described and modelled as three-dimensional structures, changing shape, and interacting with other 3D structures. Furthermore, our intuition was that the visualization should be as embodied as possible and open for collaboration.In this project we seek to create a tool for collaborative, embodied visualization of biomolecules. To achieve this interaction with targeted for hand on visualizations allowing for biomolecular exploration and scientific visualization within immersive augmented reality platforms. We anticipate a tool where components can assist both in (1) biomolecule discovery and design and a subset applicable for (2) education in biomolecules. We conducted some formative research to analyze user value and requirement.For the prototype we focused on the visualization of DNA binding protein, called Zip Proteins. These proteins are transcription factors. This system is implemented across two devices that support AR capabilities: head mount display (HMD) and the mobile phone. Key development is the porting of these molecules to immersive augmented reality environment for direct interaction.Describing the advantage of the platform for this application at the broadest level, we can say that augmented reality platforms allow for full embodied interaction with the structures at any scale and contextualized by the physical background. We also discuss future plans for this platform.

Divers methods to train motions exist in multiple domains such as sports, rehabilitation or in industrial use cases. With regard to findings considering imitation learning in real world scenarios and social interaction guidelines in extended realities (XR), in this paper the transferability of these real world effects to AR is investigated within an assembly scenario. On that basis, a comparative user study (N = 12) was conducted analysing implicit imitation learning as well as the impact on the performance. Therefore, besides the measurement of the completion time, motion data are captured and a first analysis of the data is conducted. Whereas in terms of the cumulative completion time, no significant differences between trained and untrained subjects can be measured, the avatar’s motion was imitated differently between the groups. More precise, for the similarity of motion pattern between the avatar’s motion and the motion of the subjects it was shown that the untrained imitated the motions significantly more often than the trained, but both groups imitated in general. The results provide a first insight of the possibilities and limits of imitation learning in AR. With respect to the specific assembly use case, an avatar assistance method could enhance the learning, considering performance parameters and could lead to an implicit imitation of ergonomic motion patterns while having free cognitive capacity for conducting the specific processes.

With the rapid development of science and technology, the era of “Internet of everything” is imminent, and “interaction” has emerged a new connotation. Interactive design should be reconsidered and defined in view of many new application scenarios. From the perspective of safety psychology, how to innovate interactive design methods and improve the effect of safety education become a problem that designers should take in to account. This greatly broaden the application scope of interactive design, and create more conditions for updating the application concept of interactive design and grasping the new connotation of interactive design based on the new technical environment. The traditional methods of safety education are hard to stimulate the participation of audiences, which fails to produce active and effective interactions, and then, conceptual update and innovation of interactive design is very difficult to take effect. The development of VR created the condition for the application of more novel visualization and interaction patterns, and interactive design can be innovated with the application of VR in Safety Education. Based on the characteristics of VR and its impact on interaction behaviors of human beings, this paper combining with the relevant viewpoints and requirements of safety psychology makes an exploratory analysis on the innovation methods of VR-based interactive design.

Model Based Systems Engineering (MBSE) thrived in the recent decades in order to overcome the increasing complexity within systems. MBSE envisioned replacing document centric systems engineering by model centric systems engineering. MBSE can handle systems complexity efficiently by representing each entity in the model only once and generating different viewpoints. Most commonly used MBSE language is the “Systems Modeling Language (SysML)”. Virtual Reality can play a bigger role on creating truly model centric systems engineering approaches. The core application of MBSE approach implies utilization of a single system model, which will act as “source of truth” to all embedded system analyzing tools. Thus, the system model can communicate back and forth with Virtual Reality Environment to analyze the system performance. Systems engineer can conduct interactive immersive simulation of a scenario described in the system model and return the results obtained from the Virtual Reality Environment to the system model. Requirements analysis can be performed more efficiently by ensuring participation of the customers early in the product lifecycle utilizing the VR environment. A customer can walk into VR environment and see how the system design would look like and interact with it. Based on the interaction results, the systems engineer can modify the product/service specifications and design. This paper reviewed the works been done to incorporate virtual reality environment with SysML. The paper also proposed a framework on how VR environment can be implemented with SysML to perform true MBSE practice.

Winder-operators are responsible for the lives that they move down deep into the mines and bring them back safely to the surface. So, in order to have the same sense of responsibility in the training simulator, the user experience needs to be seamless and natural, including all the risks and surprises that a winder-operator faces in his real-world scenarios. To bring this objective to life, field studies, customer experience audits (interviews) and recordings were done to gather qualitative and physical data. Contents were inventoried, audited and analysed for body-storming and drawing up of the mental model of winder-operators. User persona was defined to maintain focus, even when expert reviews were taken. A contextual design process was followed to develop the simulator which included rigorous and multiple usability testing by the developer, designer and the winder-operators (end users).

In this paper we expound our theoretical hypothesis covering Design and Cognitive factors to be considered while designing an effective Augmented/Mixed Reality system solution for industrial use cases. For demonstration purposes, the targeted scenario chosen is an offshore oil-rig maintenance scenario: this scenario involves critical tasks in the areas of equipment maintenance, quick error recognition, safety, and effectiveness. With the goal towards enhanced effective Situational Awareness of the Oil Rig industrial system, the scenario involves oil-rig stakeholder collaboration/participation and remote assistance.An additional aim of the paper is introducing new perspectives for effective operations in an industrial maintenance scenario using mixed reality systems: the perspectives extends towards the following principles of study - Experience Design, Usability, Cognitive Science and Situational Awareness. Using Cognitive Science principles and Design Thinking approach, we try to bridge the existing white gap in the research space and intend to spearhead research towards surfacing the points to be considered in effectively designing Mixed Reality systems in such Industrial scenarios. The use of multimodal sensory augmentations such as sound and haptics to aid a better experience in performing a particular task, is also given attention. Elucidation of a functional prototype built using Microsoft Hololens1 is shared and next steps and targets are showcased as well.

The Augmented Reality Space Informatics System (ARSIS) is a Microsoft HoloLens application developed for the NASA SUITS (Spacesuit User Interfaces for Students) challenge that provides an augmented reality (AR) interface for information display and communication to be used by astronauts during extravehicular activity (EVA) procedures. Astronauts are required to deal with complex and changing circumstances on EVA. The current status quo requires heavy reliance on voice communication with Mission Control Centers (MCCs) for everything from biometric data to assistance with procedures. ARSIS is designed with the dual purpose of increasing astronaut autonomy and making communication with MCCs more effective. Features of ARSIS include voice and gaze user interface (UI) navigation, anchored menus, planetary navigation aids, biometric information display, procedure instructions, and a suite of tools for enhanced communication with Mission Control including spatial telestration. User experience (UX) testing results for ARSIS have been generally positive overall, with the biggest takeaway being that users would prefer alternative modes of UI navigation to voice control. Eye tracking for UI navigation is a promising area of exploration in the future.

In this research, we develop a mobile telepresence robot displaying a light field image of the VR avatar synchronizing its posture to that of the VR performer and VR controlling application rendering a point cloud of the scene before the robot to the VR system with an RGB-D camera. Both our robot and application communicate with each other via WebRTC. Our remote communication system compresses depth data approximately 100 kB, sends every about 200 ms and decodes it for 20 ms in average as the result of our performance test. In our online survey, 105 subjects generally expected our expecting effects, but some of them worried or suspected the limitation concerned to VR technology, network, or emotional expression of avatars.

A pair of commercially available smart glasses, Epson BT-200, was evaluated with different information display modes for assisting machine operators. Two information display modes were selected, displaying animated images only, and displaying animated images with text illustrations. Direction prompts were added in the system. Whether the prompts are helpful or not was also discussed in this study. Task completion time and error rate were collected during the experiment. System Usability Scale (SUS) and NASA-Task Load Index (NASA-TLX) were used to collect subjective information after each experiment. Twenty participants (10 males and 10 females) were recruited. The results showed that the display mode of having smart glasses displaying animated images versus displaying animated images with text illustrations showed no significant difference in task completion time and error rate. But the significant difference was found in the result of SUS scores. Participants preferred the display mode of animate images with text illustrations according to the SUS scores. Moreover, the results indicated that the use of direction prompts had significant influences upon all the measures. Participants completed the tasks faster and had lower error rate by using the smart glasses with direction prompts. And the results of subjective ratings also showed higher SUS score and lower NASA-TLX score were associated with using smart glasses with direction prompts. Thus, the implementation of using smart glasses to guide machine operations should be considered the design of adding direction prompts to increase efficiency and effectiveness of the operations.

This manuscript builds a firefighting training system using AR functionality designed for home use. In the system, the user can practice manipulating a fire extinguisher to put out the fire burning in the room. The speed of fire is examined to yield the sense of urgency and a pilot experiment is conducted. The result shows that the mechanism a fire’s flame continues to spread if nothing is done, could give the subject a sense of urgency while there are opinions that the image quality of the camera is low, and that the brightness of the virtual object is different.

Immersive technologies, such as augmented and virtual reality (AR/VR), are increasingly being utilized for training in various domains, especially the military. Although immersive technology’s potential for meeting specific training needs should be analyzed prior to implementation, an in-depth analysis is not always feasible. Therefore, military acquisition personnel often take an approach focused on logistical constraints when making decisions about acquiring new technology for training though no solution exists to guide these acquisition personnel through that selection process. The goal for this effort is to develop a software tool that will equip acquisition personnel with the ability to make evidence-based decisions about technologies for training prior to their acquisition. This support tool will help users make informed acquisition decision by inquiring about parameters (e.g. group size) and practical constraint (e.g. outdoor environment) considerations through various data extraction techniques. The ultimate goal is more efficient training as a result of guidance during the training technology acquisition process.

In the last two decades, the importance of research on block-based programming education has grown. The use of block-based programming tools is receiving attention not only in computer science courses, but also in robotics education. The effects of such programming on children’s learning outcomes have been examined, but the results have been inconclusive. The purpose of this meta-analysis was to examine the mean effect of block-based programming compared with traditional instruction (i.e., text-based programming) on children’s learning outcomes, including problem-solving skills, programming skills, computational thinking and motivation. The effect size and effects of moderators (publication year, sample size, publication sources and study region) were also examined. The database search yielded 19 publications with 31 effect sizes (n = 1369). Block-based programming had a significantly larger effect size than did traditional instruction for overall learning outcomes. More specifically, we found a large effect size for problem-solving skills, small effect sizes for programming skills and computational thinking, and a trivial effect size for motivation. No moderating effect was detected. Effect sizes for outcomes were large in research conducted in the Americas and Asia, medium in studies conducted in Europe and trivial in studies conducted in the Middle East. No evidence of publication bias in the studies was detected. These study findings support the benefits of block-based programming education for children’s learning outcomes, especially their problem-solving skills. Future research should examine additional dependent variables.

The framework of the traditional plant science education has lagged, which is hard to meet the needs of various visitors in China. In the following paper, we propose a unified interaction framework to help the design of the plant science education system. That is suitable for various visitors from China. They are from young to old with different levels of education, and some of them have dialect or accents. Inspired by the background, we try to design a new framework that takes advantage of the variety of the visitors’ output, aiming to make the plant science education system very smart to understand what the visitors want to learn. Firstly, all the plants are numbered with the non-linear digital numbering method, which can be seen as the label of the training process for deep learning networks. Afterward, the interaction continuously collects the visitors’ plant-related voice data as the input of the deep learning networks during the operation of the system to improve the performance and the stability of the plant science education system. Through continuous training, the overall accuracy of the system could be improved, and the system can gradually understand the regular pattern and central issue of people cares for the plant. The framework provides a new idea for the science education of the botanical garden and further improves the level of science education in China. This framework helps achieve sustainable development and environmental protection.

During this special period, many classes have become an online curriculum, including dissemination of Journalism and Communication major. This paper starts from the problem of human-computer interaction of the online course system for “New Media Management” course of the Journalism and Communication Major, method based on an online survey from 13 teachers and 157 students, we studied the influence and role of online teaching methods on teachers and students of dissemination of news major. Based on the research purposes, the human-computer interaction behavior, demand, and usability of teachers and students in 13 online class hours are studied on field research. Research shows that the real-time interaction, integration of multi-channel interaction, situation awareness, and practical mode of dissemination of news majors are the key problem of the online teaching system of dissemination of news majors, then we improved the online course system and completed the design of the online course system.

Many universities have started to adopt online programming tools to support students’ programming practice, yet the services currently offered by the existing tools are somewhat passive with respect to considering a student’s programming skill level and providing appropriate code questions. To enhance students’ learning experience and improve their programming skills, it would be helpful to examine students’ programming abilities and provide them with the most suitable code questions and guidelines. Machine learning can play a role in modeling the level of students’ programming skills as well as the difficulty of questions by taking the students’ programming experience and code submissions into account. This paper presents a study on the development of machine learning models to classify the levels of students’ programming skills and those of programming questions, based on the data of students’ code submissions. We extracted a total of 197 features of code quality, code readability and system time. We used those features to build classification models. The model for the student level (four classes) and the question level (five classes) yielded 0.60 and 0.82 F1-scores, respectively, showing reasonable classification performance. We discuss our study highlights and their implications, such as group and question matching based on code submissions and user experience improvement.

The Quran is the holy book for millions of Muslims around the world and is read and learnt in Arabic. We are witnessing a spawning of many mobile apps claiming to provide a digitised experience of Learning Quran. In our research we present a thorough review of 37 such apps from the Google Play Store and 85 apps from iOS Apple Store. Our results shows that while most apps provide tailored interaction, in general the main target group remains adult Arabic speaking users. Moreover, real time feedback remains a sought after feature, due to limitations in speech recognition. Accreditation and authentication of the sanctity of these apps remains a key worry for most users. In conclusion, we present design implications emerging from our results that could be applied to mobile apps for Quranic teaching.

Discussion forum boards play a crucial role in the interactions among learners and supervisors on e-learning. SWAYAM (Study webs of Active Learning for Young Aspiring Minds) is the first Indian MOOC (Massive open online course) adopted in all higher education institutes, high schools, and vocational schools as a database of their learning materials and discussions. However, being in its initial stage, SWAYAM lacks a well-designed structure in its discussion forums which is necessary for encouraging student engagement in learning.In this study, we aim to redesign the discussion forum systematically by classifying queries to enhance the learner-supervisor interactions in SWAYAM. In a previous study, FENG [1] developed a model with a convolutional neural network on Rossi’s data set to classify posts in the discussion forum of Coursera which helped to improve the course quality in MOOCs and students’ learning effect. Our study initially adopted a manual classification while in the future we will implement a hybrid approach of machine learning along with the Rule-Based expert system to predict a type of query in the discussion forum of SWAYAM. This proposed system will segregate the comments of the discussion forum using specified indicators and identify repetitive comments. The learners can acquire knowledge frequently from the discussions instead of navigating all the comments separately or retrieving the visual learning materials. On the other hand, subject matter experts (SME) can answer the relevant queries at once after indicator-based segregation of queries and need not to reply to every query distinctly.

The current society of change places the human being in a situation of new learning in emerging moments, therefore, it is necessary to place the teacher and the student in this new social paradigm. The objective of the present study was directed towards understanding the role of the educational technological society as an integrated component for the promotion of citizenship within virtual spaces. The methodology used in this study had a quantitative approach, this being a research with a descriptive level and a non-experimental and field design; where it was selected a sample of forty (40) faculty from three (3) universities in the city of Barranquilla for data collection, which was supported by a 120 question questionnaire related to the use of information technologies in education and the promotion of values within these environments. The results allowed to recognize that within the educational institutions that made part of the sample there is an optimal level in the use of information technologies, as well as in the promotion of the values in these institutions by teachers. The article concludes with a reflection of how the web spaces of today are the door for new generations to be more interconnected and informed, nourishing themselves in their growth process as individuals and citizens.

In the recent years, increasing attention has been drawn toward adaptive e-learning, which is a learning method that uses educational big data to flexibly change the learning content according to the proficiency of a learner. In this study, we proposed an overview of an adaptive e-learning system that focuses on the solution procedure, which comprises the knowledges and operations used by a learner to solve a problem. Using the example of elementary geometric problems, we develop a prototype of our proposal in which the situations of learning are formalized on the basis of an adaptive e-learning context model, which is represented using a meta-network. This prototype comprises three subsystems. One is an expert system that automatically generates various solution procedures for a given problem. The other two are an inference system that identifies the procedures that a learner is using by calculating their similarity with the procedures generated by an expert system and a classification system of given problems by calculating the similarity of the procedures of each problem; the similarity is calculated using both the Levenshtein distance and Needleman–Wunsch algorithm, respectively. Compared with conventional methods, the proposed system might provide more detailed support for learners.

Mudslide disaster prevention education is important for children who face related life-threatening situations in their daily lives. The purpose of this study was to compare the effects of two virtual reality (VR) mudslide games with different usability designs on children’s presence experience and learning motivation. The games provided users with three options (riverside, bridge and hillside) for escape from a mudslide disaster. Sixty-six fifth-grade students in Taiwan participated; 18 students played Game 1 and 48 students played Game 2 individually for 10–15 min. A 24-item questionnaire assessing the students’ learning motivation and presence experience was administered after game play. Questionnaire responses indicated that the students had positive attitudes toward learning via the two games. Learning motivation, presence and total questionnaire scores were higher among students who played Game 2 than among those who played Game 1, but these differences were not significant. All three scores were higher among boys than among girls who played Game 2, as indicated by nonparametric analysis. The results indicate that the low-polygon VR mudslide game designed for usability facilitated children’s, and especially boys’, learning about mudslide disaster prevention.

The knowledge demand in the intersection between human-centered design and Artificial Intelligence (AI) has increased rapidly in both the private and public sectors. However, higher education is struggling to provide relevant content to already established senior professionals in a flexible and timely way. Mobile learning (m-learning) provides a promising way, but more research and practice is needed to design and launch efficient m-learning initiatives. In this paper, we share our experiences in designing and launching a flexible and self-paced podcast-based, free university course for established professionals on the topic of human-centered design and AI. We present our design process and highlight the findings from our on-going student survey evaluation. The questions addressed in this paper are: (1) How can educators design podcast-based courses for professionals in Higher Education? (2) What impact does a podcast-based format have on a student’s engagement in higher education? Preliminary results indicate that the podcast-based format is an appreciated form of flexible learning, and that the content of human-centered design and AI is of high interest for a multidisciplinary community of professional practitioners.

With the COVID-19 pandemic, the higher education communities throughout almost the entire world have moved from traditional face-to-face teaching to remote learning by using video conferencing software and online learning applications and platforms. With social distancing requirements, it is expected that eLearning will be part of the delivery modalities at least until an effective vaccine is widely available. Even after the pandemic is over, it is expected that remote learning and online education will be part of the “new or next normal.” Such online and remote learning modalities are not simply restricted to academic institutions. Businesses are using online and remote learning to re-train, re-tool, and re-educate their employees. The students, in general, are not enthusiastic about the virtual classroom. Niche.com surveyed 14,000 undergraduate and graduate students in April 2020 and found that more than 2/3 of them thought online classes are not as effective as in-person and teacher-centered classes. This unplanned change in teaching modes caused by COVID-19 and the negative feedback from students creates some serious concerns for educators and universities. How to enhance the eLearning experience for students? How to choose from many eLearning platforms on the market? Which eLearning platform is the most user friendly and the best suited for online classes? Which eLearning platforms enable the best class participation and student involvement? In this research, we apply the eLearning usability heuristics to evaluate the major video conferencing platforms (e.g., Cisco Webex, Microsoft Teams, and Zoom).

The purpose of this paper is to propose findings on a process for developing and designing a hybrid Virtual Reality/Augmented Reality (VR/AR) system that facilitates artifact discovery and meaning-making at the Guard dig site near the Little Miami River. Both researchers and visitors use this active archeological dig site. The content for this system uses a combination of computer-generated and photogrammetry type models to show the world of the Ft. Ancient people. The virtual presentation of these artifacts uses both a VR and AR system or Extended Reality (XR) system, to share critical cultural stories about this ancient culture. Approaching the design of this project demanded the use of designerly thinking, design research and co-design principles with archeologists, the local community, and potential visitors. This process, based on agile methodologies, ensures the needs of researchers, stakeholders, and visitors are met. The research highlights the challenges facing, exhibit, museum and historical landmark designers as they strive to create a more vibrant and dynamic experience while at the same time creating a historically accurate recreation. The main issues facing an immersive designer consist of the blending of relevance, expectations, engaging interpretation, and finally creating a cultural hub.

This research proposes to develop a design and application model for interpretative media for ecomuseums and other exhibit facilities. With perspective of information design, we consider the form of an on-site interpretative game to combine the benefits of field interpretation and game tasks. This research takes Beitou area in northern Taiwan as the sample site of ecomuseum, to develop the contents of the interpretative game. With different styles of information design, including the order of task and interpretative text, and the use of realistic or schematic image, a card-based on-site interpretative game is designed and made. We conduct on-site user tests to understand the difference and features of usability and satisfaction among combinations of different styles of information design. Quantitative and qualitative data is collected to be analyzed. The guidelines for information design of on-site interpretative game are concluded and proposed.

The objective of the project focuses on the use of Augmented Reality and free software applied to a playful, puzzle-like game that serves as didactic support in the teaching and diffusion of archaeological patrimony in museums.For the study case, we choose The Pumapungo Museum and The Archaeological Park located in the city of Cuenca, Ecuador, which have an important archaeological and patrimony reserve and are also located on one of the most important Inkas settlements in the country. According to its statistics, around 164,000 thousand tourists visit it annually, and 14.64% of them are children; therefore, based on these data, the defined audience of users for the study were primary school children between 6 and 8 years of age.As a pedagogical methodology, the learning theories in education were used, by Seymour Papert (Constructivism) and John Dewey (Learning by doing), tools that have improved the learning process through exploration and experience. Likewise, we worked with formal and non-formal learning schemes that allow improving the interaction of children with the museum environments, reinforcing the knowledge from the guided visits.The results show that the use of Augmented Reality in the learning process arouses the interest and curiosity of children in a given topic because it allows them to get involved in real time in historical contexts and therefore learn about the material and intangible heritage or patrimony of their country.The research and development of the project was carried out in 2015, so the technological and bibliographic references correspond to that date.

Products are of commemorative significance when associated with regional culture, whose differences in design themes and cultural attributes cause consumers to have different perceptions of regional culture. In view of this, this paper took the regional culture of Nanjing as an example to study the consumer cognition of two types of products in five attributes including cultural and historic natures, cultural story-based natures, local characteristics, cultural connotations, and cultural art, with cultural and creative products designed for architecture and IP roles as the research objects. The research findings show 1. Consumers had different cognitions towards design attributes of regional cultural and creative products of Nanjing, which have different design themes. 2. Consumers had different cognitions towards design attributes of regional cultural and creative products of Nanjing on different design levels. 3. Cognition of design attributes of regional cultural and creative products of Nanjing was influenced by design themes and design levels. The research can provide a reference for the design of regional cultural and creative products, help designers master consumer demands better and prevent design deviations.

This paper will examine the interactive sound sculpture The Soundomat, constructed with the aim of making an engaging experience while challenging our perception of music.The concept of music will be discussed through people’s reaction, to an installation which is based on musique concrète. Furthermore, it will analyze and investigate where the project originated from, through methods of idea development, designing, coding and conceptualization. It will illustrate and describe what inspired the work, and the reasoning behind the choices that were made. The installation is playful in its design, and It has the intention to stir imagination, curiosity and provoke playfulness, while looking intriguing with its vibrant color scheme and ‘turntable’ element. Additionally, we will discuss and analyze the interaction between The Soundomat and the participants, through interviews, observations and the feedback received. It observes that people had two different ways of interaction. Either they picked the soundblocks based on the materials and the sounds they expected from them, or they worked in a more systematic manor and only added one soundblock at a time. There was no clear indicator, why some used one method over the other. Even though some of the participants were not aware of the concept musique concrète, they described the soundscape from The Soundomat in a way that corresponds with the definition of musique concrète. Lastly it will be analyzed, whether the project fulfills its intent, as well as its future work and improvements.

On the background of innovative concepts for the creation of cultural virtual spaces, the article discusses the fiction of form and meaning of traditional culture within virtual worlds. It argues that current digital forms in virtual worlds obscure and separate their cultural context. Moreover, commercially motivated design masks the separation of form and meaning, and the growing realism of virtual reality facilitated by technology is imperceptibly contributing to the intensification of this separation. This has a negative impact on the communication of traditional cultural values within virtual worlds. That is one of the problems between traditional culture, human, machine and virtual environment in modern society. Therefore, by speculating about the relationship and characteristics of the virtual form and the real meaning of digital culture, by means of the design of virtual narrative environments, citing the traditional cultural form and meaning as props, analyzing the relationship between designers and audiences, grounding our observations on design experiments and measurements of data, we can explore the new design ideas of virtual form and meaning of traditional cultural in the virtual environment.

This article describes a practice based artistic investigation which produced a participatory installation consisting of sculptural objects informed by sound vibration patterns. The installation, called the “Chladni Wall”, brings together Chladni bricks. These are based on analogue 2D Chladni patterns made with granular material scattered on a plate and activated by sound vibrations. One or more Chladni patterns are transformed into 3D sculptures. Four areas where Chladni pattern informed 3D objects could be applied further are identified. Namely, for (1) educational curricula which integrate teaching acoustics, 3D modelling and digital fabrication, (2) for designing objects with special acoustic properties, (3) as help in voice training and with speech impediments and finally (4) for providing sonic experiences to the hearing impaired.

Following the rapid development of high-technology industries, digital applications have been applied in various technologies, bringing people a whole new daily living experience. Interactive technology has led to breakthrough innovations. Application of augmented reality (AR), in addition to people’s approaches to interact with the physical world, has changed industry owners’ methods in designing products. This reveals the criticality of designing products with AR technology and new forms of interactions. This study investigated the current status regarding the cultural and creative products in consumer markets and the effectiveness of incorporating AR technology into cultural and creative product design courses. Data were analyzed to clarify said effectiveness and confirm the feasibility of AR technology in cultural and creative product design. A survey was performed on the cultural and creative products in the National Palace Museum located in Taipei and those designed by university students majoring in design to explore the effect of AR technology employed in cultural and creative product design on consumers’ review of and purchase intentions for the products. The following conclusion was reached: (1) The scale constructed to evaluate the cultural and creative products designed using AR technology was feasible. (2) AR technology positively affected participants’ review of and purchase intentions for cultural and creative products. (3) The products created by students according to the teaching method employed exhibited business value of high interactivity. Future cultural and creative product design courses can employ AR technology in their principles and practices in training innovative talents that satisfy the market demand. The design principles developed thereof can also serve as a reference for industry owners in designing cultural and creative products.

In our previous survey (February 2018, Japan), 70 people with hearing loss indicated the lack of necessary information at museums. The d/Deaf or hard-of-hearing visitors at museums want sign language interpreters. Therefore, we organized guided tours with a curator explaining in sign language on the B1 floor [Evolution of Life -Exploring the Mysteries of Dinosaur Evolution-] at the National Museum of Nature and Science in Tokyo, Japan. The guided tours showed that many jargons did not have corresponding signs in the sign language, which made seeing some exhibitions difficult. We had to support with written text to convey these jargons. We prepared a portable digital signboard. A tablet PC attached to a portable stand displayed technical terms and jargons in Japanese. The curator was able to navigate the exhibition floor easily while carrying the signboard. As the digital signboard was self-supporting, the curator could use both the hands freely. It was effortless, involved low technology (hence, inexpensive), and very convenient to use at museums. This practice paper will report the design method to support technical terms in written Japanese. While displaying in writing, we recommend the following: 1. Use easy Japanese; 2. Proper nouns must be written; 3. Add Kana for difficult Kanji; 4. Display structured information; 5. Align text to the exhibition layout. These methods will be useful not only for the d/Deaf or hard-of-hearing but also for the hearing. We aimed at improving information accessibility of the museum based on “universal design” and “design for all.”

Continued oscillation between digital environments and physical realities has created a demand for hyper-interactive community spaces. Because the overall cost of prototyping extended reality (XR) experiences can require large initial investments of time and money, current art spaces could be used for prototyping digital environments with physical spaces. Integrating an art gallery with digital sensors, cameras, and projectors could allow for rapid prototyping of XR development, solving rudimentary problems found with spatial navigation, sensory exposure, and psychological ramifications in a more cost efficient manner.Not only do these SmArt Spaces provide pivotal community engagement centers, they act as cultural hubs transcending divisions based on location. Connecting multiple spaces with virtual, and mixed reality experiences embrace concepts of the omni-connected environments in the near future. Integrating the primal functions of both art and an art gallery in the form of experience engages those willing to enter the space and provides an excellent opportunity for XR prototyping case studies. These spaces would naturally develop as community research centers and allow for a centralization of cultural experience and organization.This writing aims to establish a baseline of artistic explorations relating to the interconnectivity of all humans through technology. Combining culture, location, and communication this seminal work attempts to better define underlying truths to how humans experience while providing a synthesis of physical and digital elements in order to create future SmArt Spaces

We present the design concept Lokahi: a soft, interactive, wearable device which aims to promote a pleasant, intimate closeness sensation between two people. Its shape is designed to encourage hugging, while the embedded electronics –through pulsating coloured lights– lets the users to visualise their own heartbeats, so that they are aware of each other’s current affective state. Lights hues mix according to the synchronisation of the two heartbeats, possibly producing a single hue: such shared visual feedback can potentially be used in relaxing and meditative exercises, where two people try to synch their own heartbeats.

This manuscript discusses an acrophobia treatment procedure using virtual reality of VR technology and conducts a pilot experiment on feasibility. The core of our treatment produce is a exposure therapy with a strong shock. In our treatment produce, patients wear a head-mounted display or HMD and they are left on the top of a mountain in a virtual space. They are given five minutes to find a way down the mountain and only a way down is to jump off. There are some artifacts for them to do that by themselves. A strong stimulation is given to the patients at the moment of jumping. The experiment results show that our treatment procedure demonstrates the potential for alleviating acrophobia and a stronger stimulation is acceptable for alleviating acrophobia.

Traditional activity trackers use acceleration sensors to trace when individuals are walking, running, or standing still. The programming in these devices then determines the number of steps, walking or running distance, floors climbed, and calories associated with the physical activity throughout the day. Other than counting steps, there is little interaction to motivate an increase in pace or to provide feedback into how vigorous the activities are, besides heart rate sensors. The same number of steps walked represent different calories burned at different speeds. Thus, more energetic exercises provide enhanced physical activity. This work describes a wearable tracking device for enhanced monitoring of physical activity. An accelerometer was used to measure the acceleration level at the wrist location and compared against a threshold to illuminate a series of LEDs based on the number of lights turned on and color emitted. This wearable prototype provides visual feedback to let the user know how energetic the activity level is.

The demand for women who are in pregnancy transforms form medical birth service to high-quality life gestation. The family-centered approaches to maternity care present an important direction for fitting the needs of pregnant women, the better pregnancy experience, and more emotional care. In this paper, we explore design issues in developing a smart service product system for pregnant women and her family. As a prototype, we have developed a service to provide the family-centered maternity health care link with a pregnant woman, her family, and maternity hospital. The prototype includes an application provide telemedicine support, antenatal report visualization, and maternity information sharing for the whole family. These features are designed and implemented based on the principle of family-centered maternity health care, which will highlight how to dissolve the boundary between the medical system and pregnant women’s family, and how to increase the emotional link between the puerpera and her family members. The contribution of this paper is to research on maternity health service lies in a complete set of smart service design processes based on FCMC mode and discussion on the relationship on medicine, puerpera, and her family.

The ABLE Music platform is co-creation project in collaboration with older adults with dementia and their caregivers, to provide unique pair or group interactions and intergenerational play by transforming movement into art experiences (digital painting and musical creation) in order to enhance wellness (physical, mood, cognitive). The ABLE Music platform advances current research on dementia that tends to, a) not be interactive, b) not exploit digital tools, and c) doesn’t take advantage of the opportunity to engage families and caregivers, reduce their stress, and restore identity, dignity and relationships. These advancements are built on research that demonstrates the benefits of: bright colour palettes’ ability to stimulate older adults with dementia, music and painting experiences that reflect the memories and preferences of older adults, and intergenerational gaming that will allow younger children and adults to teach older adults digital gaming skills. Art combined with movement has a synergistic effect – it has the power to enhance mood, physical health and cognition. Reducing depression makes us less susceptible to cognitive & memory impairment, due to depression ageing the brain. Arts-based and Montessori-based approaches increase communication, episodic memory, and relationships between People with Dementia (PwD) and their family members and caregivers. Crip approaches to design can help researchers center PwD and caregiver perspectives in treatment plans, in addition to rethinking how power can influence the research process.

Recently, in developed countries, especially in Japan, the population of elderly people are increasing. Physical function of elderly people deteriorates due to age, it causes stumble accident. A stumble accident of an elderly people leads to a bone fracture. In elderly people, it is very dangerous to become bedridden by bone fractures and injuries. Thus, it is important for elderly people to prevent stumble accident. The “Base of Support” (BoS) has been reported to be related to stumble accident. We have been executing experimental consideration regarding stumble accident and BoS. In this study, we executed measurement and analysis of walking movement for human and analyzed based on the dynamic change for BoS. This experiment, we measured and analyzed walking movement using optical and inertial sensor type motion analysis device. In addition, we measured the step length using the camera. The Subjects walked straight for approximately 10 m indoors. Moreover, they executed heel strike according to the timing when the metronome sounds. Also, we changed the rhythm of the metronome every 10 bpm from 90 bpm to 140 bpm. As for experimental results, the dynamic change of BoS increased as walking rhythm changes faster. This is considered to be increasing movement of gravity center of human body. In this study, we executed measurement and analysis of walking movement for human and analyzed the dynamic change for BoS. In the future, we aim to establish an index of the dynamic stability evaluation of human walking movement.

Adverse health outcomes (e.g., respiratory infections, lung injury, death) related to vaping were reported at significantly higher rates in healthcare systems starting in the fall of 2019. This study seeks to leverage artificial intelligence (AI) techniques, such as latent dirichlet allocation (LDA) methods, to determine whether a signal of these negative health outcomes could have been detected by the frequency of Twitter content posted about vaping and these health outcomes prior to this increase. We utilized a random sample of 3,523 tweets from 2019 and performed LDA methods on this sample to cluster the tweets and identify latent topics. We then utilized keywords from within the health-related cluster (topic) to manually verify the frequency of these tweets across previous years to approximate topic trends. LDA methods resulted in 4 distinct topics of tweets, including a health-related topic. Keywords from this topic were found to increase slightly in 2017 and 2018, with a dramatic increase in 2019. Further, the highest performing keyword combination was found to increase most significantly beginning in August 2019. The results of this study support the feasibility of leveraging artificial intelligence techniques for surveillance of public health concerns such as vaping and adverse health outcomes reported in Twitter. Further research is needed into the development of such models, which could promote earlier detection of public health issues and timely outreach to those groups most at risk.

The research studies the impact of Human-Centred Artificial Intelligence (HAI) to augment the health of older adults. The researchers are especially interested in looking at the impact of the decrease in the cognitive effort for users and an increase in the naturalness of Human-Machine interaction. A mobile application that uses Natural Language Processing to enable senior citizens to interpret doctors’ handwriting is used as the mode of experimental study. The application consists of an optical character recognition program that leverages a Recurrent Neural Network and a data set of thousands of handwritten prescriptions. The interpreted information is linked to a personal calendar that automatically sets reminders to take the medicines. Voice is used as a medium of interaction with older adults to evaluate the role of naturalness in Human-Machine interaction. Initial results showcase a reduction in the amount of cognitive effort required to comprehend prescriptions and taking timely medication, while simultaneously making the elderly more aware and self-sufficient.

Sports rehabilitation is an important branch of rehabilitation medicine and the most widely received project in the social security system. In the process of continuous improvement of rehabilitation services, the sports rehabilitation aids industry is developing towards product, industrialization and individuation. With the increasing number of rehabilitation patients and the development trend of the Internet, Traditional therapist-assisted training has been difficult to meet patients’ rehabilitation needs, and more rehabilitation training equipment and service design are needed to meet modern technology and user experience. Modern rehabilitation training equipment has greatly improved the rehabilitation efficiency and curative effect than traditional methods, but it lacks the psychological and emotional satisfaction to the rehabilitation patients. This is also an important problem to be solved in the focus of rehabilitation auxiliary man-machine interface. Through the investigation and analysis of sports rehabilitation field, rehabilitation accessories industry and products, this paper clarifies the subject direction. The process of designing and developing rehabilitation accessory products based on user research is discussed in this paper. Under the guidance of the design concept, the design and development of rehabilitation accessory products and more humanized rehabilitation treatment service flow were carried out.

We implemented an environment for daily physiological tracking using an FPGA DE10 board with integrated hand-made sensors made of an Micro:bit microprocessor. The system is capable of live streaming collected data, performing data processing and modeling, and exporting brain state to a mobile app for mental intervention. The FPGA microcomputer is capable of electroencephalogram (EEG) data preprocessing, feature extraction, and brain state classification using deep learning models. The data rendering software on the mobile phone simulates a professional medical interface through which the user is able to connect the EEG device to the phone, relay the EEG data to the FPGA board, and render the brain state curve on the phone. The mobile app runs a sleep inducing program which can help the user get into sleep more easily and deeply. The sleep-inducing technique takes advantage of Delta waves, which is proven to have the effect of deepening a person’s sleep, when it is converted to sound wave and applied to the subject. Our system monitors the user’s sleep state and adjusts the volume of Delta wave sounds, together with a user chosen background music, according to the user’s sleep depth.

The popularity of recreational sports such as running, has increased substantially due to its low threshold to start and it is attractive for a wide range of people. However, despite the growing popularity, running has a high drop-out rate due to injuries and motivational loss, especially among novice runners. To investigate factors influencing motivation among novice runners and design opportunities, we deployed an empathy probe at a women-only running event. Faye is a running shirt that reveals motivational feedback on the shirt, during the warm-up phase of the run. In this paper, we both inform on the impact of motivational feedback while warming up on running motivation and reflect on the use of an empathy probe to investigate motivational strategies among novice runners.

Significant growth in health information sharing through Twitter is making it a compelling source for health-related information. Recent health research studies show Twitter data has been used for disease surveillance, health promotion, sentiment analysis, and perhaps has potential for clinical decision support. However, identifying health-related tweets in these massive Twitter datasets is challenging. With the increasing global prevalence of diabetes, user-generated health content in Twitter can be useful. Therefore, this preliminary study aims to classify diabetes-related tweets into meaningful health-related categories. Using an ensemble of neural network and stochastic gradient descent classifiers, we classified 13,667 diabetes-related tweets into five clusters. About 25.7% of the tweets were clustered as health-related, where 9.3% were classified as Treatment & Medication, 9.9% as Preventive Measures and 6.5% as Symptoms & Causes. More than 70% were clustered as Others. Analysing hashtags of tweets clustered in each of the categories showed significant relevance to health-related information.

The World Health Organization projects a continuous growth of the elderly population, causing a shift in the demographic distribution. With this shift, healthcare plays an important role when treating conditions such as dementia, which prominently afflicts the elderly and their families. Currently, reminiscence therapy helps support alleviation of dementia symptoms through memory recollection. However, the therapy relies on traditional printed media, including pictures shown to the patients that lack immersion. In this paper, we present a preliminary study from focus group and one-on-one interviews with patients, family members and care providers to gauge their interest and perception of two non-immersive or headset-less virtual reality scenes for complementing reminiscence therapy. One system features head tracking as an input to create shifting perspectives on the output display to provide a sense of depth, while the other system employs the Looking Glass, a holographic display providing horizontal depth perspective. The proposed non-immersive VR systems were deemed feasible for therapy purposes based on our preliminary data, with insights on challenges and opportunities for future development.

Real-time monitoring of pilot mental workload has important applications in cognitive assessment and flight safety. Progress in wireless electroencephalography (EEG) and 3D flight simulation has provided novel opportunities to advance our understanding of adaptive levels of workload during flight. The present work examines neural correlates of mental workload evoked by ecologically valid flight tasks and furthers the application of human-computer interaction in aviation. Performance and EEG data were collected while 47 participants completed basic aviator tasks in a virtual reality environment where working memory load was manipulated. Analyses investigated event-related potentials (ERP) and spectral power density differences in moderate and high task load conditions across key brain regions. The subtle modulation of moderate to high workload did not reveal significant differences in frequency changes across parietal and frontal regions or participant performance; however, a frontal ERP showed a significant effect of workload. Classification by performance level showed better utility, where greater beta power was found in the parietal regions and increased delta activity was measured in the frontal regions. Results indicate that EEG analyses that exploit spectral data from the frontal and parietal regions may offer reliable approaches for classifying performance in high workload conditions in virtual aviation environments

This paper discusses Plane-Gazing Agorá, a cyber-physical environment at the airport observation deck that stimulates taking and sharing photos of airplanes. Since airports worldwide are on intense competition, airport operators are recognizing the need for differentiating themselves. Our research spotlights the airport observation deck; focusing on the customers’ activities on taking and sharing photos of airplanes. The experience at the observation deck is site-specific and unique, hence we see the potential to contribute to the need for differentiating themselves.Plane-Gazing Agorá consists of the following 2 phases. (1) Photo-taking support by interactive displays installed in various spots on the deck and app. These display information about the upcoming aircraft and suitable shooting locations. The dedicated app provides congestion information and weather forecast for the deck. (2) Photo-sharing support by a public screen and app. Photos displayed on a large public screen are selected from photos posted by customers. The airplane photographers can post these photos from a dedicated app. This paper seeks a community centered on the observation deck that shall be formed by the experience design of the hybrid of cyber-physical space.

For automated and partially automated cars, there are new crucial questions to answer: “When should the driver or the automated system take control of the vehicle?" ; and also: “Can both control the vehicle together at the same time, or can this create potential conflicts?" . These are non-trivial issues because they depend on different conditions, such as the environment, driver’s state, vehicle capabilities, and fault tolerance, among others. This paper will describe a human-machine cooperation approach for collaborative driving maneuvers, developed in the EU funded project PRYSTINE. In particular, this study presents the work-in-progress and will focus attention on the proposed architecture design and the corresponding use case for testing.

Voice control is becoming a popular technology for in-vehicle user interactions. One of the primary design issues of voice control systems is the accuracy of voice recognition. Providing a limited vocabulary set for user-system interactions is known to be a viable strategy for enhancing the system usability and user experience in voice interactions. From the human factors perspective, the development of such a vocabulary set of voice commands for system controls should be based on user intuitions rather than technical specifications. Previous researches were mostly focusing on the recognition of specifying trip destinations, less attention has been allocated on the overall interface controls of in-vehicle navigation systems.This study aims to preliminarily explore potential patterns of the vocabulary choices of voice commands for in-vehicle navigation systems from the users’ perspective. Through a comprehensive market research, a set of 17 control functions, such as map orientation, zooming, navigation-related information, and general interface operations, of user interface interactions commonly used in modern in-vehicle navigation systems was instrumented for our experiments. Video clips showing the transitions of before-and-after scenario images for every control function were presented and prompted to experimental participants for their intuitions of the voice command vocabulary. All the intuitive voice commands collected for each control function were sorted in patterns of word cloud for further analysis. Post-experiment interviews were conducted for the subjective evaluation of the easiness of prompting voice commands and the possible reasons behind.A total of 30 Mandarin-speaking subjects (19 males and 11 females) in Taiwan with at least 2 years of driving experience voluntarily participated in our experiment. A great discrepancy in the variety of vocabulary choices among control functions was demonstrated in our analysis. As our data indicated, for example, volume control functions (volume-up/-down/mute) are with the vocabulary choices with the most consistent results among participants, while the control functions regarding presenting detailed navigational information, such as real-time traffic situations and nearby POIs, were the poorest. Subjective preferences showed a slightly different pattern to the objective data in vocabulary choices. Recommendations to the voice interface design for in-vehicle navigation systems and future research venues are further discussed.

In harbor navigation and berthing, maritime pilots are facing today many challenges such as high dense traffic, changing environmental conditions and a lack of accurate information. In order to better handle this situation, systems are needed that offer improved situation awareness. This paper presents an Augmented Reality design concept for Smart Glasses to support maritime pilots in berthing and port navigation. Unlike other approaches, the extended docking support does not depend on ship-specific sensors, but benefits from a land-based infrastructure. To design the augmented berthing support for maritime pilots, the Konect method is applied. Finally, preliminary designs which serve as a basis for further research are provided in this paper.

Future oriented mobility solutions, based on digital technologies, emphasize the need for digital and flexible urban infrastructure to give guidance to users. In the case of automated mobility on-demand (AMoD) services high user experience (UX) is essential for user acceptance. One of the main challenges is to enhance the user’s competence and information supply to overcome the physical meeting problem of user and shared automated vehicle (SAV). Hence, the need for the new concept of defined virtual ride access points (RAP) derives. The objective of this study is to evaluate a first human-centered RAP design prototype regarding usability and intuitiveness. By remotely interviewing 18 participants of young age, residing in urban areas and with experience in using ride-sharing services the authors show that already a first RAP prototype was positively rated with regard to usability and intuitiveness. So the mere existence of RAP presented with means of augmented reality has the potential to improve UX of new urban mobility services. With his article the authors seek to conceive guidelines for future digital human-centered AMoD infrastructure.

Cooperation is a key in the group or team work to accomplish a mission successfully.It is a critical factor to establish good relationships between humans and artefact systems as a group or team. In other words, cooperative work is one of important part to design current complex systems. It is important to understand the details of the basic functions, roles and their tasks in the system, in order to design more reliable interfaces and training programs for team cooperative work. Distributed cognition is an effective approach for understanding the interactions across the cooperative condition in a system.In this research, we attempted to apply a method based on distributed cognition to analyze the activity that takes place in cooperative work. Distributed cognition analysis makes explicit the dependencies between humans and artefacts by examining the transformation and propagation of information through various forms of representations. We focus on aerodrome operators’ work (aerodrome flight information services) as a case study of cooperative work.

By evaluating user experience (UX) with an integrated transportation mobile application, called GoPyeongChang, which was developed for 2018 Winter Olympic and Paralympic Winter Games in South Korea (see Fig. 1), this study aims to provide insights into how the advanced networked technology could be utilized for future, global, mega events. The GoPyeongChang mobile application was primarily designed to help users from different nationalities navigate through the public transportation services they could opt for while attending the mega sports event. This mobile application integrated transportation information to provide international users the ability to navigate Korea using any combination of available transportation systems information (e.g., a shared car system, public transportations information) in four languages (i.e., Korean, English, Chinese, and Japanese). The application allowing users to purchase tickets using the in-app payment system. However, challenges integrating with local based mobile application services, (e.g., phone number-based identification system) was inevitable, and foreign visitors had to suffer from difficulties in using the application due to unfamiliarity with local services and the applications UI hierarchy. Despite these difficulties, this case study surprisingly found that the GoPyeoungChang has contributed to forging global users’ positive attitude toward the site and the mega sports event through the mediating role of trust. This effect extended to the positive evaluation of the image of host country. Based on the findings, we attempted to provide an explanation for the experiential process through which the integrated transportation application was successfully adopted by non-local users who attended the mega sport event. Fig. 1. GoPyeongChang App: The official transport app for the PyeongChang 2018 Olympic and Paralympic Winter Games (Left) Developed Persona for this case study: Sean hasn’t been in S.Korea nor Olympic games before but interested in K pop and culture. (Right) Fig. 2. Problem Found: The duplicated menu was embedded in the main page.

The introduction of highly and fully automated vehicles (SAE levels 4 and 5) will change the drivers’ role from an active driver to a more passive on-board user. Due to this shift of control, secondary tasks may become primary tasks. The question that arises is how much information needs to be conveyed via an internal Human-Machine Interface (iHMI) to fulfill users’ information requirements. Previous research on iHMI regarding lower automation levels has shown that user require different information respectively. The present study focuses on how users’ information requirements change for highly automated driving (SAE level 4) when the on-board user is distracted with a secondary task opposed to when the user is non-distracted. Twelve participants experienced different driving conditions and were asked to rate their attention distributions to other traffic participants. Results show clearly that users rated their attention distribution to other traffic participants significantly lower in automated distracted mode compared to automated non-distracted mode and manual driving. Furthermore, the question of users’ information requirements was translated into iHMI design preferences. For this purpose, four different iHMI prototypes based on a 360° LED light-band communicating via color-coded interaction design, which proved to work well for lower levels, were evaluated regarding the information richness level sufficient for users for highly automated driving (SAE level 4). Results show that the sufficient information richness level is conditional upon gender. Implications for future research and applied issues will be discussed.

More and more studies in automotive research and development are conducting user-centered development in the emerging field of external human-machine interfaces (eHMI) in virtual reality (VR). As time, cost and risk are decreasing with progressively affordable sophisticated VR technologies, researchers have shifted to virtual testing. Within this context, they use a variety of methods and technologies to develop new designs but so far little examination has been done towards validity of virtualization and description of the technical setup. As level of immersion is one of the current pillars in VR and technology evolves rapidly, study setups differ a lot in recent years, resulting in poorer comparability. In this paper, our goal is to review the current generation of VR studies in automotive eHMI development and extract in the sense of a lessons-learned approach best practices with regard to their VR setup. For that, we assessed a total of six current studies published between 2017 and April 2020 in automotive eHMI development to extract lessons learned from study designs and virtualization setups. We took a look at hardware and software used as well as study procedure. The results allow us to find useful conclusions on automotive eHMI development practices in VR.

Driver assistance (DA) technologies pose challenges to the formation and maintenance of drivers’ mental models (i.e., understanding) of their operation. The challenges can be overcome through consumer education, training methods, and interface design strategies, but an understanding of how mental models for DA technologies form and evolve is needed. Therefore, we studied drivers’ experiences under extended real-world driving conditions for the purpose of delineating their mental models of DA technologies. Participants (n = 52) who recently purchased a vehicle with at least two DA technologies were interviewed for approximately six months. Cluster analyses (hierarchical and k-means) of data elements extracted from the interviews (e.g., ratings of mental model complexity, accuracy of technology understanding, trust, perceived usefulness, satisfaction) revealed five different types of learners of DA technologies: expert, skilled, moderate, uninformed, and misinformed. This paper reviews how mental model formation and evolution of DA technologies vary across the five learner types. The results indicate that facilitating efficient and appropriate mental model development could be enhanced by incorporating variations in mental model formation and evolution in future consumer education and design approaches.

The purpose of this study was to investigate the effect of shifting among and updating of mental models in multi remote tower operations.Background. Within the development of the future workplace for air traffic controllers (ATCO), innovation shifts to multi remote tower operations. Multi remote tower means, that an ATCO serves air traffic control services for more than one airport at a time from one physically remote located workplace. This change requires adjustments in the way how the ATCO works, as multiple mental models need to be updated constantly. Frequent shifting between airports and mental models respectively is necessary. Updating and shifting are cognitive cost sensitive, as they affect workload and situational awareness negatively. In contrast, higher workload could be beneficial for alertness which could act as a countermeasure in turn.Method. Four 90-min lasting remote tower Human-In-The-Loop simulation runs with traffic and weather were performed by eight conventional tower experienced ATCOs. Each ATCO completed two runs in multi- and two runs in single-mode. Situational awareness, workload and alertness was measured via self-reports before, during and after each run.Results. No differences between both modes with respect to situational awareness and alertness could be found. However, significant workload differences could be found during the simulation runs at two times, due to a simulated snowstorm.Conclusion. The findings indicate no negative effect of shifting among and updating of mental models in multi remote tower. A possible explanation could be, that a common hybrid mental model for in multi-mode is internally developed so that shifting among mental models is perhaps not necessary.

Cooperative automated vehicles (CAV) are not able to drive automated in all situations. Each vehicle has or is going to have its own operational design domain (ODD), which exactly specifies which situations can be handled, and which cannot. Vehicles of higher levels of automation according to SAE J3016 will try to take the driver back into the control loop if the vehicle approaches the border of its ODD by issuing a transition of control (ToC). If the driver is not responding, the vehicle will perform a minimum risk maneuver (MRM), where the CAV is stopping. Instead of looking at the internal HMI of single CAVs, the H2020 project TransAID focusses on the effects of automation limitations on traffic efficiency and safety. Besides helping the CAV to reduce negative impacts of such situations by infrastructure measures, also informing the surrounding vehicles about a CAV’s current issues and about its plans to solve them will most likely improve such situations. To approach this assumption, DLR conducted a first virtual reality study, where e.g. a 360° externally mounted LED light-band as external HMI (eHMI) of a CAV and specific vehicle movements as dynamic HMI (dHMI) are used in case it needs to perform an MRM. In the study, ten participants tested different variants and combinations. Preliminary results show that the use of an eHMI is a useful and informative approach.

The satellite navigation system provides continuous, timely, and precise temporal-spatial information signals to users all over the world. Although modern satellite systems, such as GPS and Beidou, are highly autonomous, human supervision and control still play an important role in the normal functioning of the system, especially when certain parts of the system break down. Therefore, it is vital for improving the response and disposal efficiency of equipment failures. This study used real documents to examine how human factors affected the recovery process of the system. The actual data of this study came from the log files of a ground control center of the Beidou satellite navigation system. In total, 169 records of the fault handling process of an uplink equipment were of particular interest in the present study. We collected the malfunction information, including the technical description of the failure, the time it took place, the information of the team on shift, and the task completion times of two different stages (fault judgment and recovery operation). We then transformed this information into task complexity, time of day, shift handover period, and operator team skill composition as the independent variables, while the judgment time and operation time were used as the dependent variables. Multiple regression analysis showed that task complexity is the most significant predictor of the two completion times. More complex tasks took longer time to finish. Moreover, it took more time to finish the recovery operation when the time was 16:00–18:00, and when the most adept team member on shift lacked relevant expertise. Based on the results, we made corresponding suggestions for both management and interface design.

At the University of Applied Sciences Kaiserslautern, empirical user interface studies in the automotive sector are being conducted at the driving simulation environment K3F. To improve the workflow, the driving simulation software Mave is being developed specifically designed for the setup of empirical automotive studies. A major problem is the use of different technologies for the realization of the different study components. Driving tasks are implemented via scripts; user interfaces are based on Web UI components and virtual environments are defined by several geographic data formats. The sum of these technologies is difficult to handle if a user does not have in-depth technical knowledge. The goal of this contribution is to design and develop an integrated development environment that makes it easier for new users to get started with each of these elements and to make further work more efficient. An easier start is possible by providing many example scenarios that solve different tasks. Working on own scenarios is simplified through an integrated documentation and context-aware code completion. In addition, error sources are reduced by a syntactic validation of code.

Situation awareness is needed to build dynamic mental models of the environment and contributes to safety during flight. Radio communications are key sources of information used by pilots to construct situation models. The present research investigated linguistic features of messages received during flight to understand aging effects on situation awareness. Fifty licensed pilots (age 17 to 71 years) flew a 60-min flight in a Cessna 172 simulator. Situation awareness was objectively measured throughout the flight. EEG was collected using a 14-channel wireless system and produced event-related and spectral indices of mental activity at key brain regions. Linguistic features of radio messages during flight were analyzed. Results showed significant deleterious effects of age on situation awareness. The linguistic features of messages, such as their pitch and intensity, were also associated with specific segments of the messages. EEG results supported the linguistic findings and indicate that age may impact later stages of the language processing pipeline. This research is important for informing efforts, such as the development of cockpit technologies to improve communication, that address older pilot safety.