Universal Access in Human-Computer Interaction. Design Methods and User Experience

With the increasing presence in our lives of interactive and technological items, the concept of “experience” becomes crucial in both the design and the evaluation phases. When aiming to measure such experience, researchers typically adopt self-report methodology. However, it has been established that also unconscious motives (i.e., emotions, implicit attitudes) - which people are not always aware of - need to be taken into account in understanding users’ perception, by means of indirect measures. The present study investigates the possibility of measuring UX by both implicit and explicit attitudes measures. In an experimental study (N = 36), we asked participants to evaluate a conversational chatbot prototype, which had the fictional goal of pre-selecting candidates for a job, and to fix a date for the job interview. We manipulated between-participants the chatbot’s gender (male vs. female) and the tone of voice (formal vs. informal). Explicit user experience’s evaluation (i.e., UEQ) and implicit attitudes towards the chatbot (i.e., IAT) were the dependent variables, as well as self-reported measures of chatbot’s efficacy and emotional experience. Our findings showed an implicit preference towards the informal version of the chatbot, as revealed by the IAT scores, whereas no differences emerged on explicit measures of the UEQ. Results are discussed in light of the effect of implicit attitudes on overall experience, and on the importance of comparing implicit and explicit measures, as well as objective usability measures, when evaluating the UX.

Designing assistive technologies for the benefit of a population of multi-disabled users living in specialized care homes involves the implementation of a user-centered approach (UCD). The purpose of this article is to demonstrate how the ecosystem also plays a key role in the codesign and appropriation phases of a augmentative and alternative communication (AAC), mainly in the choice and adjustment of interaction techniques through a case study. Firstly, we present related works on the codesign of assistive technologies. Then, we will describe the SoKeyTo platform used for the codesign by the human-computer interaction team. We will also explain the collaboration between the ecosystem and HCI team during tests (place of the switch and type of scanning). Usability criteria were used to determine these choices. Finally, we will illustrate our approach in the codesign and appropriation activities of layout and interaction modes of the AAC.

The advancing awareness of diversity in HCI is touted as laudable in addressing inequality and the digital divide. This progress however has not been reflected in HCI research. Diversity in HCI has focused on giving underrepresented users voice and confronting homogeneity. By increasing representations, though a diversity affinity is immediately visible, the deconstruction of institutionalized prejudice fails. Additionally, underrepresented voices have the unsurmountable task of representing their group. These approaches to diversity become tokenistic fixes failing to tackle the systemic causes of inequality.In this paper, we argue for a systemic approach to diversity in HCI. We call for setting diversity at the core of HCI where human realities and experiences are embodied rather than defined by “what users must be” or “what users are not”. The paper contributes the following: a detailed overview of diversity in HCI and presents a conceptual framework for diversity driven HCI with three correlated recommendations for setting diversity at the core of HCI.

Accessibility is a right for everyone. For this, sets of guidelines are created by authors and organizations with the objective to develop technologies more inclusive. Actually, digital games are one of the most used resources for entertainment, then they need to be inclusive to all people, regardless of their disabilities. This work presents a Systematic Literature Mapping performed to identify the guidelines for digital games aimed at people with disabilities. As a result, 44 papers that propose guidelines were accepted and analyzed. It was found the disabilities, the types of games, and the age range of the target audience covered by the guidelines. With an examination more profound, it was obtained a resulting set of guidelines for the development of digital games aimed at People with Cognitive Disabilities.

Simulation of disabled user characteristics can be useful in two cases: (1) during preliminary tests of interactive systems, (2) to train designers and make them understand the difficulties encountered by these users with special needs. This paper describes a case study involving a preliminary user test of a system called ComMob (Communication and Mobility). It is a communication aid for people with motor disabilities of the dyskinetic Cerebral Palsy type. This software is usable in mobility and may be installed on a wheelchair. A preliminary discount evaluation was carried out in the laboratory with non-disabled participants in whom involuntary movements were induced. These movements were characteristic of users with a dyskinetic Cerebral Palsy disability. The paper focuses on the principles of the discount evaluation that was implemented. The first results are promising and show the feasibility of the approach, leading to numerous research perspectives.

The paper will share practices and processes followed for the design and prototyping stage of the digital accessibility storytelling collective project that showcased the stories and work done by various technology professionals around the globe working in digital accessibility. By being more inclusive in our processes, we can bring more people into the conversation. Thus, we aimed to bake accessibility into our processes to ensure an inclusive and accessible product. Our specific aims during this phase were: 1) Seek feedback from different users and stakeholders. 2) Ensure documentation and prioritization of accessibility issues. 3) Create an inclusive product through iterative inclusive design. Thus, the goal of this paper is to describe and share the process of building this storytelling product, followed by a discussion of the learnings from integrating accessibility into design.Our processes led us to various insights that not only helped us improve our product, but also taught us valuable lessons for future processes. It gave us an opportunity to iterate on our product to ensure accessibility errors were documented and prioritized. Feedback sessions with users and stakeholders improved our awareness on our processes. Our later project stages, including evaluation, showed us how these design processes led to an accessible product and allowed us to capture critical issues and errors during this stage.

One challenge of the user-centered development of accessible information systems is the conduction of cognitively impaired persons in usability tests. The paper gathers existing guidelines for the application of usability testing with cognitively impaired people and shows empirical values for modified usability tests based on a case study. In addition, the advantages and disadvantages of on site and remote usability testing are presented in a comparative study. Especially in the context of COVID-19, remote testing has gained in relevance in the present time.

Usability and accessibility evaluations with diverse users are an essential part of an iterative universal design process for digital solutions. The COVID-19 pandemic has made it difficult to run traditional local evaluations due to social distancing restrictions to reduce infections. Remote synchronous and asynchronous evaluation methods may be a solution if it can be used by various user groups, including people with impairments. Incorporating video conferencing systems into traditional remote evaluations can be a valuable supplement to—or, given the current situation, alternative to—traditional usability and accessibility evaluations. We present a protocol for remote formative usability evaluations designed with accessibility in mind. The protocol explains how to prepare for an evaluation and its technical setup, how to conduct synchronous evaluations using video conferencing systems, and how to debrief and analyze the collected data. We tested this protocol in a pilot study for an ongoing project where we ran both synchronous and asynchronous remote evaluations. In our pilot, the synchronous evaluation using video conferencing systems provided opportunities for richer qualitative data than the asynchronous evaluation as we ran it. The findings from the pilot study indicate that the protocol is feasible and can be used when having participants with diverse abilities or impairments. We also provide suggestions for others that wish to adapt the protocol.

The present work aims to expand knowledge on user needs and the motivations of people with disabilities (PwD) in order to create a technological solution for addressing barriers in the accessible tourism market. For this, a user-centered design was followed, putting PwD at the center of the development process. This was obtained by identifying personas and scenarios contributing to a better depiction of the potential users, clearly describing their requirements and accessibility needs. For obtaining the characterization of personas, a comprehensive study in the area of accessible tourism, involving various tourism stakeholders was performed. The methods applied to collect the data were questionnaires, interviews and two focus groups with PwD. Two personas are presented in this article, illustrating the needs and motivations of two groups of PwD. In addition, two scenarios concerning the personas were also elaborated, showing how a technological solution can help the integration in tourism activities.

Headphones and earbuds are seemingly more popular than ever with the wide availability of smartphones and music streaming services. Such personal audio systems are also essential for many blind and visually impaired computer users that relies on text-to-speech. Few published studies address the users’ perceptions of such personal audio output devices. However, past research shows that negative perceptions may lead to device abandonment. General-purpose equipment may therefore be more successful than special purpose assistive technologies for marginalized groups. We therefore set out to gain insight into how users generally perceive headphones and earbuds, and we wanted to base our study in two different cultural contexts. A questionnaire built on a Kahoot quiz was developed involving 12 questions related to headphones and earbuds. A total of 100 participants were recruited in Norway and Brazil. The results show that intuitiveness is the most valued feature of these devices and cost was not. Brazilians expressed skepticism regarding the use of headphones while walking and when travelling on public transport, while Norwegians expressed that headphones were safe to use in such situations. Our experiences showed that Kahoot is a promising platform for conducting such experiments, as it may appear more engaging than regular questionnaires. Moreover, they are relatively easy to set up and allow response times to be measured.

Implementing accessibility features in digital games is an activity that has been growing over the years. Although these features do not exist in all digital games produced and distributed, some games stand out for their implemented features. In this context, this research involves the game “The Last of Us Part II” (TLOU2), which has more than 60 accessibility features and is considered, by some field experts, as the most accessible game ever produced. Considering the potential consequences that statements like this can represent for the games’ industry, this research analyzes the TLOU2 accessibility features with an analysis procedure based on resources and criteria from the community of practice and inclusion principles. From this evaluation, this study proposes an extension and improvement in the analysis procedure adopted, called PANELI, to encompass topics regarding inclusive digital games. Furthermore, we discuss the significance of TLOU2 accessibility features, the relevance of analysis procedure application, and its constant improvement and updates.

Information search is a common task when interacting with computers. Many studies have investigated the characteristics that facilitate effective search, specifically the use of icons and/or text. This study reports a simple experiment involving Kahoot where the goal was to observe searching performance with images and text. The results show that image search was faster, less error prone and preferred by participants compared to text search.

Today, we are living in a digitally dependent world. Through the use of digital technologies, life is meant to be easier and streamlined. This includes giving access to services that previously were unavailable to many due to disability. Although technology has evolved immensely over the past few decades, reducing the digital divide, authentication methods have changed very little. Authentication is the forefront of securing users’ information, services and technology, yet for many it still poses issues in terms of usability and security, due to specific characteristics of different disabilities. In this paper, drawing upon a literature review, a review of recognized disabilities, and the results of a small questionnaire study, we review the current authentication methods and discuss the potential issues that users with different disabilities face when interacting with these methods. We identify the specific aspects of disabilities that lead users to struggle to authenticate themselves. The results of this study lead to several recommendations, and suggestions for extending the existing inclusive technology framework to the authentication context. Through extending the framework, this could guide the development of future technologies, systems and services, ensuring that they are not only digitally inclusive in their function, but digitally inclusive in their security.

People with disabilities are found to be severely affected by barriers in websites and other web services. This paper aims to study the web accessibility issues in Mozambique and to review the factors that contribute to it. The case study is based on results from 2 perspectives: first, automatic evaluation of five prominent national websites, and second through fieldwork, interviewing and interacting with web developers in Mozambique. The study found that none of the websites are WCAG 2.1 compatible and the web developers, in general, do not consider web accessibility in their products and services. The underlying factors responsible for web inaccessibility are found out to be extrinsic.

Digital maps have been an integral part of modern life. Whether to venture into an unknown location, check the latest traffic update, update on the weather forecast, we come across digital maps every day. While maps have successfully evolved into digital form from paper and other physical mediums, how much evolution present-day digital maps have observed to ensure accessibility and implementation of universal design principles? Maps by nature have to rely on graphical medium to present their information content. But the users who have temporary or permanent and limited to no visual ability are excluded from reading maps for this reason. In this study, we have conducted a systematic literature review to discover the research gaps of accessibility in digital maps, focusing on map exploration based on screen reader technology. To discover further accessibility issues from users, we conducted semi-structured interviews with participants with varying degrees of visual impairments. The result from these data indicates that interactive maps are not screen-reader accessible at all. There is an apparent research gap in alternative text accessibility in maps and interview participants commonly agreed with multiple accessibility issues on contemporary interactive maps on diverse platforms.

In this paper, we investigate and analyze how young people that have motor impairments experience and have access to various forms of Information and Communications Technology (ICT). The aim is to gain a deeper understanding of how various assistive technologies (AT) mediate between young people and ICT systems, exploring how the user experience is perceived in practice. We apply the lens of the cultural-historical activity theory framework via a series of mainly qualitative explorations in Sweden, studying the user experience of ATs. They obtained findings reveal that there are many breakdowns in the interaction with ICT at the micro level, mainly due to limited fine motor impairments in the hands, which the used ICT - with or without AT - could not handle properly. Still the young adults perceived a positive UX in general. When comparing their parents’ view on the access to and experience of ICT a more negative UX of the available ICT systems emerged from a macro level. The parents often mentioned the lack of appropriate ICT and AT as well as relevant support from the education institutions and other authorities. Some identified deficits framed from the perspective of sustainable accessibility are discussed from the societal level. Especially, the need of proper education that is enabled by credible ICT and AT, otherwise disabled young people are hindered to fully participate in society.

This paper provides an overview of game accessibility. It describes what game accessibility is and highlights the need to combine usability and adaptability, as a “one-size-fits-all” approach is not possible due to the interactive and dynamic nature of the video game medium and the array of (dis)abilities different users may have. The main accessibility barriers different groups of users face are also presented. Then, the main focus shifts to the current state of the art in game accessibility, including existing guidelines and research carried out in this field. Finally, future perspectives are outlined, emphasizing the need for a user-centred approach and for collaboration between users, academia, and the industry.

The growing diffusion of fashion e-commerce websites shows the appreciation by users, highlighting the importance of offering this service also to users with different disabilities. To this end, e-commerce should be not only accessible - implementing all the technical requirements for accessibility - but also usable, paying attention to the offered user experience.This study aims to investigate the current e-commerce usability considering visually impaired users’ navigation experience and understand which aspects define a good usability level for this target. An expert analysis of a set of fashion e-commerce websites and user testing were conducted, considering five different market segment categories. The analysis highlighted a gap in the consideration of visually impaired users’ navigation needs and style, as for instance non-uniformity of layout and page structure. All the findings have been structured in usability guidelines to favor the e-commerce usability improvement, with the goal of offering visually impaired users a better shopping experience.

The success of a human-machine interface (HMI) heavily depends on its usability. An highly usable interface allows the user to more easily achieve his/her goals and in general have a better User eXperience (UX). In work environments, a structured and ready-to-use usability testing protocol can encourage companies to carry out this type of study and focus on UX from the early design phases. Even though numerous methods to test usability exist, industrial companies still have great difficulties to apply them and choose the best ones for the specific purposes. They should be guided into the analysis by a universal step-by-step approach, which helps also not experienced designers selecting the most reliable and useful methods among the available ones.In this direction, the paper proposes a structured protocol to focus on UX and guide companies in testing setup, execution and debriefing in an easy and quick way. Checklists are defined to help during user testing and assure its success. As a consequence, end users can be easily involved to give an added value in design problems identification. The novelty of this paper is the definition of a ready-to-use study protocol that can also be used by non-usability experts, in order to make them familiar with UX analysis and extend this practice also in industrial HMI design. As validation, the proposed protocol was applied to the design of interfaces for agricultural tractors during two different stages of the HMI redesign process.

Generation X and Millennials are very important to marketers, as they are the country’s main workforce and consequently have the biggest purchasing power. Social networks enable these consumers to become more informed and direct them toward potential purchases. This study analyzed Generation X and Millennial consumers and their social network behavior while searching for information about commercial products and services, and investigated how these platforms contribute changing a visitor into a buyer. A survey was implemented based on the experience economy theory and was disseminated through social networks. Then a social networks user experience (SNUX) model was developed to study the UX associated with platform use as a medium to find information about products and services with intention to purchase. The data were analyzed via structural equations modeling using SmartPLS 2.0, and the results obtained showed that information disseminated with educational and/or entertainment content was the main variable that influenced purchase intention for these two generations.

Citizen science is about democratising access to science for all citizens. A crucial question in this respect however is, whether democratisation of access to research in terms of reaching and involving diverse citizens actually takes place in citizen science projects. Unfortunately, this does not seem to be the case. The EU funded project Reinforce adheres to principles of inclusiveness and monitors and evaluates whether these principles are being met. Safeguarding that citizen science projects are inclusive means addressing technical as well as psychological and sociological aspects. On the technical side, in order for people with visual impairments to participate, the Reinforce Citizen Science projects, which are implemented on the Zooniverse platform, have to be made accessible. The software SonoUno has been developed to transform images into sound signals. A second strand for reaching diverse target groups is the participatory engagement strategy of the project. The applied evaluation process systematically targets quantitative and qualitative methods to evaluate the inclusivity approach of the project. The paper seeks to discuss the inclusivity aspect of citizen science projects in general, to then describe the project Reinforce with its strategies to make citizen scientists projects more inclusive especially for people with visual impairments, technically, psychologically and sociologically. Furthermore, we will share the evaluation framework that has been elaborated to monitor the diversity of participants and inclusivity of the citizen science projects.

The employment settings for autistic individuals in the USA is grim. As more children are diagnosed with ASD, the number of adolescent and young adult with ASD will increase as well over the next decade. Based on reports, one of the main challenges in securing and retaining employment for individual with ASD is difficulty in communicating and working with others in workplace settings. Most vocational trainings focused on technical skills development and very few addresses teamwork skills development. In this study, we present the design of a collaborative virtual environment (CVE) that support autistic individual to develop their teamwork skills by working together with a partner in a shared virtual space. This paper described the CVE architecture, teamwork-based tasks design and quantitative measures to evaluate teamwork skills. A system validation was also carried out to validate the system design. The results showed that our CVE was able to support multiple users in the same shared environment, the tasks were tolerable by users, and all the quantitative measures are recorded accordingly.Note: We are using both identity-first and people-first language to respect both views by interchangeably using the term ‘autistic adults’ and ‘adults with autism’ [22].

Around 350 million color blind people in the world and 64% of people indicated that color blindness was their biggest problem and 42% considered it difficult to fully integrate into society. This type of condition is not usually understood as a disability, however it could be the cause of dangerous situations depending on the context in which the person suffering from it is found. Color blindness has various color perception conditions. In this paper, a description of the types of color blindness is presented and the development of an assistive technology that allows deuteronopia-type color blind users to view modified images to mitigate the deficiency is presented. The modification of the images is based on algebraic transformations use of the image, color saturation and linear arrays. The app was tested and validated with color blind people and achieved an 83% approval rating.

Progressive memory loss is the hallmark of Alzheimer’s disease. This memory loss has a profound impact on the activities of daily living (ADL) of people with this condition. ADL exercises are essential for maintaining the skills of people with Alzheimer’s disease in order to expand their presence at home. But such exercises require a huge personalization to be adapted to each people with Alzheimer’s disease. In this article, we report the approach and the solutions we have adopted to set up digital workshops around ADL by “personalizing” the solutions adapting the content and functionalities of the application to the specific characteristics of each Patient. This solution has been elaborated with the collaboration of caregivers of a day-care center. We put into practice a participatory design process adapted to Patients with Alzheimer’s disease. The design effort is essentially at the level of choosing the best possible representation of objects and actions to digitize the workshops.

Nowadays, more and more visually impaired people want to live independently and cook for themselves. However, cooking independently may bring lots of challenges. This paper provides a better solution by helping visually impaired users locate ingredients quickly and find more ways to cook flexibly. The design is a comprehensive process. After the stage of problem identification, brainstorming, concept development and prototyping, we finally designed an application, Tasteful, that helps visually impaired users better solve the problems encountered in the cooking process.

It has been pointed out in previous studies that information from the peripheral vision may trigger three-dimensional (3D) motion sickness. In a previous study, the authors created two types of 3D images with different background elements, and determined the regional cerebral blood flow as the 3D images were viewed via functional near-infrared spectroscopy (fNIRS). Accordingly, we verified that the differences present in the background elements of the peripheral visual field region affected cerebral hemodynamics when viewing 3D images. In addition, we reported the possibility of overload in the depth perception-related visual information process that occurs in the dorsal visual pathway when viewing 3D images with complex background elements. In this study, we focused on the background elements in the peripheral visual field region that were added to 3D images, and investigated the effects of the differences in these elements on the body balance function. Consequently, we verified that the difference in the amount of the background element protrusion influences the body balance function.

A World Health Organization study states that there are 2.2 billion people with some sort of visual impairment. Some of those people are considered blind. A way blind people can communicate is by using a tactile alphabet called Braille. The percentual of blind people that can read Braille in developing countries is very low, about 10%, since the cost of tools for teaching Braille can be high. Additionally, not enough tutors are available to teach the students. Usually, a commercial tactile display costs over 800 dollars ($US), using components that are both expensive and hard to repair. This paper proposes enhancements for the DISBRA, a low-cost, open source, single digit Braille display that is inexpensive and made using cheaper electronical components and 3D printed parts. The DISBRA can be used to train tutors without teacher input, besides being an aid in teaching students. The improvements proposed by this paper eliminate the need for an external control device for the DISBRA, reducing its cost and making it a standalone solution. The DISBRA 2.0 prototype was tested by individuals without visual impairment. After one session of less than two hours they could correctly identify a character using only touch 68.46% of the time. While looking at the Braille the success rate was 86.15%.

Parkinson’s disease (PD) is the second most prevalent neurodegenerative disease in the world. Impacts of the disease on both quality of life and social participation make it a major preoccupation in terms of public health. This disease is always associated not only to motor but also non-motors symptoms. Among these symptoms, several people with PD report communication impairments that worsen their speech intelligibility, their ability to express affective states and as consequence, their social relations. In order to improve this situation, we have been working on the ParkinsonCom project. This project aims to co-construct and make available a communication support tool for people with PD using a participatory design. To that end a user-centered design has been applied, integrating people with PD as well as their caregivers (family, friends, medical professionals…) in all stages of the development. This paper presents the whole methodology of the project and the results of the first phase related to requirements analysis.

The most effective strategy for homes to save energy is by decreasing their electricity consumption. Home Energy Management Systems connect appliances that improve households’ energy performance to thermal comfort. These systems need to take into account human behavior regarding saving energy and thermal comfort. This paper proposes a three-step framework that integrates the Smart Residential Load Simulator (SRLS), Adaptive-Network based on Fuzzy Inference System (ANFIS), and a gamification structure to develop an interface designed to reduce energy consumption without losing thermal comfort. Finally, a gamified mock-up for mobile devices is displayed for a household with high energy consumption levels and a temperature setpoint of 23 ℃. This proposal integrates the concept of social products to empower the interaction between devices and end-users.

Visual impairment refers to the partial or complete loss of one’s ability to see. It is estimated that there are 1.3 billion people in the world with some form of vision loss. In this work, we present Viva, an Android-based virtual assistant aiming to help people with visual impairment. The application provides haptic and voice navigation assistance by detecting obstacles in the user’s surroundings and calculating the potential risk. We present the architecture, as well as a proof-of-concept prototype intended to demonstrate a potential use-case for a commercial embedded product that can be integrated into a walking stick or any wearable gadget. This Android application has features such as navigation assistant, object detection, voice-controlled UI and emergency assistant. The navigation assistant analyzes a user’s surroundings by detecting and estimating distances from the user to the object. Object recognition mode includes a pre-built object recognition model that can recognize over 100 different common objects. Data collected is then processed by a risk-prediction algorithm to calculate the risk of collision. Feedback is provided to the user whenever there is a potential risk observed. The UI of the virtual assistant is uniquely designed from the ground-up to be intuitive, without the need for any usual aids via voice commands or single point touch control – where the entire screen acts as a soft button. Viva operates in a low-power mode with the screen turned off to efficiently utilize the limited battery resources on mobile phones. Viva is a prototype intended to demonstrate the potential use-cases of this idea. It can be integrated into other IoT devices such as smart walking sticks or wearable gadgets.

3D printed audio-tactile graphics are interactive materials aimed at the accessibility of people with visual impairment. There is not yet a consensus on how to design 3D printed audio-tactile graphics and most publications on this topic follow an empirical approach, experimenting first and learning with the process. However, these research publications show meaningful knowledge that can serve as the base for a set of compiled design recommendations. The goal of this paper is to present recommendations that were analyzed and extracted from 32 publications found in a systematic literature review. In total, 57 recommendations compose the set.

Augmentative and alternative communication systems are devices that seek to solve communication problems in people who require it. People with neuromuscular diseases benefit from these systems, however, there is a growing need to develop systems that adapt to people with these types of diseases due to their degenerative nature. In this paper, a development method based on fuzzy logic is presented to identify the stage of the disease based on heuristic rules whose purpose is to provide a unique and usable system throughout the disease until the death of the person.

Graphic contents are important resources to communicate information and their use is fundamental in the teaching process. Visually impaired people do not have easy access to this resource owing to their impairment. They mainly use tactile graphics to explore visual contents. In this article, we propose the development of a tool for tablets intended to aid visually impaired people in interpreting tactile graphics. After studying different exploration strategies of tactile graphics adopted by visually impaired people, we propose an application that utilizes the benefit of multimodal interactions. Through a case study, three blind students tested the prototype of the application, demonstrating that the solution is a promising approach to aid in the exploration of tactile graphics.

The multimodal interaction technique that combines eye and foot input not only provides a great opportunity for the user with busy hands or hand disabilities to interaction with computer, but can also overcome the drawbacks when using dwell time to solve the “Midas Problem” of gaze input. However, the user’s capability of eye-foot coordination was still unclear. At the same time, the human performance in the basic task of target acquisitions by eye-foot input was also uncertain, and especially a proper performance model was lacking. Motivated by this situation, an eye pointing and foot tapping task experiment had been carried out to fill these gaps. A low-cost eye tracker and a USB foot pedal switcher were used as the input devices from different modalities. The experimental results indicated that the user was soon able to coordinate her/his foot with the eyes for target acquisitions, and that the user could respond fast to tap the foot pedal to finish a task trial in the level of 600 ms. The main performance measures of eye movement time (EMT) and eye pointing time (EPT) under the eye-foot multimodal input condition were significantly increased with the increase of saccadic amplitude A and/or the decrease of target width (size) W, and vice versa. Regression analysis shown that the $$ID_{eye}$$ I D eye model was more suitable than the standard Fitts’ law to model the human performance in this multimodal interaction context.

This paper introduces a new recommendation system for museums able to profile the visitors and propose them the most suitable exhibition path accordingly, to improve visitors’ satisfaction. It consists of an interactive touch screen totem, which implements a USB camera and exploits Convolutional Neural Network to perform facial coding to measure visitors’ emotions and estimate their age and gender. Based on the detected level of emotional valence, the system associates visitors with a profile and suggests them to visit a selection of five works of art, following a specific itinerary. An extensive experimentation lasting 2 months has been carried out at the Modern Art Museum “Palazzo Buonaccorsi” of Macerata. Results evidence that the proposed system can create an interactive and emotional link with the visitors, influencing their mood in the Pre-Experience phase and in the subsequent Post-Experience phase. In particular, they highlight that the proposed system, which aims at acting as emotional leverage, has been able to improve the positiveness of the emotions experienced by the visitors.

Employment of autistic individuals is strikingly low in relation to the skill level and capabilities of this population. Roughly 65% of autistic adults are either unemployed or underemployed relative to their abilities but there is increasing recognition that this number could be greatly improved through empowering autistic individuals while simultaneously providing a boost to the economy. Much of this disparity can be attributed in part to the lack of awareness and understanding among employers regarding behavior of autistic individuals during the hiring process. Most notably, the job interview—where strong eye contact is traditionally expected but can be extremely uncomfortable for autistic individuals—presents an unreasonable initial barrier to employment for many. The current work presents a data visualization dashboard that is populated with quantitative data (including eye tracking data) captured during simulated job interviews using a novel interview simulator called Career Interview Readiness in Virtual Reality (CIRVR). We conducted a brief series of case studies wherein autistic individuals who took part in a CIRVR interview and other key stakeholders provided lived experiences and qualitative insights into the most effective design and application of such data visualization dashboard. We conclude with a discussion of the role of information related to visual attention in job interviews with an emphasis on the importance of descriptive rather than prescriptive interpretation.

The benefit of using a non-contact eye-tracking system is that it is considered as a low-burden method for measuring biological signals. The goal of this study was to develop a visual induced motion sickness (VIMS) evaluation index that uses a non-contact eye-tracking system for driving simulator (DS) experiments. The participants included nine elderly people who had visual and balance functions that did not interfere with their daily life. The gaze data of the participants were measured at rest—both before and after DS trials. The simulator sickness questionnaires (SSQ) were conducted before and after the experiment. The participants were divided into two groups based on their SSQ results. One group experienced VIMS during the DS trial (four people, with an average age of 79.0 years), whereas the other group did not experience VIMS during the DS trial (five people, average age: 71.2 years). The results of VIMS symptoms were confirmed: data concerning the locus of eye-tracking were lengthened, whereas the eye-tracking data were diffused. Moreover, in the group which experience VIMS, the regularity of gaze data during the DS trials was increased. This experiment demonstrated the usefulness of sparse density and the regularity of evaluation as a quantification index for eye-tracking data in evaluating VIMS. Regarding the application of the findings of this study, it is believed that if an eye-tracking data-based VIMS evaluation index can be used, it will be easier to detect VIMS which is caused by DS operations, thus permitting the detection of the symptoms.

This study aims to measure music tone impact on the behavior of ADHD Children in 3 kinds of behavior attention, relaxation, and concentration related to 3 kinds of brainwave 1.Theta wave - Attention 2. Alpha wave - Relaxation 3. Beta - Concentration. In the experiment, we controlled study employing 30 students with ADHD and 30 students without ADHD (total 60 students) ranging in age from 7 to 9 years in Thailand, participated in music listening sessions of comparison design in two different tones of music by the piano melody that was composed for this research in C tone (major key) and G tone (minor key) which is the tone is lower than C tone. In the experiment, all 60 students wear the macrotellect brain-link lite EEG headset V2.0 device meanwhile 3 brainwaves 1. Theta wave 2. Alpha wave 3. Beta was collected from them. The results showed that the music tone has impacted the behavior of ADHD children because the experiment found that the brain waves score of ADHD children collected from the 3 kinds of brainwaves were changed significantly.

This paper explores the theoretical aspect of providing context to contextually impaired individuals and offers some considerations on how a machine learning system can be adapted to learn contextual clues and then provide these to users. Context itself, is the clarifying component of a situation and helps people to understand what is happening and why. Some people struggle to understand the situations that they are in, due to several issues, particularly those with sensory or social impairments, such as autism. These impairments can be interpreted as form of contextual blindness, where a lack of awareness of certain contextual clues render communication or understanding of it difficult. These clues can be as simple as a person’s current location, to the more nuanced impact of a conversational partner’s body language. Since awareness of the current context of a situation can help to clarify its meaning, it is expected that provision of contextual information to contextually impaired individuals will be beneficial; by giving them a point of reference for the tone of the situation they are in and supporting them to react accordingly. The results presented demonstrate the impact of contextual information in a specific social situation (dating), giving an indication of the benefits of both context and contextual awareness, which provide the basis for further investigation.

Studies show that young autistic adults are under- or unemployed, with almost half never holding a paying job in their 20’s. Unemployment within this population leads to decreased personal growth and increased dependence on caregivers. Research suggests that the interview process is one of the largest barriers to employment for this population. Autistic individuals often struggle with emotion regulation, which can be exacerbated by the interview process. To address this, we propose the use of a stress detection model in conjunction with a virtual reality interview simulator. This combination will allow for the interview to adapt to the state of the participant to improve the skills and engagement of the user and positively influence their comfort level. Data regarding negative affective responses to categories of questions can also be used to inform employers on better interviewing techniques. A model was designed using data obtained from neurotypical participants completing a modified Computerized Paced Serial Addition Task (PASAT-C) and evaluated on a dataset obtained from Autistic participants who took part in a simulated interview. Agreement between the model and ground truth was compared based on Pearson correlation coefficients. It was found that was r(289) = 0.28, which was statistically significant (p < .001; CI: 0.17 to 0.38). Our preliminary results provide evidence for the validity of observer-based labeling of data captured using a wrist-worn physiological sensor.

There are few reports to compare gastrointestinal motility in a seated posture and that in a supine posture for the evaluation of the severity of the motion sickness. It is difficult to distinguish between postural differences by using nonlinear analysis of the electrogastrograms (EGGs) of healthy individuals. The purpose of this study was to determine the feasibility of applying a complex dynamic analysis method to the EGGs of healthy young individuals between in a seated posture caloric intake and those in a supine posture after caloric intake. We analyzed EGGs by using artificial intelligence (AI) and compared mathematical models of EGGs in the seated posture with those in the supine.

Artificial Intelligence (AI) algorithms, together with a general increased computational performance, allow nowadays exploring the use of Facial Expression Recognition (FER) as a method of recognizing human emotion through the use of neural networks. The interest in facial emotion and expression recognition in real-life situations is one of the current cutting-edge research challenges. In this context, the creation of an ecologically valid facial expression database is crucial. To this aim, a controlled experiment has been designed, in which thirty-five subjects aged 18–35 were asked to react spontaneously to a set of 48 validated images from two affective databases, IAPS and GAPED. According to the Self-Assessment Manikin, participants were asked to rate images on a 9-points visual scale on valence and arousal. Furthermore, they were asked to select one of the six Ekman’s basic emotions. During the experiment, an RGB-D camera was also used to record spontaneous facial expressions aroused in participants storing both the color and the depth frames to feed a Convolutional Neural Network (CNN) to perform FER. In every case, the prevalent emotion pointed out in the questionnaires matched with the expected emotion. CNN obtained a recognition rate of 75.02%, computed comparing the neural network results with the evaluations given by a human observer. These preliminary results have confirmed that this experimental setting is an effective starting point for building an ecologically valid database.

Frustration is a common response during game interactions, typically decreasing a user’s engagement and leading to game failure. Artificially intelligent methods capable to automatically detect a user’s level of frustration at an early stage are hence of great interest for game designers, since this would enable optimisation of a player’s experience in real-time. Nevertheless, research in this context is still in its infancy, mainly relying on the use of pre-trained models and fine-tuning tailored to a specific dataset. Furthermore, this lack in research is due to the limited data available and to the ambiguous labelling of frustration, which leads to outcomes which are not generalisable in the real-world. Meanwhile, contrastive loss has been considered instead of the traditional cross-entropy loss in a variety of machine learning applications, showing to be more robust for system stability alternative in self-supervised learning. Following this trend, we hypothesise that using a supervised contrastive loss might overcome the limitations of the cross-entropy loss yielded by the labels’ ambiguity. In fact, our experiments demonstrate that using the supervised contrastive method as a loss function, results improve for the automatic recognition (binary frustration vs no-frustration) of game-induced frustration from speech with an Unweighted Average Recall increase from 86.4 % to 89.9 %.

Parkinson’s disease (PD) is the second most common neurodegenerative and movement disorder affecting different groups of people, especially the aging community. According to research studies, a noticeable cause of PD is early and progressive deterioration of dopaminergic neurons in the Substantia Nigra Pars compacta (SNc). PD diagnosis from Magnetic Resonance Imaging (MRI) and functional MRI (fMRI) seems to offer promising results. However, comprehensive studies are needed to explore various brain atlases and their effects on accurate anomaly detection in substantia nigra basal ganglia. In this survey paper, we present the recent related works in automatic PD diagnosis using fMRI data. Our goal is to study and compare various automatic PD detection algorithms, bring up the current limitations and challenges in the literature and highlight the future possibilities to further improve this research area.

Researchers have been developing avatars to portray sign languages as a necessary component of automatic translation systems between signed and spoken languages. Although sign language avatar technology has improved significantly in recent years, there are still open questions as to how best portray the linguistic and paralinguistic information that occurs on a signer’s face. Three interdisciplinary themes influence the current state of the art. The first, linguistic discovery, defines the facial activity that an avatar must carry out. The second, Computer Generated Imagery (CGI), supplies the tools and technology required to build avatars, and which determines the fidelity of an avatar’s appearance. In contrast, the third theme, Sign Language Representation Systems, determines the fidelity of timing of facial co-occurrences. This paper discusses the current state of the art and demonstrates how these themes contribute to the overall goal of creating avatars that can produce legible signed utterances that are acceptable to viewers.