Skip to main content
Top

2025 | Book

HCI International 2025 Posters

27th International Conference on Human-Computer Interaction, HCII 2025, Gothenburg, Sweden, June 22–27, 2025, Proceedings, Part II

insite
SEARCH

About this book

The eight-volume set, CCIS 2522-2529, constitutes the extended abstracts of the posters presented during the 27th International Conference on Human-Computer Interaction, HCII 2025, held in Gothenburg, Sweden, during June 22–27, 2025.

The total of 1430 papers and 355 posters included in the HCII 2025 proceedings were carefully reviewed and selected from 7972 submissions.

The papers presented in these eight volumes are organized in the following topical sections:

Part I: Virtual, Tangible and Intangible Interaction; HCI for Health.

Part II: Perception, Cognition and Interaction; Communication, Information, Misinformation and Online Behavior; Designing and Understanding Learning and Teaching experiences.

Part III: Design for All and Universal Access; Data, Knowledge, Collaboration, Research and Technological Innovation.

Part IV: Human-Centered Security and Privacy; Older Adults and Technology; Interacting and driving.

Part V: Interactive Technologies for wellbeing; Game Design; Child-Computer Interaction.

Part VI: Designing and Understanding XR Cultural Experiences; Designing Sustainable (Smart) Human Environments.

Part VII: Design, Creativity and AI; eCommerce, Fintech and Customer Behavior.

Part VIII: Interacting with Digital Culture; Interacting with GenAI and LLMs.

Table of Contents

Frontmatter

Perception, Cognition and Interaction

Frontmatter
Cognitive and Perceptual Reliable Performance: Comparison of Psychophysiological Limitations

Changing the task presentation time can significantly affect the time of problem solving. Research shows that when the task presentation rate increases, the time to complete it can decrease due to increased concentration and mobilization of cognitive resources. However, excessive acceleration can lead to errors and poorer performance due to stress and overload. The aim of this study was to identify quantitative criteria for assessing the ultimate possibility of reliably solving the cognitive and perceptual problems based on a comparison of the time needed to solve problems and the response of the cardiovascular system. The survey included test task performance (logical-combinatorial and perceptual tasks) and heart rate. Time for every task performance was fixed during the test session and limited (5 intervals in both tests of 10 min each with modeling different “time pressure”). We have revealed in our study that the ratio of the time of the test performance for 10 min and the tension of cardiac regulation by interbit intervals can be a quantitative measure of the limits of human capabilities for processing a stream of perceptual tasks. A significant criterion can be the measurement of the time of tasks’ solution in the perceptual test reduced to 0.62 of the average time it takes to solve such problems by this person without time limits.

Oleksandr Burov, Evgeniy Lavrov, Svitlana Lytvynova, Olha Pinchuk, Svitlana Proskura, Oleksii Tkachenko, Natalia Kovalenko, Yana Chybiriak, Yana Dolgikh
NOCTURNAL: A Virtual Reality Simulation Exploring the Effects of Night Vision Goggle Configurations and Luminance Conditions on Human Performance

Even decades following their introduction to the operational community, night vision devices (NVDs) employing optoelectronic image intensification (I $$^2$$ 2 ) continue to be the dominant form of augmented vision during low light conditions in the United States Air Force (USAF). Despite this, there is relatively little research dedicated to examining the specific effects of night vision goggle (NVG) configurations on human visual and cognitive performance—an issue that is compounded by difficulties related to experimental control in naturalistic scenarios and regulations that limit the access of USAF NVGs outside of specific environments. To help resolve these difficulties, we have created NOCTURNAL, a virtual reality simulation using Unreal Engine 5 that simulates vision in naturalistic environments when using P43 (green) and P45 (white) phosphor ANVIS-9 night vision goggle types. NOCTURNAL not only provides a controlled experimental environment with performance logging (e.g., mission completion time and shooting accuracy), but it also simulates visual artifacts and lighting effects that are normally encountered when using ANVIS-9 NVGs. Specifically, it incorporates physics-based simulations of visual static (i.e., scintillation), blooming effects (or “halos”), luminance-related changes in signal-to-noise ratio (SNR), muzzle flashes, and other related effects. The simulation also provides capabilities for logging performance through a base scenario—nighttime building ingress and egress in an urban, desert environment. Overall, NOCTURNAL provides a low-cost virtual reality platform capable of realistic simulations of NVGs and performance assessment.

Taylor M. Curley, Jerry Huggins, Sharon Ellis, Frederick Meyer, Elaina Grayson, Savonnah Johnston, Dorian Lawrence, Claire Wu, Jonathan Diemunsch
Finding Isometries or Distance-Preserving Maps Between Color Spaces of Different Observers and Its Application

It is usually difficult to observe and compare the differences in colors perceived by different individuals. Recently, isometries or distance-preserving maps between psychophysical color spaces are proposed to compare and exchange color perceptions by different observers. On the other hand, existing ways to build isometries need to solve nonlinear equations and use a restricted form in order to obtain a unique solution. In this study, we show a new method to build an isometry between color spaces of different observers. We estimate local isometries using only color discrimination thresholds in a general form and with only computations of linear algebra. These local isometries are then combined by a convex interpolation to obtain a global isometry. The proposed method is then applied to visualize and compare the differences in color visions between different observers and exchange between them. In particular, we applied the method to color-weak compensation and color-weak simulation. Experiments shown that the proposed algorithm provides accurate estimates of the local isometries and is effective in color-weak compensation and simulation.

Tomoyo Kobayashi, Ryo Kamiyama, Jinhui Chao
A Comparison of Pilot Situation Awareness in Virtual Reality Flight Simulation and an Online Cognitive Health Screening Tool

Existing cognitive screening tools for aviators have not focused on domain-relevant functions of cognition, like situation awareness (SA). Without this focus, general aviation (GA) pilots may be vulnerable to undetected cognitive incapacitation, increasing their risk of an accident or mishap in flight. The current research reports on the validation of the CANFLY, an online cognitive health screening tool that indexes cognitive factors critical to safety in GA, such as SA. The CANFLY aims to be a tool that allows pilots to self-assess their pilot-cognition in an “on-demand” fashion. In the present study, pilots completed the CANFLY, which included watching and interacting with pre-recorded flight-segment videos. Pilots monitored radio communications while tracking their flight instruments through adjusting a virtual slider to match a specified instrument gauge. After each video, pilots answered SA questions about their ownship as well as details about other traffic. Pilots then flew a virtual reality (VR) simulated flight scenario, including take off, cross-country flight, and circuit/pattern activities (i.e.: touch-and-gos and landing). The VR flight served to ground true SA abilities of pilots. The VR scenario was paused three times and pilots reported on their ownship and knowledge of traffic, similar to SA tasks contained in the CANFLY. Preliminary results with 27 licensed pilots, ages 19–75 (M = 47.57, SD = 17.61) demonstrate a moderate relationship between SA scores from the CANFLY and the VR flight. These initial findings are important for external validity of the CANFLY, and efforts to make it available for the GA community.

Emily Larkin, Kathleen Van Benthem, Alexia Bierlaire, Rawan Othman, Chris M. Herdman
Effects of Visualization Transparency in Human-Machine Interface Display on Passenger Visual Perception

With the rapid development of artificial intelligence in the field of autonomous driving, the way we drive is undergoing significant transformation. Currently, the application of Trajectory-guided Control Prediction (TCP) for End-to-End Autonomous Driving enables real-time prediction of obstacle motion trajectories and provides early warning information through Human-Machine Interface (HMI) display. This potentially enhances users’ visual perception of autonomous vehicles. However, there is still no clear answer regarding how the transparency of TCP visualizations should be configured and how such configurations affect users’ visual perception. To address this gap, this study explores the impact of different transparency levels in TCP visualizations on users’ visual perception from a visual cognition perspective. Using a between-subjects experimental design with one-way ANOVA, the study examines the effects of TCP visualizations with transparency levels of 20%, 40%, 60%, and 80% on users’ visual perception. A total of 100 participants were recruited through convenience sampling and completed questionnaires and interviews. The results reveal that: The transparency of TCP visualizations in autonomous driving HMI significantly affects users’ visual perception. High-transparency TCP visualizations result in lower levels of user experience compared to low-transparency designs. TCP visualizations with 40% transparency outperform those with 20%. The results of this study can provide a valuable reference for the visualization and design of TCP in HMI in autonomous driving scenarios.

Yihan Peng, Bowen Deng, Shiyu Fan, Detong Zhang
A Flight Fatigue Monitoring Model Based on Multimodal Physiological Parameters

This study introduces a fatigue monitoring model based on multimodal physiological parameters. The proposed model utilizes a multi-branch convolutional neural network (CNN) architecture, incorporating four distinct feature extraction branches: a global index branch, a video behavior index branch, an eye movement index branch, and an electrocardiogram (ECG) index branch. By integrating 2D convolutional layers, residual modules, and 1D convolutional layers, the model effectively captures multi-scale spatial features and temporal dynamic features from diverse modalities. Through dual-level fusion of extracted features and decision-making processes, combined with adaptive decision optimization, the model achieves precise classification of multimodal physiological data. Experimental results demonstrate the model’s superior performance in 5-fold cross-validation, achieving an average accuracy of 91.63%, precision of 91.88%, recall of 91.63%, F1 score of 0.92, and a Kappa coefficient of 0.89. These findings validate the robustness and effectiveness of the model for long-endurance flight fatigue monitoring tasks.

Donghui Piao, Xiaopeng Liu, Congchong Li, Xiaomin Liu, Yan Zhang, Lihua Yu, Weiru Shi, Wenjing Gong
Cognitive Simplification in Scenic Design: Examining Background Complexity and Perceptual Load Adjustment in Tibetan Mani Stone Landscapes

This study explored the role of background complexity in cognitive and aesthetic responses toward Tibetan cultural landscapes with emphasis upon “Mani stone stacks”—the sacred stone stacks inherent in Tibetan spirituality as well as culture. Ratings of the stone stacks in pictures under simple background (neural gray), complex background (dense natural or village), as well as singularity background conditions indicated how cognitive responses varied as a function of background. Ratings of aesthetic appeal, as well as cultural value, as well as emotional resonance, as well as cognitive responses as evidenced through the use of EEG analysis, varied as a function of background. Additionally, cognitive responses as evidenced through the use of the measures of reaction times as well as accuracy of assessment varied. Findings indicated simpler background conditions (the neutral gray) produced more attention toward the stone stacks with less cognitive work as well as more favorable assessment of aesthetic as well as cultural value. More complex background conditions, like the dense natural or village, demanded more cognitive work, which can interfere with the experience of the stones as meaningful carriers of spiritual as well as cultural value. A no background condition, being less diverting, produced middling levels of attention through the elimination of background richness. Optimizing background complexity can be a factor toward enhanced cognitive as well as aesthetic experience in culturally significant landscapes.

Xiaoyu Ren, Zhijun Peng, Han Sun, Yibo Zhang, Deng Pan, Hong Zhao, Yirun Wang
Case Study: Using EEG to Assess Cognitive Load in Infinite Interfaces

In the realm of Human-Computer Interaction, cognitive load (CL) plays a crucial role in determining user satisfaction, product acquisition, and user retention. We are introducing a new type of interface - Infinite Interfaces (II), that dynamically adapts to user inputs and, therefore, offers personalized and efficient user experiences. However, the novelty of such interfaces may lead to increased cognitive effort, as most users are accustomed to traditional predetermined interfaces. This study seeks to evaluate the CL associated with II using two approaches: subjective user ratings, which are simple and accessible, and EEG technology, providing an objective measure. The research aims to compare CL in II versus traditional interfaces, using both methods to identify any similarities or differences in the results. Results indicate that while novel interfaces like II initially induce higher CL, they offer long-term potential for improving user efficiency in complex tasks.

Anastasiia Satarenko
Metacognition in Software Teams: Investigating Individual Expertise in Cognitive Knowledge and Regulation

This study explored the relationship between individuals’ metacognition and their professional background when working in a remote software team. 30 developers, working in 3-person teams, worked on two programming challenges in a controlled virtual environment. The developers completed individual surveys on metacognition and self-efficacy. Based on quantitative analysis of survey responses, we observed that PhD students demonstrated greater resilience and higher metacognition compared to professional developers and master’s students. While we did not find statistically significant differences in metacognition across professions and proficiency levels, we observed medium effect sizes.

Sandeep Sthapit, Jess Moorefield, Sandeep Kaur Kuttal
Understanding Effects and Physiological Correlates of Operator Workload Across Remote Assistance Scenarios for Automated Vehicles - Results from a User Study

Remote assistance for highly automated vehicles involves a remote operator as a fallback whenever automated fleet vehicles require coordination and maneuver-based support. Similar to other control center professions, we expect that operator workload affects their attention and thereby their performance. To validate this, we conducted a simulator study to analyze operator workload under various task load conditions using a remote assistance dashboard mock-up. We used a within-participants design varying task frequency and task complexity for a set of scenarios generated from a previously defined remote operation scenario catalog. Our findings indicate that high workload through high task frequency and high task complexity results in longer task completion times and more errors when resolving a task. Additionally, low task frequency leads to longer task initiation times after a break. Combining high complexity and low frequency task load leads to the worst performance across multiple metrics, indicating an out-of-the-loop performance problem when low task frequency does not provide a sufficient base-load for complex tasks. Finally, we collected and analyzed physiological data from an electrocardiogram and an electro-dermal activity sensor to find stable indicators of workload changes across study participants. We find that heart rate and heart rate variability show significant differences when task frequency is modulated. In addition, the number of skin conductance response (SCR) peaks and the SCR half-recovery time vary significantly with task complexity. These results provide evidence that we are able to identify workload changes using physiological indicators in time-critical scenarios as basis to provide state-based support to operators when needed.

Fabian Walocha, Andrea Valerio, Phuong Nguyen, Klas Ihme
Are Software Developers Good Decision-Makers? An Empirical Comparison of Decision-Making Skills Between Software Developers and Non-software Developers

Decisiveness arises from having to make decisions on a day-to-day basis and involves quick and well-founded decision-making behavior. The process of decision-making is particularly relevant in the field of software development, where software developers are regularly faced with important decisions. People make decisions in almost every situation in life, including the decision not to decide. The empirical comparison shows that software developers are characterized by a more proactive and vigilant willingness to make decisions than non-software developers. They show higher vigilance by conducting a more intensive search for decision alternatives and a more detailed evaluation of potential alternatives. Both groups, the software developers and non-software developers show an aversion of avoiding decision-making responsibility. The results show that there is low tendency to procrastinate decisions and that both groups approach decision-making mainly with self-confidence and conviction. Software developers and non-software developers generally do not tend to feel under pressure in decision-making situations. This indicates a certain resistance to decision-making stress. Software developers in particular show a deep cognitive and emotional engagement in decision-making scenarios, which is characterized by a critical dispute of the result of a decision and a pronounced self-confidence.

Greta Werneke, Anika Hennig, Gerrit Diersmann, Philipp M. Zähl, Christian Czarnecki
Identifying Human Out-of-the-Loop in Cruising Flights Using EEG Spectral Features with Deep Learning

Human out-of-the-loop has been a significant problem in flight operations causing two notable accidents, Asiana Flight 214 and Air France Flight 447. It is usually caused by the monotonous nature of the automation monitoring task. With the monotonous nature of the task, pilots may experience reduced vigilance and may not effectively monitor automation. However, pilots out of the flight control loop may be unable to understand the situation well and cannot cope when required to take over from the automation in emergencies. It is essential to improve human-automation interaction for timely corrective actions. As such, this paper proposes a neurophysiological and image-driven deep learning approach to identify human out-of-the-loop (OOTL) episodes during cruising flight operations, in which automation leads to flight control while pilots are significantly less involved. We collected EEG data from 24 cadet pilots during three cruising flights from Hong Kong International Airport to Fukuoka Airport on an Airbus A320 flight simulator under different levels of automation. These include the baseline with full automation (FAF), partially automated flight (PAF), and manual flight (MF). After obtaining the data, we processed the data and transformed it into spectral data for each band wave and two-second epoch. Data were then plotted on a topographical map to generate a dataset of 112,243 epochs for image-based mental state classification. Each epoch contains five (128, 128) RGB images to show the band power of the pilots during the two-second epoch. We thus employed deep learning and designed a tailor-made model structure of convolutional neural networks to classify the mental states. The results indicate that the proposed model achieves a test accuracy of 99.30%, which outperforms the baseline models by at least 33.74%. The proposed model can be applied to identify potential human OOTL in advance so that proper countermeasures can be taken.

Cho Yin Yiu, Kam K. H. Ng, Qinbiao Li, Xin Yuan
Optimising Pilot-Aircraft Interaction: A Low-Cost Projection-Enhanced Head-Up Display (PE-HUD) with Neuro-Ergonomic Validation

Modern aviation operations demand pilots to process multi-source cockpit instrumentation, creating cognitive fragmentation that degrades situational awareness (SA) and increases mental workload. While conventional head-up displays (HUDs) partially address these challenges, their static symbology and high retrofitting costs limit effectiveness. Augmented Reality (AR) solutions face adoption barriers due to visual conflicts in head-mounted displays and prohibitive waveguide HUD costs. This study proposes a low-cost Projection-Enhanced HUD (PE-HUD) using retroreflective film-screen symbiosis, achieving collimated imagery without complex optics. Neuro-ergonomic validation with functional Near-Infrared Spectroscopy (fNIRS) and eye-tracking were conducted across flight phases with ten cadet pilots under low-visibility simulations. Results demonstrated phase-specific cognitive benefits: PE-HUD significantly enhanced occipital-parietal activation during landing and improved cross-referencing between flight control units and primary displays. Take-off phases showed balanced multi-parameter monitoring, while cruise phases exhibited reduced HUD dependency. Eye-tracking revealed fewer head-down transitions compared to those without HUDs. Critically, pilots’ visual attention patterns varied dynamically across phases, necessitating adaptive interface designs. These findings establish neuro-ergonomic evidence for phase-sensitive HUD optimisation, advancing cost-effective human-machine interface (HMI) design in aviation. Future work will integrate real-time neurofeedback to enable context-aware display adaptation.

Xin Yuan, Kam K. H. Ng, Cho Yin Yiu, Qinbiao Li
A New Aircraft Workload Assessment Method for Airworthiness Compliance Verification Based on Independent Evaluation Strategy

The assessment of crew workload is normally used as an important parameter to evaluate the overall human-machine efficiency design level of the cockpit. For mature aircraft manufacturers, it is common to use a comparative approach to verify that the crew workload of new aircraft models is acceptable. However, for new aircraft manufacturers, the comparative method is less practical. This study proposes a method based on an independent evaluation strategy, which is being used in the development of the aircraft model, offering a brand-new perspective and approach for the workload validation of new aircraft types.

Yiyuan Zheng, Yuwen Jie, Shan Fu
A Study of Personality Effects on Error Processing Mechanisms Under Time Pressure

In order to investigate the impact of time pressure on error processing, two groups of subjects with distinct personality traits (high neuroticism and low neuroticism) were selected based on the Big Five personality inventory. The behavioral outcomes of these groups under time pressure were compared. By integrating event-related potentials and time-frequency analysis, differences in EEG characteristics were examined to explore how personality influences the error processing mechanism both with and without time pressure. The results indicated that personality did not have a direct effect on error awareness or behavioral correction ability. However, subjects in the high neuroticism group exhibited a greater need for cognitive control and performed better under time pressure, although they were slightly less adept at error detection and showed lower levels of alertness and attention.

Qianxiang Zhou, Jiapeng Shen, Zhongqi Liu

Communication, Information, Misinformation and Online Behavior

Frontmatter
Understanding Users’ Acceptance of Conversational AI for Teaching Deepfake Video Detection: An Extended TAM Approach

The advancement of artificial intelligence algorithms has led to the emergence of deepfakes, which are generated through these algorithms to create realistic images, videos, or audio. Although studies have shown that humans can recognise certain types of deepfakes, such as those with political figures, detection accuracy among general individuals remains only marginally above random guessing. Conversational AI presents a promising approach to interactive learning, which is characterised by systems that simulate human-like dialogue through Natural Language Processing. Chatbots are a prominent application of this technology, which can act as scalable and real-time educational tools for guiding users in specific tasks. However, there is limited research on adopting conversational AI systems specifically designed to educate individuals in deepfake video detection. In this study, a chatbot based on ChatGPT named “Deepfake Fighter” was developed to teach people about deepfake video identification. Subsequently, semi-structured interviews with 30 participants were conducted to evaluate the chatbot’s acceptance factors. The study examined the three key factors of trust, ease of use, and usefulness derived from the extended Technology Acceptance Model. Importantly, specific antecedents were identified for each factor, providing insights into how these elements shaped user acceptance.

Chen Chen, Dion Hoe-Lian Goh
Ephemeral Social Networking: Connecting People from Offline to Online

When attending in-person professional events, we often forget who we met or miss chances to connect with others. For example, we may forget to exchange business cards or add new contacts on LinkedIn. Additionally, we may be unaware of interesting people who were nearby or also attended the event. Our research explores how to capture these missed networking opportunities using opportunistic networking, inspired by the concept of the “Familiar Stranger” by Paulos and Goodman [18]. This refers to people we repeatedly encounter at events, making them feel familiar, yet we never formally meet them. We propose a platform that records social proximity and interactions at events using Bluetooth and event activity data from mobile phones. To support this, we conducted a survey on how people network at events and explored early methods for detecting and recording proximity interactions. This data can be used to recommend potential connections based on attendees’ encounters during the event.

Alvin Chin, Kevin Leicht, Philip Yu, Diego Gomez-Zara
Exploring the Factors Influencing Ambivalent Sexism Among Users of Short-Form Video Apps

As short-form video (SFV) apps like TikTok continue to gain popularity and deliver personalized content to users, they raised concern about the potential for building information cocoon and reinforcing prejudice about certain social group. Our study explores the factors influencing sexism among users of SFV platforms, focusing on how content diversity, selective exposure, and susceptibility to persuasion shape gender attitudes. The research was conducted by a questionnaire survey, based on existing theories of ambivalent sexism and information cocoons, highlighting how homogeneity in content can reinforce sexist attitudes. Our findings suggest that lower diversity in content exposure increases sexism towards both genders, reinforcing existing stereotypes. Additionally, users with lower susceptibility to persuasion and those more inclined to selective exposure are more likely to develop stronger sexist attitudes. In particular, all participants reported perceiving an increase in prejudice towards both genders after engaging with SFV content. These results provide insights into how SFV apps propagate biased content and influence gender attitudes, emphasizing the need for content diversity and media literacy to counteract biases and promote gender equality. This research contributes to the growing body of literature on digital media’s role in shaping social attitudes, offering practical implications for reducing gender prejudice in online spaces.

Rui Hu, Ruifeng Yu
Enhancing User Understanding of Entity Relationships with Knowledge Graphs: A Dataset for Multi-entity Relationship Explanation

This paper presents an initial study on explaining multi-entity relationships using Knowledge Graphs (KGs). We introduce a novel dataset ( https://github.com/aistairc/multi-entity-relationship-explanation ) designed to clarify relationships between multiple entities as an essential task given the rapidly growing number of entities across the web. As web content such as articles and blogs expands, understanding how the entities relate to one another becomes crucial for users to navigate and interpret online information while browsing the web. Often, online information is presented in a non-self-contained manner as it sometimes lacks context and does not present the connections between the entities involved. In this study, we emphasize the importance of improving user comprehension through explicit explanations of multi-entity relationships. To address this, we propose a dataset that lays the foundation for developing a system capable of providing clear and informative relationship explanations among entities as text. Unlike existing studies which typically focus on pairs of entities, our dataset includes over 9,400 entity sets, each containing 3 to 5 entities, along with textual descriptions of their relationships. Additionally, we extract Freebase KGs for each entity set using multi-hop expansion to gather supporting entities crucial for comprehensive explanations. This paper outlines the study’s objectives, the dataset construction process, and the data quality enhancements, and provides a detailed dataset analysis.

Wiradee Imrattanatrai, Makoto P. Kato, Ken Fukuda
Comparative Study of Image Presentation Methods of Websites Based on Eye-Tracking Experiment

The aim of this study is to determine the individual and interactive effects of several elements (background, model, and promotional information) of im-ages in the context of online shopping to determine the best display method for homepage images of different categories of goods (clothing, household appliances, digital products, and food), where different genders are targeted. Participants are required to browse shopping websites and select their desired products, and eye movement data are recorded during this process. Design guidelines are proposed for homepage images of different product categories. The study suggests that images for products targeted at male users should be simple and contain fewer elements, whereas those targeted at female users can be more aesthetically pleasing and informative.

Lin Jiang, Yuan Wang
Evaluating Lexicon-Based Sentiment Analysis Methods for Small Datasets on Low Compute Devices

This paper evaluates the effectiveness and computational efficiency of four lexicon-based sentiment analysis methods: AFINN, VADER, TextBlob, and SentiWordNet, on a resource-limited target device. We assessed each method’s sentiment classification accuracy, precision, recall, F1 score, memory consumption, and processing time on a limited annotated dataset. TextBlob performed well, achieving the highest accuracy and low resource usage among the candidates, whereas SentiWordNet consistently lagged across all metrics. We discuss performance-resource trade-offs for deploying sentiment analysis in low-compute applications where it is essential to efficiently manage local data processing and resources.

Scott Johnson, Farnaz Baksh, Matev Borjan Zorec
Detecting the Undetectable: The Need for a New Paradigm for Academic Writing Evaluation in the AI Era – Addressing Inconsistencies in AI and Plagiarism Detection Tools –

The growing integration of artificial intelligence (AI) in academic writing is reshaping research and composition, offering both opportunities and challenges. While AI tools enhance accessibility, concerns over academic integrity persist as AI-generated texts become increasingly difficult to distinguish from human writing. Institutions have implemented AI detection tools such as Turnitin, Originality.ai, and GPTZero, yet their accuracy remains inconsistent, and humanizing techniques further undermine their reliability. Given these limitations, relying solely on detection-based assessments is not a sustainable approach. Rather than prohibiting AI, academic evaluation could benefit from approaches that emphasize creativity, originality, and responsible AI integration. This study examines AI's role in student writing, critiques detection tools’ limitations, and proposes a process-based assessment framework that integrates AI interaction tracking, reflective analysis, and iterative revisions. By shifting from detection-focused models to AI-conscious evaluation, this research aims to establish a fairer and more effective approach to maintaining academic integrity.

Hat Nim Kim
Gen Z Perceptions on Deepfakes as an Everyday Technology: Opportunities or Constraints

As deepfake technology becomes increasingly accessible for positive and negative uses, its integration into everyday digital interactions raises critical questions about how different generational groups perceive and engage with it. While earlier generations adapted to digital advancements, Generation Z (GenZs), the first generation of digital natives, have grown up immersed in technology, shaping their perspectives and interactions with emerging tools like artificial intelligence and deepfakes in ways fundamentally different from their predecessors. Understanding how deepfakes become a feature of GenZ’s daily informational activities such as information seeking and learning, will be critical. Drawing on interviews with 34 GenZ participants, this study investigated how GenZs navigate the opportunities and constraints of deepfakes from the lens of affordances. In terms of opportunity, the findings indicate that perceived engagement was found to be associated with deepfake, and perceived deception was the main constraint.

Chei Sian Lee, Li En Tan, Dion Hoe-Lian Goh
Impact of Descriptions, Recommendations, and Online Reviews on Fashion E-Commerce Sites on Consumers’ Purchasing Decision Behavior

Online shopping sales have experienced rapid growth in recent years, with the COVID-19 pandemic further accelerating the global adoption of e-commerce (EC). For EC platforms, understanding the factors influencing consumers’ shopping decisions is crucial. Previous studies have explored decision-making behaviors through factors such as online reviews, product recommendations, and promotions. These elements, which are part of the user interface (UI) of websites, significantly influence the user experience (UX) of consumers. However, the impact of UI elements on UX during online purchasing decisions has not been fully explored. Moreover, dark patterns have become significant focus in UI design due to their tendency to drive undesirable consumer decisions. For apparel EC, it is crucial to offer a superior UX during the purchasing process compared to other platforms. This study explores the effect of UI elements with dark patterns on purchase decision-making in apparel EC. While various UI elements can be found on apparel EC sites, this research focuses on three shopping stages: product descriptions, recommendations, and online reviews, analyzing two apparel EC sites. The UX curve method and protocol analysis are used to investigate the relationship between purchasing behavior and the shopping experience. We examine UI elements such as photos, textural descriptions, and product attributes before making a purchase decision based on these elements. We propose a multi-stage UX curve method to analyze how UX evolves over time and affects purchasing decisions. The findings demonstrate the dynamic influence of UI elements with dark patterns on consumer UX, offering valuable insights for improving the design of EC platforms.

Lei Luo, Masato Takanokura
ChAerial: Chair-Type Aerial Display for Enhancing Face-to-Face Communication with Face-Anchored Comic Effects

In modern text-based communication and online meetings, graphical expressions such as emojis and visual effects are widely used as effective means of conveying emotions. In this study, we aim to enhance face-to-face communication by developing a novel display system that projects aerial images in mid-air around a person’s face. Previous research has explored communication-support interfaces using displays on tabletops or partitions; however, these methods suffer from optical challenges such as projector light entering directly into the eyes and reduced transparency of acrylic panels. In addition, head-mounted systems like HMDs not only obscure facial expressions but also pose difficulties for prolonged use due to their weight. Therefore, we have developed a chair-based aerial display that employs an optical system mounted on the chair’s backrest to project speech bubbles and manpu as if emanating from the user’s face, thereby enabling natural face-to-face communication without the need for heavy wearable equipment. To verify the effectiveness of our system, we implemented applications based on two use casesenhancing emotional expression in face-to-face communication and supporting communication for individuals with hearing impairmentsand conducted a preliminary qualitative evaluation through feedback collection. The results suggest that the system has the potential to amplify users’ emotions, facilitate conversation, and naturally direct their gaze toward the face.

Toshiki Mori, Yoshiki Furuya, Hyuckjin Choi, Yugo Nakamura, Ari Hautasaari, Shogo Fukushima
Examining the Role of Customer Reviews in Online Purchasing Using Eye Tracking

With the growing usage of the internet, there has been a steady increase in the number of people buying online products ranging from appliances to everyday-use consumer goods. In an online shopping setup, it is difficult for customers to experience the product firsthand in the virtual environment. Many ecommerce platforms provide (a) detailed pictures, 3D models, animations, and videos on their website for customers to explore the product features, and (b) customer reviews and ratings to learn about product performance in different conditions and usage periods from previous customers’ experiences. These help the customers make purchase-related decisions. A large percentage of online customers utilize online customer reviews in their decision-making. However, there is limited knowledge about how customers read and use the information in customer reviews for purchase decisions. This research aims to fill this gap in the literature by studying users’ attention toward customer reviews during online product purchase tasks using eye tracking and their consideration of reviews in the decision-making process. A user study was conducted with 30 participants, who were asked to buy a specified consumer product by searching a popular e-commerce website. A post-task study followed the survey. Participants used the search functions on the websites to locate the selected products. This included browsing product categories, using keyword searches, and evaluating different listings. Once they found a product that matched the description, they had to add it to the cart. Throughout the task, participants followed their usual online shopping behavior, which included comparing products, reading descriptions, and potentially viewing customer reviews. The participants’ gaze movements were continuously recorded using an eye tracker, capturing every action they took on the screen, including product browsing, clicking on details, and reading information. To analyze the level of utilization of reviews by participants, detailed data was collected on the time participants spent on each stage of finding the target product. This included searching for the product, viewing product details, comparing different brand options, and adding the chosen product to the cart. The time explicitly spent reading customer reviews was analyzed to understand participants’ engagement with the reviews and their potential influence on their decision-making. The post-task survey included questions regarding demographic information (such as age, gender, education, etc.) and participants’ spending activities, preferences, and shopping habits. Additionally, participants were asked questions related to their experience during the experiment, such as difficulties faced and decision-making processes.

Madhumathi Ponnusamy, Jeevithashree Divya Venkatesh, Souvik Das, Gaurav Nanda
Bangla Fake News Detection with Source-Specific Probabilities for Improved Accuracy

New research shows that the rise of fake news has translated into an estimated $39 billion annual loss in the global stock market. Fake news has become a pressing issue, particularly in low-resource languages like Bangla. This study introduces a dataset and methodologies for detecting fake news using machine learning techniques. We utilized both machine learning and deep learning methods, achieving notable improvements in accuracy by incorporating domain-specific fake news probabilities. Without these probabilities, the models achieved 70–80% accuracy, while incorporating this feature boosted performance significantly. These findings highlight the effectiveness of neural networks and statistical models in combating misinformation and disinformation.

Sakib Mohammed Sobaha, Nazmus Sakib Sami, Sadia Sharmin
Systematic Classification of Studies Investigating Social Media Conversations About Long COVID Using a Novel Zero-Shot Transformer Framework

Long COVID continues to challenge public health by affecting a considerable number of individuals who have recovered from acute SARS-CoV-2 infection yet endure prolonged and often debilitating symptoms. Social media has emerged as a vital resource for those seeking real-time information, peer support, and validating their health concerns related to Long COVID. This paper examines recent works focusing on mining, analyzing, and interpreting user-generated content on social media platforms to capture the broader discourse on persistent post-COVID conditions. A novel transformer-based zero-shot learning approach serves as the foundation for classifying research papers in this area into four primary categories: Clinical or Symptom Characterization, Advanced NLP or Computational Methods, Policy, Advocacy or Public Health Communication, and Online Communities and Social Support. This methodology achieved an average confidence of 0.7788, with the minimum and maximum confidence being 0.1566 and 0.9928, respectively. This model showcases the ability of advanced language models to categorize research papers without any training data or predefined classification labels, thus enabling a more rapid and scalable assessment of existing literature. This paper also highlights the multifaceted nature of Long COVID research by demonstrating how advanced computational techniques applied to social media conversations can reveal deeper insights into the experiences, symptoms, and narratives of individuals affected by Long COVID.

Nirmalya Thakur, Niven Francis Da Guia Fernandes, Madje Tobi Marc’Avent Tchona
Textual Analysis of Cities’ Images in Publicity: A Computational Study of Comments on YouTube

In the digital age, social media platforms have become the primary avenue for cities to promote themselves, attracting tourists and stimulating economic growth. Cities including Shanghai and New York are typical examples of this trend, leveraging social media to showcase their unique offerings. Nevertheless, comprehension of how these promotional videos resonate with potential visitors and what images they project to the audience remains a critical challenge. This study attempts to address this issue by utilising an approach that combines natural language processing (NLP) and corpus analysis techniques. We analysed the comments associated with promotional videos of both Shanghai and New York, focusing on their linguistic features and cultural nuances. Through a comparative analysis of comments data, this research seeks to identify similarities and differences in the cities’ images shaped by these promotional efforts across distinctive cultural contexts (the eastern Shanghai and the western New York). Our results have shown that both Shanghai and New York are perceived as desirable global cities of tourism and urban life. However, Shanghai's image is more strongly tied to its national identity and role within Asia, while New York emphasises its global influence and themes of opportunity and dreams. Accordingly, this research has employed computational methods to investigate textual representations within comments, offering insights into the complex interplay between social media, city branding and public perception. Furthermore, this proposed poster will contribute to future explorations in social computation and social media research by providing an analytical framework through this case study.

Yuan Zhang, Xi Chen, Yifeng Fan
User Interaction, Knowledge Exchange, and Knowledge Innovation: An Empirical Examination of User Knowledge Collaboration Based on Huawei’s Open Innovation Community

[Purpose] This study explores user knowledge collaboration in open innovation communities, focusing on how interaction characteristics influence knowledge innovation through knowledge exchange. Based on social capital and knowledge creation theories, we analyze user interactions that facilitate knowledge flow and innovation. [Methods] Using Python-based natural language processing, we process 170,000 user interactions from the Huawei Pollen Club. A knowledge synergy model is built, transforming interaction features and knowledge exchange behaviors into measurable indicators. Stepwise regression and mediation analysis are applied. [Results] (1) Network centrality, structural holes, and relationship redundancy impact knowledge exchange and innovation. (2) Relationship redundancy follows an inverted U-shape, where moderate redundancy enhances knowledge creation, but excessive redundancy hinders it. [Innovations] (1) A user knowledge collaboration model is proposed. (2) NLP techniques enable large-scale knowledge extraction and provide strategic insights for managing innovation communities.

Yueyan Zhao, Feng Yang, Yuexin Cheng, YunYue Ren
Advances in Deepfake Detection: A Focus on Emerging Datasets and Deep Learning Techniques

The proliferation of deepfake technology has transformed artificial intelligence into a double-edged sword, facilitating innovation while posing significant ethical and security risks. This study reviews the state-of-the-art in deepfake detection, emphasizing the role of deep learning techniques and the utility of emerging datasets. Key datasets such as CIFAKE, COCOFake, and DFFD are analyzed for their contributions to robust model development across diverse manipulation types. Advances in detection algorithms, including capsule networks, attention mechanisms, and binary neural networks, are evaluated for their accuracy, computational efficiency, and generalization across datasets. Furthermore, the review highlights challenges in cross-dataset generalization, real-time detection, and model explainability, proposing future directions for research. This work aims to guide the development of next-generation deepfake detection methods, addressing the dynamic landscape of generative AI.

Haidong Zheng, Rut Patel

Designing and Understanding Learning and Teaching Experiences

Frontmatter
A Study of Electronic Circuit Design in Tiny Interactive Prototypes in an Undergraduate Course

Creating an electronic circuit inside a tiny prototype can be challenging for design students. Studies on design students’ circuit assembly tasks on tiny prototypes are scarce. This study investigated how students assemble circuits in tiny prototyping and which factors influence these activities. We analyzed the students’ prototypes and discussions using the grounded theory framework. We observed that tiny prototype circuits might be characterized by volume, composition, structure, dynamics, and isolating properties. Planning and the varied availability of conductive and circuit support materials might facilitate experimentation during circuit design. Our results shed light on the need for additional study and the development of boards and electronic componentry for educational practitioners.

Andrea Alessandrini
The Interaction of Audience Response Module and Drill Module: Combining Different E-Learning Functions for Better Learning

This study analyzes how multiple functions work together in the e-learning system. Big data analysis from external systems has been studied for a long, and the basic structure for using external artificial intelligence (AI) services for e-learning modules has been developed. Therefore, we can integrate the functionality of the e-learning system with external systems, such as social media, resulting in improved functionality. To use AI services successfully, we should determine what combination of data is most effective for learning, not just the size of the data. By studying the interactions of multiple functions within the e-learning system, we investigate the effective combinations of data. Further, we analyze five functions of the e-learning system, including drill, audience response systems, recommendation engine, comment, and note-taking. 295 students were analyzed using two-way analysis of variance (ANOVA). Some interactions were statistically significant in this analysis, indicating that AI can be used as a tool for building effective e-learning functions.

Toshikazu Iitaka
The Privacy-Efficiency Tradeoff: How AI Learning Analytics Influence Performance and Reuse Intention

Online learning platforms have become popular tools for delivering education, offering convenient and affordable learning opportunities to people worldwide. However, the integration of camera monitoring and AI learning analytics raises critical questions about their pedagogical effectiveness and user acceptance. This study investigates the efficacy of two distinct AI learning analytics in camera-monitored virtual classrooms: (1) Dashboard that analysis real-time biometric data (facial expressions, body movements, etc.) into performance reports, and (2) Learning moment, with which AI captures screenshots when detecting high student engagement (via the same biometric indicators) and shows to students. Through a 2 × 2 × 2 factorial experiment (N = 268), participants completed sequential 5-min burn knowledge courses with quizzes, receiving assigned feedback after the initial session. Results demonstrated that both tools enhanced learning performances, yet reduced platform reuse intent. Critically, concurrent use of both tools diminished dashboard’s efficacy, which may be due to students’ aversion to AI surveillance and privacy violations. While class size showed no significant moderating effect on learning performance, one-to-many classes mitigated the negative impact of analytics on reuse intent, suggesting peer presence mitigates surveillance anxiety. These findings reveal a tension in AI-augmented education: real-time analytics improve pedagogical outcomes but trigger privacy-efficacy trade-off. Our research provides critical design insights for camera-monitored learning platforms.

Mandie Liu, Yi Zeng, Xuyu Fan
Learning Through Play: A Study on the Integration of the Yuexi Nianli Cultural Elements into the Design of Educational Products for Enhancing Contextual Learning and Cultural Identity

The purpose of this study is to design a set of children’s educational products integrating the Yuexi Nianli Cultural by combining collaborative learning technology and embodied cognition theory, and to explore its role in promoting children’s contextual learning and cultural identity. By analyzing the core elements of Yuexi Nianli Cultural, the cultural symbols and behavioral patterns that are in line with children’s cognitive development are extracted, and the key design elements such as entertainment, knowledge, interactivity, and safety are determined. The study adopted a group experiment method and recruited 32 children to participate in the experiment to explore the effects of collaborative learning and embodied cognition on children’s cognitive development and cultural understanding. The results show that product design based on collaborative learning and embodied cognition can effectively improve children’s cooperation ability, knowledge transfer ability, creativity, and cultural identity, and provide a design paradigm and practice path for the integration of traditional culture inheritance and children’s education.

Jinrong Liu, Li Ou-yang, Ying Guo
Effects of a Gamification-Based Activation System and Its Adaptation for Teaching Assistants

This study investigated the long-term effects of a gamification-based system designed to enhance the learning support activities of teaching assistants (TAs). The system incorporates game elements such as game balance, feedback, quests, unlocks, and ranking to encourage active TA engagement. A comparative analysis of university programming courses-one employing the system and another without it-revealed that although some TAs initially found the game elements engaging, their interest declined over time, likely due to the novelty effect. However, the system’s student information and support-target recommendation functions effectively facilitated proactive TA involvement. Notably, the study suggested that a proactive approach by TAs to students early in the course could promote questions from students later in the course.

Kanato Murobayashi, Takahiro Yoshino, Hironori Egi
Toward the Development of a Practical E-learning System to Promote Knowledge Retention

E-learning has been widely adopted due to its numerous advantages: (1) enabling self-directed learning without time and location constraints, (2) supporting personalized learning with ease, (3) reducing educational costs, and (4) facilitating adaptive learning through data analysis. However, existing e-learning systems are often criticized for insufficiently addressing knowledge retention. This study focuses on this limitation and aims to develop a system that enhances knowledge retention. To achieve this, we analyzed widely available e-learning systems and trialed services and content from commercial providers deemed beneficial. The proposed system incorporates mechanisms to reinforce knowledge retention, such as optimizing quiz timing based on Ebbinghaus’s forgetting curve. Additionally, the system was implemented among students in the authors’ academic department, and its learning effectiveness was evaluated through empirical analysis. The findings provide clear insights for developing more effective e-learning systems.

Tetsuya Nakatoh
Triple Threat Design Principles: Learning Content, Promoting Identity Development, and Improving Executive Functioning Through Digital Learning Environments

The prevalence of online learning has grown exponentially in the past five years. While it has increased access for some, these spaces are generally not designed to be inclusive of neurodivergent learners. Additionally, while there is ample research on the importance of identity for learning and the impact of digital spaces on identity development, more work is needed to determine how to intentionally design digital learning environments (DLEs) that simultaneously support positive identity development and learning. ADHD is one of several conditions that is associated with executive dysfunction, which disrupts the process of identity development. Therefore, to effectively design identity-promoting learning environments, we must integrate tools that support executive functioning. This paper proposes a set of research-based design principles for a digital learning environment that aims to support learners with executive dysfunction through the iterative process of identity development. Such a learning environment would result in better learning outcomes for neurodivergent students. By taking a principled approach, we are better able to test design conjectures and systematically improve our designs of inclusive and effective learning environments.

Mirlanda Elizabeth Prudent
Towards a Sustainable Future: Sustainability in Higher Education Institutions

Higher education (HE) institutions have undergone many waves of change in recent years to address the challenges posed by technological advancements and social issues such as the Covid-19 Pandemic. This has enabled higher education institutions to transition to hybrid learning approaches, paving the way for transnational delivery options around the world. Meanwhile, as the post-pandemic lifestyle returns to normal, teaching in tertiary education sectors has shifted from hybrid to face-to-face model. However, the higher education sector appears to be facing challenges from the transition from online to face-to-face delivery, as learning management systems are now found to offer boundaries in terms of student engagement and learning outcomes. Many programmes designed for face-to-face learning were to be modified in a matter of days to meet the needs of the online space. This urges the question, “Is our higher education system sustainable enough to deal with the issues of rapidly changing times?”. To address this issue, the current study employs a bibliometric approach using Scopus databases to refine articles on the sustainability of the higher education. In doing so, the findings of this study are visualised with VOSviewer, which showcases the six key clusters that are related to a) sustainability ans sustainable development goals within higher education is linked to digitalisation and skills development; b) Enhancing sustainability in higher education through blockchain technology, blended education; c) sustainability in higher education is linked to teaching through technology; d) sustainability in higher education is linked to digital literacy and educational technologies; e) sustainability in higher education is linked to educational innovation and industry 4.0; f) sustainability in higher education is linked to innovation and enterpreniual universities. Lastly, it is suggested that incorporation of cutting-edge technology, transformation to entrepreneurial universities, trans-disciplinary education, university-industry and university-society collaboration contribute to the foundation for sustainable higher education institutions.

Isra Sarfraz, Abhishek Sharma, Muzammil Hussain
Cross-Media Narrative Interaction Design for International Education with Generative AI: A Case Study of Dream of the Red Chamber

This study explores how generative AI enhances the international communication and educational value of classical literature through cross-media narrative design, using Dream of the Red Chamber as a case study. It proposes an interactive storytelling system integrating text, visuals, and user interaction. The design employs large language models and diffusion models to create a seamless fusion of text, imagery, and interactivity. Key components include an AI-driven dialogue system, dynamic scene visualization, and an interactive operation zone. Users engage with characters via natural language input, while Stable Diffusion generates scene imagery, rendered in Unity 3D for an immersive environment rooted in traditional aesthetics. Designed primarily in English, the platform enhances cross-cultural communication by lowering cognitive barriers to classical literature. This study highlights AIGC-driven cross-media narratives as a pathway for global appreciation of cultural heritage through digital innovation.

Zhengming Si, Qinru Bao
Leveraging Large Language Models for Robot-Assisted Learning of Morphological Structures in Preschool Children with Language Vulnerabilities

Preschool children with language vulnerabilities—such as developmental language disorders or immigration related language challenges—often require support to strengthen their expressive language skills. Based on the principle of implicit learning, speech-language therapists (SLTs) typically embed target morphological structures (e.g., third person -s) into everyday interactions or game-based learning activities. Educators are recommended by SLTs to do the same. This approach demands precise linguistic knowledge and real-time production of various morphological forms (e.g., “Daddy wears these when he drives to work”). The task becomes even more demanding when educators or parent also must keep children engaged and manage turn-taking in a game-based activity. In the TalBot project our multiprofessional team have developed an application in which the Furhat conversational robot plays the word retrieval game “Alias” with children to improve language skills. Our application currently employs a large language model (LLM) to manage gameplay, dialogue, affective responses, and turn-taking. Our next step is to further leverage the capacity of LLMs so the robot can generate and deliver specific morphological targets during the game. We hypothesize that a robot could outperform humans at this task. Novel aspects of this approach are that the robot could ultimately serve as a model and tutor for both children and professionals and that using LLM capabilities in this context would support basic communication needs for children with language vulnerabilities. Our long-term goal is to create a robust LLM-based Robot-Assisted Language Learning intervention capable of teaching a variety of morphological structures across different languages.

Stina Sundstedt, Mattias Wingren, Susanne Hägglund, Daniel Ventus
Development of ClasScoop: A Tool for Real-Time Observation of Learner Performance in Online Skill-Based Education

In online skill-based education, instructors can readily monitor individual student progress, comprehensively assessing the progress of multiple students simultaneously poses a challenge. To address this issue, we developed ClasScoop, a lightweight, browser-based web application.ClasScoop features a navigation event for each class, with students connecting via unique URLs generated by the system. Upon activation, student PCs periodically transmit their operation screens and associated conditions to the server. The server stores this data, aggregating information from all clients with minimal latency. On the client-side, users can filter and display stored data based on specified criteria. The application also allows instructors to select specific students and review their operation flow. The interval for capturing student operation screens and the display conditions can be customized. Granular control over display conditions is provided, enabling options such as faculty-only viewing, student-to-student viewing, and authorized student browsing. This design aims to replicate the experience of instructors observing multiple students simultaneously from the back of a physical classroom. Post-development experiments, including interview surveys, were conducted to evaluate the software's effectiveness. Results confirmed that ClasScoop facilitated both casual observation of student activities and instructor-directed monitoring. Feedback from these experiments led to further software refinements.

Takuya Suzuki
Scenario-Based Learning as a Pedagogical Strategy: Fostering TPACK and Design Competences in Pre-service Teachers

In today’s rapidly evolving educational landscape, pre-service teachers (PSTs) must develop technological and pedagogical competencies to integrate digital tools into their teaching effectively. However, existing teacher education programs often lack sufficient support for fostering these skills. This study introduces the Designing for Scenario-Based Learning (DSBL) model. This instructional approach integrates Technological Pedagogical Content Knowledge (TPACK) with scenario-based learning and design thinking to enhance PSTs’ technological integration and instructional design competencies. We examined the effectiveness of DSBL through a quantitative analysis of scores from a pre- and post-course TPACK questionnaire from 30 participants. The initial results show a substantive increase (d = 1.92) in participants’ self-evaluation of their TPACK and design competencies.

Moshi Wang, Peter Reimann
Are You With Us? A Real-Time Engagement Analytics with Machine Learning in Online Learning Environments

Online learning environments cannot often evaluate students’ attentiveness and engagement in minute detail. Our project addresses this gap by developing a real-time predictive engine to better understand students’ learning states, enabling more active learning and adaptive teaching in online settings. This engine leverages a combination of digital tools to predict cognitive load and engagement in real-time.In this work, we outline the procedures and some preliminary findings from developing and testing this predictive model using data from wearables and webcams. These devices captured physiological signals, facial expressions, eye gazes, and head poses to measure engagement levels in content-heavy classrooms. To build the model, we manually selected and extracted features from a pre-existing database containing sensory data. The model categorized students into two engagement levels: low and high.Following model development, we conducted an experimental study involving two 30-minute machine learning (ML) lectures with 12 college students. Alongside physiological data, self-reported survey data on engagement were collected as the ground truth. The engagement levels predicted by the engine were found to be highly associated with their self-reports on engagement (approximately 50% of variance shared). Results demonstrated that electrodermal activity (EDA), recorded via E4 wearable devices, was higher during challenging lecture segments than in easier ones. Additionally, variability in EDA levels during lectures correlated with self-reported stress, suggesting a potential link between stress and engagement.Through this work, we present insights into developing multimodal learning analytics based on real-time psychophysiological signals and learning behavioral data. These insights highlight the potential for integrating wearable and webcam data to enhance online learning.

Linghan Zhang, Jung Yeon Park, Nirup Menon, Nupoor Ranade, Bo Yu, Sheng Tan
Multimodal Situational Awareness: Neuro-Symbolic AI for Real-Time HCI in the Classroom

This research addresses the problem of an enabling AI systems to become capable of dynamically adapting to the complex, multimodal nature of classroom interactions. We introduce a novel neuro-symbolic AI approach designed to achieve real-time, multi-party situational awareness in educational environments. By integrating verbal, gestural, and physical cues from multiple students, our system constructs a coherent understanding of collaborative learning processes. This involves not only tracking verbal dialogue but also interpreting nonverbal behavior and contextual changes to analyze interaction dynamics. The proposed framework advances the development of AI systems that can effectively support and enhance classroom engagement and learning outcomes.

Yifan Zhu, Kenneth Lai, Ibrahim Khebour, Marc Verhagen, Nikhil Krishnaswamy, James Pustejovsky
Backmatter
Metadata
Title
HCI International 2025 Posters
Editors
Constantine Stephanidis
Margherita Antona
Stavroula Ntoa
Gavriel Salvendy
Copyright Year
2025
Electronic ISBN
978-3-031-94153-5
Print ISBN
978-3-031-94152-8
DOI
https://doi.org/10.1007/978-3-031-94153-5