Technologies and Applications of Artificial Intelligence

Game streaming has become global entertainment, offering viewers a combination of gameplay observation and parasocial interaction that transcends borders. This paper explores the multifaceted aspects of engagement in game streaming across language barriers as part of a bigger project that explores the utility of automatic speech translation for game streams. A comprehensive framework that categorizes streaming elements into game and non-game content examines how these elements enable viewer engagement. We delve into the social dimensions of streaming, where both linguistic and non-linguistic interactions build parasocial relationships. Our study utilizes an experimental approach to isolate the impact of linguistic and non-linguistic elements in streams. We conducted an experiment with 1,236 participants who were exposed to game streams with modified auditory and textual components to isolate the effects of non-linguistic cues on engagement. Our findings suggested that non-linguistic elements did not significantly influence viewer engagement. This paper contributes to offering a vision for understanding digital spectatorship and interaction in game streaming. Future insights are presented that could help global engagement in streaming.

In human communication, how we say things is just as important as what we say. This “how” is called paralanguage, e.g., the pitch of our voice, how we stress words, how fast we speak, and the rhythm of our speech. These elements help us understand what someone means, even if they say only one-word like “ha” In our study, we focus on how people use and understand the word “ha” in different situations as an extreme challenge for paralanguage recognition. We applied a game to collect data on how people say “ha” in eight different contexts. When people could see and hear others saying “ha,” they guessed the correct context about 63% of the time. We then used a machine learning model to analyze the acoustic features of these “ha” utterances and estimate the intended context. The model was able to estimate the correct context with an average F1-Score of about 0.58. This showed that even a machine learning model could understand some of the meaning behind how we say “ha” at almost the same level as humans watching the speaker’s faces and movements. We found some interesting differences by comparing how well humans and a machine learning model could estimate the context. The findings have helped us understand more about how we communicate with just a single word.

This paper proposes a system that supports watching Mahjong games for novices. Two factors make watching Mahjong games difficult for novices: it is difficult to know what combination of tiles the players are aiming for, and it is difficult to instantly grasp what tiles each player has and what tiles they discard. The system predicts and visualizes the combination of tiles at the goal that each player is aiming for and also visualizes the status of each player’s holding and discarded tiles side by side in the horizontal line. In the evaluation experiment, participants watched Mahjong games through the proposed system. Utterances during the experiment were analyzed based on Think-aloud protocols. For comparison, we prepared a comparison system that reproduced a typical Mahjong game, had the participants watch the game similarly, and analyzed the content of their speech. Participants watched two games, and the proposed system increased utterances indicating an understanding of the game’s situation in the second time more than that in the first time. This indicates that the proposed system can help the spectator understand the game’s contents and watch the Mahjong game.

This study aims to determine the influence of comments posted online about news on viewers. Opinions posted by news readers via the web often influence the formation of other readers’ views. In this paper, we investigate the impressions of news articles under three conditions: (a) presenting a news article only, (b) presenting a news article and comments posted in the comment section of the article by readers, and (c) presenting a news article, comments posted by readers, and the readers’ previous comments for other news articles. The result revealed that (1) readers’ comments do not affect the impression of the original news article but are more influenced by the perspective of whether the respondent is familiar with the news article, (2) The impression of the comments that viewers agreed on tend to emphasize sincerity, responsibility, and credibility, regardless of the news article’s genre, and (3) presenting readers’ past comments may influence viewers’ evaluation of the information and lead them to form different opinions.

This paper proposes a Quantum Computational Intelligence (QCI) robot with a Generative AI (GAI) knowledge graph (KG) for Taiwanese and Japanese co-learning model applications. During the 2024 IEEE CIS Summer School on QCI at Tokyo Metropolitan University (TMU) in Japan, we organized lectures and a hands-on workshop on QCI for young students to learn and experience QCI using the QCI&AI-FML learning tool and robot. Learners first observe, study, research, utilize, understand, and explain what they have learned in the heart-sutra-based human and machine co-learning model. We transcribed the collected multimodal data using the OpenAI Whisper models into English or Japanese texts. The GAI Knowledge Graph (GAIKG) agent generates the knowledge graph with concepts, relations, and communities for the transcribed texts by providing prompts with Large Language Models (LLMs), such as Trustworthy AI Dialogue Engine (TAIDE). The Sentence BERT (SBERT) similarity agent computes the similarity between the collected texts and the golden standard provided by the English/Japanese domain experts. Next, CI domain experts constructed the CI model based on the generated data from these two agents to infer the performance of the generated knowledge graph and deployed the model to the QCI robot. Finally, the QCI robot evaluates how well the generative knowledge graph leverages human and machine learning according to the constructed CI model. In addition, the QCI robot evaluates the performance of the generative knowledge graph in co-learning English/Japanese based on the constructed CI model and quantum fuzzy inference engine. In the future, we will extend the QCI robot to more countries for young students to co-learn CI, QCI, and Taiwanese/Japanese languages with smart machines.

We propose optimization methods to accept immediate tasks in automated warehouses. While Multi-Agent Pickup and Delivery (MAPD) problems for the automated warehouses suppose that all delivery tasks are given at planning phases, there are several situations where the system needs to accept immediate and emergent tasks in real-time in practical applications. We conducted four methods, Optimal Timing Search, Optimal Vertical Location Search, Optimal Horizontal Location Search, and Variance Minimization to assign an immediate task into already-planned schedules of MAPD. We demonstrate that the proposed methods can mitigate costs increases associated with accepting immediate tasks on the system.

This research describes improvements to an agent called the Intelligent Dialogue Agent (IDA) which aims to be a friendly intermediary between older adults and a preventive care system through performing natural dialogic interactions. The preventive care system consists of a fall prevention component, a cognitive training component, and the IDA, which promotes these ends by creating an environment in which those users (older adults) can participate in their own preventive care while having fun.

To achieve this, the IDA uses reinforcement learning methods to learn the appropriate policy and provide speech topics depending on the preferences of the specific user, thereby motivating him or her to use the preventive care system on a daily basis. We extend the state-action definition of the IDA by introducing the results of sentiment analysis to improve its learning performance and make the IDA possible to always output fresh topics based on distributed text representation.

Finally, we conduct several experiments with older adults to evaluate whether the IDA successfully captures their specific preferences, and to assess basic characteristics of the updated IDA incorporating sentiment analysis and distributed text representation.

This paper proposes ComicFaves, the system enabling users to organize their favorite comics in bookshelves format. Organizing comics in the form of bookshelves, which is different from the favorite list in conventional comic applications, enables us to understand users’ preferences visually and intuitively. In the user test, multiple users arranged bookshelves based on some themes. We conducted a basic analysis of the characteristics and trends of the bookshelves. The result suggested that bookshelves for affective keywords more strongly reflect users’ individuality and affection than bookshelves for genre information. Moreover, it was suggested that the arrangement of comic titles on bookshelves helped us to understand differences in users’ preferences and sensibilities. It is expected that the features obtained from bookshelves could be applied to develop a novel feature space for comics reflecting readers’ affection.

Procedural content generation has been widely deployed to automatically generate digital content with limited or indirect user input. This paper shows how ant colony optimization algorithms, a multi-agent system usually applied to shortest-path optimization problems, can be adapted to generate natural-looking villages in the video game Minecraft. To achieve this, houses are stochastically placed in a specified region of a Minecraft world, favoring flat and central areas. Next, villagers inhabiting these houses are represented by multiple ant agents each and simulate life in the village by wandering between houses. The shorter and flatter a found path between two houses is, the more likely the path will be followed by future ant agents. After some iterations, a natural path network connects all houses. This process of placing houses and connecting them with paths can be repeated, allowing the village to naturally grow. Survey results show that the generated villages are perceived as more natural than the default ones, both regarding house placements and path trajectories.

This paper proposes a method for constructing personal value-based user models from review texts using LLM (Large Language Models). The RMrate (Rating Matching Rate) has been proposed as a metric to quantitatively assess the intensity of user preferences towards item attributes when selecting items, and has been applied to personal value-based models. RMrate is defined as the proportion of cases where the evaluation polarity of an item’s attributes matches the polarity of the overall evaluation: attributes of high RMrate are considered to have a strong influence on user’s decision making. While its effectiveness in information recommendation has been demonstrated, existing methods required explicit attribute evaluations. To address this issue, the proposed method calculates RMrate by applying LLM to extract the evaluation polarity of item’s attributes mentioned in reviews through prompting. Additionally, based on the assumption that LLM can determine whether attributes are mentioned in review texts before judging their evaluation polarity, this paper also proposes to extend RMrate by adding a term representing the frequency of attribute mentions. In this paper, experiments using movies as target items are conducted to evaluate the accuracy of polarity extraction and its effectiveness for recommendation. The effectiveness of the proposed extended RMrate is also shown with experiments.

This paper proposes a learning support system that can analyse the user’s attention in real-time and provide review suggestions during video-based learning. By utilizing the front camera of the personal computer to detect users’ pupils, this study achieves a real-time analysis of users’ attention and is able to match their attentional state with the timeline of the learning video. With the visualization of attentional state along with the timeline of the learning video, users can readily grasp the parts of learning video they should review to achieve an improved review outcome. We evaluated the review effectiveness of the proposed system by measuring the percentage of correct answers of 20 video learners’ answers to the test questions. As the result, the percentage of correct answers of the inattentive parts of the experimental group was 93.7%, while the control group was 67%. It was observed that through the proposed system, the participants experienced an increase in the percentage of correct answers of their answer in 26.7%. It suggests that the proposed system has the ability to support users’ video-based learning by improving the efficiency of reviewing.

In recent years, the development of large-scale language models (LLM) has dramatically improved text generation performance. However, general-purpose LLMs have the problem that they do not always perform optimally in tasks in specific fields. The approach of fine-tuning with data from specific fields is commonly used to address this problem, but collecting training data from limited fields is difficult. In particular, data related to human values have problems such as difficulty in annotation, insufficient amount of data, and large variability. In addition, the training and inference of LLMs is expensive. This study focused on the task of human values estimation and tested the effectiveness of an approach that uses an LLM to generate pseudo-data and trains a small-scale model on that data. In the experiment, we augmented a 3,870-items human values dataset with 10 categories to four times its original size with the generated pseudo-data. The training dataset with pseudo-data increased human values estimation accuracy by 17% than the original dataset without pseudo-data. The fine-tuned small-scale model with the accuracy of 57% also outperformed the LLM with the accuracy of 27% in human values estimation. This result indicates that pseudo-data generation using an LLM is effective in the human values estimation task.

This research assumes that the CFA patterns currently used are not necessarily optimal for deep learning instead of the traditional methods used in the demosaicing process in camera image processing, and attempts to find more suitable patterns. For deep learning, we used SwinIR, a super-resolution framework that is one of the most accurate in image processing, to search for the best CFA patterns for demosaicing. This revealed that the conventional approach of increasing the green component (G) was not optimal for demosaicing using SwinIR, and that patterns containing a fourth W component performed better than conventional Bayer patterns on the cPSNR and SSIM metrics. It was also found that W does not necessarily have to be luminance, but can be the average of the RGB values with comparable results.

This paper proposes a new evaluation index for evaluating human fallibility in shogi. By analyzing the differences between the decision-making processes of AI and humans, we found that while humans rely on intuition and decisions made within a limited timeframe, AI often makes different decisions because it carries out exhaustive searches. In this study, we developed a new evaluation index that extracts features prone to human error using a policy network trained based on professional game records. This evaluation model uses logistic regression to predict the probability of making a mistake in the endgame. We have shown that the proposed indicator is effective through testing with test data. In the future, we plan to construct similar evaluation indicators for the opening and middle game phases and verify their effectiveness.

N-tuple networks are simple and efficient methods for generating evaluation functions and have been used with an increasing number of N-tuples to develop state-of-the-art 2048 computer players. Although performance is expected to improve as the number of N-tuples increases, a systematic method is needed to find a good combination of many N-tuples. In this study, we propose the third systematic method that can be applied to any number of N-tuples and take into account the interdependency between them. Our approach has two main steps: first, we formulate the problem with a function from a combination of N-tuples to an evaluation value; second, we use neural networks to approximate this function. We compare the combinations selected by existing methods and proposed methods as well as those designed manually. Although the combinations selected by the proposed method did not outperform those from our previous work, they achieved better results than the manually designed combinations.

A shift schedule is absolutely necessary to manage the work of each employee. However, manually constructing the schedule taken account of employee preference can impose a large burden on a constructor. In private tutoring school, a shift schedule should make arrangement of the requests of students in addition to teachers’ preference, being necessary to construct simultaneously a shift schedule for each teacher and a timetable for each student. In this paper, we have proposed a two-phase optimization method to find such schedules: shift scheduling using memetic algorithms and timetabling using simulated annealing. We have demonstrated that the proposed method, in particular memetic algorithms in the first phase, can be more effective for constructing a shift schedule and timetable that are taken account of both requests of teachers and students than the previous method using simple genetic algorithms and simulated annealing. In addition, we have developed an interface that allows users to interactively check and make minor modifications to the automatically created shift schedule. This user interface can be useful for the user to instantly check where constraints have been violated and assist in making corrections by the user.

The threat of COVID-19, though significantly reduced, persists. Additionally, the ongoing uncertainty regarding the nature of potential future viral outbreaks underscores the increased importance of preventive measures. High-density and high-risk facilities, such as hospitals and apartments buildings, face significant threats. Traditional virus simulations in the past required extensive time and specialized software. However, in this study, we conducted simulations using a diverse set of heuristic algorithms based on different concepts. The experimental results indicate that heuristic algorithms not only significantly reduce overall simulation runtime—with the Crow Search Algorithm (CSA) achieving the shortest runtime in this experiment at 39.3 s—but also show a substantial deviation from traditional simulations like Computational Fluid Dynamics (CFD), which often took hours. This research not only expands the application of algorithms but also provides essential insights for management personnel in preparing for virus transmission prevention.

In this work, we present a new method that incorporates physical information to enhance prediction accuracy without requiring the solution of complex partial differential equations. To validate our approach, we generated a dataset through simulations in SolidWorks and compared our predictions with the simulated values. Our experiments demonstrate that the proposed method outperforms two different neural network-based baseline models. This framework enables faster and more accurate predictions, making it an essential tool for applications where both speed and precision are critical. Furthermore, our approach simplifies the modeling process by removing the reliance on complex numerical computations, providing a more efficient and accessible solution for real-world applications.

Social media platform frequently experiences incidents where accounts are compromised. These incidents involve attempts to misappropriate personal information, deceive others into trusting fraudulent URLs, or manipulate public sentiment by exploiting stolen accounts. Little research has been done on compromised account detection. Some approaches adopt an incremental approach at message level by comparing incoming messages against established normal behavior model for detection. The other approach utilizes supervised algorithms at the account level by binary classification into normal and compromised accounts.

This paper aims to investigate compromised account detection over social media. We proposed the segmented-based supervised approach by leveraging time series change point detection from statistical community. The change point detection algorithm attempts to discover the suspicious compromised intervals. Based on the change point detection, a new feature, change point feature is proposed to capture user’s behavior changing pattern. Moreover, the interest feature based on a user’s activities in his/her interested groups and the polarity feature that captures the user’s polarity based on like/dislike behaviors are proposed. Experiments conducted on both real and synthetic datasets, demonstrate the superior accuracy of the proposed method.

Classification of bloodstain patterns at the crime scene is a crucial task in the context of forensic investigations. Traditionally, this task is carried out by forensic experts by applying, unfortunately, time-consuming procedures and subjective judgments. For this reason, artificial intelligence techniques have recently been applied to make this task faster and less subjective. This work presents, for the first time, the application of different deep learning techniques for facing the classification of bloodstain patterns as a multi-class problem. As shown in the experimental session, where a dataset created from scratch has been used, deep learning show optimal performance in facing multi-class classification of bloodstains at the crime scene.

Microsectioning is a destructive testing method extensively employed in the flexible printed circuit board (FPC) fabrication industry to assess the structural integrity and quality of FPCs. FPCs are essential components in various electronic devices due to their flexibility, lightweight nature, and ability to fit into complex shapes and spaces. A cross-section, or microsection, involves obtaining a thin slice of the FPC to expose its internal structure. During cross-section analysis, operators manually measure the thickness of FPC components, such as copper layers, OSC layers, and FSL layers. However, this manual process can lead to inconsistencies and difficulties in establishing standardized measurement procedures. To address these challenges, we propose an “AI-based Microsection Measurement Framework using ComfyUI Workflow” for FPC. This framework comprises five key modules: the target detection module, the image preprocessing and augmentation module, the AI model building and fine-tuning module, the measurement algorithm development module, and the ComfyUI visualization module. The measurement algorithm uses predicted masks from the AI model to perform precise measurements, while the visualization module plots these results directly onto the original image for easy review. In addition, we evaluate the proposed framework on two microsection types. Our experiments demonstrate that the measurement accuracy reaches an error margin of 0 pixels. Compared to the existing method, we provide a unified, faster, and lower labor cost measurement framework.

Traditional attacks such as viruses, trojans, and backdoors remain significant security challenges, especially as systems face both known and emerging threats. Current detection methods often struggle to identify unknown attacks, leaving systems vulnerable. To address this issue, we propose a transformer-based image classifier with an Out-of-Distribution (OOD) detector that uses the Swin Transformer for known attacks and the local outlier factor (LOF) for unknown attacks. Our approach utilizes the Swin Transformer’s attention mechanism to capture intricate attack patterns while LOF identifies outliers indicative of new, unseen threats. Through evaluation, our method achieved a 0.97 accuracy in classifying known attacks and about 0.7 accuracy in detecting unknown attacks, demonstrating its potential to significantly improve system security and deepen our understanding of traditional attack behaviors.