Information Systems and Neuroscience

Frontmatter

Where NeuroIS Helps to Understand Human Processing of Text: A Taxonomy for Research Questions Based on Textual Data

Abstract

Several research questions from information systems (IS) are based on textual data, such as product reviews and fake news. In this paper, we investigate in which areas NeuroIS is best suited to better understand human processing of text and subsequent human behavior or decision making. To evaluate this question, we propose a taxonomy to distinguish these research questions depending on how users’ corresponding response is formed. We first review all publications about textual data in the IS basket journals from 2010–2020. Then, we distinguish text-based research questions along two dimensions, namely, if a user’s response is influenced by subjectivity and if additional information is required to make an objective assessment. We find that NeuroIS research on textual data is still in its infancy. Existing NeuroIS studies focus on texts, where users’ responses are subject to a higher need for additional data, which is not part of the text.

Florian Popp, Bernhard Lutz, Dirk Neumann

Towards a Psychophysiological Investigation of Perceived Trustworthiness and Risk in Online Pharmacies: Results of a Pre-study

Abstract

Perceived trustworthiness and risk are crucial impact factors for a website’s success. While they have been frequently applied to diverse e-commerce contexts, an investigation of these constructs for the special case of online pharmacies is still scarce. In an attempt to measure these constructs in a neural experiment, this paper offers a pre-study with the aim to gain first insights and select appropriate stimuli for the upcoming study. Therefore, five operating online pharmacies are tested in an online survey with 121 participants which rated scales of perceived trustworthiness, perceived risk, attitude towards the website, and use intention for each of the included pharmacies. Results show that online pharmacies with high reputation are rated higher in the included constructs. Consequently, reputation, perceived risk, and trustworthiness are crucial impact factors on attitude and use intention. Thus, two promising online pharmacies could be selected for the follow-up study.

Anika Nissen, Semra Ersöz

Exploring the Influence of Personality Traits on Affective Customer Experiences in Retailing: Combination of Heart Rate Variability (HRV) and Self-report Measures

Abstract

As a result of changes in customers’ shopping behaviors and a corresponding increase in omnichannel behavior (i.e., a blend of online and offline channels), a good customer experience (CX) is crucial for retailers’ success. Affective CX responses are especially crucial in impacting a company’s marketing outcomes, such as a high level of future purchase intentions. Here, we hypothesize that a customer’s affective CX is significantly influenced by his or her personality traits. Based on this hypothesis, we plan to collect physiological data (heart rate variability) and self-report data to study affective CX. Specifically, we will examine the relationship between personality traits, affective CX, and future purchase channel choice intentions. Based on the findings, we will then formulate academic and managerial implications.

Anna Hermes, René Riedl

Motor Dysfunction Simulation in Able-Bodied Participants for Usability Evaluation of Assistive Technology: A Research Proposal

Abstract

The development of assistive technologies and guidelines for their accessibility is impeded by the limited access to disabled participants. As a consequence, performing disability simulations on able bodied participants is a common practice in usability evaluation of assistive technologies. However, still little is known about how disability simulation can influence the usability evaluation of assistive technologies by able-bodied participants. This research proposal explores the effect of a motor dysfunction simulation in able-bodied participants that impedes use of a mouse input, but supports gesture-based assistive technology. Results of this study may provide insights on how to improve, via the experimental design, the meaning and the validity of usability evaluation of assistive technologies by able-bodied participants.

Felix Giroux, Jared Boasen, Charlotte J. Stagg, Sylvain Sénécal, Constantinos Coursaris, Pierre-Majorique Léger

Exploring the Potential of NeuroIS in the Wild: Opportunities and Challenges of Home Environments

Abstract

At this stage, empirical studies in the NeuroIS field have been conducted primarily in laboratory environments. However, the continuing advances in sensor technologies and software interfaces have created novel opportunities to explore the potential of NeuroIS not only in highly controlled lab environments but also in the wild. In this exploratory study, we focus particularly on the potential of conducting NeuroIS studies in remote home environments (NeuroIS@Home) by physically sending equipment (e.g., sensors) to the participant’s location and/or utilizing existing equipment in the participants’ environment (e.g., cameras, input devices). To explore the potential of NeuroIS@Home, we conducted an online expert survey with 16 respondents. We identify higher external/ecological validity of experimental results and the potential of scalability as the most promising opportunities, whereas the lack of control over environmental factors and data quality turned out to be the most severe challenges.

Anke Greif-Winzrieth, Christian Peukert, Peyman Toreini, Marc T. P. Adam

Exploring the Recognition of Facial Activities Through Around-The-Ear Electrode Arrays (cEEGrids)

Abstract

NeuroIS scholars increasingly rely on more extensive and diverse sensor data to improve the understanding of information system (IS) use and to develop adaptive IS that foster individual and organizational productivity, growth, and well-being. Collecting such data often requires multiple recording devices, which leads to inflated study cost and decreased external validity due to greater intrusion in natural behavior. To overcome this problem, we investigated the potential of using an around-the-ear electrode array capable of capturing neural and cardiac activity for detecting an additional set of variables, namely facial muscle activity. We find that reading, speaking, chewing, jaw clenching, and six posed emotion expressions can be differentiated well by a Random Forest classifier. The results are complemented by the presentation of an open-source signal acquisition system. Thereby, an economical approach for naturalistic NeuroIS research and artefact development is provided.

Michael T. Knierim, Max Schemmer, Monica Perusquía-Hernández

Leveraging NeuroIS Tools to Understand Consumer Interactions with Social Media Content

Abstract

Social media has risen as one of the leading budget allocations for advertising within many firms, demonstrating its increasing dominance of the marketing mix. As such, many corporate entities have increased their presence on social media platforms in recent years. We seek to better understand the impact of non-consumer generated content on the social media user experience. This study presents the application of electroencephalography to uncover mental activity by consumers when processing social media content. This research continues from a larger study exploring how consumers process content based on the author of social media content. While this extension focuses on understanding how consumers process social media content based on the author of the post, it has implications for further studies in human-computer interaction and content optimization.

Jen Riley, Adriane B. Randolph

Optimizing Scatterplot-Matrices for Decision-Support:

An Experimental Eye-Tracking Study Assessing Situational Cognitive Load

Abstract

The scatterplot matrix is defined to be a standard method for multivariate data visualization; nonetheless, their use for decision-support in a corporate environment is scarce. Amongst others, longstanding criticism lies in the lack of empirical testing to investigate optimal design specifications as well as areas of application from a business related perspective. Thus, on the basis of an innovative approach to assess a visualization’s fitness for efficient and effective decision-making given a user’s situational cognitive load, this study investigates the usability of a scatterplot matrix while performing typical tasks associated with multidimensional datasets (correlation and distribution assessment). A laboratory experiment recording eye-tracking data investigates the design of the matrix and its influence on the decision-maker’s ability to process the presented information. Especially, the information content presented in the diagonal as well as the size of the matrix are tested and linked to the user’s individual processing capabilities. Results show that the design of the scatterplot as well as the size of the matrix influenced the decision-making greatly.

Lisa Perkhofer, Peter Hofer

“Overloading” Cognitive (Work)Load: What Are We Really Measuring?

Abstract

Cognitive load is one of the most studied constructs in NeuroIS [1]. Not surprisingly, we have identified 27 papers presented at NeuroIS retreats between 2012 and 2020 which included measurement of cognitive load or related constructs. This paper reviews terminology used to refer to cognitive load, mental workload and its variations, as well as their operationalizations and measurements. All 27 papers employed physiological NeuroIS measures, while six of them additionally used subjective self-ratings. The wide range of measurements prompts us to question if we are measuring the same construct. We provide an overview and a summary of cognitive load terminology and measurement used in these 27 papers and conclude with recommendations for future research.

Jacek Gwizdka

On Electrode Layout in EEG Studies: A Limitation of Consumer-Grade EEG Instruments

Abstract

There is an ongoing discussion in the NeuroIS (Neuro-Information-Systems) discipline on whether consumer-grade EEG instruments are as suitable for scientific research as research-grade instruments. Considering the increasing adoption of consumer-grade instruments along with the fact that many NeuroIS EEG papers used such tools, this debate is fundamental. We report on a study in which we contrasted a 61-channel EEG recording with a 14-channel recording that should simulate the electrode layout of the EPOC headset, the presumably worldwide most widely used consumer-grade tool. The contrast was carried out based on topographic mapping, because this kind of EEG data analysis does not only play a significant role in cognitive neuroscience, but also in NeuroIS research. Our findings show noticeable differences in the topoplots between both conditions. The current research results are limited by the fact that our task context is a non-IS context (i.e., upper limb movements). Hence, future research should validate our results based on IS tasks and situations in order to confirm, revise, or falsify the present results.

Gernot R. Müller-Putz, Ursula Tunkowitsch, Randall K. Minas, Alan R. Dennis, René Riedl

Predicting In-Field Flow Experiences Over Two Weeks from ECG Data: A Case Study

Abstract

Predicting flow intensities from unobtrusively collected sensor data is considered an important yet challenging endeavor for NeuroIS scholars aiming to understand and support flow during IS use. In this direction, a limitation has been the focus on cross-subject models built on data collected in controlled laboratory settings. We investigate the potential of predicting flow in the field through personalized models by collecting report and ECG data from a clerical worker over the course of two weeks. Results indicate that a lack of variation in flow experiences during this time likely diminished these potentials. Through pre-training feature selection methods, model accuracies could be achieved that nonetheless approach related cross-subject flow prediction work. Novel recommendations are developed that could introduce more flow variation in future flow field studies to further investigate the within-subject predictability of flow based on wearable physiological sensor data.

Michael T. Knierim, Victor Pieper, Max Schemmer, Nico Loewe, Pierluigi Reali

An Inward Focus of Attention During Information Security Decision Making: Electrophysiological Evidence

Abstract

Insider threat represents a significant source of violations of information security. Our previous research using event-related potentials (ERPs) has revealed patterns of neural activity that distinguish ethical decision making from decisions that do not involve an ethical component. In the current study, we sought to gain insight into the locus of the effect of ethical decision making on the posterior N2 component of the ERPs. The ERP data revealed that the N2 was greater in amplitude for control trials relative to ethical violation trials, and time-frequency analyses revealed that this resulted from a reduction in phase-locked activity across trials rather than a decrease in EEG power. These findings may indicate that ethical decision making related to information security is associated with a greater inward focus of attention than is the case for decision making on control trials.

Robert West, Kate Cowger

EyeTC: Attentive Terms and Conditions of Internet-Based Services with Webcam-Based Eye Tracking

Abstract

Now and then, users are asked to accept terms and conditions (T&C) before using Internet-based services. Previous studies show that users ignore reading T&C most of the time and accept them tacitly without reading, while they may include critical information. This study targets solving this problem by designing an innovative NeuroIS application called EyeTC. EyeTC uses webcam-based eye tracking technology to track users’ eye movement data in real-time and provide attention feedback when users do not read T&C of Internet-based services. We tested the effectiveness of using EyeTC to change users’ behavior for reading T&C. The results show that when users receive EyeTC-based attention feedback, they allocate more attention to the T&C, leading to a higher text comprehension. However, participants articulated privacy concerns about providing eye movement data in a real-world setup.

Peyman Toreini, Moritz Langner, Tobias Vogel, Alexander Maedche

Detecting Flow Experiences in the Field Using Video-Based Head and Face Activity Recognition: A Pilot Study

Abstract

Flow represents a valuable daily life experience as it is linked to performance, growth, and well-being. As flow support is still a major challenge due to a lack of automatic and unobtrusive detection methods, NeuroIS scholars face the opportunity to devise measurement approaches for flow experience during IS use and, moreover, flow supporting, adaptive NeuroIS. This work presents the first results from a controlled experience sampling field study in which experiences were observed using video recordings during a week of scientific writing. Novel behavioral features (face and head activity) with negative flow-report correlations are identified. Together, the results contribute to the NeuroIS community by providing an extended concept of flow as a state of behavioral efficiency, the identification of novel correlates, and recommendations for economical and feasible extensions of the study approach.

Christoph Berger, Michael T. Knierim, Christof Weinhardt

Understanding the Potential of Augmented Reality in Manufacturing Environments

Abstract

Manufacturing companies are confronted with challenges due to increasing flexibility requirements and skill gaps. Augmented Reality applications offer an efficient way to overcome these tensions by enhancing the interaction between people and technology. The positive effects of Augmented Reality solutions are often described in individual models in the scientific literature. This research-in-progress aims to aggregate the empirical findings in the usage of Augmented Reality solutions in manufacturing environments. A meta-analysis is conducted to synthesise several small studies into one large study to achieve this. In particular, the meta-analysis will focus on the impact of Augmented Reality applications on cognitive load levels. Furthermore, the effect on processing time and error rates will be evaluated. Initial results of the meta-analysis will be expected and reported at this year’s NeuroIS Retreat.

Felix Kaufmann, Laurens Rook, Iulia Lefter, Frances Brazier

On How Mind Wandering Facilitates Creative Incubation While Using Information Technology: A Research Agenda for Robust Triangulation

Abstract

Our minds tend to frequently drift away from present technology-related situations and tasks. Against this background, we seek to provide a better understanding of mind-wandering episodes while using information technology and its link to decisive variables of Information Systems research, such as performance, creativity and flow. Since the academic literature still lacks reliable and validated measurements that can fully account for all facets of mind-wandering episodes while using information technology, our work addresses this gap by presenting a way to triangulate data in the context of a digital insight problem-solving task. This new approach enables researchers to further investigate the effects of spontaneous thought in technology-related settings and is a promising building block for the development of neuroadaptive systems.

Frederike M. Oschinsky, Bjoern Niehaves, René Riedl, Michael Klesel, Selina C. Wriessnegger, Gernot R. Mueller-Putz

Consumers Prefer Abstract Design in Digital Signage: An Application of Fuzzy-Trace Theory in NeuroIS

Abstract

Visual designs of digital signage (DS) content shape and influence consumers’ decisions. Understanding the effect of DS design on consumer behavior requires a fundamental understanding of human reasoning and decision-making. This research explores the effect of different visual design cues of DS on a neural level and through the lens of Fuzzy-Trace Theory (FTT). The FTT suggests that humans have both a verbatim-based and a gist-based information processing. To explore the effect of FTT-based visual design, an experiment using functional near-infrared spectroscopy is conducted. DS are tested on three design levels: (1) verbatim: text, (2) verbatim: photographs, and (3) gist-based. Results show that only the gist-based design resulted in significantly higher self-reported results and activated brain areas in the medial prefrontal cortex, which are associated with emotional and rewarding processing. These results challenge the manifest differentiation only between image and text elements.

Anika Nissen, Gabriele Obermeier, Nadine R. Gier, Reinhard Schütte, Andreas Auinger

Topographic Analysis of Cognitive Load in Tacit Coordination Games Based on Electrophysiological Measurements

Abstract

Tacit coordination games are coordination games in which communication between the players is not possible. Various studies have shown that people succeed in these games beyond what is predicted by classical game theory. This success is attributed to the identification of focal points (also known as Schelling points). Focal points are pronounced solutions based on salient features of the game that somehow attracts the players’ attention. Experiments with tacit coordination games show that some players manage to “see” the focal points and reason about the selections made by the co-player, while others fail to do so, and might turn to guessing. According to the Cognitive Hierarchy Theory (CHT), the task of coordinating, that is, reasoning about what the other player would choose is performed on cognitive levels greater than or equal to 1. In contrast, the task of just picking an answer, without an explicit need to coordinate is done at cognitive level 0. With that in mind, our study has two main purposes. First, to examine whether the same task that is defined each time at a different cognitive level (picking or coordination) causes a different psychological cognitive load in the participating players. Second, to examine the distribution of cognitive load across the scalp during coordination tasks.

Dor Mizrahi, Ilan Laufer, Inon Zuckerman

Active Learning Techniques for Preparing NeuroIS Researchers

Abstract

The field of neuroIS is rapidly evolving, and there is a need to create a research and work force at various levels of the academy ranging from undergraduate students to professors. Motivation is not an issue with neuroIS as students are typically excited to learn, but how do we teach them the skills they need to succeed? Active learning is a pedagogical technique that has a natural fit with neuroIS. It focuses on the higher levels of learning that are essential in the field. This paper is an introduction to active learning for the benefit of the neuroIS community. It discusses examples of what can be done as well as challenges that need to be overcome.

Arjan Raven, Adriane B. Randolph

Examining the Impact of Social Video Game Tournaments on Gamers’ Mental Well-Being

Abstract

We examine the impact that gaming on a social tournament platform while playing multiplayer games has on the mental well-being of college students. In this early-stage study, we used the Scale of Positive and Negative Experiences and the Player Experience and Need Satisfaction Scale to measure well-being, gaming motivation, and enjoyment. We complement these survey tools with facial expression analysis of students during gameplay for a more holistic understanding of their emotional states and the impact of social gaming.

Tanesha Jones, Adriane B. Randolph, Sweta Sneha

Continuing Doctoral Student Training for NeuroIS and EEG During a Pandemic: A Distance Hands-On Learning Syllabus

Abstract

There is a need to train newcomers to NeuroIS on neuroscientific tools and methodologies. Due to the pandemic, existing syllabi required adaptation to enable remote training. In this paper, we present a syllabus aimed at providing hands-on distance learning and EEG training. The proposed syllabus was pretested during a Ph.D. course on Neuroscience and IT. We report in this manuscript our lessons learned and recommendations for conducting remote neuroscience training.

Théophile Demazure, Alexander Karran, Pierre-Majorique Léger

Design Mode, Color, and Button Shape: A Pilot Study on the Neural Effects of Website Perception

Abstract

The investigation of website aesthetics has a long history and has already been addressed in NeuroIS research. The extant literature predominantly studied website complexity, symmetry, and colors. However, other design factors have not yet been examined so far. We studied two new factors (design mode: light vs. dark, button shape: rounded vs. sharp angled) along with color (blue vs. red). Specifically, we examined the impact of these three factors on several outcomes. Results from a repeated-measures MANOVA indicate: (i) design mode (light vs. dark) significantly affects users’ pleasure, arousal, trust, attitude, and use intention, (ii) color (blue vs. red) significantly influences pleasure, arousal, and use intentions, while (iii) button shape (rounded vs. sharp) does not significantly influence any of the dependent measures. Based on these results, follow up functional near-infrared spectroscopy studies are developed which aim to further complement our self-report findings.

Anika Nissen, René Riedl

Does Media Richness Influence the User Experience of Chatbots: A Pilot Study

Abstract

From a user’s perspective, this pilot study investigates the contributors and irritants related to the media content format used by chatbots to assist users in an online setting. In this study, we use automated facial expression analysis (AFEA), which analyses users’ facial expressions and captures the valence of their lived experience. A questionnaire and a single-question interview were also used to measure the users’ perceived experience. All measures taken together allowed us to explore the effects of three media content formats (i.e., an interactive question and answer (Q&A), a video, and a link referring to a webpage) used in chatbots on both the lived and perceived experiences of users. In line with Media Richness Theory (MRT), our results show that an interactive Q&A might be an optimal chatbot design approach in providing users with sought-after information or assistance with transactions. Moreover, important avenues for future research emerge from this study and will be discussed.

Laurie Carmichael, Sara-Maude Poirier, Constantinos Coursaris, Pierre-Majorique Léger, Sylvain Sénécal

Development of a New Dynamic Personalised Emotional Baselining Protocol for Human-Computer Interaction

Abstract

Measuring emotional responses from users when they interact with a technological artefact is an important aspect of human-computer interaction (HCI) research. People, however, tend to react in an individualized manner to emotional stimuli, thus it is important to compare each user to a personalized baseline. We present, in this paper, the development and preliminary results of a new emotional elicitation protocol that is being validated with our remote psychophysiological measurement ecosystem.

Elise Labonté-LeMoyne, François Courtemanche, Constantinos Coursaris, Arielle Hakim, Sylvain Sénécal, Pierre-Majorique Léger

Mediators of the Relationship Between Self-control and Pathological Technology Use: Negative Affect and Cognitive Failures, but not Self-efficacy

Abstract

The widespread adoption of technologies such as smartphones, the Internet, and social media has been associated with the emergence of pathological technology use (e.g., Internet addiction). Prevalence rates of pathological technology use vary widely across age groups, cultures, and medium, although it is not uncommon for rates of mild to moderate pathological use to exceed 20%–30%. These relatively high prevalence rates have motivated researchers to identify the predictors of pathological use. The current study focuses on the relationship between self-control and pathological technology use, and demonstrates that negative affect and cognitive failures, but not self-efficacy, partially mediate the association between self-control and pathological technology use. These findings reveal some of the pathways by which poor self-control could lead to elevated levels of pathological technology use.

Robert West, Diana Jiang

High Fidelity Vibrokinetic Stimulation Augments Emotional Reactivity and Interhemispheric Coherence During Passive Multimedia Interaction

Abstract

Haptic technologies are widely used in multimedia entertainment to psychophysiologically enhance user experience. Psychometric-based research regarding vibrokinetic stimulation during multimedia viewing supports this notion. However, scant neurophysiological evidence exists to verify this effect. Using a between groups design with source-localized electroencephalography, the present study analyzed the effect of high fidelity vibrokinetic (HFVK) stimulation during passive multimedia interaction (i.e. watching a haptically enhanced movie) on self-reported emotional state and intercortical theta coherence. Results indicate that HFVK increases emotional reactivity in association with increased interhemispheric coherence between the right inferiortemporal gyrus and the left insular cortex, thereby conferring neurophysiological support for the efficaciousness of HFVK to enhance emotional response during movie watching.

Jared Boasen, Felix Giroux, Sara-Eve Renaud, Sylvain Sénécal, Pierre-Majorique Léger, Michel Paquette

Explainable Artificial Intelligence (XAI): How the Visualization of AI Predictions Affects User Cognitive Load and Confidence

Abstract

Explainable Artificial Intelligence (XAI) aims to bring transparency to AI systems by translating, simplifying, and visualizing its decisions. While society remains skeptical about AI systems, studies show that transparent and explainable AI systems result in improved confidence between humans and AI. We present preliminary results from a study designed to assess two presentation-order methods and three AI decision visualization attribution models to determine each visualization’s impact upon a user’s cognitive load and confidence in the system by asking participants to complete a visual decision-making task. The results show that both the presentation order and the morphological clarity impact cognitive load. Furthermore, a negative correlation was revealed between cognitive load and confidence in the AI system. Our findings have implications for future AI systems design, which may facilitate better collaboration between humans and AI.

Antoine Hudon, Théophile Demazure, Alexander Karran, Pierre-Majorique Léger, Sylvain Sénécal

Backmatter