Human-Computer Interaction – INTERACT 2019

Accessibility of products and services is key for people living with a disability to ensure that they are easier to use. However, web accessibility guidelines have been shown to be cumbersome to understand, which impacts on designers’ intention to use them. Several tools have been proposed in the literature, but they mostly focus on automatic accessibility testing, a process that is performed after a product has been developed. Little attention has been paid to using web accessibility guidelines during the design phase. In this paper, we present GATE, a serious game to help raise designer awareness of web accessibility guidelines, which is part of our work in progress on gamified technologies for this purpose. Its usability and perceived effectiveness were evaluated through an empirical study using a mixed methods approach. Our initial findings show that GATE is a promising solution that scored high in its playability and potential for use. This work has important potential contributions for the wider adoption of web accessibility guidelines.

Surveys and questionnaires are commonly used to capture people’s experiences with technology. However, some older users experience issues when reading and filling out forms, and inexperienced computer users may not be comfortable with web-based versions. To improve the report of user experience, we designed and implemented Aestimo, a tangible interface based on a shortened version of the AttrakDiff questionnaire. The interface was evaluated during a study with 20 older adults (age avg. = 65.6). Although completing the Aestimo questionnaire took longer than a paper-and-pen version of AttrakDiff, 60% of participants preferred Aestimo over AttrakDiff. Aestimo was found to be innovative and inviting, and to stimulate the senses and the mind. Participants liked feeling guided by the interface, and also found their experience to be playful and fun. Overall, the evaluation of Aestimo was highly positive and suggests that some user groups may benefit from the availability of innovative evaluation experiences.

A methodology for assessing the degree of accessibility of museum and science center exhibits is proposed in this work. It consists of an exhibit-centric framework with a corresponding score for the accessibility indicators vision, hearing, mobility, motor, and cognition. The scores are achieved by combining the results of an observational study with a supplementary expert evaluation for high reliability. The methodology was successfully applied with pupils in an informal-learning context at Oslo Science Center. We believe it has relevance for and can be applied to other areas such as self-service machines and interactive technical artifacts in general due to its generic nature.

Mobile device users are required to constantly learn to use new apps, features, and adapt to updates. For blind people, adapting to a new interface requires additional time and effort. At the limit, and often so, devices and applications may become unusable without support from someone else. Using tutorials is a common approach to foster independent learning of new concepts and workflows. However, most tutorials available online are limited in scope, detail, or quickly become outdated. Also, they presume a degree of tech savviness that is not at the reach of the common mobile device user. Our research explores the democratization of assistance by enabling non-technical people to create tutorials in their mobile phones for others. We report on the interaction and information needs of blind people when following ‘amateur’ tutorials. Thus, providing insights into how to widen and improve the authoring and playthrough of these learning artifacts. We conducted a study where 12 blind users followed tutorials previously created by blind or sighted people. Our findings suggest that instructions authored by sighted and blind people are limited in different aspects, and that those limitations prevent effective learning of the task at hand. We identified the types of contents produced by authors and the information required by followers during playthrough, which often do not align. We provide insights on how to support both authoring and playthrough of nonvisual smartphone tutorials. There is an opportunity to design solutions that mediate authoring, combine contributions, adapt to user profile, react to context and are living artifacts capable of perpetual improvement.

Charts such as bar or pie charts are often used to represent data and their relation. Tactile charts are widely used to enable blind and visually-impaired people to explore charts through the sense of touch. Effective tactile chart design differs from its visual counterpart due to sequential nature of touch. Accordingly, in a study with 48 blind and visually-impaired participants we investigated the preferences for chart types, design features and errors in reading data values. We developed bar, line and pie charts as well as scatterplots with different layouts and novel design properties. Participants answered questions concerning the readability, content and data, specific design aspects as well as a personal rating. Overall, participants answered 80% of nominal questions regarding minima, maxima, and comparisons, correctly. Blind participants achieved a corrected mean error rate of 4.5%, when reading single points or intersections, for example. More specifically, we directly compare chart types, and discuss the results for specific design considerations (e.g. distances between bars, width of bars, design and use of grid lines in scatterplots) by comparing different charts.

The design and development of computer assistive technologies must be tied to the needs and goals of end users and must take into account their capabilities and preferences. In this paper, we present MeDeC@, a Methodology for the Development of Computer Assistive Technologies for people with Autism Spectrum Disorders (ASD), which relies heavily in our experience working with end users with ASD. The aim of this methodology is not to design for a broad group of users, but to design highly customizable tools so that they can be easily adapted to specific situations and small user groups. We also present two applications developed using MeDeC@ in order to test its suitability: EmoTraductor, a web application for emotion recognition for people with Asperger Syndrome, and ReadIt, a web browser plug-in to help people with ASD with written language understanding difficulties to navigate the Internet. The results of our evaluation with end users show that the use of MeDeC@ helps developers to successfully design computer assistive technologies taking into account the special requirements and scenarios that arise when developing this kind of assistive applications.

ADHD has an estimated worldwide prevalence of 2–3% and is one of the most frequent neurodevelopmental disorders. Many problems in an ADHD-patient’s life arise from the lack of self-management abilities and social interaction with others. While medication is considered the most successful treatment for disorders such as ADHD, patients often seek support in therapeutic sessions with trained therapists. These aim to strengthen self-awareness of symptoms, emotional self-regulation, and self-management. However, sharing personal insights can be a burden for patients while therapists would benefit from understanding important issues a patient is facing. Our work aims to support therapy for patients and therapists by providing classification of digital diary entries for therapy sessions while protecting patients privacy. Additionally, we provide insights into important issues and topics including their affective interpretation for patients suffering from ADHD.

People with dyslexia have reported difficulties with navigating websites, yet very little research has investigated the nature of these difficulties. This study investigated the effects of two aspects of web navigation, menu organization and visibility on the eye gaze behaviour, performance and preferences of dyslexic and non-dyslexic participants. Participants undertook four tasks on a website with either unified or fragmented main menu organization and either static or dynamic submenus. Dyslexic participants had significantly longer menu scanpaths (i.e. looked at more menu items), due to looking at more different menu items and revisiting more items in comparison to non-dyslexic participants. They also had more fixations and longer dwell times on submenus than non-dyslexic participants. They were also slower to select their first menu item and complete tasks. However, the perceptions of the different menu presentations did not differ between the two participant groups, although all participants preferred the unified menu presentation to the fragmented. Directions for further research to deepen the understanding and implications of these results are discussed.

Several studies explore the use of social robots in interventions for persons with cognitive disability. This paper describes ELE, a plush social robot with an elephant appearance that has been designed as a conversational companion for persons with Neuro-Developmental Disorders (NDD). ELE speaks through the live voice of a remote caregiver, enriching the communication through body movements. It is integrated with a tool for automatic gathering and analysis of interaction data that support therapists in monitoring the users during the experience with the robotic companion. The paper describes the design and technology of ELE and presents an empirical study that involved eleven persons with NDD using the robot at a local therapeutic center. We compared user engagement in two story-telling experiences, one with ELE and one with a face-to-face human speaker. According to our results, the participants were more engaged with ELE than with the human storyteller, which indicates, although tentatively, the engagement potential of conversational social robots for persons with NDD.

Children with Autism Spectrum Disorders (ASD) have difficulties with social communication and interaction. Social Stories $$^\mathrm{TM}$$ are a well-known intervention to help them overcome these difficulties. During these interventions, practitioners must check children’s understanding of various concepts that are taught. However, this activity - comprehension checking - is often skipped as practitioners find it difficult and time consuming. Our project explores how a technology-based tool (the S $$^{2}$$ C $$^{2}$$ app) can be designed and developed to support Social Story $$^\mathrm{TM}$$ comprehension checking in children with ASD (aged 7–12) by involving typically developing children (TD) and experts in ASD and HCI. An initial pilot evaluation with sixteen TD children and five experts in ASD and HCI suggested that the S $$^{2}$$ C $$^{2}$$ app provides appropriate engaging activities for children and facilitates Social Story $$^\mathrm{TM}$$ comprehension checking. However, caution must be taken in extending the results and more studies involving children with ASD are planned to be conducted in the future.

People affected by the frozen shoulder syndrome show limited shoulder mobility which is often accompanied by pain. The frozen shoulder syndrome often lasts from months to years, and mostly affects people in the age group of 40 to 70 years. The frozen shoulder syndrome severely reduces the quality of life and the ability to work. A common treatment method is physiotherapy. Patients are referred to a physiotherapist, who selects specific exercises adapted for the specific patient. Physiotherapy requires patient compliance, time, and effort. Correct exercise performance and compliance are the main issues in physiotherapy. A smartphone app could support patients by providing detailed exercise instructions and motivation through exercise logging, as is common for fitness and sport. In this work, such an app for frozen shoulder syndrome, the ShoulderApp, is evaluated in two user studies. The main contribution is that the user studies were conducted in an ambulatory assessment setting, which allows to draw conclusions about real-world usage, usability and user acceptance. The app was regularly used and study participants were satisfied. Additionally, we researched the usability and usage of interactive 3D and multi-modal exercise instructions, motivational aspects, exercise correctness and the interplay of physiotherapy and app usage. Measurements of shoulder mobility are the key assessment tool for the state and progress of the frozen shoulder syndrome. A smartphone sensor-based measurement tool, which only required a simple band in addition to the smartphone, was developed and evaluated. Interventions with the ShoulderApp were evaluated in a three-week short-term intervention and an 18-week midterm evaluation with 5 patients each. For the evaluation of the results, we used standardized questionnaires, SUS, TAM-2, and USE. In addition, semi-structured interviews and automatic logging of user-interactions in the app were included as the outcome measurements. Overall, the results for both the short-term and mid-term user studies showed that the ShoulderApp could support physiotherapy for frozen shoulder patients. The positive results of the studies show the potential of a generalization of the ShoulderApp concept to the large group of musculoskeletal disorders such as lower back pain and knee injuries.

Persuasive technologies for physical rehabilitation have been proposed in a number of different health interventions such as post-stroke gait rehabilitation. We propose a new persuasive system, called Augmented Crutches, aimed at helping people to walk with crutches. People with injuries, or with any sort of mobility problem typically use assistive devices such as crutches, walkers or canes in order to be able to walk more independently. However, walking with crutches is a learning skill that needs continuous repetition and constant attention to detail in order to walk correctly with them and without suffering negative consequences, such as falls or injuries. In close collaboration with therapists, we identify the main issues that patients face when walking with crutches. These vary from person to person, but the most common and hardest challenges are the position and coordination of the crutches. Augmented Crutches studies human behavior aspects in these situations and augments the ground space around the user with digital visual cues where timing is the most important factor, without the need for a constant therapist providing manual help. This is performed through a mini-projector connected to a smartphone, worn by the user in a portable, lightweight manner. Our system helps people to learn how to walk using crutches with increased self-confidence and motivation. Additionally, our work identifies timing, controllability and awareness as the key design dimensions for the successful creation of persuasive, interactive experiences for learning how to walk with crutches.

This paper explores ‘A to B’ routing tools designed to chart accessible routes for wheelchair users. We develop and present a novel measurement framework based upon cost-benefit analysis in order to evaluate the real-world utility of routing systems for wheelchair users. Using this framework, we compare proposed routes generated by accessibility tools with the pedestrian routes generated by Google Maps by means of conducting expert assessments of the situation on the ground. Relative to tools aimed at pedestrians, we find that these tools are not significantly more likely to produce an accessible route, and more often than not, they present longer routes that arise from imaginary barriers that do not exist in the real world. This analysis indicates how future routing tools for wheelchair users should be designed to ensure that they genuinely ameliorate the effects of accessibility barriers in the built environment.

Post-traumatic stress disorder (PTSD) is a serious mental health condition. Prolonged Exposure (PE) therapy has been shown to have the greatest therapeutic efficacy in treating PTSD. However, clinicians face significant challenges in delivering PE therapy successfully. We conducted a two-part study with 17 clinicians where we (1) gathered requirements for a PE therapy dashboard and (2) had clinicians assess the relevance and usability of our dashboard prototypes. The first part of our study showed that clinicians wanted (1)improved workflow for their PE sessions (2)improved methods of monitoring their patients’ self-report data and (3)a way to assess their patients’ progress outside of sessions. The second part showed that our dashboard prototypes were highly usable. Our study provides preliminary steps in designing user-centered tools that assist clinicians in the delivery of PE therapy by improving data access and workflow efficiency.

According to the World Health Organization, an estimated one billion people live with a disability. Millions of them are non-verbal, and also experience motor-skill challenges. The limitations on activity and restrictions on participation due to such disabilities often lead to discrimination and social exclusion. A UNICEF study analyzing data from 15 countries found that almost 50% of children with disabilities are out of school, and 85% of them did not receive any formal education. Affording enhanced and accelerated communication for disabled people, who continue to form the world’s largest minority to experience social discrimination, is central to making the world a more inclusive place. Our LIVOX application incorporates artificial intelligence algorithms to reduce the so-called “reciprocity gap” that acts as a communication barrier between disabled people and their interlocutors, thus enabling people with disabilities, especially children, to participate in daily social and educational activities. Integrating them into the existing social structures is central to making the world a more inclusive place.

Visualization techniques such as bar graphs and pie charts let sighted users quickly understand and explore numerical data. These techniques remain by and large inaccessible for visually impaired users. Even when these are made accessible, they remain slow and cumbersome, and not as useful as they might be to sighted users. Previous research has studied two methods of improving perception and speed of navigating auditory graphs - using non-speech audio (such as tones) instead of speech to communicate data and using two audio streams in parallel instead of in series. However, these studies were done in the early 2000s and speech synthesis techniques have improved considerably in recent times, as has the familiarity of visually impaired users with smartphones and speech systems. We systematically compare user performance on four modes that can be used for the generation of auditory graphs: parallel-tone, parallel-speech, serial-tone, and serial-speech. We conducted two within-subjects studies - one with 20 sighted users and the other with 20 visually impaired users. Each user group performed point estimation and point comparison tasks with each technique on two sizes of bar graphs. We assessed task time, errors and user preference. We found that while tone was faster than speech, speech was more accurate than tone. The parallel modality was faster than serial modality and visually impaired users were faster than their sighted counterparts. Further, users showed a strong personal preference towards the serial-speech technique. To the best of our knowledge, this is the first empirical study that systematically compares these four techniques.

On-body interaction is a growing alternative way of controlling everyday interactive devices. Despite on-body gestures being natural for humans, there is still a lack of research on how this type of interaction can improve accessibility for people with visual disabilities. The first step must be to acknowledge if users from this population group are willing to use on-body interaction for mobile interaction scenarios and under what conditions. In this article we present a qualitative study with 18 interviewees to understand what factors impact the willingness of visually impaired users to control their mobile devices through on-body interaction in different environments and settings. We observed a set of factors and conditions that affect positively or negatively their willingness to use on-body interaction. The factors range from the location and type of audience to safety, privacy, and embarrassment or even the characteristics of the gestures themselves.

Swarachakra is an Abugida text input keyboard available in 12 Indian languages. We enhanced an accessible version of Swarachakra Marathi with speech input. However, speech input could be error-prone, and especially so for languages where speech recognition technologies are new. Such errors could either slow the user down due to the need for editing, or go unnoticed, leading to high uncorrected error rates. We therefore conducted a within-subject empirical study to compare the user performance of keyboard-only input method with keyboard+speech input method with 11 novice visually impaired users. We found that keyboard+speech input was almost 11 times faster, reaching 182 characters per minute, and had a lower uncorrected error rate than the keyboard-only input, and in spite of having higher corrected error rates. Though we used a wide variety of phrases in our study, we observed that all phrases were faster on average with the keyboard+speech input method. To the best of our knowledge, ours is the first empirical study to evaluate the performance of speech enabled text input in Marathi for visually impaired people. This is the highest reported speed by visually impaired users in any Indian language.

Digital Interactive Maps on touch surfaces are a convenient alternative to physical raised-line maps for users with visual impairments. To compensate for the absence of passive tactile information, they provide vibrotactile and auditory feedback. However, this feedback is ambiguous when using multiple fingers since users may not identify which finger triggered it. To address this issue, we explored the use of bilateral feedback, i.e. collocated with each hand, for two-handed map exploration. We first introduced a design space of feedback for two-handed interaction combining two dimensions: spatial location (unilateral vs. bilateral feedback) and similarity (same vs. different feedback). We implemented four techniques resulting from our design space, using one or two smartwatches worn on the wrist (unilateral vs. bilateral feedback respectively). A first study with fifteen blindfolded participants showed that bilateral feedback outperformed unilateral feedback and that feedback similarity has little influence on exploration performance. Then we did a second study with twelve users with visual impairments, which confirmed the advantage of two-handed vs. one-handed exploration, and of bilateral vs. unilateral feedback. The results also bring to light the impact of feedback on exploration strategies.

We present a design of an indoor orientation terminal for visually impaired older adults. Interaction is based on buttons, tactile symbols, and audio feedback. The terminal consists of five parts dedicated to a particular function. The tactile symbols mimic real-world objects. We performed three design iterations and conduct evaluations with a total of 17 participants, their mean age was 84.2 years. The results show that usage of real-world objects and low level of symbol abstraction leads to an unambiguous pairing of user expectations and real functions. Introduction of complicated and abstract artifacts like contour objects or complex tactile map was very hard to understand and recognize. Our final design was well accepted by all participants and allowed the participants to orient themselves in the indoor environment.

We are working towards establishing a framework to enable more people to create and play digital games. Our focus is on skills, communication, and collaboration, since these qualities can enable more people to co-create inclusive games. In this paper, we describe how the framework assisted adults involved in an alcohol and drugs rehabilitation program to co-create their own games. Ten adults in a healthcare service co-created games using the framework as a part of their rehabilitation, in ten meetings spanning four months. Two healthcare professionals evaluated the activities. Five additional collaborators (three with a Computer Science and two with a Nursing background) provided accessibility features and artistic improvements to the projects. During the meetings, we observed that game creation and playing helped the participants. They started in an uncertain frame of mind, with low-self esteem, and were scared to use computers and games, since they doubted they could succeed. However, they ended up more confident on their abilities and proud of their creations, as they were able to share their games and knowledge with their peers, and teach people how to play. The models and systems of the framework allowed the people to achieve better results. The game co-creation empowered the participants, and, hence, their abilities became opportunities for further collaborations. Co-creation consisted of a journey in which self-improvement superseded the created games.

In this paper, we present a passively responsive framework for immersive movies, called Cinévoqué. The framework seamlessly alters the narrative and visual elements within an immersive live-action movie, based on real-time passive data, such as gaze direction and system time, obtained during the experience. The paper primarily focuses on the design and storytelling aspects of Cinévoqué, such as possible narrative structures and the design challenges involved in creating responsive experiences. We further examine the potential of this framework through two prototypes of varying complexity and responsive features, and the insights from them are used to suggest approaches that can lead to effective seamless narrative experiences.

In recent years, children’s mental health problems, including Attention Deficit Hyperactivity Disorder (ADHD), have been a growing phenomenon. However, there are limited examples of designing for and with children with ADHD. This work views conditions such as ADHD through the lens of neurodiversity as different cognitive styles, focusing on “coolabilities” and enhanced competences instead of disabilities. This paper explores how to engage children with ADHD in co-design activities. Taking the Diversity for Design (D4D) framework as a starting point, an adaptation of the framework for ADHD was first driven by theoretical considerations and three expert interviews, followed by an empirical study consisting of three co-design workshops with four male participants (aged 7–10). Based on observations and audio recordings from the co-design workshops, a qualitative analysis was carried out. Our results show that when their needs, preferences, and individual desires are taken into account, children with ADHD can be meaningfully engaged in co-design activities. By offering an adapted version of the D4D framework tailored for ADHD, designers can structure the environment and provide scaffolds so that children with ADHD can become active participants in co-design workshops. This research informs the design community on how to engage and involve children with ADHD into the design process.

The paper documents the co-design methodology followed by Northern Stage theatrical company for the design of a theatrical production, rich in digital elements. Drawing on our data from fieldwork, interviews and questionnaires, we initially report on the co-design activities, then using thematic analysis, we review the impact of technology in the co-design activities, the dynamics of using digital technologies in a performance and the limitations for small-medium theatrical companies. Our work extends research and practice on co-design and participatory design in creative industries and their experimentation with technology. More specifically, we contribute by casting light on the nature of activities and the level of digital maturity and readiness. The paper concludes with considerations of co-design and HCI work in attracting a new generation of performers and audiences for the digital era.

Using highly interactive systems like computer games requires a lot of visual activity and eye movements. Eye movements are best characterized by visual fixation – periods of time when the eyes stay relatively still over an object. We analyzed the distributions of fixation duration of professional athletes, amateur and newbie players. We show that the analysis of fixation durations can be used to deduce the skill level in computer game players. Highly skilled gaming performance is characterized by more variability in fixation durations and by bimodal fixation duration distributions suggesting the presence of two fixation types in high skill gamers. These fixation types were identified as ambient (automatic spatial processing) and focal (conscious visual processing). The analysis of computer gamers’ skill level via the analysis of fixation durations may be used in developing adaptive interfaces and in interface design.

Traditional media such as television are increasingly adopting interaction logic as a way to engage viewers. In this paper, we report on the design of a smartphone application that facilitates the participatory production process of #TheDay, a two-minute item in an infotainment TV show, which is broadcasted twice a week on national television. #TheDay leverages a co-creation approach as it allows viewers to submit and discuss self-produced video content, this way empowering them to share their take on topics imposed by TV producers. We report on #TheDay’s eight-month deployment. Through a mixed methods approach, we learn how the TV producers succeeded in triggering viewer interaction, also on a long-term. From our findings, we distill three considerations for the design of interactive applications that support professional producers in deploying participatory forms of media production with sustained user commitment.

Inspired by the Maker Movement and attempts of integrating Making into formal education of children, we have examined how practice-oriented the research on Making and education is in Child-Computer Interaction field. Our results show that despite the growing interest practice-orientation is still weak. Making efforts rarely aim for longitudinal durable patterns and practices even though practice-driven research has started to gain prominence in the whole Human-Computer Interaction field. General ideas of what Making integrated with education can be and should be seem to be well shared among practitioners, however. We demonstrate what aspects should be considered when doing practice-oriented Making research and point out gaps in our current understanding of the practice. We also provide guidelines for how to study Making in order to develop sustainable practices.

Matching crowd workers to suitable tasks is highly desirable as it can enhance task performance, reduce the cost for requesters, and increase worker satisfaction. In this paper, we propose a method that considers workers’ cognitive ability to predict their suitability for a wide range of crowdsourcing tasks. We measure cognitive ability via fast-paced online cognitive tests with a combined average duration of 6.2 min. We then demonstrate that our proposed method can effectively assign or recommend workers to five different popular crowd tasks: Classification, Counting, Proofreading, Sentiment Analysis, and Transcription. Using our approach we demonstrate a significant improvement in the expected overall task accuracy. While previous methods require access to worker history or demographics, our work offers a quick and accurate way to determine which workers are more suitable for which tasks.

While digital technology adoption by older adults rises constantly, the design of new technologies often overlooks the culture of the end users, which, in turn, has an impact on the acceptance and use by many of them. Based on the fact that technology adoption by this age group in Greece remains low, compared to the vast majority of the EU countries, our goal in this paper is to gain further insights into the user requirements of older adults as web 2.0 storytellers in order for designers to better address their needs. For this purpose, we implemented participatory design with five older adults in Greece over a twelve-week period, combined with an evolutionary prototyping approach, as we noticed during our sessions that our participants had a difficulty in envisioning and proposing novel technologies. In order to analyze and interpret the feedback that we collected during the design sessions, the digital storytelling sessions and the in-depth interviews, we made use of the Activity Theory, as well as age-related and self-presentation frameworks. Through probing our participants’ usage and design preferences of the storytelling platform, we came up with a set of attributes and motives that seem to expound their online choices. Issues of loneliness and social inclusion, generativity and computer mediated communication, among others, have emerged. Additionally, differences and similarities with findings from other studies have been indicated.

The design of visual analytics tools for facilitating human-data interaction (HDI) plays a key role to help people identifying useful knowledge from large masses of data. Designing data visualization based on guidelines is relevant. However, it is necessary to further promote the engagement of people in evaluation activities in the design process. Stakeholders need to comprehend the guidelines to help with the evaluation results and design decisions. In this paper, we propose participatory evaluation practices based on HDI design guidelines. The practices aim to create the conditions to participants from any profile collaborate with the design guidelines evaluation. The practices were used on a design problem involving interactions with coordinated visualization. The context of application was a visual analytic tool supporting decisions related to the production strategy in oil reservoirs with the participation of key stakeholders. The results indicate that participants were able to understand the design guidelines and took advantage from them in the design decisions.

Visualization tools are critical components of cyber security systems allowing analyzers to better understand, detect and prevent security breaches. Security administrators need to understand which users accessed the database and what operations were performed in order to detect irregularities. The current work compares the Sankey diagram with the more commonly used node-link diagram as an alternative visualization technique for cyber security tasks in a controlled experiment. The results indicate, that the Sankey tool showed a consistent advantage in task completion time and was more effective (measured by the percent of correct answers) in synoptic tasks, while the Node-link diagram was more effective in basic, elementary tasks. Further results revealed that performance had only a small effect on user satisfaction and preferences. Our results suggest that the Sankey tool may be a viable option for cyber security visualization tools and strengthens the need to provide personalized visualization tools based on user preferences.

Internet-enabled voting introduces an element of invisibility and unfamiliarity into the voting process, which makes it very different from traditional voting. Voters might be concerned about their vote being recorded correctly and included in the final tally. To mitigate mistrust, many Internet-enabled voting systems build verifiability into their systems. This allows voters to verify that their votes have been cast as intended, stored as cast and tallied as stored at the conclusion of the voting period. Verification implementations have not been universally successful, mostly due to voter difficulties using them. Here, we evaluate two cast as intended verification approaches in a lab study: (1) “Challenge-Based” and (2) “Code-Based”. We assessed cast-as-intended vote verification efficacy, and identified usability issues related to verifying and/or vote casting. We also explored acceptance issues post-verification, to see whether our participants were willing to engage with Internet voting in a real election. Our study revealed the superiority of the code-based approach, in terms of ability to verify effectively. In terms of real-life Internet voting acceptance, convenience encourages acceptance, while security concerns and complexity might lead to rejection.

Phishing attacks are one of the most common security challenges faced by individuals and organizations today. Although many techniques exist to filter out phishing emails, they are not always effective leaving humans as the most vulnerable links in the information security chain. This paper presents a study investigating how human behavior, especially mouse movements, may reflect cybersecurity awareness, in particular to phishing emails. Using an email sorting task, we examined three key mouse movement features: hover, slow movement, and response time. The results suggest that slow mouse movements indicate high awareness of phishing emails and could be used to determine the likelihood of users falling victim to phishing attacks. However, contrary to intuition, response time and mouse hovering behaviors do not correlate with phishing awareness.

Passwords remain the most common form of authentication in the digital world. People have increasing numbers of passwords, and research suggests that many people undertake risky password management behaviours such as re-using passwords, writing them down and sharing them with friends and colleagues. It is not clear whether people persist in these behaviours because they do not understand the risks involved or the benefits of the behavior outweigh the risk. An online survey was undertaken with 120 MTurk workers in which they rated the risks, benefits and likelihood of undertaking 15 password management behaviours. They also completed the Marlow-Crowne Social Desirability Scale to investigate whether their responses, particularly of the likelihood ratings, were affected by social desirability. An interesting pattern of responses was found with some groups of behaviours more affected by perceptions of the benefits and others equally affected by the perceptions of the risks and the benefits. These results have implications for how information about risky password behaviours in presented to users and general education about password security.

Phishing emails are a widespread cybersecurity attack method. Their breadth and depth have been on the rise as they target individuals and organisations with increased sophistication. In particular, social engineering in phishing focuses on human vulnerabilities by exploiting established psychological and behavioural cues to increase the credibility of phishing emails. This work presents the results of a 56,000-participant phishing attack simulation carried out within a multi-national financial organisation. The overarching hypothesis was that strong cultural and contextual factors impact employee vulnerability. Thus, five phishing emails were crafted, based on three of Cialdini’s persuasion principles used in isolation and in combination. Our results showed that Social proof was the most effective attack vector, followed by Authority and Scarcity. Furthermore, we examined these results in the light of a set of demographic and organisational features. Finally, both click-through rates and reporting rates were examined, to provide rich insights to developers of cybersecurity educational solutions.

During discussions in collocated work it is necessary to vote for results or to rate them to reach an agreement and continue working. To ensure impartiality and to avoid social embarrassment, the assessment should then be performed anonymously in so far as other groups members should not see directly how a person votes or rates. With a growing number of digital devices in collaboration, this requirement also concerns such kinds of equipment. Our approach of ensuring anonymity of individual votes and ratings submitted on personal mobile phones is to avoid shoulder surfing activities. For this purpose, we designed four device-based interactions that aim at being easy to use and eyes-free to perform to stay in touch with the environment and potential shoulder surfers. We conducted a user study to investigate these interaction techniques and observed seven groups with four participants each while testing the interactions. Participants evaluated usability and User Experience (UX) aspects as well as unobtrusiveness of the four device-based interactions. Furthermore, participants gave valuable user feedback and stated that our proposed interactions help to avoid shoulder surfing.

Designing for experience requires designers to pay attention to reflection on design impact. However, industrial design students are observed to have difficulty in demonstrating the impact of their design concepts due to insufficient thinking and reflection on design impact. There is a lack in the literature on both reflections on design impact and the tool support. The existing tools for general reflection purpose seem not to work well for this specific purpose. In response to the calls for designing for reflection, this paper presents two exploratory studies and the design requirements of tools for reflection on design impact. The purpose aims to facilitate design students for their reflection on design impact through developing appropriate tools. The design requirements could be generally used as guidelines or reference for future work of developing tools for reflection on design impact.

We present an approach for the semi-automatic generation of gesture mappings for devices with low gestural resolution such as the Myo Armband, an off-the-shelf EMG capture device. As an exemplar interactive task, we use text-entry: a pervasive and highly complex interaction. We quantify data related to interaction combining systematic studies (i.e., error, speed, accuracy) and semi-structured workshops with experts (e.g., cognitive load, heuristics). We then formalize these factors in a mathematical model and use optimization algorithms (i.e. simulated annealing) to find an optimum gesture mapping. We demonstrated our method in a text-entry application (i.e., complex interactive dialogue) comparing our approach with other computationally determined mappings using naive cost functions. Our results showed that the designers mapping (with all factors weighted by designers) presented a good balance on performance in all factors involved (speed, accuracy, comfort, memorability, etc.), consistently performing better than purely computational mappings. The results indicate that our hybrid approach can yield better results than either pure user-driven methodologies or pure data-driven approaches, for our application context featuring a large solution space and complex high-level factors.

In a user-centered design process, graphical user interface (GUI) prototypes may be seen as an important early artifact to design and validate user requirements before making strong commitments with a full-fledged version of the user interface. Ensuring the consistency of GUI prototypes with other representations of the user requirements is then a critical aspect of the design process. This paper presents an approach which extends Behavior-Driven Development (BDD) by employing an ontology in order to provide automated assessment for GUI prototypes as design artifacts. The approach has been evaluated by exploiting user requirements described by a group of experts in the flight tickets e-commerce domain. Such requirements gave rise to a set of User Stories that have been used to automatically check the consistency of Balsamiq prototypes which were reengineered from an existing web system for booking business trips. The results have shown our approach was able to identify different types of inconsistencies in the set of analyzed artifacts, allowing to build an effective correspondence between user requirements and their representation in GUI prototypes.

Multidisciplinary design is characterized by phases of distributed work and co-design activities. An effective sharing and integration of design representations that are created by sub-teams from different disciplines is still often challenging and typically requires the reconciliation of diverging design perspectives. This paper investigates an integrated use of personas and use cases - two popular types of design representations among interaction designers and software engineers respectively. The proposed integration is particularly suitable for role-based interactive systems and differs from existing integration approaches in that it is based on a critical examination of the prevalent understandings of the goal concept in persona and use case approaches. In the paper we suggest distinguishing between organizational and user goals (while at the same time acknowledging their interplay). Corresponding adaptations to use case notations and personas are introduced and discussed. These remove the tight coupling between goals and tasks and allow integration of organizational and different persona-specific design perspectives within one use case specification and at the interaction level. As a result, interactive systems can be specified by a more compact sets of use cases. This is illustrated by an example in the context of course management systems in higher education.

The emerging infrastructure of cyber-physical systems consisting of everyday items, as product’s packaging, and advanced digital communication devices opens a new digital dimension for interaction and user experience. Consequently, the concept of human-packaging interaction goes beyond the pragmatic aspects of physical packaging attributes, and, in turn, embraces the potentials of ICT systems. Due to the new forms of human-packaging interactive systems, designers have to address the relevancy of the interaction design and the complexity in the relationship between consumer behavior and interactive system design, i.e. digitally-enhanced packaging. Therefore this research aims to describe the digitally-enhanced packaging as a digital interactive system in regards to the theories of human-computer interaction, interaction design, and user-centered design. In this paper, the critical elements of the interactive packaging design are described. This study concludes that for the interactive systems to be effective and used, designers have to build not only a reward-based, intuitive and simple interaction design that would persuade users to take actions, but also they have to think of other mediate interactions, internal and external resources that are significant to reach the final aim.

Command and control systems centralize information from multiple underlying systems to support operators in the performance of their mission. Beyond the mission itself (that may be complex), operators must also ensure the correct functioning of these systems (often called platform). Platform systems (e.g. engines or electric system) may be very different from each other and exhibit a large number of functional states. When applied to the design of command control systems, User Centered Design methods support understanding and capturing operators’ needs to perform the mission, as well as to propose solutions to design usable mission-related user interfaces. However, user interfaces for platform management need to present and organize information about the underlying complex systems. Understanding those systems and abstracting away information about their behavior (so that operators can manage them) requires deep knowledge beyond UI/UX designers and UCD methods experts. In this paper, we propose a system-centered process that would complement UCD approaches for the design of command and control systems. That process takes as input the detailed functioning of underlying systems and provides abstract and structured information to inform UCD methods. Beyond supporting usability property, the integrated process supports reliability and safety properties that UCD approaches usually overlook. We present how the proposed process has been applied for the design of a large civil commercial aircraft warning system and show generalizability to other domains.

Driving is a sensitive task that is strongly affected by the driver’s emotions. Negative emotions, such as anger, can evidently lead to more driving errors. In this work, we introduce a concept of detecting and influencing driver emotions using psycho-physiological sensing for emotion classification and ambient light for feedback. We detect arousal and valence of emotional responses from wearable bio-electric sensors, namely brain-computer interfaces and heart rate sensors. We evaluated our concept in a static driving simulator with a fully equipped car with 12 participants. Before the rides, we elicit negative emotions and evaluate driving performance and physiological data while driving under stressful conditions. We use three ambient lighting conditions (no light, blue, orange). Using a subject-dependent random forests classifier with 40 features collected from physiological data we achieve an average accuracy of $$78.9\%$$ for classifying valence and $$68.7\%$$ for arousal. Driving performance was enhanced in conditions where ambient lighting was introduced. Both blue and orange light helped drivers to improve lane keeping. We discuss insights from our study and provide design recommendations for designing emotion sensing and feedback systems in the car.

Operating heavy vehicles, for instance an excavator, requires a high level of attention to the operation done using the vehicle and awareness of the surroundings. Digital transformation in heavy vehicles aims to improve productivity and user experience, but it can also increase the operators mental load because of a higher demand of attention to instrumentation and controls, subsequently leading to reduced situation awareness. One way to mitigate this, is to display information within the operators’ field of view, which enhances information detectability through quick glances, using mixed reality interfaces. This work explores two types of mixed reality visualizations and compares them to a traditional display setup in a simulated excavator environment. We have utilized eye-tracking glasses to study users’ attention to the task, surrounding awareness, and interfaces, followed by a NASA-RTLX questionnaire to evaluate the users’ reported mental workload. The results indicate benefits for the mixed reality approaches, with lower workload ratings together with an improved rate in detection of presented information.

Commercial Virtual Reality (VR) controllers with realistic force feedback are becoming available, to increase the realism and immersion of first-person shooting (FPS) games in VR. These controllers attempt to mimic not only the shape and weight of real guns but also their recoil effects (linear force feedback parallel to the barrel, when the gun is shot). As these controllers become more popular and affordable, this paper investigates the actual effects that these properties (shape, weight, and especially directional force feedback) have on performance for general VR users (e.g. users with no marksmanship experience), drawing conclusions for both consumers and device manufacturers.We created a prototype replicating the properties exploited by commercial VR controllers (i.e. shape, weight and adjustable force feedback) and used it to assess the effect of these parameters in user performance, across a series of user studies. We first analysed the benefits on user performance of adding weight and shape vs a conventional controller (e.g. Vive controller). We then explore the implications of adding linear force feedback (LFF), as well as replicating the shape and weight. Our studies show negligible effects on the immediate shooting performance with some improvements in subjective appreciation, which are already present with low levels of LFF. While higher levels of LFF do not increase subjective appreciations any further, they lead users to reach their maximum distance skillset more quickly. This indicates that while adding low levels of LFF can be enough to influence user’s immersion/engagement for gaming contexts, controllers with higher levels of LFF might be better suited for training environments and/or when dealing with particularly demanding aiming tasks.

Creativity, a primary objective across academic disciplines, has received considerable attention over the past few decades. While much focus has been put on the measurement of individual creativity, a notable research gap remains regarding social collaborative creativity that occurs in blended learning settings. This work offers an initial validation of the psychometric properties of a self-reported instrument, the Assessment Scale for Creative Collaboration (ASCC) that can measure learner perceptions of creative collaboration in a team within a computer-supported collaborative learning (CSCL) context. In this study, 236 undergraduate and graduate students rated the key variables of creative collaboration. Exploratory factor analysis resulted in a three-factor scale (21 items) measuring ‘Synergistic Social Collaboration’, ‘Distributed Creativity’ and ‘Learning Regulation and Achievement’. Cronbach’s alphas indicated good internal consistency for the subscales. An instrument with psychometric properties for the assessment of creative collaboration is much-needed for the growing community of researchers and practitioners looking into creativity in education. It is also critical in advanced technical subjects, such as Design, HCI and Engineering, where collaboration is essential in developing innovative products.