Automotive User Interfaces

The history of automotive HMI development reveals that the development of new interactive in-car functionalities (such as infotainment systems) has often been influenced by upcoming new technologies that customers got used to in their daily lives. Examples of such technologies include the first in-car radio, which was introduced around 1922, or the first in-car phone, which was introduced around 1952. Today, a car without such functionalities is hard to imagine and the automotive industry is aiming to develop and integrate more and more innovative functionality to stay competitive on the market. The development of such functions is motivated by the construction of safer, more efficient, and more comfortable vehicle systems. Current trends in the area of in-car infotainment applications include, for instance, Internet-based applications or social network applications, whereas extendable, hybrid, adaptive, or even personalized HMI are emerging as future trends. Not only technologies have evolved throughout history, the development processes themselves have also been adapted continuously due to the challenges the automotive industry had to face with new technologies. Thus, the authors have summarized their experiences, their knowledge, and the results of literature studies in this article which covers the history of automotive HMI development from the past in 1922 to the present with an outlook on upcoming trends for future automotive user interfaces.

Regardless of punitive strategies such as fines and demerit points, drivers continue to bring their own devices into cars and use them while driving. In this chapter, we explore the opportunities for gamified safe-driving apps provided by real-time data gathered from mobile and wearable devices. The study is grounded in our interest in providing engaging experiences for drives that otherwise lack engagement, both in manual and semi-automated vehicles. We developed BrakeMaster, a smartphone app built around vehicle and road data, and evaluated it in a simulator study looking at system performance, usability, and affect. We found the app to perform responsively and accurately, and self-reported data indicate good usability and increased pleasure. Besides exploring vehicle and road data, we investigated wearable activity monitors for gathering driver data such as arousal. Consumer wearables are more cost and size effective than advanced biofeedback systems and are capable of revealing heart rate patterns and trends across drives. We conclude that road and particularly vehicle data can be leveraged to develop novel driving experiences, whereas driver data is more challenging to exploit in this unique design context.

In connection with the automobile, user experience and emotion have always been contributing to a unique selling proposition and thus an important basis for the development of the product. In addition, requirements of traffic safety and usability have of course be taken into account. The corresponding trade-offs are not easy to solve; yet, existing premium products show that this is possible. While road safety provides a clear framework through basic requirements and regulations, the usability considered the interaction of a person with a technical system for a specific task in a given context, for example a navigation device in a motor vehicle. In this context, efficiency and satisfaction are optimized effectiveness. In addition, the emotional experience of users, as joy of use or user experience gain increasing importance. How these experiences can be translated into customer experiences in combination with current technology trends, for example in the area of perception of acceleration, electric mobility or automated driving is described in this chapter.

Electric cars (EVs) are a promising alternative to combustion engine cars to lower emissions and fossil fuel dependencies. On the downside, in comparison to internal combustion engine cars (ICE), the user experience (UX) of EVs is seriously compromised due to shorter and more varied driving range depending on driving style other context of use. A further complication is that recovering from unexpectedly low battery levels is tedious due to long charging times. This causes range stress among drivers and research has highlighted a need to improve the information and tools available in order for drivers to better understand range-influencing factors and estimations, leading to increased reliability, and trust in the information. This currently leads to poor UX that may shadow all the benefits and other important environmental and experiential qualities of electric cars. In this chapter, we will provide an introduction to the subject and go through some of our studies and key lessons that have emerged from our research. In particular, we have come to the realisation that we need to energy-empower electric car drivers in order for them to be able to conceptualise how energy is intertwined with their actions and behaviour while driving. This is important, as current tools fail to provide such empowerment, causing unnecessary surprises and worries among the drivers who call the standard tool available in the electric car for the ‘guess-o-meter’. Through our designs and discussion we demonstrate how some aspects might be addressed to energy empower electric car drivers.

After explaining the motivation and presenting related experiences, an extended Usage-Centered Design approach that integrates standardized process activities from User-Centered Design approach (defined in ISO 9241-210) and uses cultural models is suggested and simultaneously it is also adapted to ASPICE Standard so that the approach is suitable for the design of intercultural user interfaces/experiences in the automotive context. This agile oriented approach makes it possible to track and trace both the culture specific requirements and the design decisions for internationalized HCI in order to produce adequate cultural interaction experiences for users of automotive user interfaces in the car.

When designing interfaces for a vehicle, the focus often lies on the driver. Since the driver always has a primary task (i.e., maneuvering the vehicle safely), interfaces for secondary tasks (e.g., entertainment systems) are designed to reduce distraction threats to a minimum. However, it is not always only the driver who is interacting with the vehicle; passengers also interact with the car. They may support the driver in the primary task (e.g., by providing navigation information) or take over secondary tasks (e.g., operating the climate control) in order to unburden the driver. Thus, we see a need for interfaces that foster the communication and collaboration between the driver and passengers but also among passengers themselves. Currently, such interfaces are usually neglected in automotive user interface research. Over the last years, we have conducted several studies focusing on communication and collaboration between drivers and passengers inside cars. Following an experience-centered approach, we started with ethnographically informed studies to gain a deeper knowledge on how drivers and passengers interact with each other inside a vehicle. Based on these insights we conceptualized and designed several prototypes that enabled collaboration between drivers and passengers. These prototypes were then studied in different studies both in a simulator setup, as well as, in real-traffic situations. In this chapter, we describe five of these research activities in more detail and present implications for designing interaction approaches that foster collaboration in the vehicle.

Today, drivers perform many non-driving-related activities while maneuvering the car. To ensure driving safety, the designers of automotive UIs have to respect the driver’s available cognitive, perceptual and motor resources to prevent overload and in turn accidents. In this chapter, we look at the different types of driver resources, how they are loaded and limited by the primary driving task, and how this affects the resources available for non-driving-related activities. We discuss aspects such as attention, driver distraction, (cognitive) workload, and other factors such as the driver’s physical and mental state to understand the limitation of the driver’s resources and how non-driving-related activities affect the primary task performance. To enable the safe execution of non-driving-related activities, we need to design the cockpit and its UI in such a way that it requires a minimal amount of resources. We will provide an outlook towards selected novel technologies such as large head-up displays and also discuss expected effects of the transition to automated driving.

The driver’s focus of attention is a key factor to be considered for building novel, intuitive user interaction concepts, and enhancing the current infotainment and safety applications in the vehicle. In this chapter we present several topics related to the development of application and systems that incorporate the user’s visual focus of attention. In the presented real-life experiments, 3D representations of both the vehicle’s interior and the outside environment are used. A real-time evaluation concerning the object in the driver’s visual focus in these environments is also performed. We describe the functionality and the accuracy of the presented systems, which is integrated in a fully functional vehicle in an actual traffic setting. In addition, several analyses concerning accuracy of the off-the-shelf eye trackers regarding peripheral vision or direct interaction with urban objects are presented.

As long as automated vehicles are not able to handle driving in every possible situation, drivers will still have to take part in the driving task from time to time. Recent research focused on handing over control entirely when automated systems reach their boundaries. Our overview on research in this domain shows that handovers are feasible, however, they are not a satisfactory solution since human factor issues such as reduced situation awareness arise in automated driving. In consequence, we suggest to implement cooperative interfaces to enable automated driving even with imperfect automation. We recommend to consider four basic requirements for driver–vehicle cooperation: mutual predictability, directability, shared situation representation, and calibrated trust in automation. We present research that can be seen as a step towards cooperative interfaces in regard to these requirements. Nevertheless, these systems are only solutions for parts of future cooperative interfaces and interaction concepts. Future design of interaction concepts in automated driving should integrate the cooperative approach in total in order to achieve safe and comfortable automated mobility.

Given the rapid advancement of technologies in the automotive domain, driver--vehicle interaction has recently become more and more complicated. The amount of research applied to the vehicle cockpit is increasing, with the advent of (highly) automated driving, as the range of interaction that is possible in a driving vehicle expands. However, as opportunities increase, so does the number of challenges that automotive user experience designers and researchers will face. This chapter focuses on the instrumentation of sensing and displaying techniques and technologies to make better user experience while driving. In the driver--vehicle interaction loop, the vehicle can sense driver states, analyze, estimate, and model the data, and then display it through the appropriate channels for intervention purposes. To improve the interaction, a huge number of new/affordable sensing (EEG, fNIRS, IR imaging) and feedback (head-up displays, auditory feedback, tactile arrays, etc.) techniques have been introduced. However, little research has attempted to investigate this area in a systematic way. This chapter provides an overview of recent advances of input and output modalities to be used for timely, appropriate driver--vehicle interaction. After outlining relevant background, we provide information on the best-known practices for input and output modalities based on the exchange results from the workshop on practical experiences for measuring and modeling drivers and driver--vehicle interactions at AutomotiveUI 2015. This chapter can help answer research questions on how to instrument a driving simulator or realistic study to gather data and how to place interaction outputs to enable appropriate driver interactions.

Informing a driver of a vehicle’s changing state and environment is a major challenge that grows with the introduction of in-vehicle assistant and infotainment systems. Even in the age of automation, the human will need to be in the loop for monitoring, taking over control, or making decisions. In these cases, poorly designed systems could lead to needless attentional demands imparted on the driver, taking it away from the primary driving task. Existing systems are offering simple and often unspecific alerts, leaving the human with the demanding task of identifying, localizing, and understanding the problem. Ideally, such systems should communicate information in a way that conveys its relevance and urgency. Specifically, information useful to promote driver safety should be conveyed as effective calls for action, while information not pertaining to safety (therefore less important) should be conveyed in ways that do not jeopardize driver attention. Adaptive ambient displays and peripheral interactions have the potential to provide superior solutions and could serve to unobtrusively present information, to shift the driver’s attention according to changing task demands, or enable a driver to react without losing the focus on the primary task. In order to build a common understanding across researchers and practitioners from different fields, we held a “Workshop on Adaptive Ambient In-Vehicle Displays and Interactions” at the AutomotiveUI‘15 conference. In this chapter, we discuss the outcomes of this workshop, provide examples of possible applications now or in the future and conclude with challenges in developing or using adaptive ambient interactions.

The steering wheel is—besides pedals for acceleration and breaking—the most prominent interaction artifact between drivers and their vehicles. All cars have a steering wheel, which translates steering instructions from drivers to cars. “Eyes on the road and hands on the wheel!” is one of the most prominent paradigms in the automotive world. The driver should always have a grip of the steering wheel, making it also the most reachable area in the car for manual interaction. Automotive interaction designers have, rightly, used the area on and around the steering wheel to position interaction elements beyond steering. Today’s cars are cluttered with buttons and switches to operate the car’s information and entertainment system. New interaction modes, such as touch screens on the steering wheel or shape changing rims offer interaction designers new perspectives on utilizing the steering wheel, as a means for interaction with the vehicle. In this chapter, we describe the design space steering wheels offer for interaction beyond steering the vehicle. We collect and analyze various approaches from industry and academia on human-steering wheel interaction beyond traditional interaction and infer potentials and risks when utilizing such novel modalities in terms of interaction design. This analysis leads to a thorough discussion of the steering wheel interaction design space, resulting in related interaction design recommendations. Finally, we provide a look into the future when evermore advanced driving assistance systems pervade the car, eventually relieving the driver from the steering task with the emergence of autonomous vehicles.

In-vehicle experiences are made up mainly of mundane small moments, repeated practices, and taken-for-granted decisions that make up daily experiences in and around private passenger vehicles. Understanding what those experiences are for drivers around the world presents an opportunity for designing novel interactive experiences, technologies, and user interfaces for vehicles. In this chapter, we present a set of tools, methodologies, and practices that will help reader create a holistic design space for future mobility. Transitioning between ethnography, insights, prototyping, experience design, and requirements decomposition is a challenging task even for experienced UX professionals. This chapter provides guidance in this matter with practical examples.

This chapter presents the development of an innovative and interactive 3D virtual reality driving simulator based on head-mounted displays, which gives the driver a near-realistic driving experience for the development and evaluation of future automotive HMI concepts. The project explores the potentials and implementation of virtual reality in the automotive sector for the analysis of new HMI concepts and safety functions in the automotive sector. Special emphasis is laid on hazardous situations which are ethically not possible to evaluate on a real road at the early stage of the concept, when the risk involved for both the driver and the prototype, for example driver distraction and autonomous vehicle studies is not yet ascertained. The 3D virtual reality approach was meant to overcome some of the limitations of conventional 3D driving simulators, such as lack of total immersion and intuitive reaction of the test driver, necessary for an effective analysis of a particular driving situation. The sense of presence offered by virtual reality is essential for the research and evaluation of safety functions, since appropriate and reliable solutions are only possible when the problem associated with a particular traffic situation is well understood. The focus was on the following aspects: 3D modeling, correct simulation of vehicle and traffic models, and integration of a motion platform to give the feel of a real car and control devices and finally, head-up display use cases. Finally, the solutions to eliminate simulation sickness were reviewed and implemented. A prototype was developed which displays dynamic head-up-display features.

To ensure safety and usability of automotive user interfaces, prospective validations during early prototyping stages are important, especially when developing innovative human-cockpit interactions (HCI). Real car driving studies are difficult to control, manipulate, replicate, and standardize. Additionally, compared to other study designs, they are also time consuming and expensive. One economizing approach is the implementation of immersive driving environments in simulator studies to provide users a more realistic awareness of the situation. Using simulator test environments puts the question of driving simulator validity forward, meaning the extent to which results generated in simulated environments can be transferred to real world environments. Thus, in this chapter the ‘Immersive model-based HCI validation method’, which was developed by the authors, will be introduced. First, the state of the art of driving simulators will be analyzed. For this, the authors defined the degree of fidelity based on the used elements. Next, findings of a series of driving simulator tests will be presented, which investigate the influence of immersive parameters in driving environments. Visual and auditory immersive parameters were used to analyze the validity of driving simulator environments, as well as different technologies (HMD, holobench, PC). Different levels of immersion (from low to high fidelity) were tested to examine this methodology. Thus, main intention was to demonstrate the generalizability and transferability of the ‘Immersive model-based HCI validation method’ for different use cases. Objective and subjective data show advantages regarding the situational awareness and perception for highly immersive driving environments while interacting with a navigation system.

The long awaited arrival of automated driving technology has the automotive industry perched on the precipice of radical change when it comes to the design of vehicle interiors and user experience. Recently, much thinking and many vehicle concepts have been devoted to demonstrating how vehicle interiors might change when vehicles reach full automation, where a human driver is neither required nor in some cases, even allowed to control the vehicle. However, looking more near term across all global market segments, we will likely see an increasing number of vehicles with widely varying automation capabilities emerging simultaneously. Any system short of full automation will still require driver control in some set of situations, and some fully automated vehicles will still allow driver control when desired. While it is unlikely that the basic seating arrangement, steering wheel, and pedals will be radically altered in this emerging segment of partial to highly automated vehicles, it is quite clear that the overall user experience during automated driving will need to evolve. Drivers will not be content to hold the steering wheel and stare at the road waiting for what may be a very infrequent request to take-over driving. The chapter presents the research conducted to develop the Valeo Mobius® Intuitive Driving solution for providing an embedded digital experience, even in lower levels of automation, and all while still promoting both shorter transition response times and better transition quality when emergency situations call for a transition from automated to manual control.

The AutoPlay prototypes have been designed to explore the implementation of non-driving activities into the context of a future autonomous driving situation. The conceptual design goal was to maintain a pleasurable situational awareness of the inactive driver by integrating the driving context as a meaningful input into the interaction system. In this chapter, we introduce the design of three experimental applications for autonomous driving and report on explorative user studies conducted to investigate the impact of the three AutoPlay prototypes: AutoGym, an in-car exertion game that translates car speed and traffic situations into an individual exercise program. AutoJam, a touch sensitive steering wheel cover to generate interactive music experiences in a creative interplay with car’s driving dynamics. AutoRoute, a discovery application for future urban commuting in autonomous cars that enables an exploration of the city based on spontaneous routing and rerouting. Furthermore, we reflect on the outcome of the user studies and propose three motivational affordances of autonomous driving: drivability, performability, and explorability. Each of these concepts, help to understand the motivational possibilities of the autonomous driving situation and facilitates a meaningful alignment of interaction systems and the driving context. We discuss the underlying concepts of the three affordances by relating them to the experiences identified in the user studies. Subsequently the contribution of this chapter is twofold: (1) We introduce the AutoPlay prototypes as inspirational concepts for aligning non-driving activities with the autonomous driving context and (2) we propose three motivational affordances as design targets for the implementation of non-driving activities in order to initiate a broader discussion on the pleasures of autonomous driving beyond instrumental motives.