Can eye-tracking data be measured to assess product design?: Visual attention mechanism should be considered

Highlights

• Eye movement is investigated when products with different user experience is displaying.

• Product with higher user experience can evoke shorter time to first fixation and smaller pupil size in the browsing task.

• Product with higher user experience can evoke larger pupil size and longer fixation time in the evaluation task.

• The effect of visual attention mechanism should be considered when using eye-tracking to assess product designs.

Abstract

Eye tracking probes user's perception of real-time reaction to products, while conventional methods (i.e. interviews, focus group, questionnaires and so on) have generally failed because they depend on users' willingness and competency to describe how they feel when they are exposed to a product. Two tasks were designed to explore the indexes of eye movement that can reflect user experience of product, and analyse the attention captured by product attributes and goal-oriented. In task one, participants just browsed two smart phone pictures and evaluated the whole user experience. Binary choices were used in task two to ask participants to select the smart phone picture with higher user experience and then click the mouse. The results showed that in the browsing task, participants had shorter time to first fixation for the smart phone picture with higher level of user experience than the lower. And pupil dilated significantly when participants browse smart phone picture with lower level of user experience. In goal-oriented task, participants' attentions were dominated by visual perception of task driven, mainly reflected on longer fixation time and larger pupil diameter when looking at the smart phone with higher level of user experience. These results support the notion that we cannot assess product design just by several eye-movement indexes without considering the effects of visual attention mechanism.

Relevance to industry

The appearance of product plays an important role to attract user's attention and stimulate their intention to experience. And vision is the main channel for users to obtain product information. Hence a thorough research on the inherent mechanism of vision perception can provide technical support for product designers, which in turn can attract more consumers to experience the product, even buy it. Moreover, the seller can find out the real buyers and predict their desired products by tracking user's eyes.

Introduction

Recently, user experience has received special attention from HCI researchers, and be manifested as a quality of product design (Law and van Schaik, 2010). This experience is recognized that user's needs go beyond functionality, and shift to experiential perspectives, not only contain usability, but socio-cognitive and affective aspects of user experience such as pleasure, aesthetic experience, desire to reuse, positive feelings interacting with products and so on (Law and van Schaik, 2010, Hassenzahl and Tractinsky, 2006). The whole experience takes behaviour, physiological reactions and psychological activities into account. A very official definition comes from ISO DIS 9241-210 (2010): “a person's perceptions and responses resulting from the anticipated use and/or use of a product”. Law et al. (2014) pointed out that user experience measurement and its multiple definitions are two foundational issues and should be solved in tandem. But user experience subsumes a range of fuzzy, vague, elusive and ephemeral experience such as affective aspects, potential and intuitive feelings, or even without rational reasoning and hard to describe with words, controversies and doubts about the definition and measurability of user experience are inevitable (Law et al., 2014).

There are so many researchers trying to explore what user experience are (Hassenzahl and Tractinsky, 2006, Scapin et al., 2012). However, give a definition of user experience to make researches reach a consensus is beyond the scope of the research objectives. In this study, we adopt the definition by ISO DIS 9241-210 (2010), the process of user experience can be divided into before purchasing and after. This paper is trying to explore the measurement method of user experience before purchasing. How to arouse user's desire to experience with the product, and then evoke the intention to interact with the product, so further behavioural could be occurred. As Norman (2004) pointed out that, only if a product catch user's first sight will “what is it?” and then “how much is it” happen. The user experience before purchasing is more important than purchase decision about product price in policy-making (Ho and Lu, 2014). Hence, a thorough research of user experience measurement in this process is critical for product design.

Law and van Schaik (2010) pointed out that current methods, techniques and tools for evaluating and measuring user experience are mainly drawn from usability researches. Questionnaires and scales containing aspects of products such as usability, emotion, and aesthetic and so on are one of the most versatile (Vermeeren et al., 2010). Then the overall user experience score was evaluated by assigning to the measures with different weights (Law and van Schaik, 2010). Moreover, data from traditional methods are easily affected by the respondent's surroundings, voluntary participation, difficulty of comparisons among product, false feeling of their inner state, or even give stated opinions object wanted (Ariely and Berns, 2010). Hence, the results of such a survey will not convey the truth information, and then mislead the direction of design (Ho and Lu, 2014). There even exists bias in the results, because users cannot recall their interactions and feelings with products exactly (Guo et al., 2015). Moreover, there are various elements and composing and affecting user experience in a certain context, such as usability of a product, affect aroused by product and user value attached to a product (Park et al., 2013). But user experience is highly subjective, dynamic and environment-dependent (Law and van Schaik, 2010). Thus quantifying overall user experience for a certain product is quite difficult (Park et al., 2013), for its dynamic with different product features and inner state.

Based on this motivation, several studies have been done to measure user experience from physiological aspects. This is mainly driven by the fact that people cannot fully explain their perception or preferences when explicitly asked; as human behaviour can be driven by processes operating below the level of conscious awareness (Calvert and Brammer, 2012). Wang and Minor (2008) summarized the validity, reliability and applicability of physiological techniques in marketing research, including eye movement, heart rate, blood pressure, facial muscle activity, voice pitch analysis and brain imaging. In such cases, the effectiveness of product form design may be evaluated by monitoring people's physiological activities resulting from observing/using different products (Khushaba et al., 2013, Ohme et al., 2009).

A user's perceptions and responses to a product are mainly affected by the product's appearance (Ho and Lu, 2014). In general, vision is the first channel to obtain information, and directly impact user's future behaviour and intention to experience with the product (Moshagen and Thielsch, 2010). Thus, product characteristics have to be inferred from product appearance. Hence a thorough dissection on how product appearance affects user's visual perception and the visual mechanism of comparison between different products can provide help for designer to grasp user's attention (Fenko et al., 2010). Moreover, eye tracking probes user's perception of real-time reaction to products, while conventional methods (i.e. interviews, focus group, questionnaires and so on) have generally failed because they depend on users' willingness and competency to describe how they feel when they are exposed to an product (Ariely and Berns, 2010, Calvert and Brammer, 2012).

As one of physiological measurements, vision receives the largest amount of information about a product quickly (Schifferstein and Desmet, 2007). “The eyes don't lie. If you want to know what people are playing attention to, follow what they are looking at” (Davenport and Beck, 2001, p.19). Therefore, most of the feelings that are elicited by a product are mediated by initial visual perception. In addition, researchers also found that vision is the most important sense in the product-buying experience (Fenko et al., 2010). Researches put forward a focus on eye movement of users and have been widely applied in web design, advertisement, and brand extension. Wedel and Pieters (2008) pointed out that more researches are needed in product, brand packages and so on.

More and more researches have applied eye tracking method to product design and explain the meaning of eye movement. Ares et al. (2013) took food labels as examples, evaluated how consumers acquire information from food labels using eye tracking by analysing eye movements (time to first fixation, percentage of consumers who fixated, total fixation duration, fixation count and visit count in each area of interest) when consumers directed their attention to selected areas. Their results discovered consumers' processing of food labels information, which can help designers to grasp consumers' attention. Ho (2014) investigated how people perceive handbags online with a task-free eye-tracking experiment. Participants were asked to look at randomly displayed pictures of handbags with predefined areas of interest. Their findings can provide eye-tracking evidence of visual behaviour in consumer when looking at products online. Ho and Lu (2014) found that pupil size can be measured to assess design product in emotional aspect. Positive and neutral products evoked significant pupil dilation than negative products. Although researches were done on the visual perception of product, but there is still a need in the study of information perception in real life, such as goal-oriented user experience and comparisons between different products displayed at the same time.

As mentioned above, user experience is highly dynamic with different product features and inner state. So perceptions and responses manifested on physiological activities will be affected by product features as well as inner feelings. In the theory of visual attention researches, generally, factors affect user's visual attention can be distinguished by bottom-up and top-down process or stimulus-driven and goal-oriented attention (Corbetta and Shulman, 2002). Top-down attention refers to voluntary attention allocated to certain object and directed by people's current goal or task, while attention caused by stimulus with visual saliency (e.g., colour, contrast) is bottom-up attention (Corbetta and Shulman, 2002, van der Laan et al., 2015). Visually more salient stimuli are looked at longer and easy to be fixated on first (Pinto et al., 2013, Orquin and Loose, 2013, van der Laan et al., 2015). When a product is exposed to users, they cannot get all information of it but merely a fast and holistic impression (Lindgaard et al., 2006). For limited perceptual capacity and huge information exposed, users will extract information they care and pay their visual attention to selective features of product as well as goal-oriented (Clement et al., 2013). Vetter and Newen (2014) pointed out that cognition penetration was embedded in a hierarchical four-stage model of perception. They thought that top-down effects can be called a cognition penetration as sensory input are influenced by higher level “cognitive” factors, and bottom-up influences can be regarded as sensory penetration as cognitive contents are influenced by sensory information.

Then eye movements will be different attributed to goal-oriented factors such as choice or judgement that result from a specific task instruction. Also there will be difference when users look at variety products with different features (e.g., aesthetics, attractiveness) (Nagai and Georgiev, 2011). And various products displayed at the same time are rarely considered. Hence, there is a need to investigate user's visual perception from free browsing and goal-oriented (Pinto et al., 2013). Orquin and Loose (2013) reviewed the visual attention and choice, and pointed out that researches should investigate people's decision from greater integration with visual attention. Shimojo et al. (2003) promoted the “cascade effect” model, suggesting that preference for objects can be predicted by analysing people's eye movement. So it cannot reveal user's real visual perception merely by looking at repeated stimulus or user cannot distinguish products unless making comparisons among different products. That is so say that different products with various levels of user experience with elicit different perceptions and responses, and the inner states also affect physiological perceptions and responses.

Without considering price and purchase, this paper just takes the use experience before purchasing of products into account. Two tasks are designed to investigate: what is the eye movement evoked by different levels of user experience? What indexes can measure the level of user experience and under what condition? Task one uses two levels of user experience as stimuli to explore the eye movement evoked by just free browsing. Task two uses pairwise stimuli to ask participants evaluate the higher user experience and click the left mouse button. Hence, the main purpose of the work is to check the capacity of eye movement indexes of physiological signals to evaluate product's user experience before purchasing.