1 Introduction
Online consumer reviews are a regular feature on most consumer websites such as Amazon or Yelp and have attracted much attention in the information systems community in recent years (e.g., Li et al. 2019). In particular, research has highlighted that certain properties of reviews determine their effects on review helpfulness, purchase intention, and product sales. In this regard, apart from the effects of review ratings (e.g., Chevalier and Mayzlin 2006; Clemons et al. 2006), a number of studies are concerned with review language, i.e., length (e.g., Pan and Zhang 2011; Schindler and Bickart 2012), content (e.g., Willemsen et al. 2011; Yin et al. 2014), and linguistic style (e.g., Li et al. 2019; Liu et al. 2008), which are arguably at least as important for review quality and effectiveness as purely numerical ratings (Archak et al. 2011; Pavlou and Dimoka 2006).
Despite the prominence of research on the effects of review language, however, little is known about why reviewers vary in the ways they use language, build arguments, and expend effort on their reviews. Particularly, while some nascent research has emerged in this field (e.g., Hu et al. 2008; Willemsen et al. 2011), only one study (Safi and Yu 2017) has so far specifically illuminated the influence of reviewer personality on the characteristics of online consumer review language. This is surprising given that personality has long been considered an important factor to explain differences in e-commerce behavior (e.g., Gefen 2000) and information systems use in general (Zmud 1979). It therefore appears reasonable to expect that personality characteristics might help explain why people vary in the way they compose reviews.
Anzeige
Our paper aims to establish a novel link between reviewer personality and online reviews. Specifically, we draw on the concept of political ideology, i.e., individuals’ leanings on a continuum between liberal and conservative. Political ideology is a particularly intriguing concept because strong evidence exists that it reflects various stable, underlying personality characteristics (see Jost et al. 2003, 2009 for reviews). In addition, political ideology contains an explicitly motivational component and thus “helps to explain why people do what they do” (Jost 2006, p. 653). As a result, the implications of ideology have often been studied in research related to information systems (Flaxman et al. 2016; Gentzkow and Shapiro 2011), e.g., with regard to its consequences for IT investment (Pang 2016), technology adoption (e.g., Chen 2010; Smith 2013), user behavior on social networking sites (Yang et al. 2017), and engagement in online piracy (Graf-Vlachy et al. 2017).
We introduce political ideology to online review research because we expect several of the associated personality characteristics and motivations to predict differences in review language. Building on previous research, we theorize that individuals’ pro-social behavior and altruism (e.g., Zettler and Hilbig 2010; Van Lange et al. 2012), cognitive complexity (e.g., Van Hiel and Mervielde 2003; Jost et al. 2003), and sensitivity to negative stimuli (e.g., Hibbing et al. 2014; Joel et al. 2013) are related to the way reviews are composed. We then link these personality characteristics associated with political ideology to three of the most-studied properties of review language which have been suggested to have a pivotal impact on review helpfulness and sales, namely review depth (e.g., Mudambi and Schuff 2010; Schindler and Bickart 2012), multifacetedness (e.g., Ghose and Ipeirotis 2006, 2011; Willemsen et al. 2011), and valence (e.g., Cao et al. 2011; Yin et al. 2014). Overall, our research thus addresses the following research question: How is reviewers’ political ideology related to the language they use when composing online reviews?
We view technology – specifically websites using online reviews – as a socially “embedded system” (Orlikowski and Iacono 2001, p. 126) and aim to contribute to research on how “different user groups [engage] with that technology” (2001, p. 127). To the best of our knowledge, our study is the first to show that the differences in review language described in extant literature are associated with differences in personality of the reviewers, as reflected in their political ideology. By adding the additional factor of reviewers’ political ideology, our study goes beyond prior research, which was limited to situational antecedents such as experience or expertise (e.g., Liu et al. 2008; Smith et al. 2005), and we reach a more granular understanding of the determinants of review language. In addition, we provide evidence for the potential of political ideology as an important construct in information systems research at large. In particular, we highlight that the political ideology of system users is closely related to how they engage with information technology, which has critical implications for the design of IT systems and user interactions.
2 Related Literature
2.1 Online Consumer Reviews
Online consumer reviews constitute a critical element of electronic word of mouth (Lis 2013; Lis and Neßler 2014) and are a regular feature of most consumer websites, especially in e-commerce. Mudambi and Schuff (2010) defined them as “peer-generated product evaluations posted on company or third party websites” (p. 186). Including reviews on websites allows customers to build stronger social rapport with the website (Kumar and Benbasat 2006) and to reduce transaction risk and search effort (Dabholkar 2006). Firms, in turn, use reviews as a feedback mechanism for product development and quality control (Dellarocas 2003).
Anzeige
As reviews play such a prominent role in decision-making processes, scholars have devoted much attention to understanding how reviews differ from one another and which consequences ensue, for instance, regarding product sales and the perceived helpfulness of reviews. On a general level and perhaps unsurprisingly, research suggests that review ratings are directly related to sales (e.g., Chevalier and Mayzlin 2006; Zhu and Zhang 2010).
On a more specific level, regarding predictors of review helpfulness, studies have focused on the length, content, and stylistic features of reviews. For instance, longer reviews are generally evaluated more positively than shorter ones (Mudambi and Schuff 2010; Pan and Zhang 2011). Content-wise, readers perceive reviews as more helpful if they contain a mixture of objective product information and subjective evaluative statements (Ghose and Ipeirotis 2006, 2011) and exhibit a high diversity of arguments, i.e., both positive and negative arguments (Willemsen et al. 2011). Perceived helpfulness has also been shown to be driven by linguistic style (Zhang and Varadarajan 2006) such as a lower level of sentence complexity or fewer grammatical errors (Ghose and Ipeirotis 2011; Liu et al. 2008; Schindler and Bickart 2012). Table A1 in the Online Appendix (available online via http://link.springer.com) provides a more extensive overview of additional research regarding the effects of review properties on further variables like purchase intention and sales.
While scholars have extensively studied the consequences of review properties, they have paid much less attention to potential antecedents. In particular, factors pertaining to reviewers’ personality are almost completely unexplored. An extensive and systematic literature review unearthed only three relevant articles1: Picazo-Vela et al. (2010) found that conscientiousness and neuroticism correlate with an individual’s intention to provide reviews in the first place. However, these authors did not study review language. Similarly, Helm et al. (2013) provide empirical evidence that introversion is related to posting of product ratings, but do not study review language. The only article explicitly relating reviewer personality to review language is Safi and Yu’s (2017) work that links reviewers’ innate innovativeness to various properties of product reviews, e.g., expressed concern with cost or degrees of uncertainty and optimism.
2.2 Political Ideology
Political ideology, i.e., the deeply rooted values, beliefs, and preferences that people hold about ideal goals for society and their beliefs about how to achieve them, has been studied extensively in political science and related fields (Jost 2006). Usually, it is conceptualized as a liberal–conservative continuum which captures the most relevant interpersonal differences regarding political ideology (Jost et al. 2009). Specifically, researchers frequently denote a given individual’s position on the spectrum as their degree of conservatism (Jost et al. 2003).
A core tenet of political ideology research is that differences in ideology are grounded in differences in underlying personality traits and motivations (Jost 2006; Jost et al. 2003, 2009). Thus, individuals’ political ideologies are the reflection of stable personality characteristics rather than of merely situational circumstances (Alford et al. 2005; Block and Block 2006).
The two most important types of motives underlying political ideology are epistemic and existential (Jost et al. 2003, 2009). Epistemic motives include elements of how humans deal with uncertainty, ambiguity, and complexity, how strongly they need to order and structure information and how mentally rigid they are. For instance, a more conservative ideology is positively correlated with intolerance of ambiguity (e.g., Budner 1962; Sidanius 1978), lower openness to experience (e.g., Van Hiel and Mervielde 2004), and stronger individualistic and less altruistic tendencies (e.g., Van Lange et al. 2012; Zettler and Hilbig 2010).
Existential motives relate to how individuals perceive and cope with threats to the current societal system as well as their position within it. Research has shown that, for example, more conservative worldviews result from greater responsiveness to negative stimuli (e.g., Hibbing et al. 2014; Joel et al. 2013), greater fear of threat and loss (e.g., Jost et al. 2007), as well as more anger and aggression (e.g., Altemeyer 1998; Tomkins 1995).
Scholars have shown that political ideology directly impacts not only every-day human behavior beyond the immediately political sphere (e.g., Carney et al. 2008; Jost et al. 2008), but also people’s behavior related to information technology. One especially substantial line of inquiry explores the impact of online platforms on ideological segregation (Flaxman et al. 2016; Gentzkow and Shapiro 2011; Himelboim et al. 2013). For instance, users’ political ideology was found to meaningfully impact their “unfriending” of other users of opposing political ideology on social networking sites (Yang et al. 2017). Another line of inquiry relates to the effect of ideology on technology investment and adoption (Baxter and Marcella 2012; Chen 2010; Smith 2013). Researchers have, for example, studied how political ideology is related to IT investments (Pang 2016), the adoption of e-participatory government (García-Sánchez et al. 2011), and the adoption of e-voting systems (Choi and Kim 2012). In addition, prior research has studied various other consequences of information system users’ political ideologies. One study, for example, links Internet users’ political ideology to online piracy (Graf-Vlachy et al. 2017) and another study found that the ideology of Twitter users is related to the valence of the content they posted (Himelboim et al. 2016). Jointly, these studies suggest that various behavioral differences can be traced to the inherent differences in personality which are the underlying drivers of political ideology. Below, we elaborate on how political ideology and its associated personality characteristics may impact a so-far disregarded element of online behavior: The composition of online reviews.
3 Linking Political Ideology and Online Reviews
3.1 Altruism and Review Depth
While the benefits of online reviews are apparent and have been widely discussed, one could argue that the benefits of posting a review for the reviewer are limited compared to its costs. Benefits generally associated with online information sharing such as social status enhancement (Lee and Ma 2012; Lu and Hsiao 2007; Wasko and Faraj 2005) or reciprocity (Chiu et al. 2006) are potentially less pronounced in the context of online reviews because reviews are often anonymous and lack direct one-to-one interactions (Wasko and Faraj 2005). On the cost side, however, reviewers must allocate attention, time, and effort to composing reviews (Hew and Hara 2007; Sun et al. 2014).
For prospective customers, the amount and quality of information are important factors to consider when evaluating the benefits of a review. Mudambi and Schuff (2010) and Pan and Zhang (2011), for example, found that the longer the online review – i.e., the more words it contains – the more helpful and beneficial it is to prospective customers. Likewise, Willemsen et al. (2011) have shown that the larger the number of arguments included, i.e., the greater the argument density of a review, the more useful it is to prospective customers. Thus, while the benefits for the customer tend to increase with the number of words and the number of arguments in a review, so do the costs for the reviewer. This raises the question of what kind of person is willing to write longer reviews or such with a greater number of arguments.
We build on research which suggests that altruism affects people’s inclination to share information (Hew and Hara 2007). More generally, altruistic individuals are willing to “pay a personal cost to provide benefits to others in general, regardless of the identity of the beneficiaries” (Fowler and Kam 2007). Hence, we believe that the more altruistic an individual, the more likely it is that he or she puts a great deal of effort into composing an online review.
Notably, a host of research on political ideology suggests that such self-sacrificial tendencies are associated with a non-conservative ideology (Farwell and Weiner 2000; Van Lange et al. 2012). The psychological underpinnings of this phenomenon are preferences for equality. Whereas more conservative individuals are more likely to accept inequality as natural, less conservative individuals tend to favor greater equality (Jost et al. 2003; Van Lange et al. 2012). As Bobbio (1996, p. 40) put it: “The left favours greater equality, while the right sees society as inevitably hierarchical.” From these assumptions, it follows that less conservative individuals would be more inclined to extensively share goods or knowledge, i.e., behave altruistically, whereas more conservative individuals might not feel a need to increase equality by sharing. We therefore conclude that since more conservative individuals tend to be less altruistic, they will be less motivated to expend effort on composing a review, and thus, will submit reviews that are shorter and contain fewer arguments.
H1a
Greater reviewer conservatism is associated with a lower number of words in reviews.
H1b
Greater reviewer conservatism is associated with a lower number of arguments in reviews.
3.2 Cognitive Complexity and Review Multifacetedness
Consumers consult online reviews during the decision-making process to reduce the information asymmetry between the seller and themselves (Hu et al. 2008; Kumar and Benbasat 2006; Mudambi and Schuff 2010). In this pursuit, review multifacetedness, i.e., the degree to which multiple perspectives are considered in the review, has been shown to be important. Reviews that present both positive and negative information signal that the reviewer is independent and truthful (Crowley and Hoyer 1994) and are therefore perceived as more helpful than reviews that are one-sided (Willemsen et al. 2011).
While this aspect of balanced argumentation is relatively novel in online review research, it has been the subject of a major research stream for political ideology scholars in the form of cognitive complexity and related constructs. Cognitive complexity captures how sophisticatedly and balanced individuals process information (e.g., Suedfeld and Rank 1976; Van Hiel and Mervielde 2003). As such, an individual exhibiting low cognitive complexity is characterized by “rigid evaluations of stimuli [and] the rejection of dissonant information” (Suedfeld and Rank 1976). An individual with high cognitive complexity, in contrast, will interpret information in a flexible fashion, combine and integrate stimuli, as well as consider multiple viewpoints.
More conservative individuals have been found to exhibit a greater need for closure (Chirumbolo 2002; Chirumbolo et al. 2004) and a greater tendency for uncertainty and ambiguity avoidance (Jost et al. 2007), largely because they have a greater propensity to interpret ambiguous situations as threatening (Hibbing et al. 2014). Such tendencies, in turn, lead to a more rigid, black-and-white view of the world and to potentially premature closure, i.e., possibly ending data gathering before all information is known and thus forming opinions and making decisions without incorporating all available information (Furnham and Ribchester 1995).
Consequently, despite some recent suggestions that the reality of political ideology and cognitive complexity might be slightly more intricate (Conway et al. 2016), the overwhelming current scholarly consensus is that individuals that are more conservative tend to display lower cognitive complexity (Jost et al. 2003). Since cognition and communication are hard to separate (Slatcher et al. 2007), cognitive complexity is also reflected in language use. Multiple studies have examined cognitive complexity in oral and written communication and have found support for this rigidity-of-the-right hypothesis (e.g., Tetlock 1983; Tetlock et al. 1984).
We therefore hypothesize that reviewers’ conservatism will be linked to the degree of cognitive complexity in the language they use in reviews.
H2a
Greater reviewer conservatism is associated with less cognitively complex language used in reviews.
Furthermore, we hypothesize that more conservative individuals, who likely process information in a less complex way, will formulate online reviews that are less balanced concerning positive and negative arguments.
H2b
Greater reviewer conservatism is associated with lower argument diversity in reviews.
Finally, we expect argument diversity to be the result of thinking and expressing oneself in a more cognitively complex fashion. Correspondingly, the less balanced information processing of more conservative individuals, which makes them prone to using less complex language, may result in a reduced propensity to provide balanced argumentation. We thus hypothesize that cognitively complex language will act as a mediator for the effect of political ideology on argument diversity.
H2c
Cognitively complex language mediates the effect of political ideology on argument diversity.
3.3 Sensitivity to Negative Stimuli and Review Language Valence
The valence of an online review plays a major role in how it is received by a prospective customer. An individual is more likely to purchase a product if he or she reads a positive review compared to a negative review (e.g., Clemons et al. 2006). Furthermore, research suggests that negative reviews are perceived as more helpful than positive ones (Cao et al. 2011).
Various researchers have noted that more conservative individuals exhibit a heightened sensitivity to negative stimuli. Mendez, for example, describes a neurological “conservative-complex” (2017, p. 92) involving brain structures that are particularly responsive to negativity per se, threat, and disgust, and give rise to a propensity for avoidance (as opposed to approach). Jost and Amodio (2012) additionally review behavioral evidence and find similar results. Most prominently perhaps, Hibbing et al. (2014) review an extensive body of literature and find that conservative individuals tend to allocate more attention to negative stimuli and experience stronger reactions – both psychologically and physiologically – to those stimuli. For example, conservative individuals tend to pay more attention to negatively valenced language than more liberal individuals do (Carraro et al. 2011). Greater conservatism is also related to experiencing greater emotional reactions to negative personal outcomes (Joel et al. 2013). In fact, Hibbing et al. (2014) suggest that such negativity bias is the fundamental distinction between more and less conservative individuals. While other authors tend to take slightly different positions or argue for narrower or wider boundary conditions, most agree that there is some relationship between individuals’ political ideology and their reaction to negative stimuli (also see the extensive peer commentary in the same issue as Hibbing et al.’s article).
We expect the particular sensitivity to negative stimuli displayed by more conservative individuals to translate into the valence of their communication. Empirical evidence shows, e.g., that there is a parallel between a stronger negativity bias and more pronounced linguistic use of negatively valenced emotive intensifiers (e.g., “terribly”) across different cultures (Jing-Schmidt 2007). Similarly, prior research on political ideology and the valence of social media messages shows that conservatism is linked to less positive language (Himelboim et al. 2016). Thus, we hypothesize that more conservative individuals make use of language that is overall less positively valenced in their reviews, independent of review rating.
H3
Greater reviewer conservatism is associated with less positively valenced language used in reviews.
4 Methodology
4.1 Data Sample
To test our hypotheses, we rely on two data sources: clickstream data and manually collected customer reviews. We use the clickstream data to measure the political ideology of the individuals in the sample, i.e., our main independent variable. We use the online customer reviews written by individuals in our sample to measure our dependent variables.
The clickstream data we use is derived from a panel maintained by Comscore, a ratings service. Our initial dataset comprises 17,097 individuals from 9933 households in the US. Their home computer Internet activity was tracked from March until August 2014. After removing individuals that did not provide all demographic information or did not meet the criteria for the measurement of political ideology (see next section), our clickstream sample comprises 3873 individuals from 3361 households.
Clickstream data has several advantages over traditional data sources such as surveys. First, as we track actual behavior of the subjects, we can at least partially avoid self-report biases such as the consistency motif, social desirability, or priming effects (Podsakoff et al. 2003). Second, as clickstream data collection is rather unobtrusive, we likely capture genuine behavior (Bucklin and Sismeiro 2009). Third, we can minimize temporal behavioral biases through a longitudinal data collection over a period of 6 months.
The online reviews we analyze were written by individuals in our sample on Amazon.com, Tripadvisor.com, and Yelp.com. We chose these websites since they used URLs that allowed us to identify when an online review was being composed and because they are popular enough in our dataset (ranked 7, 249, and 507 by page views, respectively) to provide a sufficiently large sample. Furthermore, reviews from Amazon.com and Tripadvisor.com have been used in previous studies (Chevalier and Mayzlin 2006; Mudambi and Schuff 2010; Willemsen et al. 2011; Wu 2013). The reviews were extracted in a three-step process. First, we identified URLs in our data that indicated the posting of an online review on either of the three platforms. Second, as these URLs identify the reviewed product or service but not the reviewer, we manually identified the respective user accounts using the information we have on the reviewed product or service and the review date, as well as demographic data on the user such as age, gender, and location. When we could not unambiguously identify the reviewer or when a review had not actually been posted, we discarded the data. Third, we extracted the most recent reviews the user had submitted (up to 10 reviews). Our final sample consists of 245 reviews containing 23,459 words, written by 37 reviewers. Some reviewers only wrote a single review and the highest number of reviews per reviewer was 20. The median number of reviews per reviewer was 8, while the mean was at 6.62. Three reviewers had written reviews on more than one platform.
Although the sample may appear small, our analyses are likely sufficiently powerful to detect relevant effects. We used G*Power 3.1.9.4 (Faul et al. 2007) to perform a power analysis for a multiple OLS regression, assuming a “large” (Cohen 1992) true effect of f2 = 0.35. Setting alpha to 0.05, desired power to 0.80, and the total number of predictors to 10 (the maximum used in any of our models), a sample of 26 observations is shown to be sufficient to test a single predictor, i.e., the political ideology of a reviewer. The number of independent reviewers in our study is 37, clearly exceeding this threshold.
4.2 Measuring Political Ideology
We measure political ideology using a behavioral approach, employing data on news media consumption to infer political ideology. This is possible since empirical evidence suggests that the political preferences of news media outlets and their audience are very similar (Chiang 2010; Gentzkow and Shapiro 2010; Gentzkow et al. 2014; Iyengar and Hahn 2009; Stroud 2008). Flaxman et al. (2016) estimate the political slant of news outlets by assigning a “conservative share” to the top 100 online news outlets based on the fraction of readership that voted for the Republican candidate in the 2012 US presidential election (see Table A2 in the Online Appendix). We can thus perform an unobtrusive measurement based on actual human behavior, namely online news consumption, avoiding many of the biases that plague self-report measures (Podsakoff et al. 2003).
To approximate the political ideology of the individuals in our sample, we calculate the average conservative share of online news outlets they visited in the entire 6-month period weighted with the page views each outlet accounts for. The following formula depicts the calculation of political ideology for a given individual i, with w being an index over the news outlets:Consequently, our measure of political ideology captures conservatism on a scale from 0 to 1, with increasing values indicating greater conservatism. To ensure reliability of our measure, we only include individuals who regularly consumed online news and thus, similar to Flaxman et al. (2016), we limit our sample to individuals with on average at least four monthly page views on these news outlets.
$$Political\,Ideology_{i} = \frac{{\mathop \sum \nolimits_{w = 1}^{100} \left( {conservativeshare_{w} * pageviews_{iw} } \right)}}{{\mathop \sum \nolimits_{w = 1}^{100} pageviews_{iw} }}$$
We scrutinized our political ideology measure by comparing our distribution to the one found in the sample of Flaxman et al. (2016), as well as to the voting records of the 2012 presidential election. Both comparisons strengthen our conviction in the validity of our measure. First, while Flaxman et al. (2016) find that 66% of users have a political ideology score between 0.41 and 0.54, we find 65% of our entire clickstream sample in that range. Additionally, the ideological distance between two randomly selected individuals in their sample is 0.11 compared to 0.12 our entire sample. Second, similar to the voting records, we find that less conservative individuals have a stronger representation in young age groups as well as in metropolitan areas (New York Times 2012; Roper Center 2012).
4.3 Measuring Review Length and Number of Arguments
We measure review length as the simple count of words in the review. This approach was, for example, used by Mudambi and Schuff (2010). We measure the number of arguments in an online review as the sum of all positive and negative indirect statements. To this end, we manually coded all indirect valenced statements (e.g., “The pictures this camera takes are amazing”). Similar to Willemsen et al. (2011) we only consider indirect valenced statements as arguments, ignoring direct valenced statements (e.g., “This camera is amazing”). Two raters coded all reviews independently. Cohen’s kappa was 0.86, indicating very good intercoder reliability (Landis and Koch 1977).
4.4 Measuring Cognitively Complex Language
We measure cognitive complexity in review language with a linguistic measure developed by Pennebaker and King (1999) using the word count dictionaries from LIWC (Pennebaker et al. 2001). The measure has been frequently used to measure cognitive complexity (e.g., Abe 2011; Slatcher et al. 2007) as it captures the degree to which an individual differentiates and weighs multiple perspectives. When doing so, individuals use more exclusive words (e.g., “but”, “if”), tentative words (e.g., “almost”, “perhaps”), negations (e.g., “can’t”, “wouldn’t”), and discrepancies (e.g., “must”, “ought”), and fewer inclusive words (e.g., “with”, “and”). We first count the words belonging to the LIWC categories “exclusive”, “tentative”, “negations”, “discrepancies”, and “inclusion” used in online reviews. In line with Slatcher et al. (2007) we then compute cognitive complexity using the z-scores (variables transformed to have a mean of zero and a standard deviation of one) of the categories according to the following formula:In our sample, the reliability of the measure indicated by Cronbach’s alpha is 0.61, which is above the reliability of the cognitive complexity measure (0.52) in the sample of Slatcher et al. (2007). Furthermore, it is above the threshold of 0.60, which indicates acceptable reliability (Hair et al. 2009).
$$Cognitively\,complex\,language = zExcl + zTentat + zNegate + zDiscrep - zIncl$$
4.5 Measuring Argument Diversity
We measure argument diversity by calculating the proportion of positive (p) and negative indirect statements (n) in an online review (see Sect. 4.3), according to the following formula:As do Willemsen et al. (2011), we measure argument diversity on a scale from 0 (low diversity) to 1 (high diversity). Note that low diversity can be achieved by having either positive or negative indirect statements strongly dominate a given review.
$$Argument\, diversity = \left\{ \begin{array}{ll}
\frac{n}{p} \quad if\,p > n\\
1\quad if\,p = n \\
\frac{p}{n}\quad if\,n > p \end{array}\right.$$
4.6 Measuring Language Valence
We measure language valence with the Janis–Fadner coefficient of imbalance (Janis and Fadner 1943), which is frequently used by scholars in the context of content analysis (e.g., Pollock and Rindova 2003). Specifically, we employ the following formula:As we aim to capture the emotional tenor of the language our subjects use, we consider each individual word as our recording unit. To classify the words in conveying positive (p in the formula) or negative emotions (n), we use the categories “positive emotions” (e.g., “beautiful”, “sharing”) and “negative emotions” (e.g., “awkward”, “nasty”) from the LIWC dictionary. As is evident, our measure is positive when positive words dominate over negative words and negative when the reverse is true. Note that the extremity of language valence is not only influenced by the ratio of positive to negative words but also by all words that are neither positive nor negative because they are still counted towards total words in the formula.
$$Language\,valence = \left\{
\begin{array}{ll}
\frac{{p^{2} - pn}}{{\left( {total \,words} \right)^{2}}} &if\,p > n\\
\quad 0 & if\,p = n \\
\frac{{pn - n^{2} }}{{\left( {total\,words} \right)^{2} }}& if\, n > p \end{array}\right.$$
4.7 Control Variables
All regressions control for age, gender, and annual household income, which is a valid predictor for socio-economic status and education (Chiou-Wei and Inman 2008). Household income was measured on an ordinal scale from 1 through 13, indicating household income brackets from below 15,000 US$ to above 250,000 US$. We further controlled for Internet usage, which was measured in brackets coded as 1 through 3, indicating less than 5 h, between 5 and 16 h, and more than 16 h of Internet usage per week, respectively. These data were based on user self-reports. In addition, we control for the source website of the review using dummy variables (one for Amazon, one for Yelp, TripAdvisor is the reference category). We also include review ratings, i.e., the numerical star rating (ranging from 1 to 5) indicating the satisfaction of the reviewer with the product or service (Mudambi and Schuff 2010), in our model as online reviews on e-commerce sites are overwhelmingly positive (Chevalier and Mayzlin 2006). Indeed, we find that in our sample, the average rating of the reviews is 4.2 out of 5 stars. Furthermore, for H3, controlling for the review rating is paramount to isolating the effect of valenced language use from the reviewer’s satisfaction with the reviewed product or service. Lastly, we control for the review word count in the models for H2a–c. We do not do so for H3 as the word count is already included in the language valence measure or H1a-b as the word count is used as the measure for the dependent variable.
5 Estimation Approach and Results
Table 1 contains summary statistics and pairwise correlations for all variables used in our analyses. To test for multicollinearity, we calculated the mean variance inflation factors, which, at values between 1.70 (Model 9) and 1.81 (Model 4) for the saturated models, are well below the suggested threshold of 10.0 (Hair et al. 2009; Kutner et al. 2004).
Table 1
Descriptives and correlations (n = 245)
Mean | S.D. | (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | (10) | (11) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
(1) Cognitively complex language | 0.00 | 3.11 | 1.00 | ||||||||||
(2) Argument diversity | 0.24 | 0.39 | 0.07 | 1.00 | |||||||||
(0.29) | |||||||||||||
(3) Language valence | 0.01 | 0.03 | − 0.06 | 0.21 | 1.00 | ||||||||
(0.38) | (0.00) | ||||||||||||
(4) Wordcount | 95.75 | 106.19 | − 0.00 | − 0.04 | − 0.20 | 1.00 | |||||||
(0.97) | (0.49) | (0.00) | |||||||||||
(5) Number of arguments | 2.96 | 2.47 | − 0.08 | − 0.23 | − 0.20 | 0.64 | 1.00 | ||||||
(0.23) | (0.00) | (0.00) | (0.00) | ||||||||||
(6) Political ideologya | 0.43 | 0.05 | − 0.14 | − 0.08 | − 0.17 | − 0.16 | − 0.20 | 1.00 | |||||
(0.03) | (0.22) | (0.01) | (0.01) | (0.00) | |||||||||
(7) Age | 44.89 | 17.95 | − 0.21 | 0.11 | 0.05 | 0.03 | 0.05 | 0.09 | 1.00 | ||||
(0.00) | (0.08) | (0.41) | (0.60) | (0.41) | (0.18) | ||||||||
(8) Genderb | 0.44 | 0.50 | 0.16 | 0.21 | 0.03 | − 0.11 | − 0.27 | 0.17 | − 0.13 | 1.00 | |||
(0.01) | (0.00) | (0.61) | (0.09) | (0.00) | (0.01) | (0.04) | |||||||
(9) Internet usage | 2.43 | 0.56 | 0.04 | − 0.08 | − 0.26 | 0.18 | 0.21 | 0.15 | − 0.04 | − 0.14 | 1.00 | ||
(0.58) | (0.20) | (0.00) | (0.00) | (0.00) | (0.02) | (0.58) | (0.03) | ||||||
(10) Income | 6.64 | 3.16 | − 0.07 | 0.03 | 0.20 | − 0.08 | − 0.10 | 0.00 | − 0.04 | 0.12 | − 0.73 | 1.00 | |
(0.24) | (0.61) | (0.00) | (0.23) | (0.13) | (0.95) | (0.55) | (0.07) | (0.00) | |||||
(11) Review stars | 4.26 | 1.25 | − 0.19 | − 0.09 | 0.10 | − 0.27 | − 0.17 | 0.03 | − 0.02 | − 0.10 | − 0.03 | 0.08 | 1.00 |
(0.00) | (0.17) | (0.13) | (0.00) | (0.01) | (0.59) | (0.77) | (0.12) | (0.62) | (0.20) |
To test H1a, H1b, and H2a, we use panel random effects regression models to accommodate the panel structure of our dataset. To test H2b and H2c, we employ a pooled fractional probit model, as argument diversity, the dependent variable, is a fractional outcome variable (Baum 2008; Papke and Wooldridge 2008). To test H3, we use a pooled Tobit model, as language valence, the dependent variable, is a censored variable (Wooldridge 2001). To account for the fact that our observations are not independent but are nested in reviewers, we clustered standard errors at the individual reviewer in all models. The results for all models are presented in Table 2. Models 1, 3, 5, 7 and 10 are control models for H1a, H1b, H2a, H2b, and H3 respectively.
Table 2
Regression results
We find marginal support for H1a in Model 2 and support for H1b in Model 4. As anticipated, the results in Model 2 show that the word count, i.e., the review length, is higher for reviews submitted by less conservative reviewers (p < 0.10). Model 4 supports our hypothesis that less conservative reviewers also make use of more arguments in their online reviews (p < 0.05). On average, reviews by less conservative reviewers (political ideology score < 0.5) contain 97 words and 3 arguments, while reviews by more conservative reviewers (political ideology score > 0.5) contain only 71 words and 2 arguments.
Model 6 provides support for H2a as we find a negative and significant (p < 0.01) coefficient for political ideology, suggesting that the more conservative a reviewer, the less cognitively complex language will their online reviews contain. Similarly, we find support for H2b in Model 8, albeit with a slightly less significant coefficient (p < 0.05) for political ideology. Contrary to our expectations, we do not find a mediating effect of cognitively complex language for the effect of political ideology on argument diversity. As depicted in Model 9, when cognitively complex language is added to Model 8, political ideology still has a significant effect on the dependent variable (p < 0.05). A possible explanation could be that the greater argument diversity exhibited by less conservative reviewers is not only a result of greater cognitive complexity, but also directly of greater ambiguity tolerance (Jost et al. 2003).
Finally, the results of Model 11 lend support to H3, as political ideology has a negative and significant (p < 0.05) coefficient. This suggests that more conservative individuals tend to use language with a less positive valence in online reviews.
We ran several robustness checks to see whether our models were robust to the use of alternative estimators. Using OLS models with clustered standard errors provided consistent results. We further obtained very similar results when we re-ran our analyses using multilevel models. While both approaches neglect the fact that our dependent variables are not always continuous, the similar results further increase our confidence in the reported models.
6 Discussion
6.1 Contribution
Our research contributes to theory and practice in several ways. We explain relevant differences in how individuals write reviews based on differences in their personality, as reflected in political ideology. Prior research has only examined non-personality-induced differences such as expertise, experience, and social connections; or, if it has studied personality, it has not done so in relation to review language but rather attitudes such as intentions to provide a review. Given that the impact of differences in online reviews on sales and helpfulness has been a major topic in information systems research in past years (e.g., Forman et al. 2008), the lack of scholarly attention to explanatory variables of such differences is surprising. Our research addresses this gap and specifically contributes to research on online reviews (e.g., Goes et al. 2014) and user content generation (e.g., Baeza-Yates and Saez-Trumper 2015) by highlighting how the personality of the reviewer is predictive of the way he or she uses language, builds arguments, and expends effort on a review.
Furthermore, we contribute to theory on information systems research more generally by highlighting the role that users’ political ideology can play in the complex webs in which technology is embedded (Orlikowski and Iacono 2001). Specifically, we show that the construct of political ideology, as the result of stable underlying personality traits and motivational structures (Jost et al. 2003, 2009), is related to everyday human behavior not only in politics and the offline world (e.g., Carney et al. 2008; Jost et al. 2008), but also in online settings beyond mere news media consumption (e.g., Flaxman et al. 2016; Gentzkow and Shapiro 2011). We therefore offer an additional user-specific factor that researchers of technology adoption and use (Graf-Vlachy et al. 2018; Venkatesh et al. 2012) might wish to consider in future work. Our study highlights how the construct of political ideology – which may initially appear far-removed from systems design – is relevant for creators of information systems who need to anticipate how different users may engage with a system. Overall, our evidence reaffirms and broadens the argument of Carney et al. (2008) that “the political divide extends far beyond overtly ideological opinions to much subtler and more banal personal interests, tastes, preferences, and interaction styles” (p. 835). In fact, it also applies to information systems use.
6.2 Practical Implications
As online reviews have become an integral success factor for online retailers (e.g., Dellarocas 2003; Kumar and Benbasat 2006), such firms rely heavily on their customers to provide helpful reviews. Our findings suggest that reviewer personality influences reviews’ depth and multifacetedness, both of which have previously been linked to review helpfulness (e.g., Mudambi and Schuff 2010; Willemsen et al. 2011) and which, in turn, affect purchase intention (Coursaris et al. 2018). As online retailers may be able to infer customers’ political ideology, they can use this information productively. While they may not have access to a customer’s clickstream across the web, retailers could, for example, obtain ideology information directly by way of survey or infer it from past purchasing behavior (potentially using models trained on survey data from a subset of users) or social media posts (e.g., Preoţiuc-Pietro et al. 2017) – as far as this is ethically and legally permissible.
They can then use information on would-be-reviewers’ ideology to possibly improve the data quality (Tilly et al. 2017) of their reviews by subtly nudging (Weinmann et al. 2016) and supporting them in various ways, e.g., by providing personalized input methods, guidance, and incentives. For instance, as more conservative individuals gravitate towards lower argument diversity, firms could provide such reviewers not only with a free text field but additionally with a structured review template, in which reviewers can provide positive and negative feedback. Specifically, the template could allow reviewers to select from pre-populated input fields, displaying frequently used elements of feedback (Lukyanenko et al. 2014). Less conservative reviewers might only be offered a standard free text input field. Appropriately tuned autocomplete features or personalized defaults (Goldstein et al. 2008) might also be helpful in improving review quality. Further, instructions could be tailored to reviewers’ political ideology in that less conservative reviewers might be reminded to include a clear “buy or don’t buy” recommendation (to offset their tendency towards balance in their arguments) and more conservative reviewers to write a certain number of words (to counter their tendency towards writing shorter reviews). Nudges could also include customized appeals to motivate users to write a review in the first place. For example, building on the notion that people tend to assume that others are like them (Marks and Miller 1987), instructions for less conservative individuals might include appeals to write a review to help others make better decisions (leveraging their greater altruism) and try out new products or services (appealing to their greater openness to experience; Jost et al. 2003). Conversely, instructions to more conservative individuals might highlight the ability of online reviews to prevent poor decisions by other customers (appealing to their greater fear of loss and prevention focus; Jost et al. 2003) and discipline providers of poor quality and service (appealing to their greater support of punitive measures; Sargent 2004). Finally, to increase review depth, more conservative individuals could be incentivized to write longer reviews by rewarding them symbolically, for example by giving visual feedback about whether their review is considered sufficiently long to be helpful, or materially, for example with coupons, if their reviews exceed a specified length.
Naturally, these suggestions are contingent on the specific objectives of the online retailer and the precise consequences of review depth and multifacetedness on the achievement of these objectives. Since prior research has, for example, found conflicting evidence on the consequences of review valence on review helpfulness (Cao et al. 2011; Pan and Zhang 2011; Wu 2013), we not only refrain from making suggestions regarding review valence but also caution the reader that the effects of other properties of review language like review depth and multifacetedness might very well be context-dependent.
6.3 Limitations and Future Research
As any empirical study, ours also has limitations. Most critically, we base our measure of political ideology on a relatively novel methodology by Flaxman et al. (2016). While there is substantial evidence that the measure’s foundation, i.e., people’s preference for news outlets that conform to their own political ideology, is solid in the aggregate, no attempt has been made to validate the measure on an individual level. In particular, further validation of this measure would be welcome to ensure that it has tolerable measurement error and that it is valid across the entire liberal–conservative spectrum, even when including not only news coverage that is overtly political. We therefore propose further research that links clickstream data with detailed self-reports of political ideology. This is particularly important since, although our measure avoids common problems of self-reports, it is subject to other potential biases like reactivity or technical difficulties and errors in data collection (Jürgens et al. 2019).
Additional opportunities for future research emerge from our findings. For instance, since we established a link between political ideology and review language, and prior literature had linked review language to review helpfulness, purchase intention, and sales (e.g., Coursaris et al. 2018; Ghose and Ipeirotis 2006, 2011; Mudambi and Schuff 2010; Yin et al. 2014), we wonder if review language might mediate the relationship between reviewers’ political ideology and such substantive outcomes. Further, it would be interesting to study if the relationship between political ideology and review language is conditional on other personality characteristics like the Big Five personality traits. For example, it is conceivable that conscientiousness may dampen the relationship specifically for review depth, as relatively conscientious individuals may be likely to write thorough and balanced reviews irrespective of their political ideology.
Moving beyond the subject of online reviews, our study more broadly raises the question of how political ideology influences users’ creation of and interaction with user-generated content. For example, contributions to open source software projects or efforts like Wikipedia might be influenced by motives reflected in individuals’ political ideologies, such as altruism (Wagner and Prasarnphanich 2007). Also, there is clearly an opportunity for further general research on how system designers can leverage users’ political ideology to improve the quality of user-generated content (Lukyanenko et al. 2014; Tilly et al. 2017).
6.4 Conclusion
Overall, this paper contributes by introducing personality, as reflected in political ideology, as a predictor of online review language. We primarily view our study as a first step towards a deeper, more nuanced understanding of how reviews are created. However, it also suggests that political ideology might be an important construct for research on online behavior on a broader level and even for information systems research in general. Specifically, we theoretically and empirically underscore that the political ideology of system users is closely linked to how they engage with information technology – a notion that has potentially vital implications for the design of IT systems and user interactions. We thus encourage scholars to not only empirically further validate our findings, but also explore additional potential effects of personality in the context of online reviews, online behavior, and beyond.
Acknowledgement
Open Access funding provided by Projekt DEAL. This project
benefitted from a collaboration with McKinsey & Company's Global iConsumer Research Initiative.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.