Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Journal of the Academy of Marketing Science

25-01-2024 | Original Empirical Research

(Mis)alignment between facial and textual emotions and its effects on donors engagement behavior in online crowdsourcing platforms

Authors: Elham Yazdani, Anindita Chakravarty, Jeff Inman

Published in: Journal of the Academy of Marketing Science

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Although textual and facial emotion expressions in prosocial appeals have been studied separately in past research, there has been little work on the misalignment (compared to alignment) between them. Using a multi-method approach comprising experiments and panel data of 6,098 crowdfunding projects from Gofundme, we investigate how emotions expressed in the cover image and the project description interact to increase potential donors’ engagement as measured by metrics relevant to the platform, i.e., donation amount per donor. More specifically, we focus on the misalignment (vs. alignment) between happy and sad facial expressions in images and the positive and negative emotions in the project description. Our empirical analysis shows that misalignment scenarios have a bigger positive impact on the donor’s contributed amount, compared to alignment scenarios. We also show that such a relative impact is heterogeneous and differs based on expressed urgency, beneficiary age, and progress towards donation goal. We make a theoretical contribution via assessing the extent of cognitive processing in misalignment vs alignment cases. We propose situations with a tradeoff such that the extent of cognitive processing of multimodal information content can be reduced due to simultaneous activation of affective processing. Our research provides strategic recommendations for platforms to increase donors’ engagement and platform revenue management.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Footnotes
3
There is no dominant emotion in the remaining 11% of the sample.
 
4
We use this data for the robustness analysis.
 
5
The remaining 15% of projects belong to either of the categories “Animals and pets,” “Business and entrepreneurs,” “Creative arts and music,” “Travel and adventure,” “Mission and faith,” “Weddings and honeymoon,” “Sports and clubs,” “Competitions and pageants,” and “Environment.”
 
6
The emotion scores from Google Cloud API and Amazon Rekognition are between 0–5 and 0–100 respectively.
 
7
Since Azure API over-represents the happiness scores, we re-scaled this variable in our empirical analysis.
 
8
We also applied an alternate endogeneity correction using the control function approach. Our results remain consistent.
 
9
The two misalignment cases are not significantly different from each other.
 
10
We run Study 1a analysis with number of donors as the DV, but the results do not hold.
 
11
We treat progress toward the goal as a dummy variable for the ease of interpretation, however our results hold if we consider it a continuous variable.
 
12
As a robustness check, we got similar results using categorical measures (1 = Less than $25, 2 = $26-$50, 3 = $51-$75, 4 = $76-$100, 5 = $101-$125, 6 = $126-$150, 7 = $151-$175, 8 = $176-$200).
 
13
The effects remain significant without including gender as covariate.
 
14
We use SEM command in STATA to do the parallel mediation analysis.
 
15
The correlation between the two cognitive scales is .83.
 
17
The calculations are based on the results from Study 1a, more details are provided in the last row of Table 5.
 
18
Since 2010, Gofundme has hosted more than one million campaigns.
 
Literature
Baberini, M., Coleman, C. L., Slovic, P., & Västfjäll, D. (2015). Examining the effects of photographic attributes on sympathy, emotions, and donation behavior. Visual Communication Quarterly, 22(2), 118–128.CrossRef
Bagozzi, R. P., & Moore, D. J. (1994). Public Service Advertisements: Emotions and Empathy guide Prosocial Behavior. Journal of Marketing, 58(1), 56–70.CrossRef
Bansal, R., Raj, G., & Choudhury, T. (2016). Blur Image Detection using Laplacian operator and Open-CV. International Conference System Modeling & Advancement in Research Trends (SMART). IEEE.
Bechler, C. J., Tormala, Z. L., & Rucker, D. D. (2019). Perceiving attitude change: How qualitative shifts augment change perception. Journal of Experimental Social Psychology, 82, 160–175.CrossRef
Bezryadin, S., Bourov, P., & Ilinih, D. (2007). Brightness Calculation in Digital Image Processing. International Symposium on Technologies for Digital Photo Fulfillment (pp. 10–15). Society for Imaging Science and Technology.
Bhati, A., & Hansen, R. (2020). A literature review of experimental studies in fundraising. Journal of Behavioral Public Administration, 3(1)
Bum, C. H., & Lee, K. (2016). The relationships among non-verbal communication, emotional response, satisfaction, and participation adherence behavior in sports participants. Journal of Physical Education and Sport, 16, 1052–1057.
Burman, E. (2016). Deconstructing developmental psychology. Taylor & Francis.CrossRef
Carr, M. B., & Lutjemeier, J. A. (2005). The relation of facial affect recognition and empathy to delinquency in youth offenders. Adolescence, 40(159), 601.
Carroll, R., & Kachersky, L. (2019). Service fundraising and the role of perceived donation efficacy in individual charitable giving. Journal of Business Research, 99, 254–263.CrossRef
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016). Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: history, trends, and affect-related applications. IEEE Transations on Pattern Analysis and Machine Intelligence, 38(8), 1548–1568.CrossRef
Courtney, C., Dutta, S., & Li, Y. (2017). Resolving information asymmetry: Signaling, endorsement, and crowdfunding success. Entrepreneurship Theory and Practice, 41(2), 265–290.CrossRef
Dewaele, J.-M. (2019). When elephants fly: The lift-off of emotion research in applied linguistics. The Modern Language Journal, 103(2), 533–536.CrossRef
Diemand-Yauman, C., Oppenheimer, D. M., & Vaughan, E. B. (2011). Fortune favors the (): Effects of disfluency on educational outcomes. Cognition, 118(1), 111–115.CrossRef
Dovidio, J. F., Piliavin, J. A., Schroeder, D. A., & Penner, L. A. (2017). The social psychology of prosocial behavior. Psychology Press.CrossRef
Eidels, A., Townsend, J. T., & Algom, D. (2010). Comparing perception of Stroop stimuli in focused versus divided attention paradigms: Evidence for dramatic processing differences. Cognition, 114(2), 129–150.CrossRef
Eisenberg, N. (2014). Altruistic emotion, cognition, and behavior (PLE: Emotion). Psychology Press.CrossRef
Farwell, L., & Weiner, B. (2000). Bleeding Hearts and the Heartless: Popular Perceptions of Liberal and Conservative Ideologies. Personality and Social Psychology Bulletin, 26(7), 845–852.CrossRef
Fennell, P. B., Coleman, J. T., & Kuo, A. (2020). The moderating role of donation quantifiers on price fairness judgments. Journal of Business Research, 110, 464–473.CrossRef
Fisher, R. J., Vandenbosch, M., & Antia, K. D. (2008). An Empathy-helping Perspective on Consumers’ Responses to Fund-raising Appeals. Journal of Consumer Research, 35(3), 519–531.CrossRef
French, M. M. J., Blood, A., Bright, N. D., Futak, D., Grohmann, M. J., Hasthorpe, A., Heritage, J., Poland, R. L., Reece, S., & Tabor, J. (2013). Changing fonts in education: How the benefits vary with ability and dyslexia. The Journal of Educational Research, 106, 301–304.CrossRef
Goldin-Meadow, S. (2003). Thought before language: Do we think ergative. Language in mind: Advances in the study of language and thought, 493–522.
Halpern-Felsher, B. L., Millstein, S. G., Ellen, J. M., Adler, N. E., Tschann, J. M., & Biehl, M. (2001). The role of behavioral experience in judging risks. Health Psychology, 20(2), 120.CrossRef
Hans, A., & Hans, E. (2015). Kinesics, haptics and proxemics: Aspects of non-verbal communication. IOSR Journal of Humanities and Social Science (IOSR-JHSS), 20(2), 47–52.
Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica: Journal of the Econometric Society, 153–161.
Henry, P. J., Reyna, C., & Weiner, B. (2004). Hate welfare but help the poor: How the attributional content of stereotypes explains the paradox of reactions to the destitute in America 1. Journal of Applied Social Psychology, 34(1), 34–58.CrossRef
Hoffman, M. L. (2001). Empathy and Moral Development: Implications for Caring and Justice. Cambridge University Press.
Hou, J.-R., Zhang, J., & Zhang, K. (2023). Pictures that are worth a thousand donations: How emotions in project images drive the success of online charity fundraising campaigns? An image design perspective. Management Information Systems Quarterly, 47(2), 535–584.CrossRef
Iyer, A., Webster, J., Hornsey, M. J., & Vanman, E. J. (2014). Understanding the power of the picture: The effect of image content on emotional and political responses to terrorism. Journal of Applied Social Psychology, 44(7), 511–521.CrossRef
Jack, R. E., Blais, C., Scheepers, C., Schyns, P. G., & Caldara, R. (2009). Cultural confusions show that facial expressions are not universal. Current Biology, 19(18), 1543–1548.CrossRef
Jack, R. E., Garrod, O. G. B., Hui, Yu., Caldara, R., & Schyns, P. G. (2012). Facial Expressions of Emotion are not Culturally Universal. Proceedings of the National Academy of Sciences, 109(19), 7241–7244.CrossRef
Jensen, J. D., King, A. J., & Carcioppolo, N. (2013). Driving toward a goal and the goal-gradient hypothesis: The impact of goal proximity on compliance rate, donation size, and fatigue. Journal of Applied Social Psychology, 43(9), 1881–1895.CrossRef
Jin, F., Zheng, Z., & Banggang, Wu. (2021). God Helps Those Who Help Themselves: How Recipients’ Efforts Perception Affects Donation. Frontiers in Psychology, 12, 695332.CrossRef
Kensinger, E. A., & Schacter, D. L. (2006). Processing emotional pictures and words: Effects of valence and arousal. Cognitive, Affective, & Behavioral Neuroscience, 6(2), 110–126.CrossRef
Kivetz, R., Urminsky, O., & Zheng, Y. (2006). The goal-gradient hypothesis resurrected: Purchase acceleration, illusionary goal progress, and customer retention. Journal of Marketing Research, 43(1), 39–58.CrossRef
Kjeldsen, J., & Hess, A. (2021). Experiencing multimodal rhetoric and argumentation in political advertisements: A study of how people respond to the rhetoric of multimodal communication. Visual Communication, 20(3), 327–352.CrossRef
Kranzbühler, A.-M., Zerres, A., Kleijnen, M. H. P., & Verlegh, P. W. J. (2020). Beyond valence: A meta-analysis of discrete emotions in firm-customer encounters. Journal of the Academy of Marketing Science, 48, 478–498.CrossRef
Kronrod, A., & Danziger, S. (2013). “Wii Will Rock You!” The use and effect of figurative language in consumer reviews of hedonic and utilitarian consumption. Journal of Consumer Research, 40(4,1), 726–739.CrossRef
Kuperman, V., Estes, Z., Brysbaert, M., & Warriner, A. B. (2014). Emotion and language: valence and arousal affect word recognition. Journal of Experimental Psychology: General, 143(3), 1065.CrossRef
Kuppuswamy, V., & Bayus, B. L. (2017). Does my contribution to your crowdfunding project matter? Journal of business venturing, 32(1), 72–89.CrossRef
Kurien, D. N. (2010). Body language: Silent communicator at the workplace. IUP Journal of Soft Skills, 1, 4.
Lee, D., & Park, J. (2020). The relationship between a charity crowdfunding project’s contents and donors’ participation: An empirical study with deep learning methodologies. Computers in Human Behavior, 106, 106–261.CrossRef
Lee, S., Winterich, K. P., & Ross Jr, W. T. (2014). I’m moral, but I won’t help you: The distinct roles of empathy and justice in donations. Journal of Consumer Research, 41(3), 678–696.CrossRef
Lefcourt, H. M. (2014). Locus of control: Current trends in theory & research. Psychology Press.CrossRef
Lehmann, J., Goussios, C., & Seufert, T. (2016). Working memory capacity and disfluency effect: An aptitude-treatment-interaction study. Metacognition and Learning, 11, 89–105.CrossRef
Levett, A. (2003). Problems of cultural imperialism in the study of child sexual abuse. New Feminist Stories of Child Sexual Abuse (pp. 64–88). Routledge.
Li, Y., & Xie, Y. (2020). Is a Picture Worth a Thousand Words? An Empirical Study of Image Content and Social Media Engagement. Journal of Marketing Research, 57(1), 1–19.CrossRef
Liang, J., Chen, Z., & Lei, J. (2016). Inspire Me to Donate: The Use of Strength Emotion in Donation Appeals. Journal of Consumer Psychology, 26(2), 283–288.CrossRef
Little, D. R., Eidels, A., Fifić, M., & Wang, T. S. (2018). How do information processing systems deal with conflicting information? Differential predictions for serial, parallel, and coactive models. Computational Brain & Behavior, 1(1), 1–21.CrossRef
Liu, F., Zhao, Q., Liu, X., & Zeng, D. (2018). Joint face alignment and 3D face reconstruction with application to face recognition. IEEE transactions on pattern analysis and machine intelligence, 42(3), 664–678.CrossRef
Ludwig, S., De Ruyter, K., Friedman, M., Brüggen, E. C., Wetzels, M., & Pfann, G. (2013). More than words: The influence of affective content and linguistic style matches in online reviews on conversion rates. Journal of Marketing, 77(1), 87–103.CrossRef
Maia, B., & Santos, D. (2018). Language, emotion, and the emotions: The multidisciplinary and linguistic background. Language and Linguistics Compass, 12(6), e12280.CrossRef
Mangaroska, K., Sharma, K., Gašević, D., & Giannakos, M. (2022). Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity. Journal of Computer Assisted Learning, 38(1), 40–59.CrossRef
Martinec, R., & Salway, A. (2005). A system for image–text relations in new (and old) media. Visual Communication, 4(3), 337–371.CrossRef
Moriuchi, E., & Chung, C. (2018). Young Americans’ ethnic identity toward disaster advertising: The effects of disaster location and message types. International Journal of Nonprofit and Voluntary Sector Marketing, 23(1), e1596.CrossRef
Oppenheimer, D. M., & Olivola, C. Y. (2011). The science of giving: Experimental approaches to the study of charity. Psychology Press.CrossRef
Park, B., Flowerday, T., & Brünken, R. (2015). Cognitive and affective effects of seductive details in multimedia learning. Computers in Human Behavior, 44, 267–278.CrossRef
Parton, N. (Ed.). (1997). Child protection and family support: tensions, contradictions, and possibilities. Psychology Press.
Peng, L., Cui, G., Bao, Z., & Liu, S. (2022). Speaking the same language: the power of words in crowdfunding success and failure. Marketing Letters, 33(2), 311–323.CrossRef
Pham, C., & Septianto, F. (2019). A smile–the key to everybody’s heart? The interactive effects of image and message in increasing charitable behavior. European Journal of Marketing, 54(2), 261–281.CrossRef
Plant, E. A., Hyde, J. S., Keltner, D., & Devine, P. G. (2000). The gender stereotyping of emotions. Psychology of Women Quarterly, 24(1), 81–92.CrossRef
Raab, M., Schlauderer, S., Overhage, S., & Friedrich, T. (2020). More than a feeling: Investigating the contagious effect of facial emotional expressions on investment decisions in reward-based crowdfunding. Decision Support Systems, 135, 113326.CrossRef
Ren, J., Raghupathi, V., & Raghupathi, W. (2020). Understanding the dimensions of medical crowdfunding: A visual analytics approach. Journal of Medical Internet Research, 22(7), e18813.CrossRef
Rhue, L., & Robert, L. P. (2018). Emotional delivery in pro-social crowdfunding success. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–6).
Rincón Quintero, Y. A., Sukier, H., Contreras Capella, J., & Ramírez Molina, R. I. (2019). Responsible communication strategies for small and medium-sized enterprises.
Robiady, N. D., Windasari, N. A., & Nita, A. (2021). Customer engagement in online social crowdfunding: The influence of storytelling technique on donation performance. International Journal of Research in Marketing, 38(2), 492–500.CrossRef
Septianto, F., & Paramita, W. (2021). Sad but smiling? How the combination of happy victim images and sad message appeals increase prosocial behavior. Marketing Letters, 32(1), 91–110.CrossRef
Sharkov, F. I., Silkin, V. V., & Kireeva, O. F. (2022). Non-verbal signs of personality: Communicative meanings of facial expressions. Becтник Poccийcкoгo yнивepcитeтa дpyжбы нapoдoв. Cepия: Coциoлoгия, 22(2), 387–403.
Skitka, L. J., & Tetlock, P. E. (1992). Allocating Scarce Resources: A Contingency Model of Distributive Justice. Journal of Experimental Social Psychology, 28(6), 491–522.CrossRef
Small, D. A., & Verrochi, N. M. (2009). The Face of Need: Facial Emotion Expression on Charity Advertisements. Journal of Marketing Research, 46(6), 777–787.CrossRef
Small, D. A., Loewenstein, G., & Slovic, P. (2007). Sympathy and callousness: The impact of deliberative thought on donations to identifiable and statistical victims. Organizational Behavior and Human Decision Processes, 102(2), 143–153.CrossRef
Small, D. A., Loewenstein, G., & Slovic, P. (2013). Sympathy and callousness: The impact of deliberative thought on donations to identifiable and statistical victims. In The Feeling of Risk (pp. 51–68). Routledge.
Soyer, E., & Hogarth, R. M. (2011). The size and distribution of donations: Effects of number of recipients. Judgment and Decision Making, 6(7), 616–628.CrossRef
Sundar, S. S. (2000). Multimedia effects on processing and perception of online news: A study of picture, audio, and video downloads. Journalism & Mass Communication Quarterly, 77(3), 480–499.CrossRef
Teixeira, T., Wedel, M., & Pieters, R. (2012). Emotion-induced engagement in internet video advertisements. Journal of Marketing Research, 49(2), 144–159.CrossRef
Touré-Tillery, M., & Fishbach, A. (2011). The course of motivation. Journal of Consumer Psychology, 21(4), 414–423.CrossRef
Trope, Y., & Liberman, N. (2003). Temporal construal. Psychological Review, 110(3), 403.CrossRef
Tsiros, M., & Irmak, C. (2020). Lowering the minimum donation amount increases consumer purchase likelihood of products associated with cause-related marketing campaigns. Journal of Marketing Research, 57(4), 755–770.CrossRef
Vieites, Y., Goldszmidt, R., & Andrade, E. B. (2022). Social Class Shapes Donation Allocation Preferences. Journal of Consumer Research, 48(5), 775–795.CrossRef
Weiner, B. (1993). On Sin versus Sickness: A Theory of Perceived Responsibility and Social Motivation. American Psychologist, 48(9), 957.CrossRef
Williams, L. E., Stein, R., & Galguera, L. (2014). The distinct affective consequences of psychological distance and construal level. Journal of Consumer Research, 40(6), 1123–1138.CrossRef
Xu, L., Mehta, R., & Dahl, D. W. (2022). Leveraging creativity in charity marketing: The impact of engaging in creative activities on subsequent donation behavior. Journal of Marketing, 86(5), 79–94.CrossRef
Yang, J., Li, Y., Calic, G., & Shevchenko, A. (2020). How multimedia shape crowdfunding outcomes: The overshadowing effect of images and videos on text in campaign information. Journal of Business Research, 117, 6–18.CrossRef
Yin, D., Bond, S. D., & Zhang, H. (2014). Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS Quarterly, 38(2), 539–560.CrossRef
Zagefka, H., Noor, M., Brown, R., de Moura, G. R., & Hopthrow, T. (2011). Donating to disaster victims: Responses to natural and humanly caused events. European Journal of Social Psychology, 41(3), 353–363.CrossRef
Zemack-Rugar, Y., & Klucarova-Travani, S. (2018). Should donation ads include happy victim images? The moderating role of regulatory focus. Marketing Letters, 29(4), 421–434.CrossRef
Zhao, K., Zhou, L., & Zhao, X. (2022). Multi-modal emotion expression and online charity crowdfunding success. Decision Support Systems, 163, 113842.CrossRef
Metadata
Title
(Mis)alignment between facial and textual emotions and its effects on donors engagement behavior in online crowdsourcing platforms
Authors
Elham Yazdani
Anindita Chakravarty
Jeff Inman
Publication date
25-01-2024
Publisher
Springer US
Published in
Journal of the Academy of Marketing Science
Print ISSN: 0092-0703
Electronic ISSN: 1552-7824
DOI
https://doi.org/10.1007/s11747-024-01002-3