Top

Published in:

2021 | OriginalPaper | Chapter

Recommendations for Conducting Longitudinal Experience Sampling Studies

Authors : Niels van Berkel, Vassilis Kostakos

Published in: Advances in Longitudinal HCI Research

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The Experience Sampling Method is used to collect participant self-reports over extended observation periods. These self-reports offer a rich insight into the individual lives of study participants by intermittently asking participants a set of questions. However, the longitudinal and repetitive nature of this sampling approach introduces a variety of concerns regarding the data contributed by participants. A decrease in participant interest and motivation may negatively affect study adherence, as well as potentially affecting the reliability of participant data. In this chapter, we reflect on a number of studies that aim to understand better participant performance with Experience Sampling. We discuss the main issues relating to participant data for longitudinal studies and provide hands-on recommendations for researchers to remedy these concerns in their own studies.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Longitudinal Studies in Information Systems

next chapter Longitudinal First-Person HCI Research Methods

Larson R, Csikszentmihalyi M (1983) The experience sampling method. New Directions Methodol Soc Behav Sci

Csikszentmihalyi M, Larson R (1987) Validity and reliability of the Experience-Sampling method. J Nerv Ment Dis 175:526–536CrossRef

Csikszentmihalyi M, Larson R, Prescott S (1977) The ecology of adolescent activity and experience. J Youth Adolescence 6:281–294CrossRef

van Berkel N, Ferreira D, Kostakos V (2017) The experience sampling method on mobile devices. ACM Comput Surv 50(6):93:1–93:40

Smyth JM, Stone AA (2003) Ecological momentary assessment research in behavioral medicine. J Happiness Stud 4:35–52CrossRef

Hsieh G, Li I, Dey A, Forlizzi J, Hudson SE (2008) Using visualizations to increase compliance in experience sampling. In: Proceedings of the 10th international conference on ubiquitous computing, UbiComp ’08, (New York, NY, USA). ACM, pp 164–167

van Berkel N, Goncalves J, Hosio S, Kostakos V (2017) Gamification of mobile experience sampling improves data quality and quantity. In: Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT), vol 1, no 3, pp 107:1–107:21

Raento M, Oulasvirta A, Eagle N (2009) Smartphones: an emerging tool for social scientists. Sociol Methods Res 37(3):426–454MathSciNetCrossRef

Ferreira D, Kostakos V, Schweizer I (2017) Human sensors on the move. Springer International Publishing, pp 9–19

10.

van Berkel N, Goncalves J, Wac K, Hosio S, Cox AL (2020) Human accuracy in mobile data collection. Int J Hum-Comput Stud, p 102396

11.

Westerink J, Ouwerkerk M, de Vries G, de Waele S, van den Eerenbeemd J, van Boven M (2009) Emotion measurement platform for daily life situations. In: 3rd international conference on affective computing and intelligent interaction and workshops, pp 1–6

12.

Guha S, Wicker SB (2015) Spatial subterfuge: an experience sampling study to predict deceptive location disclosures. In: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing, UbiComp ’15, (New York, NY, USA). Association for Computing Machinery, pp 1131–1135

13.

Heron KE, Smyth JM (2010) Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Brit J Health Psychol 15(1):1–39CrossRef

14.

Shaughnessy JJ, Zechmeister EB, Zechmeister JS (2011) Research methods in psychology. McGraw-Hill, New York

15.

Armantier O, Topa G, Van der Klaauw W, Zafar B (2017) An overview of the survey of consumer expectations. Econ Policy Rev 23–2:51–72

16.

Stein RE, Horwitz SM, Storfer-Isser A, Heneghan A, Olson L, Hoagwood KE (2008) Do pediatricians think they are responsible for identification and management of child mental health problems? Results of the AAP periodic survey. Ambulatory Pediatr 8(1):11–17CrossRef

17.

van Berkel N, Budde M, Wijenayake S, Goncalves J (2018) Improving accuracy in mobile human contributions: an overview. In: Adjunct proceedings of the ACM international joint conference on pervasive and ubiquitous computing, pp 594–599

18.

van Berkel N (2019) Data quality and quantity in mobile experience sampling. Phd thesis, The University of Melbourne

19.

Hektner JM, Schmidt JA, Csikszentmihalyi M (2007) Experience sampling method: measuring the quality of everyday life. Sage

20.

van Berkel N, Goncalves J, Hosio S, Sarsenbayeva Z, Velloso E, Kostakos V (2020) Overcoming compliance bias in self-report studies: a cross-study analysis. Int J Hum-Comput Stud 134:1–12CrossRef

21.

Reiss S (2012) Intrinsic and extrinsic motivation. Teach Psychol 39(2):152–156

22.

Eveleigh A, Jennett C, Blandford A, Brohan P, Cox AL (2014) Designing for dabblers and deterring drop-outs in citizen science. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’14, (New York, NY, USA). ACM, pp 2985–2994

23.

Rotman D, Preece J, Hammock J, Procita K, Hansen D, Parr C, Lewis D, Jacobs D (2012) Dynamic changes in motivation in collaborative citizen-science projects. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, CSCW ’12, (New York, NY, USA). ACM, pp 217–226

24.

Measham TG, Barnett GB (2008) Environmental volunteering: motivations, modes and outcomes. Australian Geographer 39(4):537–552CrossRef

25.

Deci E, Ryan RM (1985) Intrinsic motivation and self-determination in human behavior. Springer, BerlinCrossRef

26.

Consolvo S, Walker M (2003) Using the experience sampling method to evaluate ubicomp applications. IEEE Pervas Comput 2:24–31CrossRef

27.

Musthag M, Raij A, Ganesan D, Kumar S, Shiffman S (2011) Exploring micro-incentive strategies for participant compensation in high-burden studies. In: Proceedings of the 13th international conference on ubiquitous computing, UbiComp ’11, (New York, NY, USA). ACM, pp 435–444

28.

Stone AA, Kessler RC, Haythomthwatte JA (1991) Measuring daily events and experiences: decisions for the researcher. J Personal 59(3):575–607CrossRef

29.

Shih F, Liccardi I, Weitzner D (2015) Privacy tipping points in smartphones privacy preferences. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, CHI ’15, (New York, NY, USA). Association for Computing Machinery, pp 807–816

30.

Tollmar K, Huang C (2015) Boosting mobile experience sampling with social media. In: Proceedings of the 17th international conference on human-computer interaction with mobile devices and services, MobileHCI ’15, (New York, NY, USA). Association for Computing Machinery, pp 525–530

31.

van Berkel N, Goncalves J, Lovén L, Ferreira D, Hosio S, Kostakos V (2019) Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. Int J Hum-Comput Stud 125:118–128CrossRef

32.

Eisele G, Vachon H, Lafit G, Kuppens P, Houben M, Myin-Germeys I, Viechtbauer W (2020) The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

33.

Wheeler L, Reis HT (1991) Self-recording of everyday life events: origins, types, and uses. J Personal 59(3):339–354CrossRef

34.

Barrett LF, Barrett DJ (2001) An introduction to computerized experience sampling in psychology. Soc Sci Comput Rev 19(2):175–185CrossRef

35.

Bolger N, Davis A, Rafaeli E (2003) Diary methods: capturing life as it is lived. Ann Rev Psychol 54(1):579–616CrossRef

36.

Kapoor A, Horvitz E (2008) Experience sampling for building predictive user models: a comparative study. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’08, (New York, NY, USA). Association for Computing Machinery, pp 657–666

37.

Rosenthal S, Dey AK, Veloso M (2011) Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In: Lyons K, Hightower J, Huang EM (eds) Pervasive computing. Springer, Berlin, Heidelberg, pp 170–187CrossRef

38.

Mehrotra A, Vermeulen J, Pejovic V, Musolesi V (2015) Ask, but don’t interrupt: the case for interruptibility-aware mobile experience sampling. In: Adjunct proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2015 ACM international symposium on wearable computers, UbiComp/ISWC’15 Adjunct, (New York, NY, USA). Association for Computing Machinery, pp 723–732

39.

van Berkel N, Goncalves J, Koval P, Hosio S, Dingler T, Ferreira D, Kostakos V (2019) Context-informed scheduling and analysis: improving accuracy of mobile self-reports. In: Proceedings of ACM SIGCHI conference on human factors in computing systems, pp 51:1–51:12

40.

Lathia N, Rachuri KK, Mascolo C, Rentfrow PJ (2013) Contextual dissonance: design bias in sensor-based experience sampling methods. In: Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing, UbiComp ’13, (New York, NY, USA). Association for Computing Machinery, pp 183–192

41.

Carstensen LL, Turan B, Scheibe S, Ram N, Ersner-Hershfield H, Samanez-Larkin GR, Brooks KP, Nesselroade JR (2011) Emotional experience improves with age: evidence based on over 10 years of experience sampling. Psychol Aging 26(1):21–33CrossRef

42.

Savin-Williams RC, Demo DH (1983) Situational and transituational determinants of adolescent self-feelings. J Personal Soc Psychol 44(4):824CrossRef

43.

Bailon C, Damas M, Pomares H, Sanabria D, Perakakis P, Goicoechea C, Banos O (2019) Smartphone-based platform for affect monitoring through flexibly managed experience sampling methods. Sensors 19(15):3430CrossRef

44.

Berrocal A, Wac K (2018) Peer-vasive computing: leveraging peers to enhance the accuracy of self-reports in mobile human studies. In: Proceedings of the 2018 ACM international joint conference and 2018 international symposium on pervasive and ubiquitous computing and wearable computers. ACM, pp 600–605

45.

Chang Y-L, Chang Y-J, Shen C-Y (2019) She is in a bad mood now: leveraging peers to increase data quantity via a chatbot-based ESM. In: Proceedings of the 21st international conference on human-computer interaction with mobile devices and services, MobileHCI ’19 (New York, NY, USA). Association for Computing Machinery

46.

Solymosi R, Bowers K, Fujiyama T (2015) Mapping fear of crime as a context-dependent everyday experience that varies in space and time. Legal Criminol Psychol 20(2):193–211CrossRef

47.

Kittur A, Chi EH, Suh B (2008) Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’08, (New York, NY, USA). Association for Computing Machinery, pp 453–456

48.

Oleson D, Sorokin A, Laughlin G, Hester V, Le J, Biewald L (2011) Programmatic gold: targeted and scalable quality assurance in crowdsourcing. In: Workshops at the Twenty-Fifth AAAI conference on artificial intelligence

49.

Rabbi M, Li K, Yan HY, Hall K, Klasnja P, Murphy S (2019) Revibe: a context-assisted evening recall approach to improve self-report adherence. In: Proceedings of the ACM Interaction Mobile Wearable Ubiquitous Technology, vol 3

50.

Intille S, Kukla C, Ma X (2002) Eliciting user preferences using image-based experience sampling and reflection. In: CHI ’02 extended abstracts on human factors in computing systems, CHI EA ’02, (New York, NY, USA). Association for Computing Machinery, pp 738–739

51.

Yue Z, Litt E, Cai CJ, Stern J, Baxter KK, Guan Z, Sharma N, Zhang GG (2014) Photographing information needs: the role of photos in experience sampling method-style research. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’14, (New York, NY, USA). Association for Computing Machinery, pp 1545–1554

52.

Sprangers MA, Schwartz CE (1999) Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 48(11):1507–1515CrossRef

53.

Schwartz CE, Sprangers MA, Carey A, Reed G (2004) Exploring response shift in longitudinal data. Psychol Health 19(1):51–69CrossRef

54.

Ring L, Höfer S, Heuston F, Harris D, O’Boyle CA (2005) Response shift masks the treatment impact on patient reported outcomes (PROs): the example of individual quality of life in edentulous patients. Health Qual Life Outcomes 3(1):55CrossRef

55.

Schwartz CE, Sprangers MA (2010) Guidelines for improving the stringency of response shift research using the thentest. Qual Life Res 19(4):455–464CrossRef

56.

Christensen TC, Barrett LF, Bliss-Moreau E, Lebo K, Kaschub C (2003) A practical guide to experience-sampling procedures. J Happiness Stud 4(1):53–78CrossRef

57.

Paruthi G, Raj S, Gupta A, Huang C-C, Chang Y-J, Newman MW (2017) Heed: situated and distributed interactive devices for self-reporting. In: Proceedings of the 2017 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2017 ACM international symposium on wearable computers, UbiComp ’17, (New York, NY, USA). Association for Computing Machinery, pp 181–184

58.

Gong Q, He X, Xie Q, Lin S, She G, Fang R, Han R, Chen Y, Xiao Y, Fu X et al (2018) LBSLAB: a user data collection system in mobile environments. In: Proceedings of the 2018 ACM international joint conference and 2018 international symposium on pervasive and ubiquitous computing and wearable computers, UbiComp ’18, (New York, NY, USA). Association for Computing Machinery, pp 624–629

59.

Adams AT, Murnane EL, Adams P, Elfenbein M, Chang PF, Sannon S, Gay G, Choudhury T (2018) Keppi: a tangible user interface for self-reporting pain. In: Proceedings of the 2018 CHI conference on human factors in computing systems, CHI ’18 (New York, NY, USA). Association for Computing Machinery

60.

Hernandez J, McDuff D, Infante C, Maes P, Quigley K, Picard R (2016) Wearable ESM: differences in the experience sampling method across wearable devices. In: Proceedings of the 18th international conference on human-computer interaction with mobile devices and services, MobileHCI ’16, (New York, NY, USA). Association for Computing Machinery, pp 195–205

61.

Liono J, Salim FD, van Berkel N, Kostakos V, Qin AK (2019) Improving experience sampling with multi-view user-driven annotation prediction. In: IEEE international conference on pervasive computing and communications (PerCom, pp 1–11

62.

Pielot M, Vradi A, Park S (2018) Dismissed! a detailed exploration of how mobile phone users handle push notifications. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services, MobileHCI ’18 (New York, NY, USA). Association for Computing Machinery

63.

Visuri A, van Berkel N, Okoshi T, Goncalves J, Kostakos V (2019) Understanding smartphone notifications’ user interactions and content importance. Int J Hum-Comput Stud 128:72–85CrossRef

64.

Weber D, Voit A, Auda J, Schneegass S, Henze N (2018) Snooze! investigating the user-defined deferral of mobile notifications. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services, MobileHCI ’18 (New York, NY, USA). Association for Computing Machinery

Title: Recommendations for Conducting Longitudinal Experience Sampling Studies
Authors: Niels van Berkel
Vassilis Kostakos
Publisher: Springer International Publishing
Book: Advances in Longitudinal HCI Research
Print ISBN: 978-3-030-67321-5

Electronic ISBN: 978-3-030-67322-2

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-67322-2_4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"