Robert LiKamWa
Yunxin Liu
ABSTRACT
We report a first-of-its-kind smartphone software system, MoodScope, which infers the mood of its user based on how the smartphone is used. Compared to smartphone sensors that measure acceleration, light, and other physical properties, MoodScope is a "sensor" that measures the mental state of the user and provides mood as an important input to context-aware computing. We run a formative statistical mood study with smartphone-logged data collected from 32 participants over two months. Through the study, we find that by analyzing communication history and application usage patterns, we can statistically infer a user's daily mood average with an initial accuracy of 66%, which gradu-ally improves to an accuracy of 93% after a two-month personal-ized training period. Motivated by these results, we build a service, MoodScope, which analyzes usage history to act as a sensor of the user's mood. We provide a MoodScope API for developers to use our system to create mood-enabled applications. We further create and deploy a mood-sharing social application.
- J. A. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), pp. 1161--1178, 1980.Google Scholar
Cross Ref
- C. D. Batson, L. L. Shaw and K. C. Oleson. Distinguishing affect, mood and emotion: toward functionally-based conceptual distinctions. Review of Personality and Social Psychology, 13, Newbury Park, 1992.Google Scholar
- P. Ekman. Universal and cultural differences in facial expressions of emotion. In Proc. Nebraska Symp. Motivation, 1971.Google Scholar
- S. S. Tomkins. Affect, Imagery, Consciousness. Springer, 1962.Google Scholar
- J. A. Russell. Affect grid: a single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57(3), 1989.Google ScholarCross Ref
- J. Tao and T. Tan. Affective computing: a review. Affective Computing and Intelligent Interaction, LNCS 3784, Springer, 2005. Google ScholarDigital Library
- M. Yik, Studying Affect among the Chinese: The circular way. Journal of Personality Assessment, 91(5), 2009.Google ScholarCross Ref
- K. Church, E. Hoggan and N. Oliver. A study of mobile mood awareness and communication through MobiMood. In Proc. NordiCHI, 2010. Google ScholarDigital Library
- A. Gluhak, M. Presser, L. Zhu, S. Esfandiyari and S. Kupschick. Towards mood based mobile services and applica-tions. In Proc. European Conference on Smart Sensing & Context (EuroSSC), 2007. Google ScholarDigital Library
- D. Watson, L.A. Clark, and A. Tellegen. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, pages 1063--1070, 1988.Google ScholarCross Ref
- J. R. Crawford and J. D. Henry. The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 43, pages 245--265, 2004.Google ScholarCross Ref
- J. T. Larsen, A. P. McGraw and J. T. Cacioppo, Can people feel happy and sad at the same time? Journal of Personality and Social Psychology, 81(4), 1988.Google Scholar
- Z. Zeng, M. Pantic, G. I. Roisman and T. S. Huang. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 2009. Google ScholarDigital Library
- A. Vinciarelli, M. Pantic and H. Bourlard. Social signal processing: survey of an emerging domain. Image and Vision Computing, 2009. Google ScholarDigital Library
- J. F. Cohn. Foundations of human computing: Facial expression and emotion. In Proc. ACM Int'l Conf. Multimodal Inter-faces (ICMI), 2006. Google ScholarDigital Library
- M. Pantic and M.S. Bartlett. Machine analysis of facial expressions. Face Recognition, K. Delac and M. Grgic, eds., pages 377--416, I-Tech Education and Publishing, 2007.Google Scholar
- P. Ekman, W.V. Friesen, and J.C. Hager. Facial action coding system. A Human Face, 2002.Google Scholar
- C.M. Lee and S.S. Narayanan. Toward detecting emotions in spoken dialogs. IEEE Trans. Speech and Audio Processing, 13(2), pages 293--303, 2005.Google Scholar
- B. Schuller, R.J. Villar, G. Rigoll, and M. Lang. Meta-classifiers in acoustic and linguistic feature fusion-based affect recognition. In Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing (ICASSP '05), pages 325--328, 2005.Google ScholarCross Ref
- R. W. Picard, Affective computing, MIT Press, 1997. Google ScholarDigital Library
- The Affectiva Q sensor, http://www.affectiva.com/q-sensor/.Google Scholar
- A. Gluhak, M. Presser, L. Zhu, S. Esfandiyari and S. Kupschick. Towards mood based mobile services and applications. In Proc. European Conference on Smart Sensing & Context (EuroSSC), 2007. Google ScholarDigital Library
- K. K. Rachuri, M. Musolesi, C. Mascolo, P. J. Rentfrow, C. Longworth and A. Aucinas. EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2010. Google ScholarDigital Library
- H. Lee, Y. Choi, S. Lee and I. P. Park. Towards unobtrusive emotion recognition for affective social communication. In Proc. IEEE Consumer Communications & Networking Conference (CCNC), 2012.Google ScholarCross Ref
- The mappiness application, http://www.mappiness.org.uk/.Google Scholar
- The gottaFeeling application, http://gottafeeling.com/.Google Scholar
- C. Shepard, A. Rahmati, C. Tossell, L. Zhong and P. Kortum. LiveLab: measuring wireless networks and smartphone users in the field. ACM SIGMETRICS Perform. Eval. Rev., 38(3), 2010. Google ScholarDigital Library
- J. P. Forgas, G. H. Bower and S.E. Krantz. The influence of mood on perceptions of social interactions, Journal of Experimental Social Psychology, 20(6), 1984.Google Scholar
- L. A. Clark and D. Watson. Mood and the mundane: Rela-tions between daily life events and self-reported mood. Journal of Personality and Social Psychology, 54(2), 1988.Google ScholarCross Ref
- M. Ester, H.-P. Kriegel, J. Sander and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. Second International Conference on Knowledge Discovery and Data Mining, 1996.Google ScholarDigital Library
- C. J. Beedie, P. C. Terry and A. M. Lane. Distinctions between emotion and mood. Cognition and Emotion, 19(6), 2005.Google Scholar
- Hernandez, Javier, et al. "Mood meter: counting smiles in the wild." In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2012. Google ScholarDigital Library
- G. Chittaranjan, J. Blom and D. Gatica-Perez. Who's who with Big-Five: Analyzing and Classifying personality traits with smartphones. In Proc. Int'l Symposium on Wearable Computers (ISWC), 2011. Google ScholarDigital Library
- L. Lu, D. Liu and H. Zhang. Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio, Speech and Language Processing, 14(1), 2006. Google ScholarDigital Library
- N. Japkowicz and S. Stephen. The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 2002. Google ScholarDigital Library
- A. Jain and D. Zongker. Feature selection: Evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2), 1997. Google ScholarDigital Library
- A. Narayanan and V. Shmatikov. Myths and fallacies of Personally Identifiable Information. From Commun. ACM 53(6), pages 24--26, June 2010. Google ScholarDigital Library
- N. D. Lane, Y. Xu, H. Lu, S. Hu, T. Choudhury, A. T. Camp-bell and F. Zhao. Enabling large-scale human activity inference on smartphones using community similarity networks (CSN). In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2011. Google ScholarDigital Library
- R. LiKamWa, Y. Liu, N. D. Lane and L. Zhong. "Can your smartphone infer your mood?", In Proc. PhoneSense Work-shop, 2011.Google Scholar
- Hong Lu, A. J. Bernheim Brush, Bodhi Priyantha, Amy K. Karlson, and Jie Liu. SpeakerSense: energy efficient unobtrusive speaker identification on mobile phones. In Proc. Int'l Conf. on Pervasive Computing, 2011. Google ScholarDigital Library
Index Terms
- MoodScope: building a mood sensor from smartphone usage patterns
Recommendations
MoodScope: building a mood sensor from smartphone usage patterns
MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and servicesWe present MoodScope, a software system which infers the mood of its user based on how the smartphone is used. Similar to smartphone sensors that measure acceleration, light, and other physical properties, MoodScope is a "sensor" that measures the ...
Comparing ESM Timings for Emotional Estimation Model with Fine Temporal Granularity
UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable ComputersTowards wellbeing-awareness in computing, researches for estimating users' emotions using smartphone sensor data have been actively conducted as smartphones are getting more and more ubiquitous. Most studies have constructed emotion estimation models ...
A Large-Scale Study of iPhone App Launch Behaviour
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsThere have been many large-scale investigations of users' mobile app launch behaviour, but all have been conducted on Android, even though recent reports suggest iPhones account for a third of all smartphones in use. We report on the first large-scale ...
Comments