Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access July 22, 2020

AutisMitr: Emotion Recognition Assistive Tool for Autistic Children

  • Akansha Singh EMAIL logo and Surbhi Dewan
From the journal Open Computer Science

Abstract

Assistive technology has proven to be one of the most significant inventions to aid people with Autism to improve the quality of their lives. In this study, a real-time emotion recognition system for autistic children has been developed. Emotion recognition is implemented by executing three stages: Face identification, Facial Feature extraction, and feature classification. The objective is to frame a system that includes all three stages of emotion recognition activity that executes expeditiously in real time. Thus, Affectiva SDK is implemented in the application. The propound system detects at most 7 facial emotions: anger, disgust, fear, joy, sadness, contempt, and surprise. The purpose for performing this study is to teach emotions to individuals suffering from autism, as they lack the ability to respond appropriately to others emotions. The proposed application was tested with a group of typical children aged 6–14 years, and positive outcomes were achieved.

References

[1] Kanner, L. (1943). Autistic disturbances of affective contact. Nervous child, 2(3), 217-250.Search in Google Scholar

[2] American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub.Search in Google Scholar

[3] Baio, J. (2014). Prevalence of autism spectrum disorder among children aged 8 years-autism and developmental disabilities monitoring network, 11 sites, United States, 2010.Search in Google Scholar

[4] Baron-Cohen, S., & Wheelwright, S. (2004). The empathy quotient: an investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. Journal of autism and developmental disorders, 34(2), 163-175.10.1023/B:JADD.0000022607.19833.00Search in Google Scholar

[5] Magdin, M., & Prikler, F. (2018). Real time facial expression recognition using webcam and SDK affectiva. IJIMAI, 5(1), 7-15.10.9781/ijimai.2017.11.002Search in Google Scholar

[6] Capps, L., Kasari, C., Yirmiya, N., & Sigman, M. (1993). Parental perception of emotional expressiveness in children with autism. Journal of Consulting and Clinical Psychology, 61(3), 475.10.1037/0022-006X.61.3.475Search in Google Scholar

[7] McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & Kaliouby, R. E. (2016, May). AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems (pp. 3723-3726). ACM.10.1145/2851581.2890247Search in Google Scholar

[8] Mazefsky, C. A., Herrington, J., Siegel, M., Scarpa, A., Maddox, B. B., Scahill, L., & White, S. W. (2013). The role of emotion regulation in autism spectrum disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 52(7), 679-688.10.1016/j.jaac.2013.05.006Search in Google Scholar

[9] Lopresti, E. F., Bodine, C., & Lewis, C. (2008). Assistive technology for cognition [Understanding the Needs of Persons with Disabilities. IEEE Engineering in Medicine and Biology Magazine, 27(2), 29-39.10.1109/EMB.2007.907396Search in Google Scholar

[10] Knight, V., McKissick, B. R., & Saunders, A. (2013). A review of technology-based interventions to teach academic skills to students with autism spectrum disorder. Journal of autism and developmental disorders, 43(11), 2628-2648.10.1007/s10803-013-1814-ySearch in Google Scholar

[11] Aresti-Bartolome, N., & Garcia-Zapirain, B. (2014). Technologies as support tools for persons with autistic spectrum disorder: a systematic review. International journal of environmental research and public health, 11(8), 7767-7802.10.3390/ijerph110807767Search in Google Scholar

[12] Blasco, S., Cerro, P., Elena, M., & Uceda, J. D. (2009). Autism and Technology: an approach to new technology-based therapeutic tools. In World Congress on Medical Physics and Biomedical Engineering, September 7-12, 2009, Munich, Germany (pp. 340-343). Springer, Berlin, Heidelberg.10.1007/978-3-642-03893-8_97Search in Google Scholar

[13] Mirenda, P. (2003). Toward functional augmentative and alternative communication for students with autism. Language, speech, and hearing services in schools.10.1044/0161-1461(2003/017)Search in Google Scholar

[14] Liddle, K. (2001). Implementing the picture exchange communication system (PECS). International journal of language & communication disorders, 36(S1), 391-395.10.3109/13682820109177917Search in Google Scholar PubMed

[15] Cortiella, C., & Horowitz, S. H. (2014). The state of learning disabilities: Facts, trends and emerging issues. New York: National Center for Learning Disabilities, 25.Search in Google Scholar

[16] Hayes, G. R., Hirano, S., Marcu, G., Monibi, M., Nguyen, D. H., & Yeganyan, M. (2010). Interactive visual supports for children with autism. Personal and ubiquitous computing, 14(7), 663-680.10.1007/s00779-010-0294-8Search in Google Scholar

[17] Noor, H. A. M., Shahbodin, F., & Pee, N. C. (2012). Serious game for autism children: review of literature. World Academy of Science, Engineering and Technology, 64(124), 647-652.Search in Google Scholar

[18] Rahman, M. M., Ferdous, S. M., & Ahmed, S. I. (2010, December). Increasing intelligibility in the speech of the autistic children by an interactive computer game. In 2010 IEEE International Symposium on Multimedia (pp. 383-387). IEEE.10.1109/ISM.2010.64Search in Google Scholar

[19] Alqahtani, A., Jaafar, N., & Alfadda, N. (2011, October). Interactive speech based games for autistic children with asperger syndrome. In The 2011 International Conference and Workshop on Current Trends in Information Technology (CTIT 11) (pp. 126-131). IEEE.10.1109/CTIT.2011.6107947Search in Google Scholar

[20] Anwar, A., Rahman, M. M., Ferdous, S. M., Anik, S. A., & Ahmed, S. I. (2011, July). A computer game based approach for increasing fluency in the speech of the autistic children. In 2011 IEEE 11th International Conference on Advanced Learning Technologies (pp. 17-18). IEEE.10.1109/ICALT.2011.13Search in Google Scholar

[21] Hassan, A. Z., Zahed, B. T., Zohora, F. T., Moosa, J. M., Salam, T., Rahman, M. M., ... & Ahmed, S. I. (2011, December). Developing the concept of money by interactive computer games for autistic children. In 2011 IEEE International Symposium on Multimedia (pp. 559-564). IEEE.10.1109/ISM.2011.99Search in Google Scholar

[22] Gates, K. (2004). The past perfect promise of facial recognition technology. ACDIS Occasional Paper.Search in Google Scholar

[23] Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM computing surveys (CSUR), 35(4), 399-458.10.1145/954339.954342Search in Google Scholar

[24] Baron-Cohen, S., Golan, O., & Ashwin, E. (2009). Can emotion recognition be taught to children with autism spectrum conditions?. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3567-3574.Search in Google Scholar

[25] Cheng, L., Kimberly, G., & Orlich, F. (2002). KidTalk: online therapy for Asperger’s syndrome. Microsoft Research.Search in Google Scholar

[26] Goldsmith, T. R., & LeBlanc, L. A. (2004). Use of technology in interventions for children with autism. Journal of Early and Intensive Behavior Intervention, 1(2), 166.10.1037/h0100287Search in Google Scholar

[27] Dautenhahn, K., & Werry, I. (2004). Towards interactive robots in autism therapy: Background, motivation and challenges. Pragmatics & Cognition, 12(1), 1-35.10.1075/pc.12.1.03dauSearch in Google Scholar

[28] Robins, B., Dautenhahn, K., & Dickerson, P. (2009, February). From isolation to communication: a case study evaluation of robot assisted play for children with autism with a minimally expressive humanoid robot. In 2009 Second International Conferences on Advances in Computer-Human Interactions(pp. 205-211). IEEE.10.1109/ACHI.2009.32Search in Google Scholar

[29] El Kaliouby, R., & Robinson, P. (2005). The emotional hearing aid: an assistive tool for children with Asperger syndrome. Universal Access in the Information Society, 4(2), 121-134.10.1007/s10209-005-0119-0Search in Google Scholar

[30] Ekman, P., & Friesen, W. V. (1978). Manual for the facial action coding system. Consulting Psychologists Press.Search in Google Scholar

[31] Pantic, M., & Rothkrantz, L. J. (2000). Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis & Machine Intelligence, (12), 1424-1445.10.1109/34.895976Search in Google Scholar

[32] Barrett, L. F., Mesquita, B., Ochsner, K. N., & Gross, J. J. (2007). The experience of emotion. Annu. Rev. Psychol., 58, 373-403.10.1146/annurev.psych.58.110405.085709Search in Google Scholar

[33] Magdin, M., Benko, Ľ., & Koprda, Š. (2019). A Case Study of Facial Emotion Classification Using Affdex. Sensors, 19(9), 2140.10.3390/s19092140Search in Google Scholar

[34] Batty, M., & Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cognitive Brain Research, 17(3), 613-620.10.1016/S0926-6410(03)00174-5Search in Google Scholar

[35] Leony, D., Muñoz-Merino, P. J., Pardo, A., & Kloos, C. D. (2013). Provision of awareness of learners’ emotions through visualizations in a computer interaction-based environment. Expert Systems with Applications, 40(13), 5093-5100.10.1016/j.eswa.2013.03.030Search in Google Scholar

[36] Wells, L. J., Gillespie, S. M., & Rotshtein, P. (2016). Identification of emotional facial expressions: Effects of expression, intensity, and sex on eye gaze. PloS one, 11(12), e0168307.10.1371/journal.pone.0168307Search in Google Scholar PubMed PubMed Central

[37] Bartlett, M. S., Viola, P. A., Sejnowski, T. J., Golomb, B. A., Larsen, J., Hager, J. C., & Ekman, P. (1996). Classifying facial action. In Advances in neural information processing systems (pp. 823-829).Search in Google Scholar

[38] Wong, B., Cronin-Golomb, A., & Neargarder, S. (2005). Patterns of visual scanning as predictors of emotion identification in normal aging. Neuropsychology, 19(6), 739.10.1037/0894-4105.19.6.739Search in Google Scholar PubMed

[39] Jensen, O. H. (2008). Implementing the Viola-Jones face detection algorithm (Master’s thesis, Technical University of Denmark, DTU, DK-2800 Kgs. Lyngby, Denmark).Search in Google Scholar

[40] Dalal, N., & Triggs, B. (2005, June). Histograms of oriented gradients for human detection. In international Conference on computer vision & Pattern Recognition (CVPR’05) (Vol. 1, pp. 886-893). IEEE Computer Society.Search in Google Scholar

[41] Magdin, M., & Prikler, F. (2018). Real Time Facial Expression Recognition Using Webcam and SDK Affectiva. International Journal of Interactive Multimedia & Artificial Intelligence, 5(1).10.9781/ijimai.2017.11.002Search in Google Scholar

[42] McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & Kaliouby, R. E. (2016, May). AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems (pp. 3723-3726). ACM.10.1145/2851581.2890247Search in Google Scholar

Received: 2019-09-26
Accepted: 2019-12-05
Published Online: 2020-07-22

© 2020 Akansha Singh et al., published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 3.6.2024 from https://www.degruyter.com/document/doi/10.1515/comp-2020-0006/html
Scroll to top button