skip to main content
10.1145/3160504.3160548acmotherconferencesArticle/Chapter ViewAbstractPublication PagesihcConference Proceedingsconference-collections
research-article

Beyond traditional evaluations: user's view in app stores

Published:23 October 2017Publication History

ABSTRACT

The choice of apps by users in App Stores is influenced by other users' ratings (reviews and star ratings). However, it is unclear what the relationship between user appraisal and an Usability and User eXperience (UUX) evaluation. In this paper, we will make the following investigations: i) what is the result of an UUX evaluation of mobile applications using user reviews in App Stores? ii) does the star rating represent the opinions of users about an app? and iii) is there a relationship between the ranking of apps and an evaluation of the UUX? Some results: i) the result of an UUX evaluation was different for the same app on different systems; ii) the star rating was in agreement with the opinions of the users, in two of the three apps tested; and iii) the results of an UUX evaluation are similar to the ranking of apps.

References

  1. Atualização do Waze no Windows 10 Mobile. Acessado em 12 de junho de 2016. Disponível em: https://pplware.sapo.pt/microsoft/windows/actualizacoes-waze-windows-10-mobile-nao/Google ScholarGoogle Scholar
  2. J. Bargas-Avila, K. Hornbæk 2011. Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'11), 2689--2698. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. N. Bevan, 2008. Classifying and selecting UX and usability measures. In International Workshop on Meaningful Measures: Valid Useful User Experience Measurement, 13--18.Google ScholarGoogle Scholar
  4. Como funcionam os rankings das Apps Stores. Acessado em 15 de abril de 2017. Disponível em: http://www.rankmyapp.com.br/blog/como-funcionamos-rankings-das-app-stores/Google ScholarGoogle Scholar
  5. L. Freitas; T. Silva; M. Mendes 2016. Avaliação do Spotify -- uma experiência de avaliação textual utilizando a metodologia MALTU. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computer Systems (IHC'16), 50.Google ScholarGoogle Scholar
  6. B. Fu; J. Lin; L. Li; C. Faloutsos; J. Hong; N. Sadeh 2013. Why People Hate Your App: Making Sense of User Feedback in a Mobile App Store. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD'13), 1276--1284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Google Maps. Acesso em 28 de fevereiro de 2017. Disponível em: https://play.google.com/store/apps/details?id=com.google.android.apps.maps&hl=pt_BRGoogle ScholarGoogle Scholar
  8. HackStore. Acesso em 22 de maio de 2016. Disponível em: http://hack-store.comGoogle ScholarGoogle Scholar
  9. M. Hassenzahl; S. Diefenbach; A. Göritz 2010. Needs, affect, and interactive products -- Facets of user experience. Interacting with Computers vol 22, 5: 353--362. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Hedegaard; J. Simonsen 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13), 2089--2098. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. L. Hoon; R. Vasa; G. Martino; J. Schneider; K. Mouzakis 2013. Awesome! Conveying Satisfaction on the App Store. In Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration (OZCHI'13), 229--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. ISO 9241-11. ISO 9241-11. Geneve: International Organization For Standardization, 1998.Google ScholarGoogle Scholar
  13. P. Ketola; V. Roto 2008. Exploring user experience measurement needs. In Proceedings of the 5th COST294-MAUSE Open Workshop on Valid Useful User Experience Measurement (VUUM), 23--26.Google ScholarGoogle Scholar
  14. D. Krutz; N. Munaiah; A. Mennely; S. Malachowsky 2016. Examining the Relationship between Security Metrics and User Ratings of Mobile Apps: A Case Study. In Proceedings of the International Workshop on App Market Analytics (WAMA'16), 8--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. H. Korhonen; J. Arrasvuori; K. Väänänen-Vainio-Mattila 2010. Let Users Tell the Story: Evaluating User Experience with Experience Reports. In Proceeding Extended Abstracts on Human Factors in Computing Systems (CHI'10), 4051--4056. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Lima; P. Silva; L. Cruz; M. Mendes 2017. Investigating the polarity of user postings in a Social System. In 19th International Conference on Human-Computer Interaction (HCII 2017).Google ScholarGoogle ScholarCross RefCross Ref
  17. M. Mendes; E. Furtado; M. Castro 2014. Do users write about the system in use? An investigation from messages in Natural Language on Twitter. In Proceedings of the 7th Euro American Conference on Telematics and Information Systems (EATIS'14), 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. M. Mendes; E. Furtado; M. Castro 2015. Investigating Usability and User Experience from the user postings in Social Systems. In 17th International Conference on Human-Computer Interaction (HCII 2015).Google ScholarGoogle ScholarCross RefCross Ref
  19. M. Mendes; E. Furtado 2017. UUX-Posts: a tool for extracting and classifying postings related to the use of a system. In VIII Latin American Conference on Human-Computer Interaction (CLIHC 2017). Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Mendes 2015. MALTU -- Um modelo para avaliação da interação em sistemas sociais a partir da linguagem textual do usuário. Tese (doutorado). Universidade Federal do Ceará, Programa de Pós-Graduação em Ciência da Computação, Fortaleza, CE.Google ScholarGoogle Scholar
  21. O que é landing page. Acesso em 25 de abril de 2017. Disponível em: http://www.seomaster.com.br/blog/oque-e-landing-pageGoogle ScholarGoogle Scholar
  22. T. Olsson; M. Salo 2012. Narratives of satisfying and unsatisfying experiences of current mobile augmented reality applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2012), 2779--2788. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. T. Partala; A. Kallinen. Understanding the most satisfying and unsatisfying user experiences: Emotions, psychological needs, and context. Interacting with Computers vol 24, 1: 25--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Preece; Y. Rogers; H. Sharp 2013. Design de interação: além da interação homem-computador. Bookman.Google ScholarGoogle Scholar
  25. Problemas que geram reviews negativos na App Store. Acesso em 25 de abril de 2017. Disponível em: http://blog.onedaytesting.com.br/problemas-e-reviews-negativos-na-app-store/Google ScholarGoogle Scholar
  26. J. Russell 1989. Measures of emotion. In R. Plutchik & H. Kellerman (Eds.), Emotion: Theory, research, and experience. Toronto: Academic.Google ScholarGoogle Scholar
  27. Sobre o Google Maps. Acesso em 04 de junho de 2017. Disponível em: https://www.google.com/intl/pt-BR/maps/about/Google ScholarGoogle Scholar
  28. A. Tuch; R. Trusell; K. Hornbæk 2013. Analyzing Users' Narratives to Understand Experience with Interactive Products. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), 2079--2088. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Waze para Android. Acesso em 28 de fevereiro de 2017. Disponível em: https://play.google.com/store/apps/details?id=com.waze&hl=ptGoogle ScholarGoogle Scholar

Index Terms

  1. Beyond traditional evaluations: user's view in app stores

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems
      October 2017
      622 pages
      ISBN:9781450363778
      DOI:10.1145/3160504

      Copyright © 2017 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 23 October 2017

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed limited

      Acceptance Rates

      IHC '17 Paper Acceptance Rate66of184submissions,36%Overall Acceptance Rate331of973submissions,34%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader