Florian Lettner
Clemens Holzmann
ABSTRACT
The evaluation of mobile user interfaces can be a tedious task, especially if usability tests under real-world conditions should be performed. In particular, the evaluation of high-fidelity prototypes provides valuable measures about the quality of mobile applications, which helps designers to identify potentials of improvement for the next revision. Due to their costs or missing expert knowledge evaluation techniques such as cognitive walkthroughs or heuristic evaluation are often not used. Additionally, commercial frameworks provide insufficient details on usability as they only address commercial statistics regarding user loyalty, in-app purchases or demographics.
In this paper, we present a novel approach and toolkit for automated and unsupervised evaluation of mobile applications that, in contrast to existing frameworks, is able to trace any user interaction during the entire lifecycle of an application. As a major novelty, our toolkit can be added to mobile applications without changing application source code, which makes it flexible and scalable for all types of applications. Also, our toolkit is able to identify and visualize design flaws such as navigational errors or efficiency for mobile applications.
- Balagtas-Fernandez, F., and Hussmann, H. A methodology and framework to simplify usability analysis of mobile applications. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (2009). Google Scholar
Digital Library
- Bevan, N., and Macleod, M. Usability measurement in context. Behaviour and Information Technology 13 (1994), 132--145.Google ScholarCross Ref
- Gupta, S., Hanssens, D., Hardie, B., Kahn, W., Kumar, V., Lin, N., and Sriram, N. R. S. Modeling customer lifetime value. Journal of Service Research 9 (2006), 139--155.Google ScholarCross Ref
- Hertzum, M. User testing in industry: A case study of laboratory, workshop, and field tests. In Proceedings of the 5th ERCIM Workshop on "User Interfaces for All" (1999).Google Scholar
- Horton, S. Access by Design: A Guide to Universal Usability for Web Designers. New Riders Publishing, 2005. Google ScholarDigital Library
- Kaikkonen, A., Kallio, T., Kekäläinen, A., Kankainen, A., and Cankar, M. Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability Studies 1 (2005), 4--16.Google ScholarDigital Library
- Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C. V., Loingtier, J.-M., and Irwin, J. Aspect-oriented programming. In Proceedings of the European Conference on Object-Oriented Programming (ECOOP) (1997).Google ScholarDigital Library
- Lettner, F., and Holzmann, C. Sensing mobile phone interaction in the field. In Proceedings of the 4th International Workshop on Sensor Networks and Ambient Intelligence (SeNAml 2012) (2012).Google ScholarCross Ref
- Lidwell, W., Holden, K., and Butler, J. Universal Principles of Design: 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions, and Teach through Design. Rockport Publishers, 2010.Google Scholar
- MacKenzie, S., and Read, J. C. Using paper mockups for evaluating soft keyboard layouts. In Proceedings of the 2007 Conference of the Center for Advanced Studies on Collaborative Research (2007). Google ScholarDigital Library
- Madrigal, D., and McClain, B. Usability for mobile devices, September 2010. http://www.uxmatters.com/mt/archives/2010/09/usability-for-mobile-devices.php.Google Scholar
- Nael, M. Design issues for usability of residential multifunction terminals. IEEE Journal on Selected Areas in Communications 9 (1991), 518--523. Google ScholarDigital Library
- Nielsen, J. Usability Engineering. Morgan Kaufmann Publishers Inc., 1993. Google ScholarDigital Library
- Nielsen, J. Usability metrics: Tracking interface improvements. IEEE Software 13 (1996), 12--13. Google ScholarDigital Library
- Nielsen, J. Usability 101: Introduction to usability, August 2003.Google Scholar
- Nielsen, J. Medical usability: How to kill patients through bad design, April 2005.Google Scholar
- Oztoprak, A., and Erbug, C. Field versus laboratory usability testing: A first comparison. Tech. rep., Department of Industrial Design - Middle East Technical University, Faculty of Architecture, 2008.Google Scholar
- Polson, P. G., Lewis, C., Rieman, J., and Wharton, C. Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies 36 (1992), 741--773. Google ScholarDigital Library
- Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Addison-Wesley Longman Publishing Co., Inc., 1986. Google ScholarDigital Library
- Tullis, T., and Albert, W. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann Publishers Inc., 2008. Google ScholarDigital Library
- WebAnalyticsAssociation. Webanalyticsdefinitions vol1. Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License, August 2007. http://www.webanalyticsassociation.org/.Google Scholar
- Wilson, P. I. Active story: A low fidelity prototyping and distributed usability testing tool for agile teams. Master's thesis, University of Calgary - Department of Computer Science, 2008.Google Scholar
- Xerox. The AspectJ programming guide. Palo Alto Research Center, 2003.Google Scholar
- Zduniak, M. Automated gui testing of mobile java applications. Master's thesis, Poznan University of Technology Faculty of Computer Science and Management, 2007.Google Scholar
Index Terms
- Automated and unsupervised user interaction logging as basis for usability evaluation of mobile applications
Recommendations
Evaluating Usability for Mobile Application: A MAUEM Approach
ICSEB '17: Proceedings of the 2017 International Conference on Software and e-BusinessMany traditional usability metrics were purposely created for desktop computer applications and may not be directly applicable to mobile device applications. Usability metrics refer as means of measuring or evaluating the usability factors, attributes ...
Gamification in mobile applications usability evaluation: A New Approach
MobileHCI '15: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services AdjunctUsability evaluation of mobile application interfaces is continuously gaining importance, due to the spread of mobile devices in everyday life and to the varied nature of mobile applications. Unfortunately, usability is always a secondary aspect when ...
Usability risk level evaluation for physical user interface of mobile phone
The mobile phone has become an indispensable device in everyday life. However, many usability problems were generated by the multi-functionality of mobile phones. There was an important increase on the user interface (UI) design and usability. With the ...
Comments