Kerry Rodden
ABSTRACT
More and more products and services are being deployed on the web, and this presents new challenges and opportunities for measurement of user experience on a large scale. There is a strong need for user-centered metrics for web applications, which can be used to measure progress towards key goals, and drive product decisions. In this note, we describe the HEART framework for user-centered metrics, as well as a process for mapping product goals to metrics. We include practical examples of how HEART metrics have helped product teams make decisions that are both data-driven and user-centered. The framework and process have generalized to enough of our company's own products that we are confident that teams in other organizations will be able to reuse or adapt them. We also hope to encourage more research into metrics based on large-scale behavioral data.
- Akers, D. et al. (2009). Undo and Erase Events as Indicators of Usability Problems. Proc of CHI 2009, ACM Press, pp. 659--668. Google Scholar
Digital Library
- Burby, J. & Atchison, S. (2007). Actionable Web Analytics. Indianapolis: Wiley Publishing, Inc.Google Scholar
- Chi, E. et al. (2002). LumberJack: Intelligent Discovery and Analysis of Web User Traffic Composition. Proc of WebKDD 2002, ACM Press, pp. 1--15.Google Scholar
- Dean, J. & Ghemawat, S. (2008). MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51 (1), pp. 107--113. Google ScholarDigital Library
- Google Analytics: http://www.google.com/analyticsGoogle Scholar
- Grimes, C. et al. (2007). Query Logs Alone are not Enough. Proc of WWW 07 Workshop on Query Log Analysis: http://querylogs2007.webir.orgGoogle Scholar
- Gwizdka, J. & Spence, I. (2007). Implicit Measures of Lostness and Success in Web Navigation. Interacting with Computers 19(3), pp. 357--369. Google ScholarDigital Library
- Hadoop: http://hadoop.apache.org/coreGoogle Scholar
- Kaushik, A. (2007). Web Analytics: An Hour a Day. Indianapolis: Wiley Publishing, Inc. Google ScholarDigital Library
- Kohavi, R. et al. (2007). Practical Guide to Controlled Experiments on the Web. Proc of KDD 07, ACM Press, pp. 959--967. Google ScholarDigital Library
- Omniture: http://www.omniture.comGoogle Scholar
- Pike, R. et al. (2005). Interpreting the Data: Parallel Analysis with Sawzall. Scientific Programming (13), pp. 277--298. Google ScholarDigital Library
- Tullis, T. & Albert, W. (2008). Measuring the User Experience. Burlington: Morgan Kaufmann. Google ScholarDigital Library
- UserZoom: http://www.userzoom.comGoogle Scholar
- Weischedel, B. & Huizingh, E. (2006). Website Optimization with Web Metrics: A Case Study. Proc of ICEC 06, ACM Press, pp. 463--470. Google ScholarDigital Library
Index Terms
- Measuring the user experience on a large scale: user-centered metrics for web applications
Recommendations
Quantitatively measuring a large-scale agile transformation
We provide a quantitative metrics model to evaluate the impact of an agile and lean transformation.We propose eight rigorously described metrics within that model.We establish and apply the model in a large international telecommunication organization ...
Measuring OO Systems: A Critical Analysis of the MOOD Metrics
TOOLS '99: Proceedings of the Technology of Object-Oriented Languages and SystemsIn parallel with the rise to prominence of the OO paradigm has come the acceptance that conventional software metrics are not adequate to measure object-oriented systems. This has inspired a number of software practitioners and academics to develop new ...
Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical Study
Background. Slice-based cohesion metrics leverage program slices with respect to the output variables of a module to quantify the strength of functional relatedness of the elements within the module. Although slice-based cohesion metrics have been ...
Comments