Hidetake Uwano
Akito Monden
ABSTRACT
This paper proposes to use eye movements to characterize the performance of individuals in reviewing source code of computer programs. We first present an integrated environment to measure and record the eye movements of the code reviewers. Based on the fixation data, the environment computes the line number of the source code that the reviewer is currently looking at. The environment can also record and play back how the eyes moved during the review process. We conducted an experiment to analyze 30 review processes (6 programs, 5 subjects) using the environment. As a result, we have identified a particular pattern, called scan, in the subjects' eye movements. Quantitative analysis showed that reviewers who did not spend enough time for the scan tend to take more time for finding defects.
- Basili, V. R., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sørumgärd, S., and Zelkowitz, M. V. 1996. The Empirical Investigation of Perspective-Based Reading, Empirical Software Engineering: An International Journal, 1, 2, 133--163.Google Scholar
Cross Ref
- Boehm, B. W. 1981. Software Engineering Economics. Prentice Hall. Google ScholarDigital Library
- Bojko, A., and Stephenson, A. 2005. Supplementing Conventional Usability Measures with Eye Movement Data in Evaluating Visual Search Performance. In Proceedings of the 11th International Conference on Human-Computer Interaction (HCII 2005).Google Scholar
- Ciolkowski, M., Laitenberger, O., Rombach, D., Shull, F., and Perry, D. 2002. Software Inspection, Reviews & Walkthroughs. In Proceedings of the International Conference on Software Engineering (ICSE), 641--642. Google ScholarDigital Library
- Crosby, M. E., and Stelovsky, J. 1990. How Do We Read Algorithms? A Case Study. IEEE Computer, 23, 1, 24--35. Google ScholarDigital Library
- Fagan, M. E. 1976. Design and Code Inspections to Reduce Errors in Program Development. IBM Systems Journal, 15, 3, 182--211.Google ScholarDigital Library
- Fusaro, P., Lanubile, F., and Visaggio, G. 1997. A Replicated Experiment to Assess Requirements Inspection Techniques. Empirical Software Engineering: An International Journal, 2, 1, 39--57. Google ScholarDigital Library
- Halling, M., Biffl, S., Grechenig, T., and Kohle, M. 2001. Using Reading Techniques to Focus Inspection Performance. In Proceedings of 27th Euromicro Workshop Software Process and Product Improvement, 248--257.Google Scholar
- Jacob, R. J. K. 1995. Eye Tracking in Advanced Interface Design. Virtual environments and advanced interface design, Oxford University Press, 258--288. Google ScholarDigital Library
- Kasarskis, P., Stehwien, J., Hichox, J., Aretz, A., and Wickens C. 2001. Comparison of Expert and Novice Scan Behaviors during VFR Flight. In Proceedings of the 11th International Symposium on Aviation Psychology, http://www.aviation.uiuc.edu/UnitsHFD/conference/proced01.pdfGoogle Scholar
- Lanubile, F., and Visaggio, G. 2000. Evaluating Defect Detection Techniques for Software Requirements Inspections. ISERN Technical Report, 00, 08.Google Scholar
- Law, B., Atkins, M. S., Kirkpatrick, A. E., Lomax, A. J., and Mackenzie, C. L. 2004. Eye Gaze Patterns Differentiate Novice and Expert in a Virtual Laparoscopic Surgery Training Environment. In Proceedings of ACM Symposium of Eye Tracking Research and Applications (ETRA), 41--48. Google ScholarDigital Library
- Miller, J., Wood, M., Roper, M., and Brooks, A. 1998. Further Experiences with Scenarios and Checklists. Empirical Software Engineering: An International Journal, 3, 3, 37--64. Google ScholarDigital Library
- Nakamichi, N., Sakai, M., Hu, J., Shima, K., Nakamura, M., and Matsumoto, K. 2003. Web-Tracer: Evaluating web usability with browsing history and eye movement. In Proceedings of 10th International Conference on Human-Computer Interaction (HCI International 2003), 813--817.Google Scholar
- Porter, A., and Votta, L. 1998. Comparing Detection Methods for Software Requirements Inspection: A Replication Using Professional Subjects. Empirical Software Engineering: An International Journal, 3, 4, 355--380. Google ScholarDigital Library
- Porter, A. A., Votta, L. G., and Basili, V. R. 1995. Comparing Detection Methods for Software Requirements Inspection - A Replicated Experiment. IEEE Transaction on Software Engineering, 21, 6, 563--575. Google ScholarDigital Library
- Sandahl, K., Blomkvist, O., Karlsonn, J., Krysander, C., Lindvall, M., and Ohlsson, N. 1998. An Extended Replication of an Experiment for Assessing Methods for Software Requirements Inspections. Empirical Software Engineering: An International Journal, 3, 4, 218--406. Google ScholarDigital Library
- Shull, F. J. 1998. Developing Techniques for Using Software Documents: A Series of Empirical Studies. PhD thesis, Univ. of Maryland. Google ScholarDigital Library
- Shull, F., Rus, I., and Basili, V. 2000. How Perspective-Based Reading Can Improve Requirements Inspections. IEEE Computer, 33, 7, 73--79. Google ScholarDigital Library
- Stein, R., and Brennan, S. E. 2004. Another Person's Eye Gaze as a Cue in Solving Programming Problems. In Proceedings of the 6th International Conference on Multimodal Interface, 9--15. Google ScholarDigital Library
- Thelin, T., Andersson, C., Runeson, P. and Dzamashvili-Fogelström, N. 2004. A Replicated Experiment of Usage-Based and Checklist-Based Reading. In Proceedings of 10th IEEE International Symposium on Software Metrics (METRICS'04), 246--256. Google ScholarDigital Library
- Thelin, T., Runeson, P., and Regnell, B. 2001. Usage-Based reading - An Experiment to Guide Reviewers with Use Cases. Information and Software Technology, 43, 15, 925--938.Google ScholarCross Ref
- Thelin, T., Runeson, P., and Wohlin, C. 2003. An Experimental Comparison of Usage-Based and Checklist-Based Reading. IEEE Transaction on Software Engineering, 29, 8, 687--704. Google ScholarDigital Library
- Torii, K., Matsumoto, K., Nakakoji, K. Takada, Y., Takada, S., and Shima, K. 1999. Ginger2: An Environment for Computer-Aided Empirical Software Engineering. IEEE Transactions on Software Engineering, 25, 4, 474--492. Google ScholarDigital Library
- Weigers, K. 2002. Peer Reviews in Software -- A Practical Guide. Addison-Wesley (in Japanese). Google ScholarDigital Library
- Zhai, S., Morimoto, C., and Ihde, S. 1999. Manual and Gaze Input Cascaded (MAGIC) Pointing. In Proceedings of the SIGCHI conference on Human factors in computing systems, 246--253. Google ScholarDigital Library
Index Terms
- Analyzing individual performance of source code review using reviewers' eye movement
Recommendations
Exploiting Eye Movements for Evaluating Reviewer's Performance in Software Review
This paper proposes to use eye movements to characterize the performance of individuals in reviewing software documents. We design and implement a system called DRESREM, which measures and records eye movements of document reviewers. Based on the eye ...
Analysis of eye tracking movements using FIR median hybrid filters
ETRA '00: Proceedings of the 2000 symposium on Eye tracking research & applicationsThis paper presents an approach of using FIR Median Hybrid Filters for analysis of eye tracking movements. The proposed filter can remove the eye blink artifact from the eye movement signal. The background of the project is described first. The whole ...
Hierarchical HMM for Eye Movement Classification
Computer Vision – ECCV 2020 WorkshopsAbstractIn this work, we tackle the problem of ternary eye movement classification, which aims to separate fixations, saccades and smooth pursuits from the raw eye positional data. The efficient classification of these different types of eye movements ...
Comments