Muhammad Usman
ABSTRACT
Context: There are numerous studies on effort estimation in Agile Software Development (ASD) and the state of the art in this area has been recently documented in a Systematic Literature Review (SLR). However, to date there are no studies on the state of the practice in this area, focusing on similar issues to those investigated in the above-mentioned SLR. Objectives: The aim of this paper is to report on the state of the practice on effort estimation in ASD, focusing on a wide range of aspects such as the estimation techniques and effort predictors used, to name a few. Method: A survey was carried out using as instrument an on-line questionnaire answered by agile practitioners who have experience in effort estimation. Results: Data was collected from 60 agile practitioners from 16 different countries, and the main findings are: 1) Planning poker (63%), analogy (47%) and expert judgment (38%) are frequently practiced estimation techniques in ASD; 2) Story points is the most frequently (62%) employed size metric, used solo or in combination with other metrics (e.g., function points); 3) Team's expertise level and prior experience are most commonly used cost drivers; 4) 52% of the respondents believe that their effort estimates on average are under/over estimated by an error of 25% or more; 5) Most agile teams take into account implementation and testing activities during effort estimation; and 6) Estimation is mostly performed at sprint and release planning levels in ASD. Conclusions: Estimation techniques that rely on experts' subjective assessment are the ones used the most in ASD, with effort underestimation being the dominant trend. Further, the use of multiple techniques in combination and story points seem to present a positive association with estimation accuracy, and team-related cost drivers are the ones used by most agile teams. Finally, requirements and management related issues are perceived as the main reasons for inaccurate estimates.
- D. Azhar, P. Riddle, E. Mendes, N. Mittas, and L. Angelis. Using ensembles for web effort estimation. In Empirical Software Engineering and Measurement, 2013 ACM/IEEE International Symposium on, pages 173--182. IEEE, 2013.Google Scholar
Cross Ref
- F. O. Bjørnson and T. Dingsøyr. Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used. Information and Software Technology, 50(11): 1055--1068, 2008. Google ScholarDigital Library
- L. Cao. Estimating agile software project effort: an empirical study. 2008.Google Scholar
- M. Cohn. Agile estimating and planning. Pearson Education, 2005. Google ScholarDigital Library
- A. Fink. The survey handbook, volume 1. Sage, 2003.Google Scholar
- D. Gotterbarn, K. Miller, and S. Rogerson. Computer society and acm approve software engineering code of ethics. Computer, 32(10): 84--88, 1999.Google ScholarCross Ref
- S. Grimstad, M. Jorgensen, and K. Molokken-Ostvold. The clients' impact on effort estimation accuracy in software development projects. In Software Metrics, 2005. 11th IEEE International Symposium, pages 10--pp. IEEE, 2005. Google ScholarDigital Library
- M. Jorgensen and K. Molokken-Ostvold. Reasons for software effort estimation error: impact of respondent role, information collection approach, and data analysis method. Software Engineering, IEEE Transactions on, 30(12): 993--1007, 2004. Google ScholarDigital Library
- S. Keaveney and K. Conboy. Cost estimation in agile development projects. 2006.Google Scholar
- S. Keele. Guidelines for performing systematic literature reviews in software engineering. Technical report, Technical report, EBSE Technical Report EBSE-2007-01, 2007.Google Scholar
- B. Kitchenham and S. L. Pfleeger. Principles of survey research parts 1--6. ACM SIGSOFT Software Engineering Notes, Nov 2001 to Mar 2003.Google Scholar
- O. Liskin, R. Pham, S. Kiesling, and K. Schneider. Why we need a granularity concept for user stories. In Agile Processes in Software Engineering and Extreme Programming, pages 110--125. Springer, 2014.Google Scholar
- V. Mahnič and T. Hovelja. On using planning poker for estimating user stories. Journal of Systems and Software, 85(9): 2086--2095, 2012. Google ScholarDigital Library
- K. Molokken-Ostvold and K. M. Furulund. The relationship between customer collaboration and software project overruns. In Agile Conference (AGILE), 2007, pages 72--83. IEEE, 2007. Google ScholarDigital Library
- K. Molokken-Ostvold and M. Jorgensen. A comparison of software project overruns-flexible versus sequential development models. Software Engineering, IEEE Transactions on, 31(9): 754--766, 2005. Google ScholarDigital Library
- K. Moløkken-Østvold, M. Jørgensen, S. S. Tanilkan, H. Gallis, A. C. Lien, and S. Hove. A survey on software estimation in the norwegian industry. In Software Metrics, 2004. Proceedings. 10th International Symposium on, pages 208--219. IEEE, 2004. Google ScholarDigital Library
- T. Punter, M. Ciolkowski, B. Freimut, and I. John. Conducting on-line surveys in software engineering. In Empirical Software Engineering, 2003. ISESE 2003. Proceedings. 2003 International Symposium on, pages 80--88. IEEE, 2003. Google ScholarDigital Library
- C. Robson. Real world research: a resource for users of social research methods in applied settings. Wiley Chichester, 2011.Google Scholar
- M. Usman, E. Mendes, F. Weidt, and R. Britto. Effort estimation in agile software development: A systematic literature review. In Proceedings of the 10th International Conference on Predictive Models in Software Engineering, PROMISE '14, pages 82--91, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- K. E. Wiegers. More about software requirements. Microsoft Press, 2006. Google ScholarDigital Library
- C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in software engineering. Springer, 2012. Google ScholarDigital Library
Index Terms
- Effort estimation in agile software development: a survey on the state of the practice
Recommendations
Investigating documented information for accurate effort estimation in agile software development
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringAn Agile development team estimates the effort of a work item to plan a sprint (i.e., an iteration in Scrum). Hence, reliable effort estimation would help the team create a reliable sprint plan. Prior studies proposed automated approaches to help the ...
Understanding and improving effort estimation in Agile software development: an industrial case study
ICSSP '16: Proceedings of the International Conference on Software and Systems ProcessEffort estimation is more challenging in an agile context, as instead of exerting strict control over changes in requirements, dynamism is embraced. Current practice relies on expert judgment, where the accuracy of estimates is sensitive to the ...
Effort estimation for agile software development: comparative case studies using COSMIC functional size measurement and story points
IWSM Mensura '17: Proceedings of the 27th International Workshop on Software Measurement and 12th International Conference on Software Process and Product MeasurementAgile methodologies have gained significant popularity among software development organizations during the last decade. Although agile methodologies are regarded as minimizing formal processes, they still utilize an estimation methodology for proper ...
Comments