Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Empirical Software Engineering Models: Can They Become the Equivalent of Physical Laws in Traditional Engineering? Read chapter Read first chapter

2013 | OriginalPaper | Chapter

Empirical Practice in Software Engineering

Authors : Andreas Jedlitschka, Liliana Guzmán, Jessica Jung, Constanza Lampasona, Silke Steinbach

Published in: Perspectives on the Future of Software Engineering

Publisher: Springer Berlin Heidelberg

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Experimental software engineering has been defined as the scientific approach to systematically evaluating software technologies by referring to predefined hypotheses using sound empirical methods.
The purpose of this chapter is to give an overview of the history, current practice, and future of empirical practice in Software Engineering. In particular, based on what we have learned from 20 years of research in empirical software engineering, we describe the empirical approach we are currently using in terms of a scientific approach to applied research and as a means for systematic evaluation.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Can We Trust Software Repositories?
next chapter What Is Software? The Role of Empirical Methods in Answering the Question
Footnotes
1
In this chapter, we use the word study for all kinds of empirical studies. We use experiment as the generic term for controlled and quasi-experiments.
 
2
Synthesis is the umbrella term covering different strategies for combining empirical evidence [34].
 
3
We use the term technology to refer to technique, method, and tool, following Basili et al. [44].
 
Literature
1.
Basili, V., Selby, R., Hutchens, D.: Experimentation in software engineering. IEEE Trans. Softw. Eng. 12(7), 733–743 (1986)CrossRef
2.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: An Introduction. Kluwer (2000). ISBN: 0-7923-8682-5
3.
IEEE Computer Society: IEEE: standard glossary of software engineering terminology. IEEE Standard 610.12-1990
4.
Juristo, N., Moreno, A.: Basics of Software Engineering Experimentation. Kluwer, Boston (2001)MATHCrossRef
5.
Schulz, W.: Kausalität und Experiment in den Sozialwissenschaften – Methodologie und Forschungstechnik. V. Hase & Koehler Verlag, Mainz (1970)
6.
Rombach, H.D., Basili, V.R., Selby, R.W. (eds.): Experimental Software Engineering Issues: Critical Assessment and Future Directions. International Workshop Dagstuhl Castle, Germany. Lecture Notes in Computer Science, vol. 706. Springer (1992)
7.
8.
Rombach, D.: Fraunhofer: the German model for applied research and technology transfer. In: Proceedings of International Conference on Software Engineering 2000, Limerick, pp. 531–537. (2000)
9.
10.
Fenton, N.E., Pfleeger, S.L., Glass, R.: Science and substance: a challenge to software engineers. IEEE Softw. 11(4), 86–95 (1994)CrossRef
11.
Pfleeger, S.L.: Design and analysis in software engineering. Part 1: the language of case studies and formal experiments. ACM SIGSOFT Softw. Eng. Notes 19(4), 16–20 (1994)CrossRef
12.
Kitchenham, B., Linkman, S., Law, D.: DESMET: a methodology for evaluating software engineering methods and tools. Comput. Contr. Eng. J. 8(3), 120–126 (1997)CrossRef
13.
Singer, J.: (APA) Style guidelines to report experimental results. In: Proceedings of Workshop on Empirical Studies in Software Maintenance, pp. 71–77. Oxford, UK (1999)
14.
American Psychological Association: Publication Manual of the American Psychological Association, 5th edn. American Psychological Association, Washington, DC (2001)
15.
Pfleeger, S.L., Kitchenham, B.A.: Principles of survey research. Part 1: turning lemons into lemonade. ACM SIGSOFT Softw. Eng. Notes 26(6), 16–18 (2001)CrossRef
16.
Kitchenham, B.A., Pfleeger, S.L., Pickard, L.M., Jones, P.W., Hoaglin, D.C., El Emam, K., Rosenberg, J.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28(8), 721–734 (2002)CrossRef
17.
Shaw, M.: Writing good software engineering research papers. In: Proceedings of the 25th International Conference on Software Engineering (ICSE’03), pp. 726–736. IEEE CS, Portland (2003)
18.
Ruhe, G.: Software engineering decision support – a new paradigm for learning software organizations. In: Henninger, S., Maurer, F. (eds.) Advances in LSO. 4th International Workshop, LSO 2002, Chicago, 6 Aug 2002: Revised papers, Lecture Notes in Computer Science, vol. 2640, pp. 104–115. Springer (2003)
19.
Aurum, A., Petersson, H., Wohlin, C.: State-of-the-art: software inspections after 25 years. Softw. Test. Verif. Reliab. 12(3), 133–154 (2002)CrossRef
20.
Juristo, N., Moreno, A., Vegas, S.: Reviewing 25 years of testing technique experiments. J. Empirical Softw. Eng. 9(1–2), 7–44 (2004)CrossRef
21.
Jedlitschka, A., Ciolkowski, M.: Towards evidence in software engineering. In: Proceedings of International Symposium on Empirical SE 2004 (ISESE2004), pp. 261–270. Redondo Beach (2004)
22.
Wohlin, C., Petersson, H., Aurum, A.: Combining data from reading experiments in software inspections. In: Juristo, N., Moreno, A. (eds.) Lecture Notes on Empirical Software Engineering, pp. 85–132. World Scientific Publishing, River Edge (2003)
23.
Conradi, R., Wang, A.I. (eds.): Empirical Methods and Studies in Software Engineering – Experiences from ESERNET. Lecture Notes in Computer Science, vol. 2765. Springer, Berlin (2003)MATH
24.
Conradi, R., Dybå, T., Sjøberg, D., Ulsund, T.: Lessons learned and recommendations from two large Norwegian SPI programmes. In: Oquendo, F. (ed.) 9th European Workshop on Software Process Technology (EWSPT 2003), Helsinki, 1–2 Sept 2003, Lecture Notes in Computer Science, vol. 2786, pp. 32–45. Springer (2003)
25.
Glass, R.L.: Matching methodology to problem domain. Column Pract. Programmer Commun. ACM 47(5), 19–21 (2004)CrossRef
26.
Turner, R.: Why we need empirical information on best practices. CROSSTALK – J. Defense Softw. Eng. 17(4), 9–11 (2004)
27.
Kitchenham, B.A., Dybå, T., Jørgensen, M.: Evidence-based software engineering. In: Proceedings of 26th International Conference on Software Engineering (ICSE’04), pp. 273–281. Edinburgh (2004)
28.
Kitchenham, B.A.: Procedures for performing systematic reviews. Keele University Technical Report TR/SE-0401; ISSN 1353-7776.1 (2004)
29.
Sjøberg, D., Hannay, J., Hansen, O., By Kampenes, V., Karahasanovic, A., Liborg, N.-K., Rekdal, A.: A survey of controlled experiments in software engineering. Trans. Softw. Eng. 31(9), 733–753 (2005)CrossRef
30.
Jedlitschka, A., Pfahl, D.: Reporting guidelines for controlled experiments in software engineering. In: Proceedings of International Symposium on Empirical SE 2005 (ISESE2005), Noosa Heads, Australia, Nov 2005, pp. 95–104. IEEE CS (2005)
31.
Basili, V.R., Rombach, D., Schneider, K., Kitchenham, B., Pfahl, D., Selby, R.W. (eds.): Empirical Software Engineering Issues: Critical Assessment and Future Directions. International Workshop Dagstuhl Castle, Germany. Lecture Notes in Computer Science, vol. 4336. Springer (2007)
32.
Dieste, O., Juristo, N.: Systematic review and aggregation of empirical studies on elicitation techniques. IEEE Trans. Softw. Eng. 37(2), 283–304 (2011)CrossRef
33.
Dybå, T., Dingsøyr, T.: Empirical studies of agile software development: a systematic review. Inf. Softw. Technol. 50(9–10), 833–859 (2008)CrossRef
34.
Chalmers, I., Hedges, L., Cooper, H.: A brief history of research synthesis. Eval. Health Prof. 25(1), 12–37 (2002)CrossRef
35.
Ciolkowski, M.: What do we know about perspective-based reading? An approach for quantitative aggregation in software engineering. In: Proceedings of ESEM 2009, Lake Buena Vista, pp. 133–144
36.
Jedlitschka, A., Ciolkowski, M., Pfahl, D.: Reporting controlled experiments in software engineering. In: Shull, F., et al. (eds.) Guide to Advanced Empirical Software Engineering. Springer, New York (2008)
37.
Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in SE. J. Empirical Softw. Eng. 14(2), 131–164 (2009)CrossRef
38.
Carver, J.: Towards reporting guidelines for experimental replications: a proposal. In: Proceedings of the 1st International Workshop on Replication in ESE Research (RESER) @ ICSE, Cape Town, 4 May 2010
39.
Ciolkowski, M.: An approach for quantitative aggregation of evidence from controlled experiments in software engineering. Kaiserslautern University, Dissertation, 2011, 231 pp, Fraunhofer Verlag, Stuttgart (2012)
40.
Cruzes, D.S., Dybå, T.: Recommended steps for thematic synthesis in software engineering. In: Proceedings of ESEM 2011, Banff, pp. 275–284
41.
Rombach, D.: Empirical software engineering models: can they become the equivalent of physical laws in traditional engineering? Int. J. Softw. Inf. 5(3), 525–534 (2011)
42.
Linkman, S., Rombach, D.: Experimentation as a vehicle for software technology transfer – a family of software reading techniques. Inf. Softw. Technol. 39(11), 777–780 (1997)CrossRef
43.
Basili, V.R., Caldiera, G., Rombach, H.D.: Goal question metric paradigm. In: Marciniak, J.J. (ed.) Encyclopedia of Software Engineering, vol. 1, 2nd edn, pp. 528–532. Wiley-Interscience, New York (2001a). doi:10.​1002/​0471028959. ISBN 10: 0471377376
44.
Basili, V.R., Caldiera, G., Rombach, H.D.: Experience factory. In: Marciniak, J.J. (ed.) Encyclopedia of Software Engineering, vol. 1, pp. 511–519. Wiley-Interscience, New York (2001b). doi:10.​1002/​0471028959. ISBN 10: 0471377376
45.
Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd edn. Sage, Thousand Oaks (2003)
46.
Shadish, W.R., Cook, T.D., Campbell, D.T.: Experimental and Quasi-experimental Design for Generalized Causal Inference. Houghton-Mifflin, Boston (2002)
47.
Yin, R.K.: Case Study Research. Design and Methods, 3rd edn. Sage, Thousand Oaks (2003)
48.
49.
Ciolkowski, M., Laitenberger, O., Biffl, S.: Software reviews: the state of the practice. IEEE Softw. 20(6), 46–51 (2003)CrossRef
50.
Jedlitschka, A., Ciolkowski, M., Denger, C., Freimut, B., Schlichting, A.: Relevant information sources for successful technology transfer: a survey using inspections as an example. In: Proceedings of the International Symposium on Empirical SE and Measurement 2007 (ESEM2007), pp. 31–40. Madrid (2007)
51.
Jedlitschka, A., Hamann, D., Göhlert, T., Schröder, A.: Adapting PROFES for use in an agile process: an industry experience report. In: Bomarius, F., et al. (ed.) Proceedings of 6th International Conference on Product Focused Software Process Improvement. Profes’2005. Lecture Notes in Computer Science, vol. 3547, pp. 502–516. (2005)
52.
Kläs, M., Nakao, H., Elberzhager, E., Münch, J.: Support planning and controlling of early quality assurance by combining expert judgment and defect data – a case study. J. Empir. Softw. Eng. 15(4), 423–454, Springer (2010)
53.
Kerth, N.: Project Retrospectives: A Handbook for Team Reviews. Dorset House Publishing, New York (2001)
54.
Sjøberg, D.I.K., Dybå, T., Anda, B.C.D., Hannay, J.E.: Building theories in software engineering. In: Shull, F. et al. (eds.) Guide to advanced empirical software engineering. Springer (2008)
55.
Basili, V., Heidrich, J., Lindvall, M., Münch, J., Regardie, M., Rombach, D., Seaman, C., Trendowicz, A.: Linking software development and business strategy through measurement. IEEE Comput. 43(4), 57–65 (2010)CrossRef
56.
Easterbrook, S.M., Singer, J., Storey, M., Damian, D.: Selecting empirical methods for software engineering research. In: Shull, F., et al. (eds.) Guide to Advanced Empirical Software Engineering. Springer, New York (2008)
57.
Kleinberger, T., Jedlitschka, A., Storf, H., Steinbach-Nordmann, S., Prueckner, S.: An approach to and evaluations of assisted living systems using ambient intelligence for emergency monitoring and prevention. In: Universal Access in HCI. Intelligent and Ubiquitous Interaction Environments. Lecture Notes in Computer Science, vol. 5615, pp. 199–208. (2009)
58.
Venkatesh, V., Bala, H.: Technology acceptance model 3 and a research agenda on interventions. Decis. Sci. 39(2), 273–315 (2008)CrossRef
59.
Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology. MIS Q. 27(3), 425–478 (2003)
60.
Nunnenmacher, S., Jung, J., Chehrazi, G., Klaus, A., Lampasona, C., Webel, C., Ciolkowski, M.: A preliminary survey on subjective measurements and personal insights into factors of perceived future project success. In: Proceedings of 5th International Symposium on Empirical SE and Measurement, pp. 396–399, IEEE CS, Los Alamitos (2011)
61.
Harris, P.: Designing and Reporting Experiments in Psychology, 2nd edn. Open University Press, Berkshire (2002)
62.
Wagner, S., Lochmann, K., Heinemann, L., Kläs, M., Trendowicz, A., Plösch, R., Seidl, A., Goeb, A., Streit, J.: The Quamoco product quality modeling and assessment approach. In: Proceedings of 34th International Conference on Software Engineering (ICSE 2012), pp. 1133–1142. ACM/IEEE, Zurich, 2–9 June 2012
63.
Jedlitschka, A.: An empirical model of software managers information needs for software engineering technology selection. Kaiserslautern University Dissertation, 435 pp. Fraunhofer Verlag, Stuttgart (2009)
64.
Trendowicz, A.: Software Cost Estimation, Benchmarking, and Risk Assessment. The Software Decision-Makers’ Guide to Predictable Software Development. The Fraunhofer IESE Series on Software and Systems Engineering. Springer, Berlin (2013)CrossRef
65.
Lampasona, C., Rostanin, O., Maus, H.: Seamless integration of order processing in MS outlook using SmartOffice: an empirical evaluation. In: Proceedings of International Symposium on Empirical SE and Measurement, pp. 165–168. ACM Press, New York (2012)
66.
Jedlitschka, A.: Evaluating a model of software managers’ information needs: an experiment. In: Proceedings of ACM-IEEE International Symposium on Empirical SE and Measurement (ESEM’10). ACM, Bozen, No. 19, 10 pp (2010)
67.
Jung, J., Höfig, K., Hiller, M., Jedlitschka, A., Domis, D.: Are Ph.D.-Students with Domain Knowledge Appropriate Subjects for Experiments? Kaiserslautern, IESE-Report; 037.12/E (2012)
68.
Rombach, D., Achatz, R.: Research collaborations between academia and industry. In: Proceedings of WS on future of Software Engineering, Minneapolis, pp. 29–36. (2007)
69.
Jedlitschka, A., Pfahl, D.: Experience-based model-driven improvement management with combined data sources from industry and academia. In: Proceedings of International Symposium on Empirical SE ISESE 2003, pp. 154–161. Roman Castles (2003)
70.
Tichy, W.: Empirical software research: an interview with Dag Sjøberg, University of Oslo, Norway. Ubiquity 2011, June, Article 2, 14 pp (2011)
71.
72.
Lincoln, Y., Guba, E.G.: Naturalistic Inquiry. Sage, London/Thousand Oaks/New Delhi (1985)
73.
Pope, C., Mays, N., Popay, J.: Synthesizing Qualitative and Quantitative Health Evidence. A Guide to Methods. Open University Press, Berkshire (2007)
74.
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., Sutton, A.: Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10, 45–53 (2005)CrossRef
75.
Miles, M.B., Huberman, A.M.: Qualitative Data Analysis: An Expanded Sourcebook. Sage, Thousand Oaks (1994)
Metadata
Title
Empirical Practice in Software Engineering
Authors
Andreas Jedlitschka
Liliana Guzmán
Jessica Jung
Constanza Lampasona
Silke Steinbach
Copyright Year
2013
Publisher
Springer Berlin Heidelberg
Book
Perspectives on the Future of Software Engineering

Print ISBN: 978-3-642-37394-7

Electronic ISBN: 978-3-642-37395-4

Copyright Year: 2013

DOI
https://doi.org/10.1007/978-3-642-37395-4_15

Premium Partner

    Image Credits