nach oben

Software Quality Journal

Erschienen in:

01.06.2016

Industry–academia collaborations in software testing: experience and success stories from Canada and Turkey

verfasst von: Vahid Garousi, Matt M. Eskandar, Kadir Herkiloğlu

Erschienen in: Software Quality Journal | Ausgabe 4/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. However, many researchers and practitioners believe that the level of joint industry–academia collaborations (IAC) in software engineering (SE) is still relatively very low, compared to the amount of activity in each of the two communities. The goal of the empirical study reported in this paper is to characterize a set of collaborative industry–academia R&D projects in the area of software testing conducted by the authors (based in Canada and Turkey) with respect to a set of challenges, patterns and anti-patterns identified by a recent Systematic Literature Review study, with the aim of contributing to the body of evidence in the area of IAC, for the benefit of SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general. To address the above goal, a pool of ten IAC projects (six completed, two failed and two ongoing) all in the area of software testing, which the authors have led or have had active roles in, were selected as objects of study and were analyzed (both quantitatively and qualitatively) with respect to the set of selected challenges, patterns and anti-patterns. As outputs, the study presents a set of empirical findings and evidence-based recommendations, e.g.: it has been observed that even if an IAC project may seem perfect from many aspects, one single major challenge (e.g., disagreement in confidentiality agreements) can lead to its failure. Thus, we recommend that both parties (academics and practitioners) consider all the challenges early on and proactively work together to eliminate the risk of challenges in IAC projects. We furthermore report correlation and interrelationship of challenges, patterns and anti-patterns with project success measures. This study hopes to encourage and benefit other SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general in the future.

Vorheriger Artikel Special issue on collaboration in software testing between industry and academia

Nächster Artikel Toward the characterization of software testing practices in South America: looking at Brazil and Uruguay

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Adler, M., & Ziglio, E. (1996). Gazing into the Oracle: The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers.

Ahimbisibwe, A., Cavana, R. Y., & Daellenbach, U. (2015). A contingency fit model of critical success factors for software development projects. Journal of Enterprise Information Management, 28, 7–33.CrossRef

Akdur, D., & Garousi, V. (2015). Model-driven engineering in support of development, test and maintenance of middleware communication protocols: A preliminary approach. In international conference on model-driven engineering and software development (MODELSWARD), Doctoral Consortium (pp. 11–19).

Amannejad, Y., Garousi, V., Irving, R., & Sahaf, Z. (2014). A search-based approach for cost-effective software test automation decision support and an industrial case study. In Proceedings of international workshop on regression testing, co-located with the sixth ieee international conference on software testing, verification, and validation (pp. 302–311).

Ankrah, S., & Al-Tabbaa, O. (2015). Universities–industry collaboration: A systematic review. Scandinavian Journal of Management, 31, 387–408.CrossRef

Avison, D. E., Lau, F., Myers, M. D., & Nielsen, P. A. (1999). Action research. Communications of the ACM, 42, 94–97.CrossRef

Baldassarre, M.T., Caivano, D., & Visaggio, G. (2013). Empirical studies for innovation dissemination: Ten years of experience. In: Presented at the Proceedings of the International Conference on Evaluation and Assessment in Software Engineering, Porto de Galinhas, Brazil.

Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., & Grünbacher, P. (2006). Value-based software engineering. Berlin: Springer.CrossRef

Briand, L.C. (2011). Useful software engineering research-leading a double-agent life. In IEEE international conference on software maintenance (pp. 2–2).

Briand, L. (2012). Embracing the Engineering Side of Software Engineering. IEEE Software, 29, 96.CrossRef

Charette, R. N. (2005). Why software fails. IEEE Spectrum, 42, 42–49.CrossRef

Chow, T., & Cao, D.-B. (2008). A survey study of critical success factors in agile software projects. Journal of Systems and Software, 81, 961–971.CrossRef

Connor, A.M., Buchan, J., & Petrova, K. (2009). Bridging the research-practice gap in requirements engineering through effective teaching and peer learning. In International conference on information technology: new generations (pp. 678–683).

Cruzes, D. S., & Dyba, T. (2011). Research synthesis in software engineering: A tertiary study. Information and Software Technology, 53, 440–455.CrossRef

Deutsch, M. S. (1991). An exploratory analysis relating the software project management process to project success. IEEE Transactions on Engineering Management, 38, 365–375.CrossRef

Eldh, S. (2013). Some researcher considerations when conducting empirical studies in industry. In International workshop on conducting empirical studies in industry (pp. 69–70).

Enoiu, E.P. & Čaušević, A. (2014). Enablers and impediments for collaborative research in software testing: An empirical exploration. In Presented at the Proceedings of the international workshop on Long-term industrial collaboration on software engineering, Vasteras, Sweden.

Fairhurst, M. (2013). Academia and industry collaboration to drive biometrics boom. Biometric Technology Today, 2013, 10–11.CrossRef

Franch Gutiérrez, J., Ameller, D., Ayala Martínez, C. P., & Cabot Sagrera, J. (2012). Bridging the gap among academics and practitioners in non-functional requirements management: Some reflections and proposals for the future. In M. U. Vannerdat (Ed.), Modelling and quality in requirements engineering (pp. 267–273). Münster, Germany: Monsenstein und Vannerdat.

Garousi, V. (2008). Empirical analysis of a genetic algorithm-based stress test technique for distributed real-time systems. In Proceedings of the genetic and evolutionary computation conference, search-based software engineering (SBSE) track (pp. 1743–1750).

Garousi, V. (2010a). An open modern software testing laboratory courseware: An experience report. In Proceedings of the 23rd IEEE conference on software engineering education and training (pp. 177–184).

Garousi, V. (2010b). A genetic algorithm-based stress test requirements generator tool and its empirical evaluation. IEEE Transactions on Software Engineering Special Issue on Search-Based Software Engineering (SBSE), 36, 778–797.

Garousi, V. (2011). Incorporating real-world industrial testing projects in software testing courses: Opportunities, challenges, and lessons learned. In Proceedings of the IEEE conference on software engineering education and training (CSEE&T) (pp. 396–400).

Garousi, V. (2012). Success stories in industrial software testing research and education: A Canadian perspective. In Marmara Research Center, Scientific and Technological Research Council of Turkey (BILGEM, TÜBITAK), Istanbul, Turkey.

Garousi, V., Briand, L., & Labiche, Y. (2008). Traffic-aware stress testing of distributed real-time systems Based on UML models using genetic algorithms. Elsevier Journal of Systems and Software, Special Issue on Model-Based Software Testing, 81, 161–185.

Garousi, V., Coşkunçay, A., Can, A.B., & Demirörs, O. (2013). A survey of software testing practices in Turkey. In Proceedings of the Turkish national software engineering symposium “Ulusal Yazılım Mühendisliği Sempozyumu” (UYMS).

Garousi, V., Coşkunçay, A., Can, A. B., & Demirörs, O. (2015a). A survey of software engineering practices in Turkey. Journal of Systems and Software, 108, 148–177.CrossRef

Garousi, V., Petersen, K., Özkan, B. (2015b). Industry–Academia collaborations in software engineering: A systematic literature review. In Technical report, Hacettepe University software engineering research group, HUSE-2015-01. https://drive.google.com/open?id=0B6dKdxaNjBENSWRwRlNJbExYUWc.

Garousi, V., Petersen, K., & Özkan, B. (2015c). Challenges, success patterns and anti-patterns in industry–academia collaborations in software engineering: A systematic literature review. In Technical Report, Hacettepe University Software Engineering Research Group, HUSE-2015-01.

Garousi, V., & Pfahl, D. (2015). When to automate software testing? A decision-support approach based on process simulation. In Press, Wiley Journal of Software: Evolution and Process.

Garousi, V., & Varma, T. (2010). A replicated survey of software testing practices in the Canadian Province of Alberta: What has changed from 2004 to 2009? Journal of Systems and Software, 83, 2251–2262.CrossRef

Garousi, V., & Zhi, J. (2013). A survey of software testing practices in Canada. Journal of Systems and Software, 86, 1354–1376.CrossRef

Glass, R.L. (2006). Software creativity 2.0: Developer.* Books.

Glass, R.L., & Hunt, A. (2006). Software conflict 2.0: The art and science of software engineering: Developer.* Books.

Gorschek, T., Wohlin, C., Carre, P., & Larsson, S. (2006). A model for technology transfer in practice. IEEE Software, 23, 88–95.CrossRef

Grunbacher, P. & Rabiser, R. (2013). Success factors for empirical studies in industry-academia collaboration: A reflection. In International workshop on conducting empirical studies in industry (pp. 27–32).

Guruprasad, K. (2013). Multi-faceted industry–academia collaboration. In Evolving corporate education strategies for developing countries: The Role of Universities (pp. 90–102).

Harrigan, M. T. (2003). Industrial biotechnology education: A model of collaboration between industry and academia. Biochemistry and Molecular Biology Education, 31, 142–144.CrossRef

Jain, S., Ali Babar, M., & Fernandez, J. (2013). Conducting empirical studies in industry: Balancing rigor and relevance. In International workshop on conducting empirical studies in industry (pp. 9–14).

Jolly, S.A., Garousi, V., & Eskandar, M.M. (2012). Automated unit testing of a SCADA control software: An industrial case study based on action research. In IEEE international conference on software testing, verification and validation (ICST) (pp. 400–409).

Jugdev, K., & Müller, R. (2005). A retrospective look at our evolving understanding of project success. Project Management Journal, 36, 19–31.

Kaindl, H., Brinkkemper, S., Bubenko, J. A, Jr, Farbey, B., Greenspan, S. J., Heitmeyer*, C. L., et al. (2002). Requirements engineering and technology transfer: Obstacles, incentives and improvement agenda. Requirements Engineering, 7, 113–123.CrossRef

Kanso, A., & Monette, D. (2014). Foundations for long-term collaborative research. In Presented at the proceedings of the international workshop on Long-term industrial collaboration on software engineering, Vasteras, Sweden.

Kato, Y. (2005). Industry–academia collaboration program for commercialization of new dry cleaning system. In AIST Today (International Edition) (p. 11).

Krishnan, P., Ross, K., & Pari-Salas, P. (2009). Industry academia collaboration: An experience report at a small University. In Proceedings of conference on software engineering education and training (pp. 117–121).

Lamprecht, S.J., & Van Rooyen, G.J. (2012). Models for technology research collaboration between industry and academia in South Africa. In Proceedings of the software engineering colloquium (pp. 11–17).

Leszak, M., Perry, D. E., & Stoll, D. (2002). Classification and evaluation of defects in a project retrospective. Journal of Systems and Software, 61, 173–187.CrossRef

Lunsford, T. R., & Lunsford, B. R. (1995). The research sample, part I: Sampling. Journal of Prosthetics and Orthotics, 7, 105–112.

Martínez-Fernández, S. & Marques, H.M. (2014). Practical experiences in designing and conducting empirical studies in industry–academia collaboration. In Presented at the proceedings of the international workshop on conducting empirical studies in industry, Hyderabad, India.

Maxwell, J. C., & Dornan, J. (2013). How to influence people: Make a difference in your world. Nashville: Thomas Nelson Inc.

Misirli, A.T., Erdogmus, H., Juristo, N. & Dieste, O. (2014). Topic selection in industry experiments. In Presented at the proceedings of the international workshop on conducting empirical studies in industry, Hyderabad, India.

Morris, P., Masera, M., & Wilikens, M. (1998). Requirements engineering and industrial uptake. In Proceedings of international conference on requirements engineering (pp. 130–137).

Mujumdar, A. S. (2010). Editorial on industry–academia collaboration in R&D. Drying Technology, 28, 431–432.CrossRef

Mullins, C. B. (1985). A university administrator’s view of academia–industry collaboration. Circulation, 72, I8–12.CrossRef

O’Leary, C., Lawless, D., Gordon, D., Li, H., & Bechkoum, K. (2006). Developing a software engineering curriculum for the emerging software industry in China. In 19th Conference on software engineering education and training, 2006. Proceedings. (pp. 115–122).

Osterweil, L. J., Ghezzi, C., Kramer, J., & Wolf, A. L. (2008). Determining the impact of software engineering research on practice. IEEE Computer, 41, 39–49.CrossRef

Palkar, S. (2013). Industry–academia collaboration, expectations, and experiences. ACM Inroads, 4, 56–58.CrossRef

Perkmann, M., Tartari, V., McKelvey, M., Autio, E., Broström, A., D’Este, P., et al. (2013). Academic engagement and commercialisation: A review of the literature on university–industry relations. Research Policy, 42, 423–442.CrossRef

Petersen, K., & Engstre, E. (2014). Finding relevant research solutions for practical problems: The serp taxonomy architecture. In presented at the Proceedings of the international workshop on Long-term industrial collaboration on software engineering, Vasteras, Sweden.

Petersen, K., Gencel, C., Asghari, N., Baca, D., & Betz, S. (2014). Action research as a model for industry–academia collaboration in the software engineering context. In Proceedings of the ACM international workshop on long-term industrial collaboration on software engineering (pp. 55–62).

Pfleeger, S. L. (1999). Understanding and improving technology transfer in software engineering. Journal of Systems and Software, 47, 111–124.CrossRef

Pinheiro, C., Garousi, V., Maurer, F., & Sillito, J. (2010). Introducing automated environment configuration testing in an industrial setting. In Proceedings of the international conference on software engineering and knowledge engineering, workshop on software test automation, practice, and standardization (pp. 186–191).

Price, F. D. (1985). Industry and academia in collaboration: the Pfizer experience. Circulation, 72, I13–I17.

Raschke, W., Zilli, M., Loinig, J., Weiss, R., Steger, C., & Kreiner, C. (2014). Embedding research in the industrial field: A case of a transition to a software product line. In Presented at the Proceedings of the international workshop on Long-term industrial collaboration on software engineering, Vasteras, Sweden.

Reel, J. S. (1999). Critical success factors in software projects. IEEE Software, 16, 18–23.CrossRef

Rombach, D., & Achatz, R. (2007). Research collaborations between academia and industry. In Symposium on future of software engineering (pp. 29–36).

Rombach, D., Ciolkowski, M., Jeffery, R., Laitenberger, O., McGarry, F., & Shull, F. (2008). Impact of research on practice in the field of inspections, reviews and walkthroughs: Learning from successful industrial uses. ACM SIGSOFT Software Engineering Notes, 33, 26–35.CrossRef

Runeson, P. (2012). It takes two to tango: An experience report on industry–academia collaboration. In Proceedings of international conference on software testing, verification and validation, (pp. 872–877).

Runeson, P., Host, M., Rainer, A., & Regnell, B. (2012). Case study research in software engineering: Guidelines and examples. New York: Wiley.CrossRef

Runeson, P., & Minör, S. (2014). The 4 + 1 view model of industry-academia collaboration. In Proceedings of the ACM international workshop on long-term industrial collaboration on software engineering, co-located with ASE 2014 (pp. 21–24).

Runeson, P., Minör, S., & Svenér, J. (2014). Get the cogs in synch-time horizon aspects of industry–academia collaboration. In Proceedings of the ACM international workshop on long-term industrial collaboration on software engineering, co-located with ASE 2014 (pp. 25–28).

Sahaf, Z., Garousi, V., Pfahl, D., Irving, R., & Amannejad, Y. (2014). When to automate software testing? Decision support based on system dynamics—An industrial case study. In Proceedings of international conference on software and systems process (pp. 149–158).

Sandberg, A., Pareto, L., & Arts, T. (2011). Agile collaborative research: Action principles for industry–academia collaboration. IEEE Software, 28, 74–83.CrossRef

Solingen, R. V., & Berghout, E. (1999). The Goal/Question/Metric Method: A practical guide for quality improvement of software development. New York: McGraw-Hill.

Subrahmanyam, G.V.B. (2009). A dynamic framework for software engineering education curriculum to reduce the gap between the software organizations and software educational institutions. In CSEET ‘09. 22nd conference on software engineering education and training, 2009 (pp. 248–254).

Sullivan, G. M., & Artino, A. R. (2013). Analyzing and interpreting data from likert-type scales. Journal of Graduate Medical Education, 5, 541–542.CrossRef

Thomas, C. J., & McKew, J. C. (2014). Playing well with others! Initiating and sustaining successful collaborations between industry, academia and government. Current Topics in Medicinal Chemistry, 14, 291–293.CrossRef

Wiederseiner, C., Garousi, V., & Smith, M. (2011). Tool support for automated traceability analysis in embedded software systems. In Proceedings of IEEE international conference on embedded software and system (ICESS) (pp. 1109–1117).

Wiederseiner, C., Jolly, S.A., & Garousi, V. (2012). AutoBBUT—Automated code generation for black-box unit testing. http://code.google.com/p/autobbut/. Last accessed Sept. 2012.

Wiederseiner, C., Jolly, S.A., Garousi, V., & Eskandar, M.M. (2010). An open-source tool for automated generation of black-box xunit test code and its industrial evaluation. In Proceedings of the international conference on testing: academic and industrial conference—Practice and research techniques (TAIC PART) (pp. 118–128).

Wohlin, C. (2013a). Empirical software engineering research with industry: Top 10 challenges. In International workshop on conducting empirical studies in industry (pp. 43–46).

Wohlin, C. (2013b). Software Engineering Research under the Lamppost. In Proceedings of the international joint conference on software technologies.

Wohlin, C., Aurum, A., Angelis, L., Phillips, L., Dittrich, Y., Gorschek, T., et al. (2012). The success factors powering industry–academia collaboration. IEEE Software, 29, 67–73.CrossRef

Wohlin, C., & Regnell, B. (1999). Strategies for industrial relevance in software engineering education. Journal of Systems and Software, 49, 125–134.CrossRef

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2000). Experimentation in software engineering: An introduction. Berlin: Kluwer.CrossRefMATH

Yeo, K. T. (2002). Critical failure factors in information system projects. International Journal of Project Management, 20, 241–246.CrossRef

Titel: Industry–academia collaborations in software testing: experience and success stories from Canada and Turkey
verfasst von: Vahid Garousi
Matt M. Eskandar
Kadir Herkiloğlu
Publikationsdatum: 01.06.2016
Verlag: Springer US
Erschienen in: Software Quality Journal / Ausgabe 4/2017
Print ISSN: 0963-9314
Elektronische ISSN: 1573-1367
DOI: https://doi.org/10.1007/s11219-016-9319-5

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 4/2017

In this issue

SERP-test: a taxonomy for supporting industry–academia communication

Special issue on collaboration in software testing between industry and academia

Toward the characterization of software testing practices in South America: looking at Brazil and Uruguay

Onboarding inexperienced developers: struggles and perceptions regarding automated testing

Model-based testing of digital TVs: an industry-as-laboratory approach

Premium Partner