Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
International Journal on Software Tools for Technology Transfer
Erschienen in: International Journal on Software Tools for Technology Transfer 5/2014

01.10.2014 | RBT

Dynamic test planning: a study in an industrial context

verfasst von: Gabriella Carrozza, Roberto Pietrantuono, Stefano Russo

Erschienen in: International Journal on Software Tools for Technology Transfer | Ausgabe 5/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Testing accounts for a relevant part of the production cost of complex or critical software systems. Nevertheless, time and resources budgeted to testing are often underestimated with respect to the target quality goals. Test managers need engineering methods to perform appropriate choices in spending testing resources, so as to maximize the outcome. We present a method to dynamically allocate testing resources to software components minimizing the estimated number of residual defects and/or the estimated residual defect density. We discuss the application to a real-world critical system in the homeland security domain. We describe a support tool aimed at easing industrial technology transfer by hiding to practitioners the mathematical details of the method application.
Vorheriger Artikel Risk-based testing via active continuous quality control
Nächster Artikel A multiple case study on risk-based testing in industry

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
1
Note that detecting more faults does not imply improving reliability: this requires removing the faults occurring more frequently.
 
2
The term fault (defect) is preferred in the fault tolerance (software engineering) community; here, we use them as synonymous.
 
3
Improving operational reliability is more desirable, but it requires the knowledge of the operational profile, seldom available.
 
4
We assume that the number of man-weeks worked per week is fixed; namely, the assigned resources are spent uniformly across the weeks.
 
5
This is the worst case; it may happen indeed that past data exist and can be used as starting point to build a model.
 
6
Authors in [35] indicate a time of 25 % to have an SRGM with a definitive accuracy deviation of 20 %; however, in our method, we practically need much less time, since we do not select a definitive model now, but we will iteratively select the best model (and thus improve the initial accuracy deviation) as testing time proceeds.
 
7
We assure them to receive a certain amount of resources, in order to not stop completely their testing, so as to have more data for it in the next iteration. This amount will be the same as the previous iteration but diminished by a factor \(\alpha \in [0,1]\). The latter is computed as: \(\alpha _j = [\textit{DR}_j-min_i(\textit{DR})]/[max_i(DR)-min_i(\textit{DR})]\), where \(\textit{DR}_j\) is the current detection rate (defect/weeks) of the component \(j\) and the index \(i\) spans across components. Thus, if \(C_j\) had the best detection rate, it receives the same amount as the previous cycle; otherwise, it is penalized.
 
8
\(B^*\) also takes into account the possible resources already allocated to nonstatistically valid components (see note 7).
 
9
This function also manages the case in which the tester wants to go on with even a subset of statistically valid components; in such a case, the function sets ready to true when at least \(x\) components are valid, with \(x\) decided by the tester (omitted for simplicity).
 
10
It also applies the detection rate proportional allocation described above (note 6), in case there is some component with no valid SRGM.
 
11
We assume to have a budget lower than 326 man-weeks, so as to avoid to allocate a number of man-weeks to a CSCI greater than the amount actually available (e.g., CSCI 1 has been tested with 32 man-weeks, if the number of man-weeks allocated by the scheme is greater than 32, the experiment may be invalid).
 
12
The models that more often fitted data have been: exponential, especially in the beginning, truncated logistic, and truncated normal.
 
13
The Analyzer is left out from the explanation, since it does not regard the test planning issue presented here.
 
14
The re-allocation step input can also be empty, giving the possibility to the tester of requiring the re-allocation at any time she/he wants by just pressing a button.
 
15
The best fitting SRGM is chosen by default.
 
Literatur
1.
Cotroneo, D., Pietrantuono, R., Russo, S.: Testing techniques selection based on ODC fault types and software metrics. Journal of Systems and Software 86(6), 1613–1637 (2013)CrossRef
2.
Goel, A.L.: Software Reliability Models: Assumptions, Limitations and Applicability. IEEE Transactions on Software Engineering SE–11(12), 1411–1423 (1985)CrossRef
3.
Cotroneo, D., Pietrantuono, R., Russo, S.: Combining operational and debug testing for improving reliability. IEEE Transactions on Reliability 62(2), 408–423 (2013)CrossRef
4.
Catal, C., Diri, B.M.: A systematic review of software fault prediction studies. Expert Systems with Applications 36(4), 7346–7354 (2009)CrossRef
5.
Halstead, M.: Elements of Software Science. Elsevier Science, New York (1977)MATH
6.
Chidamber, S.R., Kemerer, C.F.: A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering 20(6), 476–493 (1994)CrossRef
7.
Gokhale, S.S., Lyu, M.R.: Regression Tree Modeling for the Prediction of Software Quality. In: Proc. 3rd ISSAT (1997)
8.
Subramanyam, R., Krishnan, M.S.: Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects. IEEE Transactions on Software Engineering 29(4), 297–310 (2003)CrossRef
9.
Basili, V.R., Briand, L.C., Melo, W.L.: A Validation of Object-Oriented Design Metrics as Quality Indicators. IEEE Transactions on Software Engineering 22(10), 751–761 (1996)CrossRef
10.
Ohlsson, N., Alberg, H.: Predicting fault-prone software modules in telephone switches. IEEE Transactions on Software Engineering. 22(12), 886–894 (1996)CrossRef
11.
Denaro, G., Pezzè, M.: An Empirical Evaluation of Fault-proneness Models. In: Proc. 24th Int. Conference on Software Engineering (ICSE), pp. 241–251 (2002)
12.
Nagappan, N., Ball, T., Zeller, A.: Mining Metrics to Predict Component Failures. In: Proc. 28th Int. Conference on Software Engineering (ICSE), pp. 452–461 (2006)
13.
Ostrand, T., Weyuker, E., Bell, R.: Predicting the Location and Number of Faults in Large Software Systems. IEEE Transactions on Software Engineering 31(4), 340–355 (2005)CrossRef
14.
Menzies, T., Greenwald, J., Frank, A.: Data Mining Static Code Attributes to Learn Defect Predictors. IEEE Transactions on Software Engineering 33(1), 2–13 (2007)CrossRef
15.
Nam, J., Jialin Pan, S., Kim, S.: Transfer Defect Learning. In: Proc. 35th Int. Conference on Software Engineering (ICSE), pp. 382–391 (2013)
16.
Zimmermann T., et al.: Cross-project Defect Prediction: A Large Scale Experiment on Data vs. Domain vs. Process. In: Proc. 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Eng., pp. 91–100 (2009)
17.
Dugan, J.B.: Automated Analysis of Phase-Mission Reliability. IEEE Transactions on Reliability 40, 45–52 (1991)CrossRefMATH
18.
Garzia, M.R.: Assessing the Reliability of Windows Servers. In: Proc. of IEEE Dependable Systems and Networks conference (2002)
19.
Pietrantuono, R., Russo, S., Trivedi, K.S.: Online Monitoring of Software System Reliability. In: Proc. of the European Dependable Computing Conference (EDCC), 209–218 (2010)
20.
Goel, A.L., Okumoto, K.: Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability R–28(3), 206–211 (1979)CrossRef
21.
Yamada, S., Ohba, M., Osaki, S.: S-Shaped Reliability Growth Modeling for Software Error Detection. IEEE Transactions on Reliability R–32(5), 475–485 (1983)CrossRef
22.
Gokhale, S.S., Trivedi, K.S.: Log-logistic software reliability growth model. In: Proc. 3rd Int. High-Assurance Systems Engineering Symposium, pp. 34–41 (1998)
23.
Mullen, R.E.: The lognormal distribution of software failure rates: application to software reliability growth modeling. In: Proc. 9th Int. Symposium on Software Reliability Engineering (ISSRE), pp. 134–142 (1998)
24.
Okamura, H., Dohi, T., Osaki, S.: EM algorithms for logistic software reliability models. In: Proc. 22nd IASTED Int. Conference on Software Engineering, pp. 263–268 (2004)
25.
Yamada, S., Ichimori, T., Nishiwaki, M.: Optimal Allocation Policies for Testing-Resource Based on a Software Reliability Growth Model. Int. Journal of Mathematical and Computer Modeling. 22(10–12), 295–301 (1995)CrossRefMATH
26.
Huang, C., Kuo, S., Lyu, M.R.: An Assessment of Testing-Effort Dependent Software Reliability Growth Models. IEEE Transactions on Reliability 56(2), 198–211 (2007)CrossRef
27.
Yamada, S., Ohtera, H., Narihisa, H.: Software reliability growth models with testing effort. IEEE Transactions on Reliability R–35, 19–23 (1986)CrossRef
28.
Lyu, M.R., Rangarajan, S., van Moorsel, A.P.A.: Optimal Allocation of Test Resources for Software Reliability Growth Modeling in Software Development. IEEE Transactions on Reliability 51(2), 336–347 (2002)CrossRef
29.
Huang, C.Y., Lo, J.H., Kuo, S.Y., Lyu, M.R.: Optimal Allocation of Testing Resources for Modular Software Systems. In: Proc. 13th Int. Symposium on Software Reliability Engineering (ISSRE), pp. 129–138 (2002)
30.
Huang, C.Y., Lo, J.H.: Optimal Resource Allocation for Cost and Reliability of Modular Software Systems in the Testing Phase. Journal of Systems and Software 79(5), 653–664 (2006)CrossRef
31.
Hou, R.H., Kuo, S.Y., Chang, Y.P.: Efficient allocation of testing resources for software module testing based on the hyper-geometric distribution software reliability growth model. In: Proc. 7th Int. Symposium on Software Reliability Engineering (ISSRE), pp. 289–298 (1996)
32.
Everett, W.: Software Component Reliability Analysis. In: Proc. Symposium on Application-specific Systems and Software Eng. and Techn. (ASSET), pp. 204–211 (1999)
33.
Pietrantuono, R., Russo, S., Trivedi, K.S.: Software Reliability and Testing Time Allocation: An Architecture-Based Approach. IEEE Transactions on Software Engineering 36(3), 323–337 (2010)
34.
35.
Almering, V., Van Genuchten, M., Cloudt, G., Sonnemans, P.J.M.: Using Software Reliability Growth Models in Practice. IEEE Software 24(6), 82–88 (2007)CrossRef
36.
Stringfellow, C., Amschler, A.: Andrews: An Empirical Method for Selecting Software Reliability Growth Models. Empirical Software Engineering 7(4), 319–343 (2002)
37.
Farr, W.: Handbook of Software Reliability Engineering, M.R. Lyu (Ed.), chapter: Software Reliability Modeling Survey, pp. 71–117. McGraw-Hill, New York, NY (1996)
38.
Musa, J.D., Okumoto, K.: A logarithmic Poisson execution time model for software reliability measurement. In: Proc. 7th Int. Conference on Software Engineering (ICSE), pp. 230–238 (1984)
39.
Zachariah, B., Rattihalli, R.N.: Failure Size Proportional Models and an Analysis of Failure Detection Abilities of Software Testing Strategies. IEEE Transactions on Reliability 56(2), 246–253 (2007)CrossRef
40.
Okamura, H., Watanabe, Y., Dohi, T.: An iterative scheme for maximum likelihood estimation in software reliability modeling. In: Proc. 14th Int. Symposium on Software Reliab. Eng. (ISSRE). IEEE CS Press, pp. 246–256 (2003)
41.
Ohishi, K., Okamura, H., Dohi, T.: Gompertz software reliability model: Estimation algorithm and empirical validation. Journal of Systems and Software 82(3), 535–543 (2009)CrossRef
42.
Okamura, H., Dohi, T., Osaki, S.: Software reliability growth model with normal distribution and its parameter estimation. In: Proc. Int. Conference on Quality, Reliability, Risk, Maintenance, and Safety, Engineering (ICQR2MSE), pp. 411–416 (2011)
Metadaten
Titel
Dynamic test planning: a study in an industrial context
verfasst von
Gabriella Carrozza
Roberto Pietrantuono
Stefano Russo
Publikationsdatum
01.10.2014
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal on Software Tools for Technology Transfer / Ausgabe 5/2014
Print ISSN: 1433-2779
Elektronische ISSN: 1433-2787
DOI
https://doi.org/10.1007/s10009-014-0319-0

Weitere Artikel der Ausgabe 5/2014

International Journal on Software Tools for Technology Transfer 5/2014 Zur Ausgabe

RBT

Risk-based testing via active continuous quality control

Introduction

Rigorous examination of reactive systems

RERS

Thoughtful brute-force attack of the RERS 2012 and 2013 Challenges

RBT

A multiple case study on risk-based testing in industry

RERS

Tailored generation of concurrent benchmarks

RERS

BDD-based software verification

Premium Partner

    Bildnachweise