nach oben

Empirical Software Engineering

Erschienen in:

01.06.2012

Location pairs: a test coverage metric for shared-memory concurrent programs

verfasst von: Serdar Tasiran, M. Erkan Keremoğlu, Kivanç Muşlu

Erschienen in: Empirical Software Engineering | Ausgabe 3/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We present a coverage metric targeted at shared-memory concurrent programs: the Location Pairs (LP) coverage metric. The goals of this metric are (i) to measure how thoroughly a program has been tested from a concurrency standpoint, i.e., whether enough qualitatively different thread interleavings have been explored, and (ii) to guide testing towards unexplored concurrency scenarios. This metric was inspired by an access pattern known to lead to high-level concurrency errors in industrial software and in the literature. We built a monitoring tool to measure LP coverage of test programs. We used the LP metric for interactive debugging, and compared LP coverage with other concurrency coverage metrics on Java benchmarks. We demonstrated that LP coverage corresponds better to concurrency errors, is a better measure of how well a program is exercised concurrency-wise by a test set, reaches saturation later than other coverage metrics, and is viable and useful as an interactive testing and debugging tool.

Nächster Artikel A qualitative human-centric evaluation of flexibility in middleware implementations

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Ben-Asher Y, Eytani Y, Farchi E, Ur S (2006) Producing scheduling that causes concurrent programs to fail. In: Proceedings of the 2006 workshop on parallel and distributed systems: testing and debugging, PADTAD ’06. ACM, New York, NY, USA, pp 37–40CrossRef

Bradbury JS, Cordy JR, Dingel J (2006) Mutation operators for concurrent java (j2se 5.0). In: Proceedings of the second workshop on mutation analysis, MUTATION ’06. IEEE Computer Society, Washington, DC, USA, pp 11–11CrossRef

Bron A, Farchi E, Magid Y, Nir Y, Ur S (2005) Applications of synchronization coverage. In: PPoPP ’05: proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming. ACM Press, New York, NY, USA, pp 206–212CrossRef

Chen J, MacDonald S (2007) Testing concurrent programs using value schedules. In: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, ASE ’07. ACM, New York, NY, USA, pp 313–322CrossRef

Daly C, Horgan J, Power J, Waldron J (2001) Platform independent dynamic java virtual machine analysis: the java grande forum benchmark suite. In: JGI ’01: proceedings of the 2001 joint ACM-ISCOPE conference on Java Grande. ACM Press, New York, pp 106–115CrossRef

Elmas T, Tasiran S, Qadeer S (2005) Vyrd: verifying concurrent programs by runtime refinement-violation detection. In: PLDI ’05: proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation. ACM Press, New York, pp 27–37CrossRef

Elmas T, Qadeer S, Tasiran S (2010) Goldilocks: a race-aware java runtime. Commun. ACM 53:85–92CrossRef

Factor M, Farchi E, Lichtenstein Y, Malka Y (1996) Testing concurrent programs: a formal evaluation of coverage criteria. In: ICCSSE ’96: proceedings of the 7th Israeli conference on computer-based systems and software engineering. IEEE Computer Society, Washington, DC, p 119CrossRef

Farchi E, Nir Y, Ur S (2003) Concurrent bug patterns and how to test them. In: IPDPS ’03: proceedings of the 17th international symposium on parallel and distributed processing. IEEE Computer Society, Washington, DC, p 286.2

Farzan A, Madhusudan P (2008) Monitoring atomicity in concurrent programs. In: Proceedings of the 20th international conference on computer aided verification, CAV ’08. Springer, Berlin, pp 52–65CrossRef

Flanagan C, Freund SN (2004) Atomizer: a dynamic atomicity checker for multithreaded programs. SIGPLAN Not 39(1):256–267CrossRef

Flanagan C, Qadeer S (2003) Types for atomicity. In: TLDI ’03: proceedings of the 2003 ACM SIGPLAN international workshop on types in language design and implementation. ACM, New York, pp 1–12CrossRef

Flanagan C, Freund SN, Yi J (2008) Velodrome: a sound and complete dynamic atomicity checker for multithreaded programs. In: Proceedings of the 2008 ACM SIGPLAN conference on programming language design and implementation, PLDI ’08. ACM, New York, pp 293–303CrossRef

Frankl PG, Weiss SN (1993) An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans Softw Eng 19(8):774–787CrossRef

Havelund K, Stoller SD, Ur S (2003) Benchmark and framework for encouraging research on multi-threaded testing tools. In: IPDPS ’03: proceedings of the 17th international symposium on parallel and distributed processing. IEEE Computer Society, Washington, DC, p 286.1

Kaminsky A (2007) Parallel java: a unified api for shared memory and cluster parallel programming. In: 100 parallel and distributed processing symposium, international 0:231

Kulkarni M, Burtscher M, Cascaval C, Pingali K (2009) Lonestar: a suite of parallel irregular programs. In: ISPASS, pp 65–76

Keremoglu ME, Tasiran S, Elmas T (2006) A classification of concurrency bugs in java benchmarks by developer intent. In: PADTAD ’06: proceeding of the 2006 workshop on parallel and distributed systems: testing and debugging. ACM Press, New York, pp 23–26CrossRef

Lai Z, Cheung SC, Chan WK (2008) Inter-context control-flow and data-flow test adequacy criteria for nesc applications. In: Proceedings of the 16th ACM SIGSOFT international symposium on foundations of software engineering, SIGSOFT ’08/FSE-16. ACM, New York, pp 94–104CrossRef

Lu S, Jiang W, Zhou Y (2007) A study of interleaving coverage criteria. In: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, ESEC-FSE ’07. ACM, New York, pp 533–536

Manson J, Pugh W, Adve SV (2005) The java memory model. In: POPL ’05: proceedings of the 32nd ACM SIGPLAN-SIGACT symposium on principles of programming languages. ACM Press, New York, pp 378–391CrossRef

Musuvathi M, Qadeer S, Ball T, Basler G, Nainar PA, Neamtiu I (2008) Finding and reproducing heisenbugs in concurrent programs. In: Proceedings of the 8th USENIX conference on Operating systems design and implementation, OSDI’08. USENIX Association, Berkeley, pp 267–280

Naik M, Aiken A, Whaley J (2006) Effective static race detection for java. In: PLDI ’06: proceedings of the 2006 ACM SIGPLAN conference on programming language design and implementation. ACM Press, New York, pp 308–319CrossRef

Nir-Buchbinder Y, Ur S (2007) Contest listeners: a concurrency-oriented infrastructure for java test and heal tools. In: Fourth international workshop on software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting, SOQUA ’07. ACM, New York, pp 9–16CrossRef

Sherman E, Dwyer MB, Elbaum S (2009) Saturation-based testing of concurrent programs. In: Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, ESEC/FSE ’09. ACM, New York, pp 53–62

Smith LA, Bull JM, Obdrzálek J (2001) A parallel java grande benchmark suite. In: Proceedings of the 2001 ACM/IEEE conference on supercomputing (CDROM), Supercomputing ’01. ACM, New York, pp 8–8CrossRef

Sung A, Srisa-an W, Rothermel G, Yu T (2010) Testing inter-layer and inter-task interactions in rtes applications. In: Asia-Pacific software engineering conference, pp 260–269

Tasiran S, Bogdanov A, Ji M (2004) Detecting concurrency errors in file systems by runtime refinement checking. Technical Report HPL-2004-177, HP Laboratories

Taylor RN, Levine DL, Kelly CD (1992) Structural testing of concurrent programs. IEEE Trans Softw Eng 18(3):206–215CrossRef

Tretmans J (1999) Testing concurrent systems: A formal approach. In: CONCUR ’99: proceedings of the 10th international conference on concurrency theory. Springer, London, pp 46–65CrossRef

Visser W, Havelund K, Brat G, Park S, Lerda F (2003) Model checking programs. Autom Softw Eng 10(2):203–232CrossRef

Wang L, Stoller SD (2006) Accurate and efficient runtime detection of atomicity errors in concurrent programs. In: PPoPP ’06: proceedings of the eleventh ACM SIGPLAN symposium on principles and practice of parallel programming. ACM, New York, pp 137–146CrossRef

Yang C-SD, Souter AL, Pollock LL (1998) All-du-path coverage for parallel programs. In: Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, ISSTA ’98. ACM, New York, pp 153–162CrossRef

Yu T, Sung A, Srisa-an W, Rothermel G (2011) Using property-based oracles when testing embedded system applications. In: Proceedings of the fourth IEEE international conference on software testing

Zhu H, Hall PAV, May JHR (1997) Software unit test coverage and adequacy. ACM Comput Surv 29(4):366–427CrossRef

Titel: Location pairs: a test coverage metric for shared-memory concurrent programs
verfasst von: Serdar Tasiran
M. Erkan Keremoğlu
Kivanç Muşlu
Publikationsdatum: 01.06.2012
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 3/2012
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-011-9166-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 3/2012

A qualitative human-centric evaluation of flexibility in middleware implementations

An exploratory study of the impact of antipatterns on class change- and fault-proneness

The ability of object-oriented metrics to predict change-proneness: a meta-analysis

Program comprehension of domain-specific and general-purpose languages: comparison using a family of experiments

Replication of an experiment on linguistic tool support for consolidation of requirements from multiple sources

Premium Partner