Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Automated Software Engineering
Published in: Automated Software Engineering 1/2017

13-01-2016

Unit testing performance with Stochastic Performance Logic

Authors: Lubomír Bulej, Tomáš Bureš, Vojtěch Horký, Jaroslav Kotrč, Lukáš Marek, Tomáš Trojánek, Petr Tůma

Published in: Automated Software Engineering | Issue 1/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Unit testing is an attractive quality management tool in the software development process, however, practical obstacles make it difficult to use unit tests for performance testing. We present Stochastic Performance Logic, a formalism for expressing performance requirements, together with interpretations that facilitate performance evaluation in the unit test context. The formalism and the interpretations are implemented in a performance testing framework and evaluated in multiple experiments, demonstrating the ability to identify performance differences in realistic unit test scenarios.
previous article Automated QoS-oriented cloud resource optimization using containers
next article Continuous validation of performance test workloads

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Footnotes
1
A. Shipilev. Assorted performance presentations. http://​shipilev.​net.
 
2
Note that the effect of the performance observation transformation function on distribution parameters is potentially complex. We assume that in practical applications, the performance observation transformation functions will be limited to linear shift and scale.
 
Literature
ANSI/IEEE: IEEE standard for software unit testing. ANSI/IEEE Std 1008-1987 (1986)
Beck, K.: Simple Smalltalk Testing. Cambridge University Press, Cambridge (1997)CrossRef
Beck, K.: Test Driven Development: By Example. Addison-Wesley, Boston (2002)
Bourque, P., Fairley, R.: Guide to the Software Engineering Body of Knowledge, Version 3.0. IEEE Computer Society Press, Los Alamitos (2014)
Bulej, L., Bureš, T., Gerostathopoulos, I., Horký, V., Keznikl, J., Marek, L., Tschaikowski, M., Tribastone, M., Tůma, P.: Supporting Performance Awareness in Autonomous Ensembles. Lecture Notes in Computer Science, pp. 291–322. Springer, Berlin (2015)
Bulej, L., Bureš, T., Keznikl, J., Koubkovš, A., Podzimek, A. Tůma, P.: Capturing performance assumptions using stochastic performance logic. In: Proceedings of 3rd ACM/SPEC International Conference on Performance Engineering (ICPE), pp. 311–322. ACM (2012)
Chen, T., Guo, Q., Temam, O., Wu, Y., Bao, Y., Xu, Z., Chen, Y.: Statistical performance comparisons of computers. IEEE Trans. Comput. 64(5), 1442–1455 (2015)MathSciNetCrossRef
Dice, D., Moir, M., Scherer, W.: Quickly reacquirable locks. Technical report, Sun Microsystems, Inc. (2003)
Foo, K., Jiang, Z.M., Adams, B., Hassan, A., Zou, Y., Flora, P.: Mining performance regression testing repositories for automated performance analysis. In: Proceedings of 10th International Conference on Quality Software (QSIC), pp. 32–41. IEEE (2010)
Ghaith, S., Wang, M., Perry, P. and Murphy, J.: Profile-based, load-independent anomaly detection and analysis in performance regression testing of software systems. In: Proceedings of 17th European Conference on Software Maintenance and Reengineering (CSMR), pp. 379–383. IEEE (2013)
Heger, C., Happe, J., Farahbod, R.: Automated root cause isolation of performance regressions during software development. In: Proceedings of 4th ACM/SPEC International Conference on Performance Engineering (ICPE), pp.s 27–38. ACM (2013)
Horký, V., Haas, F., Kotrč, J., Lacina, M., Tůma, P.: Performance regression unit testing: a case study. In: Proceedings of 10th European Performance Engineering Workshop (EPEW), Lecture Notes in Computer Science, vol. 8168, pp. 149–163. Springer, Berlin (2013)
Horký, V., Libič, P., Steinhauser, A., Tůma, P.: DOs and DON’Ts of conducting performance measurements in Java (tutorial). In: Proceedings of 6th ACM/SPEC International Conference on Performance Engineering (ICPE), pp. 337–340. ACM (2015)
Kalibera, T., Bulej, L., Tůma, P.: Automated detection of performance regressions: the Mono experience. In: Proceedings of 13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 183–190. IEEE (2005)
Kalibera, T., Bulej, L., Tůma, P.: Benchmark precision and random initial state. In: Proceedings of International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), pp. 853–862. SCS (2005)
Liu, X., Guo, Z., Wang, X., Chen, F., Lian, X., Tang, J., Wu, M., Kaashoek, M.F., Zhang, Z.: D3s: debugging deployed distributed systems. In: Proceedings of 5th USENIX Symposium on Networked Systems Design & Implementation (NSDI), pp. 423–437. USENIX (2008)
Mytkowicz, T., Diwan, A., Hauswirth, M., Sweeney, P. F.: Producing wrong data without doing anything obviously wrong. In: Proceedings of 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 265–276. ACM (2009)
Oliveira, A., Petkovich, J.-C., Reidemeister, T., Fischmeister, S.: Datamill: rigorous performance evaluation made easy. In: Proceedings of 4th ACM/SPEC International Conference on Performance Engineering (ICPE), pp. 137–148. ACM (2013)
Perl, S.E., Weihl, W.E.: Performance assertion checking. In: Proceedings of 14th ACM Symposium on Operating Systems Principles (SOSP), pp. 134–145. ACM (1993)
Porter, A., Yilmaz, C., Memon, A.M., Schmidt, D.C., Natarajan, B.: Skoll: a process and infrastructure for distributed continuous quality assurance. IEEE Trans. Softw. Eng. 33(8), 510–525 (2007)CrossRef
Reynolds, P., Killian, C., Wiener, J.L., Mogul, J.C., Shah, M.A., Vahdat, A.: Pip: Detecting the unexpected in distributed systems. In: Proceedings of 3rd Conference on Networked Systems Design & Implementation (NSDI), pp. 9–9. USENIX (2006)
Sewe, A., Mezini, M., Sarimbekov, A., Binder, W.: Da Capo con Scala: design and analysis of a Scala benchmark suite for the Java virtual machine. In: Proceedings of 26th ACM conference on Object-oriented Programming Systems, Languages, and Applications (OOPSLA), pp. 657–676. ACM (2011)
Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures. CRC Press, Boca Raton (2011)MATH
Tjang, A., Oliveira, F., Bianchini, R., Martin, R., Nguyen, T.: Model-based validation for internet services. In: Proceedings of 28th IEEE International Symposium on Reliable Distributed Systems (SRDS), pp. 61–70. IEEE (2009)
Trojánek, T.: Capturing performance assumptions using stochastic performance logic. Master’s Thesis, Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic (2012)
Vetter, J.S., Worley, P.H.: Asserting performance expectations. In: Proceedings of 15th ACM/IEEE Conference on Supercomputing (SC), pp. 1–13. IEEE (2002)
Welch, B.L.: The generalization of “Student’s” problem when several different population variances are involved. Biometrika 34(1/2), 28–35 (1947)MathSciNetCrossRefMATH
Wirsing, M., Hölzl, M., Koch, N., Mayer, P. (eds.): Software Engineering for Collective Autonomic Systems (The ASCENS Approach). Lecture Notes in Computer Science, vol. 8998. Springer, Berlin (2015)
Metadata
Title
Unit testing performance with Stochastic Performance Logic
Authors
Lubomír Bulej
Tomáš Bureš
Vojtěch Horký
Jaroslav Kotrč
Lukáš Marek
Tomáš Trojánek
Petr Tůma
Publication date
13-01-2016
Publisher
Springer US
Published in
Automated Software Engineering / Issue 1/2017
Print ISSN: 0928-8910
Electronic ISSN: 1573-7535
DOI
https://doi.org/10.1007/s10515-015-0188-0

Other articles of this Issue 1/2017

Automated Software Engineering 1/2017 Go to the issue

OriginalPaper

MaramaAIC: tool support for consistency management and validation of requirements

EditorialNotes

Guest Editorial: Automation in Software Performance Engineering

OriginalPaper

Label propagation based semi-supervised learning for software defect prediction

OriginalPaper

A Petri net tool for software performance estimation based on upper throughput bounds

OriginalPaper

Continuous validation of performance test workloads

OriginalPaper

Automated QoS-oriented cloud resource optimization using containers

Premium Partner

    Image Credits