Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Learning a Behavior Model of Hybrid Systems Through Combining Model-Based Testing and Machine Learning Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Authors : Osvaldo Huerta-Guevara, Vanessa Ayala-Rivera, Liam Murphy, A. Omar Portillo-Dominguez

Published in: Testing Software and Systems

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Performance testing is a critical task to ensure an acceptable user experience with software systems, especially when there are high numbers of concurrent users. Selecting an appropriate test workload is a challenging and time-consuming process that relies heavily on the testers’ expertise. Not only are workloads application-dependent, but also it is usually unclear how large a workload must be to expose any performance issues that exist in an application. Previous research has proposed to dynamically adapt the test workloads in real-time based on the application behavior. By reducing the need for the trial-and-error test cycles required when using static workloads, dynamic workload adaptation can reduce the effort and expertise needed to carry out performance testing. However, such approaches usually require testers to properly configure several parameters in order to be effective in identifying workload-dependent performance bugs, which may hinder their usability among practitioners. To address this issue, this paper examines the different criteria needed to conduct performance testing efficiently using dynamic workload adaptation. We present the results of comprehensively evaluating one such approach, providing insights into how to tune it properly in order to obtain better outcomes based on different scenarios. We also study the effects of varying its configuration and how this can affect the results obtained.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter A Model Checking Based Approach for Detecting SDN Races
next chapter DYNAMOJM: A JMeter Tool for Performance Testing Using Dynamic Workload Adaptation
Footnotes
Literature
1.
Altman, E., Arnold, M., Fink, S., Mitchell, N.: Performance analysis of idle programs. ACM SIGPLAN Not. 45(10), 739–753 (2010)CrossRef
2.
Angelopoulos, V., Parsons, T., Murphy, J., O’Sullivan, P.: GcLite: an expert tool for analyzing garbage collection behavior. In: 2012 IEEE 36th Annual Computer Software and Applications Conference Workshops, pp. 493–502. IEEE (2012)
3.
Ayala-Rivera, V., Kaczmarski, M., Murphy, J., Darisa, A., Portillo-Dominguez, A.O.: One size does not fit all. In: ICPE 2018, pp. 211–222. ACM Press, New York (2018)
4.
5.
Basak, J., Wadhwani, K., Voruganti, K.: Storage workload identification. ACM Trans. Storage 12(3), 1–30 (2016)
6.
Bourne, D.A., Chung, A.A., Price, D.L.: Capturing trace information using annotated trace output, 31 May 2016, US Patent 9,355,002
7.
Bures, M., Miroslav: Metrics for automated testability of web applications. In: CompSysTech 2015, pp. 83–89. ACM Press, New York (2015)
8.
de Camargo, A., Salvadori, I., Mello, R.d.S., Siqueira, F.: An architecture to automate performance tests on microservices. In: iiWAS 2016, pp. 422–429. ACM Press, New York (2016)
9.
Carothers, C.D., et al.: Durango: scalable synthetic workload generation for extreme-scale application performance modeling and simulation. In: SIGSIM-PADS 2017, pp. 97–108. ACM Press, New York (2017)
10.
Chen, T.H., et al.: Analytics-driven load testing: an industrial experience report on load testing of large-scale systems. In: ICSE-SEIP. IEEE (2017)
11.
Conley, M., Vahdat, A., Porter, G.: Achieving cost-efficient, data-intensive computing in the cloud. In: SoCC 2015, pp. 302–314. ACM (2015)
12.
Curiel, M., Pont, A.: Workload generators for web-based systems: characteristics, current status, and challenges. IEEE Commun. Surv. Tutor. 20(2), 1526–1546 (2018)CrossRef
13.
Drave, I., et al.: Smardt modeling for automotive software testing. Softw.: Pract. Exp. 49(2), 301–328 (2019)
14.
Dulz, W.: A versatile tool environment to perform model-based testing of web applications and multilingual websites. In: ICSOFT, pp. 45–56. SciTePress (2018)
15.
Ferme, V., Pautasso, C.: A declarative approach for performance tests execution in continuous software development environments. In: ICPE 2018, pp. 261–272. ACM Press, New York (2018)
16.
Furda, A., Fidge, C., Barros, A., Zimmermann, O.: Reengineering data-centric information systems for the cloud-a method and architectural patterns promoting multitenancy. In: Software Architecture for Big Data and the Cloud. Elsevier (2017)
17.
Grechanik, M., Luo, Q., Poshyvanyk, D., Porter, A.: Enhancing rules for cloud resource provisioning via learned software performance models. In: ICPE 2016. ACM Press, New York (2016)
18.
Henein, M.M.R., Shawky, D.M., Abd-El-Hafiz, S.K.: Clustering-based Under-sampling for software defect prediction. In: ICSOFT. SciTePress (2018)
19.
Hooda, I., Chhillar, R.S.: Software test process, testing types and techniques. Int. J. Comput. Appl. 111(13) (2015)CrossRef
20.
21.
Jiang, Z.M., Ming, Z.: Automated analysis of load testing results. In: ISSTA 2010. p. 143. ACM Press, New York (2010)
22.
Kaczmarski, M., Perry, P., Murphy, J., Portillo-Dominguez, A.O.: In-test adaptation of workload in enterprise application performance testing. In: ICPE 2017 (2017)
23.
Krichen, M., Maâlej, A.J., Lahami, M.: A model-based approach to combine conformance and load tests: an ehealth case study. Int. J. Crit. Comput.-Based Syst. 8(3–4), 282–310 (2018)CrossRef
24.
Luo, Q., Poshyvanyk, D., Nair, A., Grechanik, M.: FOREPOST: a tool for detecting performance problems with feedback-driven learning software testing. In: 38th ICSE-C, pp. 593–596. ACM (2016)
25.
Maâlej, A.J., Krichen, M.: A model based approach to combine load and functional tests for service oriented architectures. In: VECoS, pp. 123–140 (2016)
26.
Markthaler, M., et al.: Improving model-based testing in automotive software engineering. In: ICSE-SEIP, pp. 172–180. SciTePress (2018)
27.
Meier, J.D., Farre, C., Bansode, P., Barber, S., Rea, D.: Performance testing guidance for web applications: patterns & Practices. Microsoft (2007)
28.
Portillo-Dominguez, A.O., Ayala-Rivera, V.: Improving the testing of clustered systems through the effective usage of java benchmarks. In: CONISOFT (2017)
29.
Portillo-Dominguez, A.O., Perry, P., Magoni, D., Murphy, J.: PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems. Softw.: Pract. Exp. 47, 1837–1874 (2017)
30.
Portillo-Dominguez, A.O., Wang, M., Murphy, J., Magoni, D.: Automated WAIT for cloud-based application testing. ICSTW (2014)
31.
Portillo-Domínguez, A.O., Murphy, J., O’Sullivan, P.: Leverage of extended information to enhance the performance of JEE systems. In: IT&T 2012 (2012)
32.
Ramakrishnan, R., Shrawan, V., Singh, P.: Setting realistic think times in performance testing: a practitioner’s approach. In: ISEC 2017, pp. 157–164. ACM (2017)
33.
Reichelt, D.G., Kühne, S.: Better early than never. In: ICPE 2018. ACM Press, New York (2018)
34.
Reichelt, D.G., Kühne, S.: How to detect performance changes in software history. In: ICPE 2018. ACM Press, New York (2018)
35.
Sánchez, A.B., Delgado-Pérez, P., Segura, S., Medina-Bulo, I.: Performance mutation testing: hypothesis and open questions. Inf. Softw. Technol. 103, 159–161 (2018)CrossRef
36.
Segura, S., Troya, J., Duran, A., Ruiz-Cortes, A.: Performance metamorphic testing: motivation and challenges. In: ICSE-NIER (2017)
37.
Shams, M., Krishnamurthy, D., Far, B.: A model-based approach for testing the performance of web applications. In: SOQUA 2006, p. 54. ACM Press, New York (2006)
38.
Spear, W., et al.: Making performance analysis and tuning part of the software development cycle. In: DoD High Performance Computing Modernization Program Users Group Conference (2009)
39.
Tang, W., Fu, Y., Cherkasova, L., Vahdat, A.: Medisyn: a synthetic streaming media service workload generator. In: NOSSDAV 2003. p. 12. ACM Press, New York (2003)
40.
Troya, J., Segura, S., Ruiz-Cortés, A.: Automated inference of likely metamorphic relations for model transformations. J. Syst. Softw. 136, 188–208 (2018)CrossRef
41.
Wu, H., Tantawi, A.N., Yu, T.: A self-optimizing workload management solution for cloud applications. In: ICWS 2013, pp. 483–490. IEEE (2013)
Metadata
Title
Towards an Efficient Performance Testing Through Dynamic Workload Adaptation
Authors
Osvaldo Huerta-Guevara
Vanessa Ayala-Rivera
Liam Murphy
A. Omar Portillo-Dominguez
Copyright Year
2019
Book
Testing Software and Systems

Print ISBN: 978-3-030-31279-4

Electronic ISBN: 978-3-030-31280-0

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-030-31280-0_13

Premium Partner

    Image Credits