Skip to main content
Erschienen in: Empirical Software Engineering 6/2018

21.03.2018 | Experience Report

A systematic approach for performance assessment using process mining

An industrial experience report

verfasst von: Simona Bernardi, Juan L. Domínguez, Abel Gómez, Christophe Joubert, José Merseguer, Diego Perez-Palacin, José I. Requeno, Alberto Romeu

Erschienen in: Empirical Software Engineering | Ausgabe 6/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Software performance engineering is a mature field that offers methods to assess system performance. Process mining is a promising research field applied to gain insight on system processes. The interplay of these two fields opens promising applications in the industry. In this work, we report our experience applying a methodology, based on process mining techniques, for the performance assessment of a commercial data-intensive software application. The methodology has successfully assessed the scalability of future versions of this system. Moreover, it has identified bottlenecks components and replication needs for fulfilling business rules. The system, an integrated port operations management system, has been developed by Prodevelop, a medium-sized software enterprise with high expertise in geospatial technologies. The performance assessment has been carried out by a team composed by practitioners and researchers. Finally, the paper offers a deep discussion on the lessons learned during the experience, that will be useful for practitioners to adopt the methodology and for researcher to find new routes.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
Unified Modeling Language
 
2
Modeling and Analysis of Real-time and Embedded Systems
 
3
XES is the XML-based IEEE 1849-2016 standard format for event logs.
 
4
In the observation period from June, 18th to August, 11th in 2017, the dates of the logs are not contiguous since logs were not available for few days.
 
5
Observe that the normal distribution is considered instead of the Student’s one since the sample size is large (i.e., N ≫ 30).
 
6
The Student t-distribution with N-1 = 15 degrees of freedom has been used.
 
7
The percentage corresponds to the workload that is completely processed by the parsing process and it has been computed as the ratio between the filtered and unfiltered logs.
 
8
The minimum (maximum) values of the interval correspond to the 20% of the minimum (maximum) throughput of the parsing process, estimated with the analysis of the parsing scenario.
 
Literatur
Zurück zum Zitat Ajmone-Marsan M, Balbo G, Conte G, Donatelli S, Franceschinis G (1994) Modelling with Generalized Stochastic Petri Nets, 1st edn. Wiley, New YorkMATH Ajmone-Marsan M, Balbo G, Conte G, Donatelli S, Franceschinis G (1994) Modelling with Generalized Stochastic Petri Nets, 1st edn. Wiley, New YorkMATH
Zurück zum Zitat Averill ML (2015) Simulation Modeling and Analysis. McGraw-Hill, New York Averill ML (2015) Simulation Modeling and Analysis. McGraw-Hill, New York
Zurück zum Zitat Balsamo S, Di Marco A, Inverardi P, Simeoni M (2004) Model-based performance prediction in software development: A survey. IEEE Trans Softw Eng 30(5):295–310CrossRef Balsamo S, Di Marco A, Inverardi P, Simeoni M (2004) Model-based performance prediction in software development: A survey. IEEE Trans Softw Eng 30(5):295–310CrossRef
Zurück zum Zitat Bass L, Weber I, Zhu L (2015) DevOps: A software architect’s perspective. Addison-Wesley Professional, Boston Bass L, Weber I, Zhu L (2015) DevOps: A software architect’s perspective. Addison-Wesley Professional, Boston
Zurück zum Zitat Becker S, Koziolek H, Reussner R (2009) The Palladio Component Model for Model-driven Performance Prediction. J Syst Softw 82(1):3–22CrossRef Becker S, Koziolek H, Reussner R (2009) The Palladio Component Model for Model-driven Performance Prediction. J Syst Softw 82(1):3–22CrossRef
Zurück zum Zitat Bernardi S, Campos J, Merseguer J (2011) Timing-Failure Risk Assessment of UML Design Using Time Petri Net Bound Techniques. IEEE Trans Ind Inf 7(1):90–104CrossRef Bernardi S, Campos J, Merseguer J (2011) Timing-Failure Risk Assessment of UML Design Using Time Petri Net Bound Techniques. IEEE Trans Ind Inf 7(1):90–104CrossRef
Zurück zum Zitat Bernardi S, Merseguer J, Petriu DC (2012) Dependability modeling and analysis of software systems specified with UML. ACM Comput Surv 45(1):1–48CrossRef Bernardi S, Merseguer J, Petriu DC (2012) Dependability modeling and analysis of software systems specified with UML. ACM Comput Surv 45(1):1–48CrossRef
Zurück zum Zitat Brünink M, Rosenblum DS (2016) Mining performance specifications. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ACM, pp 39–49 Brünink M, Rosenblum DS (2016) Mining performance specifications. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ACM, pp 39–49
Zurück zum Zitat Brunnert A, Krcmar H (2017) Continuous performance evaluation and capacity planning using resource profiles for enterprise applications. J Syst Softw 123:239–262CrossRef Brunnert A, Krcmar H (2017) Continuous performance evaluation and capacity planning using resource profiles for enterprise applications. J Syst Softw 123:239–262CrossRef
Zurück zum Zitat Casale G, Ardagna D, Artac M, Barbier F, Nitto ED, Henry A, Iuhasz G, Joubert C, Merseguer J, Munteanu V I, Perez JF, Petcu D, Rossi M, Sheridan C, Spais I, Vladuic D (2015) DICE: Quality-Driven Development of Data-Intensive Cloud Applications. In: 7th IEEE/ACM International Workshop on Modeling in Software Engineering, MiSE 2015, Florence, Italy, May, vol 16-17, pp 78–83 Casale G, Ardagna D, Artac M, Barbier F, Nitto ED, Henry A, Iuhasz G, Joubert C, Merseguer J, Munteanu V I, Perez JF, Petcu D, Rossi M, Sheridan C, Spais I, Vladuic D (2015) DICE: Quality-Driven Development of Data-Intensive Cloud Applications. In: 7th IEEE/ACM International Workshop on Modeling in Software Engineering, MiSE 2015, Florence, Italy, May, vol 16-17, pp 78–83
Zurück zum Zitat Ceravolo P, Damiani E, Torabi M, Barbon S (2017) Toward a new generation of log pre-processing methods for process mining. In: International Conference on Business Process Management, Springer, pp 55–70 Ceravolo P, Damiani E, Torabi M, Barbon S (2017) Toward a new generation of log pre-processing methods for process mining. In: International Conference on Business Process Management, Springer, pp 55–70
Zurück zum Zitat Cortellessa V, Marco AD, Inverardi P (2011) Model-based software performance analysis, 1st edn. Springer Publishing Company Incorporated, BerlinCrossRef Cortellessa V, Marco AD, Inverardi P (2011) Model-based software performance analysis, 1st edn. Springer Publishing Company Incorporated, BerlinCrossRef
Zurück zum Zitat Diwan A, Hauswirth M, Mytkowicz T, Sweeney PF (2011) TraceAnalyzer: A system for processing performance traces. Soft Practice and Experience 41(3):267–282CrossRef Diwan A, Hauswirth M, Mytkowicz T, Sweeney PF (2011) TraceAnalyzer: A system for processing performance traces. Soft Practice and Experience 41(3):267–282CrossRef
Zurück zum Zitat Ferme V, Pautasso C (2017) Towards holistic continuous software performance assessment. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ACM, New York, NY, USA, ICPE ’17 Companion, pp 159–164 Ferme V, Pautasso C (2017) Towards holistic continuous software performance assessment. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ACM, New York, NY, USA, ICPE ’17 Companion, pp 159–164
Zurück zum Zitat Gómez A, Joubert C, Merseguer J (2016) A tool for assessing performance requirements of data-intensive applications. In: Proceedings of the XXIV National Conference of Concurrency and Distributed Systems (JCDS 2016), pp 159–169. https://github.com/dice-project/DICE-Simulation, accessed 01/23/2017 Gómez A, Joubert C, Merseguer J (2016) A tool for assessing performance requirements of data-intensive applications. In: Proceedings of the XXIV National Conference of Concurrency and Distributed Systems (JCDS 2016), pp 159–169. https://​github.​com/​dice-project/​DICE-Simulation, accessed 01/23/2017
Zurück zum Zitat Günther CW, Rozinat A (2012) Disco: Discover Your Processes. BPM (Demos) 940:40–44 Günther CW, Rozinat A (2012) Disco: Discover Your Processes. BPM (Demos) 940:40–44
Zurück zum Zitat Hernȧndez S, van Zelst SJ, Ezpeleta J, van der Aalst W (2015) Handling big(ger) logs: Connecting ProM 6 to apache hadoop. In: Proceedings of the BPM Demo Session 2015, CEUR-WS.org, vol 1418 , pp 80–84 Hernȧndez S, van Zelst SJ, Ezpeleta J, van der Aalst W (2015) Handling big(ger) logs: Connecting ProM 6 to apache hadoop. In: Proceedings of the BPM Demo Session 2015, CEUR-WS.org, vol 1418 , pp 80–84
Zurück zum Zitat Huber N, Walter J, Bähr M, Kounev S (2015) Model-based autonomic and performance-aware system adaptation in heterogeneous resource environments: A case study. In: International Conference on Cloud and Autonomic Computing, pp 181–191 Huber N, Walter J, Bähr M, Kounev S (2015) Model-based autonomic and performance-aware system adaptation in heterogeneous resource environments: A case study. In: International Conference on Cloud and Autonomic Computing, pp 181–191
Zurück zum Zitat Huber N, Brosig F, Spinner S, Kounev S, Bähr M (2017) Model-based self-aware performance and resource management using the descartes modeling language. IEEE Trans Softw Eng 43(5):432–452CrossRef Huber N, Brosig F, Spinner S, Kounev S, Bähr M (2017) Model-based self-aware performance and resource management using the descartes modeling language. IEEE Trans Softw Eng 43(5):432–452CrossRef
Zurück zum Zitat ISO (2008) Systems and software engineering – High-level Petri nets – Part 2: Transfer format. ISO/IEC 15909-2:2011, International Organization for Standardization, Geneva, Switzerland ISO (2008) Systems and software engineering – High-level Petri nets – Part 2: Transfer format. ISO/IEC 15909-2:2011, International Organization for Standardization, Geneva, Switzerland
Zurück zum Zitat Joishi J, Sureka A (2015) Vishleshan: performance comparison and programming process mining algorithms in graph-oriented and relational database query languages. In: Proceedings of the 19th International Database Engineering & Applications Symposium. ACM, pp 192–197 Joishi J, Sureka A (2015) Vishleshan: performance comparison and programming process mining algorithms in graph-oriented and relational database query languages. In: Proceedings of the 19th International Database Engineering & Applications Symposium. ACM, pp 192–197
Zurück zum Zitat Kounev S (2006) Performance modeling and evaluation of distributed component-based systems using queueing petri nets. IEEE Trans Softw Eng 32 (7):486–502CrossRef Kounev S (2006) Performance modeling and evaluation of distributed component-based systems using queueing petri nets. IEEE Trans Softw Eng 32 (7):486–502CrossRef
Zurück zum Zitat Lazowska ED, Zahorjan J, Graham GS, Sevcik KC (1984) Quantitative system performance: Computer system analysis using queueing network models. Prentice-Hall, Inc, Upper Saddle River Lazowska ED, Zahorjan J, Graham GS, Sevcik KC (1984) Quantitative system performance: Computer system analysis using queueing network models. Prentice-Hall, Inc, Upper Saddle River
Zurück zum Zitat Lȯpez-Grao J, Merseguer J, Campos J (2004) From UML activity diagrams to Stochastic Petri nets: application to software performance engineering. In: Proceedings of the Fourth International Workshop on Software and Performance, WOSP 2004, Redwood Shores, California, USA, January 14-16, 2004, pp 25–36 Lȯpez-Grao J, Merseguer J, Campos J (2004) From UML activity diagrams to Stochastic Petri nets: application to software performance engineering. In: Proceedings of the Fourth International Workshop on Software and Performance, WOSP 2004, Redwood Shores, California, USA, January 14-16, 2004, pp 25–36
Zurück zum Zitat Menasce DA, Almeida VA, Dowdy LW, Dowdy L (2004) Performance by design: computer capacity planning by example. Prentice Hall Professional, Upper Saddle River Menasce DA, Almeida VA, Dowdy LW, Dowdy L (2004) Performance by design: computer capacity planning by example. Prentice Hall Professional, Upper Saddle River
Zurück zum Zitat Rubin VA, Mitsyuk AA, Lomazova IA, van der Aalst WM (2014) Process mining can be applied to software too!. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ACM Rubin VA, Mitsyuk AA, Lomazova IA, van der Aalst WM (2014) Process mining can be applied to software too!. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ACM
Zurück zum Zitat Shavor S, D’Anjou J, Fairbrother S, Kehn D, Kellerman J, McCarthy P (2003) The java developer’s guide to eclipse. Addison-Wesley Longman Publishing Co., Inc., Boston Shavor S, D’Anjou J, Fairbrother S, Kehn D, Kellerman J, McCarthy P (2003) The java developer’s guide to eclipse. Addison-Wesley Longman Publishing Co., Inc., Boston
Zurück zum Zitat Smith CU (1990) Performance Engineering of Software Systems, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston Smith CU (1990) Performance Engineering of Software Systems, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston
Zurück zum Zitat Smith CU (2002) Williams LG. Addison Wesley Longman Publishing Co., Inc., Redwood City Smith CU (2002) Williams LG. Addison Wesley Longman Publishing Co., Inc., Redwood City
Zurück zum Zitat Tarvo A, Reiss SP (2018) Automatic performance prediction of multithreaded programs: a simulation approach. Autom Softw Eng 25(1):101–155CrossRef Tarvo A, Reiss SP (2018) Automatic performance prediction of multithreaded programs: a simulation approach. Autom Softw Eng 25(1):101–155CrossRef
Zurück zum Zitat UML2 (2015) Unified Modeling Language: Infrastructure. Version 2.5, OMG document: formal/2015- 03-01 UML2 (2015) Unified Modeling Language: Infrastructure. Version 2.5, OMG document: formal/2015- 03-01
Zurück zum Zitat Van Dongen BF et al (2005) The ProM framework: A new era in process mining tool support. In: Applications and Theory of Petri Nets 2005, Springer, pp 444–454 Van Dongen BF et al (2005) The ProM framework: A new era in process mining tool support. In: Applications and Theory of Petri Nets 2005, Springer, pp 444–454
Zurück zum Zitat Walter J, van Hoorn A, Koziolek H, Okanovic D, Kounev S (2016) Asking “What”?, Automating the “How”?: The Vision of Declarative Performance Engineering. In: Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016, Delft, The Netherlands, March 12-16, 2016, pp 91–94 Walter J, van Hoorn A, Koziolek H, Okanovic D, Kounev S (2016) Asking “What”?, Automating the “How”?: The Vision of Declarative Performance Engineering. In: Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016, Delft, The Netherlands, March 12-16, 2016, pp 91–94
Zurück zum Zitat Walter J, Stier C, Koziolek H, Kounev S (2017) An expandable extraction framework for architectural performance models. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ACM, New York, NY, USA, ICPE ’17 Companion, pp 165–170 Walter J, Stier C, Koziolek H, Kounev S (2017) An expandable extraction framework for architectural performance models. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ACM, New York, NY, USA, ICPE ’17 Companion, pp 165–170
Zurück zum Zitat XES (2016) Extensible Event Stream. IEEE Task FOrce on Process Mining, [Online; accessed 18-April-2016] XES (2016) Extensible Event Stream. IEEE Task FOrce on Process Mining, [Online; accessed 18-April-2016]
Zurück zum Zitat Yu X, Han S, Zhang D, Xie T (2014) Comprehending performance from real-world execution traces: A device-driver case. ACM SIGPLAN Notices 49(4):193–206 Yu X, Han S, Zhang D, Xie T (2014) Comprehending performance from real-world execution traces: A device-driver case. ACM SIGPLAN Notices 49(4):193–206
Zurück zum Zitat Van der Aalst W (2014) Business intelligence: Third european summer school, eBISS 2013, Dagstuhl Castle, Germany, July 7-12, 2013, Tutorial Lectures. In: Springer International Publishing, chap Process Mining in the Large: A Tutorial, pp 33–76 Van der Aalst W (2014) Business intelligence: Third european summer school, eBISS 2013, Dagstuhl Castle, Germany, July 7-12, 2013, Tutorial Lectures. In: Springer International Publishing, chap Process Mining in the Large: A Tutorial, pp 33–76
Zurück zum Zitat Van der Aalst W, Adriansyah A, van Dongen B (2012) Replaying history on process models for conformance checking and performance analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2(2):182–192 Van der Aalst W, Adriansyah A, van Dongen B (2012) Replaying history on process models for conformance checking and performance analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2(2):182–192
Metadaten
Titel
A systematic approach for performance assessment using process mining
An industrial experience report
verfasst von
Simona Bernardi
Juan L. Domínguez
Abel Gómez
Christophe Joubert
José Merseguer
Diego Perez-Palacin
José I. Requeno
Alberto Romeu
Publikationsdatum
21.03.2018
Verlag
Springer US
Erschienen in
Empirical Software Engineering / Ausgabe 6/2018
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-018-9606-9

Weitere Artikel der Ausgabe 6/2018

Empirical Software Engineering 6/2018 Zur Ausgabe