Skip to main content
Top
Published in: The Journal of Supercomputing 5/2021

26-10-2020

ACEP: an adaptive strategy for proactive and elastic processing of complex events

Authors: Mohammadmehdi Talebi, Mohsen Sharifi, Mohammadhesam Kalantari

Published in: The Journal of Supercomputing | Issue 5/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The execution of complex event processing (CEP) applications on a set of clustered homogenous computing nodes is latency-sensitive, especially when workload conditions widely change at runtime. To manage the varying workloads of nodes in a scalable and cost-effective manner, adjusting the application parallelism at runtime is critical. To tackle the scalability challenge, we have extended an existing parallelization model called PARS that only supports stateless CEP operators and runs operators in parallel regions without changing the number of computing nodes assigned to parallel regions. We have added new features to PARS in support of stateful operators by introducing local controllers and new initiator and terminator event types, making partitioning fully transparent to application developers. We have proved the correctness of this extended model, called PARS+ , with respect to its presented formal definition. We have then used PARS+ as the base parallelization model in formulation of an adaptive strategy called ACEP to auto-scale operators including the stateful operators. Scaling decisions are governed by a predictive performance model that uses a control-theoretic method for estimating the resource and latency costs of each operator at runtime. The loads of clustered compute nodes are monitored, and compute nodes in a parallel region are reconfigured at runtime to ensure a balanced load on all compute nodes, accruing minimum cost to parallelize a stateful operator. ACEP minimizes network delays because it does not force using the shared state or techniques that employ state migration. We have built an event generator to simulate event sources and experimentally evaluate the ACEP strategy in terms of response time and resource costs. Two variant implementations of ACEP have been compared with the elastic strategy presented by Xiao et al. (2018), and the findings demonstrated that our implementations had adapted themselves better in different resource and response-time-sensitive scenarios, with lower response time (6%) and lower resource cost (8%).

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Luthra M, Koldehofe B, Steinmetz R (2018) Adaptive complex event processing over fog-cloud infrastructure supporting transitions. KuVS-Fachgespräch Fog Comput 2018:17 Luthra M, Koldehofe B, Steinmetz R (2018) Adaptive complex event processing over fog-cloud infrastructure supporting transitions. KuVS-Fachgespräch Fog Comput 2018:17
4.
go back to reference Antonić A, Marjanović M, Žarko IP (2017) Modeling aggregate input load of interoperable smart city services. In: Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems, New York, NY, USA, 2017, pp 34–43, https://doi.org/10.1145/3093742.3093928 Antonić A, Marjanović M, Žarko IP (2017) Modeling aggregate input load of interoperable smart city services. In: Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems, New York, NY, USA, 2017, pp 34–43, https://​doi.​org/​10.​1145/​3093742.​3093928
7.
go back to reference Saleh O, Betz H, Sattler K-U (2015) “Partitioning for scalable complex event processing on data streams”, in New Trends in Database and Information Systems II. Springer, Cham, pp 185–197 Saleh O, Betz H, Sattler K-U (2015) “Partitioning for scalable complex event processing on data streams”, in New Trends in Database and Information Systems II. Springer, Cham, pp 185–197
8.
go back to reference Gulisano V, Jimenez-Peris R, Patino-Martinez M, Valduriez P (2010) StreamCloud: a large scale data streaming system. In: Proceedings of the 2010 IEEE 30th International Conference on Distributed Computing Systems, Washington, DC, USA, 2010, pp 126–137, https://doi.org/10.1109/icdcs.2010.72 Gulisano V, Jimenez-Peris R, Patino-Martinez M, Valduriez P (2010) StreamCloud: a large scale data streaming system. In: Proceedings of the 2010 IEEE 30th International Conference on Distributed Computing Systems, Washington, DC, USA, 2010, pp 126–137, https://​doi.​org/​10.​1109/​icdcs.​2010.​72
9.
go back to reference Castro Fernandez R et al (2013) Integrating scale out and fault tolerance in stream processing using operator state management. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, 2013, pp 725–736, https://doi.org/10.1145/2463676.2465282 Castro Fernandez R et al (2013) Integrating scale out and fault tolerance in stream processing using operator state management. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, 2013, pp 725–736, https://​doi.​org/​10.​1145/​2463676.​2465282
10.
go back to reference Heinze T, Jerzak Z, Hackenbroich G, Fetzer C (2014) Latency-aware elastic scaling for distributed data stream processing systems. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2014, pp 13–22, https://doi.org/10.1145/2611286.2611294 Heinze T, Jerzak Z, Hackenbroich G, Fetzer C (2014) Latency-aware elastic scaling for distributed data stream processing systems. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2014, pp 13–22, https://​doi.​org/​10.​1145/​2611286.​2611294
11.
go back to reference Camacho EF, Bordons C (2007) Nonlinear model predictive control: an introductory review. In: Assessment and Future Directions of Nonlinear Model Predictive Control, R Findeisen, F Allgöwer, and LT Biegler, eds. Heidelberg: Springer, 2007, pp 1–16 Camacho EF, Bordons C (2007) Nonlinear model predictive control: an introductory review. In: Assessment and Future Directions of Nonlinear Model Predictive Control, R Findeisen, F Allgöwer, and LT Biegler, eds. Heidelberg: Springer, 2007, pp 1–16
14.
15.
go back to reference Wu S, Kumar V, Wu K-L, Ooi BC (2012) Parallelizing stateful operators in a distributed stream processing system: how, should you and how much? In: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2012, pp 278–289, https://doi.org/10.1145/2335484.2335515 Wu S, Kumar V, Wu K-L, Ooi BC (2012) Parallelizing stateful operators in a distributed stream processing system: how, should you and how much? In: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2012, pp 278–289, https://​doi.​org/​10.​1145/​2335484.​2335515
18.
27.
go back to reference Fernandez RC, Weidlich M, Pietzuch P, Gal A (2014) Scalable stateful stream processing for smart grids. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2014, pp 276–281, https://doi.org/10.1145/2611286.2611326 Fernandez RC, Weidlich M, Pietzuch P, Gal A (2014) Scalable stateful stream processing for smart grids. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, New York, NY, USA, 2014, pp 276–281, https://​doi.​org/​10.​1145/​2611286.​2611326
31.
go back to reference Weisenburger P et al (2017) Quality-aware runtime adaptation in complex event processing. In: 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) (pp 140–151). 2017. IEEE Weisenburger P et al (2017) Quality-aware runtime adaptation in complex event processing. In: 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) (pp 140–151). 2017. IEEE
32.
go back to reference Zhao B (2018) Complex event processing under constrained resources by state-based load shedding. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp 1699–1703. IEEE, 2018 Zhao B (2018) Complex event processing under constrained resources by state-based load shedding. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp 1699–1703. IEEE, 2018
36.
go back to reference Fülöp LJ et al (2012) Predictive complex event processing: a conceptual framework for combining complex event processing and predictive analytics. In: Proceedings of the Fifth Balkan Conference in Informatics, New York, NY, USA, 2012, pp 26–31, https://doi.org/10.1145/2371316.2371323 Fülöp LJ et al (2012) Predictive complex event processing: a conceptual framework for combining complex event processing and predictive analytics. In: Proceedings of the Fifth Balkan Conference in Informatics, New York, NY, USA, 2012, pp 26–31, https://​doi.​org/​10.​1145/​2371316.​2371323
37.
39.
go back to reference Camacho EF, Bordons C (2007) Nonlinear model predictive control: an introductory review. In: Assessment and Future Directions of Nonlinear Model Predictive Control, R Findeisen, F Allgöwer, and LT Biegler, eds. Heidelberg: Springer, 2007, pp 1–16 Camacho EF, Bordons C (2007) Nonlinear model predictive control: an introductory review. In: Assessment and Future Directions of Nonlinear Model Predictive Control, R Findeisen, F Allgöwer, and LT Biegler, eds. Heidelberg: Springer, 2007, pp 1–16
Metadata
Title
ACEP: an adaptive strategy for proactive and elastic processing of complex events
Authors
Mohammadmehdi Talebi
Mohsen Sharifi
Mohammadhesam Kalantari
Publication date
26-10-2020
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 5/2021
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-020-03454-0

Other articles of this Issue 5/2021

The Journal of Supercomputing 5/2021 Go to the issue

Premium Partner