Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Buchtitelbild

2018 | OriginalPaper | Buchkapitel

Moderated Resource Elasticity for Stream Processing Applications

verfasst von : Michael Borkowski, Christoph Hochreiner, Stefan Schulte

Erschienen in: Euro-Par 2017: Parallel Processing Workshops

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In stream processing, elasticity is often realized by adapting the system scale and topology according to the volume of input data. However, this volume is often fluctuating, with a high degree of noise, which can trigger a high amount of scaling operations. Since these scaling operations introduce additional overhead and cost, systems employing such approaches are at risk of spending a significant amount of time scaling up and down, nullifying the positive effects of scalability.
To overcome this, we propose an approach for moderating the scaling behavior of stream processing applications by reducing the number of scaling operations, while still providing quick responses to changes in input data volume. Contrary to existing approaches, instead of using linear smoothing techniques, we show how to employ non-linear filtering techniques from the field of signal processing to pre-process the raw volume measurements, mitigating superfluous scaling operations, and effectively reducing the number of such operations by up to 94%.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel Erratum to: Euro-Par 2017: Parallel Processing Workshops
Nächstes Kapitel Container-Based Support for Autonomic Data Stream Processing Through the Fog
Fußnoten
1
Note that in the original work [21], the raw measured signal was named \(u_0\), and the filtered signal was named u. We have adapted the names to v and \(\overline{v}\), respectively, to maintain consistency within our work.
 
2
Again, in the original approach [15], the control function is denoted as u(t), and the noise is denoted as v(t). We have changed the names to c(t) and n(t), respectively, in order to avoid overloading and maintain consistency.
 
Literatur
1.
Abadi, D.J., et al.: The design of the borealis stream processing engine. In: CIDR, vol. 5, pp. 277–289 (2005)
2.
Beloglazov, A., Buyya, R.: Energy efficient allocation of virtual machines in cloud data centers. In: International Conference on Cluster, Cloud and Grid Computing, pp. 577–578. IEEE (2010)
3.
Buyya, R., Ranjan, R., Calheiros, R.N.: InterCloud: utility-oriented federation of cloud computing environments for scaling of application services. In: Hsu, C.-H., Yang, L.T., Park, J.H., Yeo, S.-S. (eds.) ICA3PP 2010 Part I. LNCS, vol. 6081, pp. 13–31. Springer, Heidelberg (2010). https://​doi.​org/​10.​1007/​978-3-642-13119-6_​2 CrossRef
4.
Castro Fernandez, R., et al.: Integrating scale out and fault tolerance in stream processing using operator state management. In: International Conference on Management of Data, pp. 725–736. ACM (2013)
5.
Corradi, A., Fanelli, M., Foschini, L.: VM consolidation: a real case based on OpenStack Cloud. Future Gener. Comput. Syst. 32, 118–127 (2014)CrossRef
6.
Dustdar, S., et al.: Principles of elastic processes. Internet Comput. 15(5), 66–71 (2011)CrossRefMATH
7.
Figueiredo, M.A.T., et al.: On total variation denoising: a new majorization-minimization algorithm and an experimental comparison with wavalet denoising. In: International Conference on Image Processing, pp. 2633–2636. IEEE (2006)
8.
Ghahremani, E., Kamwa, I.: Dynamic state estimation in power system by applying the extended Kalman filter with unknown inputs to phasor measurements. Trans. Power Syst. 26(4), 2556–2566 (2011)CrossRef
9.
Gong, Z., Gu, X., Wilkes, J.: PRESS: predictive elastic resource scaling for cloud systems. In: International Conference on Network and Service Management (CNSM), pp. 9–16. IEEE (2010)
10.
Gulisano, V., et al.: StreamCloud: an elastic and scalable data streaming system. Trans. Parallel Distrib. Syst. 23(12), 2351–2365 (2012)CrossRef
11.
Heinze, T., et al.: Online parameter optimization for elastic data stream processing. In: Symposium on Cloud Computing, pp. 276–287. ACM, New York (2015)
12.
Hochreiner, C., et al.: Elastic stream processing for distributed environments. Internet Comput. 19(6), 54–59 (2015)CrossRef
13.
Hochreiner, C., et al.: Elastic stream processing for the Internet of Things. In: International Conference on Cloud Computing (CLOUD), pp. 100–107 (2016)
14.
Islam, S., et al.: How a consumer can measure elasticity for cloud platforms. In: 3rd International Conference on Performance Engineering, pp. 85–96. ACM/SPEC (2012)
15.
Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction theory. J. Basic Eng. 83(3), 95–108 (1961)MathSciNetCrossRef
16.
Khan, A., et al.: Workload characterization and prediction in the cloud: a multiple time series approach. In: Network Operations and Management Symposium (NOMS), pp. 1287–1294. IEEE (2012)
17.
Little, M.A., Jones, N.S.: Sparse Bayesian step-filtering for high-throughput analysis of molecular machine dynamics. In: International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 4162–4165. IEEE (2010)
18.
Mao, M., Humphrey, M.: Auto-scaling to minimize cost and meet application deadlines in cloud workows. In: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–12. IEEE (2011)
19.
Mencagli, G., Vanneschi, M., Vespa, E.: A cooperative predictive control approach to improve the reconfiguration stability of adaptive distributed parallel applications. Trans. Auton. Adapt. Syst. (TAAS) 9(1), 2 (2014)
20.
Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27(4), 45009–45038 (2011)MathSciNetCrossRefMATH
21.
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D: Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRefMATH
22.
Selesnick, I.: Total variation denoising (an MM algorithm). In: NYU Polytechnic School of Engineering Lecture Notes (2012)
23.
Shen, Z., et al.: CloudScale: elastic resource scaling for multi-tenant cloud systems. In: 2nd Symposium on Cloud Computing, pp. 5–18. ACM (2011)
24.
Thomas, P.J., Midgley, P.A.: Image-spectroscopy-I. The advantages of increased spectral information for compositional EFTEM analysis. Ultramicroscopy 88(3), 179–186 (2001)CrossRef
25.
Xu, J., et al.: T-storm: traffic-aware online scheduling in storm. In: 34th International Conference on Distributed Computing Systems (ICDCS), pp. 535–544. IEEE (2014)
26.
Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (2010)CrossRef
Metadaten
Titel
Moderated Resource Elasticity for Stream Processing Applications
verfasst von
Michael Borkowski
Christoph Hochreiner
Stefan Schulte
Copyright-Jahr
2018
Buch
Euro-Par 2017: Parallel Processing Workshops

Print ISBN: 978-3-319-75177-1

Electronic ISBN: 978-3-319-75178-8

Copyright-Jahr: 2018

DOI
https://doi.org/10.1007/978-3-319-75178-8_1

Neuer Inhalt

    Bildnachweise