Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
The Journal of Supercomputing
Published in: The Journal of Supercomputing 11/2018

22-06-2016

Data stream processing via code annotations

Authors: Marco Danelutto, Tiziano De Matteis, Gabriele Mencagli, Massimo Torquati

Published in: The Journal of Supercomputing | Issue 11/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Time-to-solution is an important metric when parallelizing existing code. The REPARA approach provides a systematic way to instantiate stream and data parallel patterns by annotating the sequential source code with \({\mathtt {C}}\)++\({\mathtt {11}}\) attributes. Annotations are automatically transformed in a target parallel code that uses existing libraries for parallel programming (e.g., FastFlow). In this paper, we apply this approach for the parallelization of a data stream processing application. The description shows the effectiveness of the approach in easily and quickly prototyping several parallel variants of the sequential code by obtaining good overall performance in terms of both throughput and latency.
previous article MeterPU: a generic measurement abstraction API
next article Assessing and discovering parallelism in C code for heterogeneous platforms

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now
Footnotes
1
At the time of writing, this phase is hand-made and not fully automatized.
 
2
REPARA imposes restrictions on the source code when targeting specific hardware.
 
4
The trades and quotes NASDAQ tracefile of 30 Oct 2014, downloadable at http://​www.​nyxdata.​com.
 
Literature
1.
Andrade H, Gedik B, Turaga D (2014) Fundamentals of stream processing. Cambridge University Press, CambridgeCrossRef
2.
Cugola G, Margara A (2012) Processing flows of information: From data stream to complex event processing. ACM Comput Surv 44(3):15:1–15:62CrossRef
3.
Castro Fernandez R, Migliavacca M, Kalyvianaki E, Pietzuch P (2013) Integrating scale out and fault tolerance in stream processing using operator state management. In: Proc. of the 2013 ACM SIGMOD international conference on management of data, SIGMOD ’13. ACM, New York, pp 725–736
4.
Chapman B, Jost G, Pas Rvd (2007) Using OpenMP: portable shared memory parallel programming (scientific and engineering computation). The MIT Press, USA
5.
Danelutto M, De Matteis T., Mencagli G, Torquati M (2015) Parallelizing high-frequency trading applications by using c++11 attributes. In: Proc. of the 1st IEEE Inter. workshop on reengineering for parallelism in heterogeneous parallel platforms
6.
Danelutto M, Garcia JD, Sanchez LM, Sotomayor R, Torquati, M (2016) Introducing parallelism by using repara c++11 attributes. In: Proc. of the 17th Euromicro PDP 2016: parallel distributed and network-based processing. IEEE, Crete
7.
Danelutto M, Torquati M (2015) Structured parallel programming with “core” fastflow. In: Zsók V, Horváth Z, Csató L (eds) Central European functional programming school. vol 8606, Springer, LNCS, pp 29–75
8.
De Matteis T, Mencagli G (2016) Keep calm and react with foresight: strategies for low- latency and energy-efficient elastic data stream processing. In: Proceedings of the 21th ACM SIGPLAN symposium on principles and practice of parallel programming, PPoPP 2016. ACM, New York
9.
Enterprise C, Inc. (2011) C, NVIDIA, the Portland Group: The OpenACC Application Programming Interface, v1.0a
10.
11.
Gulisano V, Jimenez-Peris R, Patino-Martinez M, Soriente C, Valduriez P (2012) Streamcloud: An elastic and scalable data streaming system. IEEE Trans Parallel Distrib Syst 23(12):2351–2365CrossRef
12.
13.
14.
15.
16.
Leijen D, Schulte W, Burckhardt S (2009) The design of a task parallel library. In: Proc. of the 24th ACM SIGPLAN conference on object oriented programming systems languages and applications, OOPSLA ’09, ACM, New York, pp 227–242
17.
Blumofe RD, Joerg CF, Kuszmaul BC, Leiserson CE, Randall KH, Zhou Y (1995) Cilk: an efficient multithreaded runtime system. SIGPLAN Not 30(8):207–216CrossRef
18.
Kramer P, Egloff D, Blaser L (2016) The alea reactive dataflow system for gpu parallelization. In: Proc. of the HLGPU 2016 Workshop, HiPEAC 2016, Prague
19.
20.
ISO/IEC (2011) Information technology—Programming languages—C++. International Standard ISO/IEC 14882:20111, ISO/IEC, Geneva
21.
REPARA Project Deliverable, “D2.1: REPARA C++ Open Specification document” (2015)
22.
Andrade H, Gedik B, Wu KL, Yu PS (2011) Processing high data rate streams in system s. J Parallel Distrib Comput 71(2):145–156CrossRef
23.
Babcock B, Babu S, Datar M, Motwani R, Widom J (2002) Models and issues in data stream systems. In: Proc. of the 21st ACM SIGMOD-SIGACT-SIGART Symp. on principles of database systems, PODS ’02, ACM, New York, pp 1–16
24.
Aldinucci M, Campa S, Danelutto M, Kilpatrick P, Torquati M (2014) Design patterns percolating to parallel programming framework implementation. Int J Parallel Program 42(6):1012–1031CrossRef
25.
Balkesen C, Tatbul N (2011) Scalable data partitioning techniques for parallel sliding window processing over data streams. In: VLDB Inter. workshop on data management for sensor networks (DMSN’11), Seattle
26.
Mattson T, Sanders B, Massingill B (2004) Patterns for parallel programming, 1st edn. Addison-Wesley Professional, USA
27.
Thies W, Karczmarek M, Amarasinghe SP (2002) Streamit: a language for streaming applications. In: Proc. of the 11th Inter. conference on compiler construction, CC ’02. Springer-Verlag, London, pp 179–196
28.
29.
REPARA Project Deliverable, “D3.3: Static partitioning tool” (2015)
Metadata
Title
Data stream processing via code annotations
Authors
Marco Danelutto
Tiziano De Matteis
Gabriele Mencagli
Massimo Torquati
Publication date
22-06-2016
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 11/2018
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-016-1793-9

Other articles of this Issue 11/2018

The Journal of Supercomputing 11/2018 Go to the issue

OriginalPaper

A camera-based attention level assessment tool designed for classroom usage

OriginalPaper

Approximation of conformal mappings and novel applications to shape recognition of planar domains

OriginalPaper

Implementation of secret disclosure attack against two IoT lightweight authentication protocols

OriginalPaper

A parallel pattern for iterative stencil + reduce

OriginalPaper

An adaptive breadth-first search algorithm on integrated architectures

EditorialNotes

Reengineering for parallelism in heterogeneous parallel platforms

Premium Partner

    Image Credits