survey

Runtime Adaptation of Data Stream Processing Systems: The State of the Art

Authors:
Valeria Cardellini

University of Rome Tor Vergata, Rome, Italy

University of Rome Tor Vergata, Rome, Italy

0000-0002-6870-7083
View Profile

,
Francesco Lo Presti

University of Rome Tor Vergata, Rome, Italy

University of Rome Tor Vergata, Rome, Italy

0000-0002-7461-6276
View Profile

,
Matteo Nardelli

University of Rome Tor Vergata, Rome, Italy

University of Rome Tor Vergata, Rome, Italy

0000-0002-9519-9387
View Profile

,
Gabriele Russo Russo

University of Rome Tor Vergata, Rome, Italy

University of Rome Tor Vergata, Rome, Italy

0000-0001-8233-4570
View Profile

Authors Info & Claims

ACM Computing Surveys Volume 54 Issue 11sArticle No.: 237pp 1–36https://doi.org/10.1145/3514496

Published:09 September 2022Publication History

ACM Computing Surveys

Abstract

Data stream processing (DSP) has emerged over the years as the reference paradigm for the analysis of continuous and fast information flows, which often have to be processed with low-latency requirements to extract insights and knowledge from raw data. Dealing with unbounded dataflows, DSP applications are typically long running and thus, likely experience varying workloads and working conditions over time. To keep a consistent service level in face of such variability, a lot of effort has been spent studying strategies for runtime adaptation of DSP systems and applications. In this survey, we review the most relevant approaches from the literature, presenting a taxonomy to characterize the state of the art along several key dimensions. Our analysis allows us to identify current research trends as well as open challenges that will motivate further investigations in this field.

Supplemental Material

Available for Download

pdf

3514496.app.pdf (947.9 KB)

Supplementary appendix

REFERENCES

[1] Abadi Daniel J., Ahmad Yanif, Balazinska Magdalena, Çetintemel Ugur, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag S. Maskey, et al. 2005. The design of the Borealis stream processing engine. In Proc. of CIDR’05. 277–289.Google Scholar
[2] Abadi Daniel J., Carney Don, Çetintemel Ugur, Cherniack Mitch, Convey Christian, Sangdon Lee, Michael Stonebraker, Nesime Tatbul, and Stan Zdonik. 2003. Aurora: A new model and architecture for data stream management. VLDB J. 12, 2 (2003), 120–139.Google ScholarDigital Library
[3] Abdelhamid Ahmed S., Mahmood Ahmed R., Daghistani Anas, and Aref Walid G.. 2020. Prompt: Dynamic data-partitioning for distributed micro-batch stream processing systems. In Proc. of ACM SIGMOD’20. ACM, New York, NY, 2455–2469.Google Scholar
[4] Akidau Tyler, Bradshaw Robert, Chambers Craig, Chernyak Slava, Fernández-Moctezuma Rafael, Reuven Lax, Sam McVeety, et al. 2015. The dataflow model: A practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc. VLDB Endow. 8, 12 (2015), 1792–1803.Google ScholarDigital Library
[5] Aljoby Walid A. Y., Wang Xin, Fu Tom Z. J., and Ma Richard T. B.. 2019. On SDN-enabled online and dynamic bandwidth allocation for stream analytics. IEEE J. Sel. Areas Commun. 37, 8 (2019), 1688–1702.Google ScholarDigital Library
[6] Amini Lisa, Jain Navendu, Sehgal Anshul, Silber Jeremy, and Verscheure Olivier. 2006. Adaptive control of extreme-scale stream processing systems. In Proc. of IEEE ICDCS’06.Google Scholar
[7] Aniello Leonardo, Baldoni Roberto, and Querzoni Leonardo. 2013. Adaptive online scheduling in storm. In Proc. of ACM DEBS’13. 207–218.Google Scholar
[8] Aral Atakan, Erol-Kantarci Melike, and Brandic Ivona. 2020. Staleness control for edge data analytics. Proc. ACM Meas. Anal. Comput. Syst. 4, 2 (2020), Article 38, 24 pages.Google ScholarDigital Library
[9] Assunção Marcos D. de, Veith Alexandre da Silva, and Buyya Rajkumar. 2018. Distributed data stream processing and edge computing: A survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103 (2018), 1–17.Google ScholarDigital Library
[10] Babcock Brian, Babu Shivnath, Datar Mayur, Motwani Rajeev, and Widom Jennifer. 2002. Models and issues in data stream systems. In Proc. of ACM PODS’02. 1–16.Google Scholar
[11] Babcock Brian, Datar Mayur, and Motwani Rajeev. 2004. Load shedding for aggregation queries over data streams. In Proc. of ICDE’04. IEEE, Los Alamitos, CA, 350–361.Google Scholar
[12] Balazinska Magdalena, Balakrishnan Hari, Madden Samuel, and Stonebraker Michael. 2008. Fault-tolerance in the Borealis distributed stream processing system. ACM Trans. Database Syst. 33, 1 (2008), Article 3, 44 pages.Google ScholarDigital Library
[13] Balazinska Magdalena, Balakrishnan Hari, and Stonebraker Mike. 2004. Contract-based load management in federated distributed systems. In Proc. of USENIX NSDI’04.Google Scholar
[14] Balkesen Cagri, Tatbul Nesime, and Özsu M. Tamer. 2013. Adaptive input admission and management for parallel stream processing. In Proc. of ACM DEBS’13. 15–26.Google Scholar
[15] Begoli Edmon, Akidau Tyler, Chernyak Slava, Hueske Fabian, Knight Kathryn, Kenneth Knowles, Daniel Mills, and Dan Sotolongo. 2021. Watermarks in stream processing systems: Semantics and comparative analysis of Apache Flink and Google Cloud dataflow. Proc. VLDB Endow. 14, 12 (2021), 3135–3147.Google ScholarDigital Library
[16] Belkhiria Mehdi M., Bertier Marin, and Tedeschi Cédric. 2020. Group mutual exclusion to scale distributed stream processing pipelines. In Proc. of IEEE/ACM UCC’20. 247–256.Google Scholar
[17] Bellavista Paolo, Corradi Antonio, Kotoulas Spyros, and Reale Andrea. 2014a. Adaptive fault-tolerance for dynamic resource provisioning in distributed stream processing systems. In Proc. of EDBT’14. 85–96.Google Scholar
[18] Bellavista Paolo, Corradi Antonio, Reale Andrea, and Ticca Nicola. 2014b. Priority-based resource scheduling in distributed stream processing systems for big data applications. In Proc. of IEEE/ACM UCC’14. 363–370.Google Scholar
[19] Bolch Gunter, Greiner Stefan, Meer Hermann de, and Trivedi Kishor S.. 2006. Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications (2nd ed.). Wiley.Google ScholarCross Ref
[20] Borkowski Michael, Hochreiner Christoph, and Schulte Stefan. 2019. Minimizing cost by reducing scaling operations in distributed stream processing. Proc. VLDB Endow. 12, 7 (2019), 724–737.Google ScholarDigital Library
[21] Buddhika Thilina, Stern Ryan, Lindburg Kira, Ericson Kathleen, and Pallickara Shrideep. 2017. Online scheduling and interference alleviation for low-latency, high-throughput processing of data streams. IEEE Trans. Parallel Distrib. Syst. 28, 12 (2017), 3553–3569.Google ScholarDigital Library
[22] Cammert Michael, Kramer Jurgen, Seeger Bernhard, and Vaupel Sonny. 2008. A cost-based approach to adaptive resource management in data stream systems. IEEE Trans. Knowl. Data Eng. 20, 2 (2008), 230–245.Google ScholarDigital Library
[23] Caneill Matthieu, El-Rheddane Ahmed, Leroy Vincent, and Palma Noël De. 2016. Locality-aware routing in stateful streaming applications. In Proc. of ACM/IFIP/USENIX MIDDLEWARE’16. ACM, New York, NY, Article 4, 13 pages.Google Scholar
[24] Carbone Paris, Ewen Stephan, Fóra Gyula, Haridi Seif, Richter Stefan, and Tzoumas Kostas. 2017. State management in Apache Flink®: Consistent stateful distributed stream processing. Proc. VLDB Endow. 10, 12 (2017), 1718–1729.Google ScholarDigital Library
[25] Cardellini Valeria, Presti Francesco Lo, Nardelli Matteo, and Russo Gabriele Russo. 2018b. Decentralized self-adaptation for elastic data stream processing. Future Gener. Comput. Syst. 87 (2018), 171–185.Google ScholarDigital Library
[26] Cardellini Valeria, Presti Francesco Lo, Nardelli Matteo, and Russo Gabriele Russo. 2018a. Optimal operator deployment and replication for elastic distributed data stream processing. Concurr. Comp. Pract. Exp. 30, 9 (2018).Google Scholar
[27] Cardellini Valeria, Nardelli Matteo, and Luzi Dario. 2016. Elastic stateful stream processing in storm. In Proc. of HPCS’16. IEEE, Los Alamitos, CA, 583–590.Google Scholar
[28] Carminati Barbara, Ferrari Elena, Cao Jianneng, and Tan Kian Lee. 2010. A framework to enforce access control over data streams. ACM Trans. Inf. Syst. Secur. 13, 3 (2010), Article 28, 31 pages.Google ScholarDigital Library
[29] Castro Paul, Ishakian Vatche, Muthusamy Vinod, and Slominski Aleksander. 2019. The rise of serverless computing. Commun. ACM 62, 12 (2019), 44–54.Google ScholarDigital Library
[30] Cerviño Javier, Kalyvianaki Evangelia, Salvachúa Joaquín, and Pietzuch Peter R.. 2012. Adaptive provisioning of stream processing systems in the cloud. In Proc. of IEEE ICDE’12. 295–301.Google Scholar
[31] Chandrasekaran Sirish, Cooper Owen, Deshpande Amol, Franklin Michael J., Hellerstein Joseph M., Wei Hong, Sailesh Krishnamurthy, Samuel R. Madden, and Fred Reiss. 2003. TelegraphCQ: Continuous dataflow processing. In Proc. of ACM SIGMOD’03. 668.Google Scholar
[32] Chao Mengyuan and Stoleru Radu. 2020. R-MStorm: A resilient mobile stream processing system for dynamic edge networks. In Proc. of IEEE ICFC’20. 64–72.Google Scholar
[33] Chao Mengyuan, Yang Chen, Zeng Yukun, and Stoleru Radu. 2018. F-MStorm: Feedback-based online distributed mobile stream processing. In Proc. of IEEE/ACM SEC’18. 273–285.Google Scholar
[34] Chaturvedi Shilpa and Simmhan Yogesh. 2019. Toward resilient stream processing on clouds using moving target defense. In Proc. of IEEE ISORC’19. 134–142.Google Scholar
[35] Chaturvedi Shilpa, Tyagi Sahil, and Simmhan Yogesh. 2021. Cost-effective sharing of streaming dataflows for IoT applications. IEEE Trans. Cloud Comput. 9, 4 (2021), 1391–1407.Google ScholarCross Ref
[36] Chatzistergiou Andreas and Viglas Stratis D.. 2014. Fast heuristics for near-optimal task allocation in data stream processing over clusters. In Proc. of ACM CIKM’14. 1579–1588.Google Scholar
[37] Chen Xin, Vigfusson Ymir, Blough Douglas M., Zheng Fang, Wu Kun-Lung, and Hu Liting. 2017. GOVERNOR: Smoother stream processing through smarter backpressure. In Proc. of IEEE ICAC’17. 145–154.Google Scholar
[38] Cheng Dazhao, Zhou Xiaobo, Wang Yu, and Jiang Changjun. 2018. Adaptive scheduling parallel jobs with dynamic batching in spark streaming. IEEE Trans. Parallel Distrib. Syst. 29, 12 (2018), 2672–2685.Google ScholarCross Ref
[39] Cugola Gianpaolo and Margara Alessandro. 2012. Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44, 3 (2012), Article 15, 62 pages.Google ScholarDigital Library
[40] Cugola Gianpaolo and Margara Alessandro. 2013. Deployment strategies for distributed complex event processing. Computing 95, 2 (2013), 129–156.Google ScholarDigital Library
[41] Das Tathagata, Zhong Yuan, Stoica Ion, and Shenker Scott. 2014. Adaptive stream processing using dynamic batch sizing. In Proc. of ACM SoCC’14. Article 16, 13 pages.Google Scholar
[42] Dayarathna Miyuru and Perera Srinath. 2018. Recent advancements in event processing. ACM Comput. Surv. 51, 2 (2018), Article 33, 36 pages.Google Scholar
[43] Matteis Tiziano De and Mencagli Gabriele. 2017b. Elastic scaling for distributed latency-sensitive data stream operators. In Proc. of PDP’17. IEEE, Los Alamitos, CA, 61–68.Google Scholar
[44] Matteis Tiziano De and Mencagli Gabriele. 2017a. Proactive elasticity and energy awareness in data stream processing. J. Syst. Softw. 127 (2017), 302–319.Google ScholarDigital Library
[45] Souza Felipe R. de, Veith Alexandre da Silva, Assunção Marcos D. de, and Caron Eddy. 2020. Scalable joint optimization of placement and parallelism of data stream processing applications on cloud-edge infrastructure. In Service-Oriented Computing. Lecture Notes in Computer Science, Vol. 12571. Springer, 149–164.Google Scholar
[46] Du Guangxiang and Gupta Indranil. 2016. New techniques to curtail the tail latency in stream processing systems. In Proc. of DCC@PODC’16. ACM, New York, NY, Article 7, 6 pages.Google Scholar
[47] Eibel Christopher, Gulden Christian, Schröder-Preikschat Wolfgang, and Distler Tobias. 2018. Strome: Energy-aware data-stream processing. In Distributed Applications and Interoperable Systems. Lecture Notes in Computer Science, Vol. 10853. Springer, 40–57.Google Scholar
[48] Eskandari Leila, Huang Zhiyi, and Eyers David M.. 2016. P-scheduler: Adaptive hierarchical scheduling in Apache Storm. In Proc. of ACSW’16.ACM, New York, NY, Article 26, 10 pages.Google Scholar
[49] Fang Junhua, Chao Pingfu, Zhang Rong, and Zhou Xiaofang. 2019. Integrating workload balancing and fault tolerance in distributed stream processing system. World Wide Web 22, 6 (2019), 2471–2496.Google ScholarCross Ref
[50] Fang Junhua, Zhang Rong, Fu Tom Z. J., Zhang Zhenjie, Zhou Aoying, and Zhou Xiaofang. 2018. Distributed stream rebalance for stateful operator under workload variance. IEEE Trans. Parallel Distrib. Syst. 29, 10 (2018), 2223–2240.Google ScholarCross Ref
[51] Farhat Omar, Daudjee Khuzaima, and Querzoni Leonardo. 2021. Klink: Progress-aware scheduling for streaming data systems. In Proc. of ACM SIGMOD’21. 485–498.Google Scholar
[52] Fernandez Raul Castro, Migliavacca Matteo, Kalyvianaki Evangelia, and Pietzuch Peter R.. 2013. Integrating scale out and fault tolerance in stream processing using operator state management. In Proc. of ACM SIGMOD’13. 725–736.Google Scholar
[53] Floratou Avrilia, Agrawal Ashvin, Graham Bill, Rao Sriram, and Ramasamy Karthik. 2017. Dhalion: Self-regulating stream processing in Heron. Proc. VLDB Endow. 10, 12 (2017), 1825–1836.Google ScholarDigital Library
[54] Fragkoulis Marios, Carbone Paris, Kalavri Vasiliki, and Katsifodimos Asterios. 2020. A survey on the evolution of stream processing systems. CoRR abs/2008.00842 (2020).Google Scholar
[55] Fu Tom Z. J., Ding Jianbing, Ma Richard T. B., Winslett Marianne, Yang Yin, and Zhang Zhenjie. 2017. DRS: Auto-scaling for real-time stream analytics. IEEE/ACM Trans. Netw. 25, 6 (2017), 3338–3352.Google ScholarDigital Library
[56] Fu Xinwei, Ghaffar Talha, Davis James C., and Lee Dongyoon. 2019. EdgeWise: A better stream processing engine for the edge. In Proc. of USENIX ATC’19. 929–946.Google Scholar
[57] Gedik Bugra, Schneider Scott, Hirzel Martin, and Wu Kun-Lung. 2014. Elastic scaling for data stream processing. IEEE Trans. Parallel Distrib. Syst. 25, 6 (2014), 1447–1463.Google ScholarDigital Library
[58] Golab Lukasz and Özsu M. Tamer. 2003. Issues in data stream management. ACM SIGMOD Rec. 32, 2 (2003), 5–14.Google ScholarDigital Library
[59] Gu Xiaohui, Yu Philip S., and Nahrstedt Klara. 2005. Optimal component composition for scalable stream processing. In Proc. of IEEE ICDCS’05. 773–782.Google Scholar
[60] Gulisano Vincenzo, Jiménez-Peris Ricardo, Patiño-Martinez Marta, Soriente Claudio, and Valduriez Patrick. 2012. StreamCloud: An elastic and scalable data streaming system. IEEE Trans. Parallel Distrib. Syst. 23, 12 (2012), 2351–2365.Google ScholarDigital Library
[61] Gulisano Vincenzo, Papatriantafilou Marina, and Papadopoulos Alessandro Vittorio. 2019. Elasticity. In Encyclopedia of Big Data Technologies. Springer.Google ScholarCross Ref
[62] Guo Qingsong and Zhou Yongluan. 2017a. CBP: A new parallelization paradigm for massively distributed stream processing. In Database Systems for Advanced Applications. Lecture Notes in Computer Science, Vol. 10178. Springer, 304–320.Google Scholar
[63] Guo Qingsong and Zhou Yongluan. 2017b. Stateful load balancing for parallel stream processing. In Euro-Par 2017: Parallel Processing Workshops. Lecture Notes in Computer Science, Vol. 10659. Springer, 80–93.Google Scholar
[64] Han Zheng, Chu Rui, Mi Haibo, and Wang Huaimin. 2014. Elastic allocator: An adaptive task scheduler for streaming query in the cloud. In Proc. of IEEE SOSE’14. 284–289.Google Scholar
[65] Havet Aurélien, Pires Rafael, Felber Pascal, Pasin Marcelo, Rouvoy Romain, and Schiavoni Valerio. 2017. SecureStreams: A reactive middleware framework for secure data stream processing. In Proc. of ACM DEBS’17. 124–133.Google Scholar
[66] Heintz Benjamin, Chandra Abhishek, and Sitaraman Ramesh K.. 2020. Optimizing timeliness and cost in geo-distributed streaming analytics. IEEE Trans. Cloud Comput. 8, 1 (2020), 232–245.Google ScholarCross Ref
[67] Heinze Thomas, Jerzak Zbigniew, Hackenbroich Gregor, and Fetzer Christof. 2014a. Latency-aware elastic scaling for distributed data stream processing systems. In Proc. of ACM DEBS’14. 13–22.Google Scholar
[68] Heinze Thomas, Pappalardo Valerio, Jerzak Zbigniew, and Fetzer Christof. 2014b. Auto-scaling techniques for elastic data stream processing. In Proc. of IEEE ICDEW’14. 296–302.Google Scholar
[69] Heinze Thomas, Roediger Lars, Meister Andreas, Ji Yuanzhen, Jerzak Zbigniew, and Fetzer Christof. 2015a. Online parameter optimization for elastic data stream processing. In Proc. of ACM SoCC’15. 276–287.Google Scholar
[70] Heinze Thomas, Zia Mariam, Krahn Robert, Jerzak Zbigniew, and Fetzer Christof. 2015b. An adaptive replication scheme for elastic data stream processing systems. In Proc. of ACM DEBS’15. 150–161.Google Scholar
[71] Herodotou Herodotos, Chen Yuxing, and Lu Jiaheng. 2020. A survey on automatic parameter tuning for big data processing systems. ACM Comput. Surv. 53, 2 (2020), Article 43, 37 pages.Google Scholar
[72] Hidalgo Nicolas, Wladdimiro Daniel, and Rosas Erika. 2017. Self-adaptive processing graph with operator fission for elastic stream processing. J. Syst. Softw. 127 (2017), 205–216.Google ScholarDigital Library
[73] Hirzel Martin, Soulé Robert, Schneider Scott, Gedik Bugra, and Grimm Robert. 2013. A catalog of stream processing optimizations. ACM Comput. Surv. 46, 4 (2013), Article 46, 34 pages.Google Scholar
[74] Hochreiner Christoph, Vögler Michael, Schulte Stefan, and Dustdar Schahram. 2016. Elastic stream processing for the Internet of Things. In Proc. of IEEE CLOUD’16. 100–107.Google Scholar
[75] Hoffmann Moritz, Lattuada Andrea, McSherry Frank, Kalavri Vasiliki, Liagouris John, and Roscoe Timothy. 2019. Megaphone: Latency-conscious state migration for distributed streaming dataflows. Proc. VLDB Endow. 12, 9 (2019), 1002–1015.Google ScholarDigital Library
[76] Farahabady Mohammad R. Hoseiny, Jannesari Ali, Taheri Javid, Bao Wei, Zomaya Albert Y., and Tari Zahir. 2020. Q-Flink: A QoS-aware controller for Apache Flink. In Proc. of IEEE/ACM CCGRID’20. 629–638.Google Scholar
[77] Farahabady Mohammad R. Hoseiny, Samani Hamid R. Dehghani, Wang Yidan, Zomaya Albert Y., and Tari Zahir. 2016. A QoS-aware controller for Apache storm. In Proc. of IEEE NCA’16. 334–342.Google Scholar
[78] Farahabady Mohammad R. Hoseiny, Zomaya Albert Y., and Tari Zahir. 2017. QoS- and contention- aware resource provisioning in a stream processing engine. In Proc. of IEEE CLUSTER’17. 137–146.Google Scholar
[79] Huang Qun and Lee Patrick P. C.. 2016. Toward high-performance distributed stream processing via approximate fault tolerance. Proc. VLDB Endow. 10, 3 (2016), 73–84.Google ScholarDigital Library
[80] Huang Xi, Shao Ziyu, and Yang Yang. 2020. POTUS: Predictive online tuple scheduling for data stream processing systems. IEEE Trans. Cloud Comput.To appear.Google Scholar
[81] Hwang Jeong-Hyon, Çetintemel Ugur, and Zdonik Stan. 2008. Fast and highly-available stream processing over wide area networks. In Proc. of IEEE ICDE’08. 804–813.Google Scholar
[82] Imai Shigeru, Patterson Stacy, and Varela Carlos A.. 2018. Uncertainty-aware elastic virtual machine scheduling for stream processing systems. In Proc. of IEEE/ACM CCGRID’18. 62–71.Google Scholar
[83] Jia Changjiang, Cai Yan, Yu Yuen-Tak, and Tse T. H.. 2016. 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing. J. Syst. Softw. 116 (2016), 206–219.Google ScholarDigital Library
[84] Jlassi Aymen and Tedeschi Cédric. 2020. Merge, split, and cluster: Dynamic deployment of stream processing applications. In Proc. of IEEE/ACM CCGRID’20. 71–80.Google Scholar
[85] Jonathan Albert, Chandra Abhishek, and Weissman Jon B.. 2020. WASP: Wide-area adaptive stream processing. In Proc. of ACM/IFIP MIDDLEWARE’20. ACM, New York, NY, 221–235.Google Scholar
[86] Kahveci Basri and Gedik Bugra. 2020. Joker: Elastic stream processing with organic adaptation. J. Parallel Distrib. Comput. 137 (2020), 205–223.Google ScholarDigital Library
[87] Kalavri Vasiliki, Liagouris John, Hoffmann Moritz, Dimitrova Desislava C., Forshaw Matthew, and Roscoe Timothy. 2018. Three steps is all you need: Fast, accurate, automatic scaling decisions for distributed streaming dataflows. In Proc. of USENIX OSDI’18. 783–798.Google Scholar
[88] Kalim Faria, Xu Le, Bathey Sharanya, Meherwal Richa, and Gupta Indranil. 2018. Henge: Intent-driven multi-tenant stream processing. In Proc. of ACM SoCC’18. 249–262.Google Scholar
[89] Kalyvianaki Evangelia, Charalambous Themistoklis, Fiscato Marco, and Pietzuch Peter. 2012. Overload management in data stream processing systems with latency guarantees. In Proc. of FCW’12.Google Scholar
[90] Kalyvianaki Evangelia, Fiscato Marco, Salonidis Theodoros, and Pietzuch Peter R.. 2016. THEMIS: Fairness in federated stream processing under overload. In Proc. of ACM SIGMOD’16. 541–553.Google Scholar
[91] Kalyvianaki Evangelia, Wiesemann Wolfram, Vu Quang H., Kuhn Daniel, and Pietzuch Peter R.. 2011. SQPR: Stream query planning with reuse. In Proc. of IEEE ICDE’11. 840–851.Google Scholar
[92] Katsipoulakis Nikos R., Labrinidis Alexandros, and Chrysanthis Panos K.. 2017. A holistic view of stream partitioning costs. Proc. VLDB Endow. 10, 11 (2017), 1286–1297.Google ScholarDigital Library
[93] Katsipoulakis Nikos R., Labrinidis Alexandros, and Chrysanthis Panos K.. 2018. Concept-driven load shedding: Reducing size and error of voluminous and variable data streams. In Proc. of IEEE Big Data’18. 418–427.Google Scholar
[94] Katsipoulakis Nikos R., Labrinidis Alexandros, and Chrysanthis Panos K.. 2020. SPEAr: Expediting stream processing with accuracy guarantees. In Proc. of IEEE ICDE’20. 1105–1116.Google Scholar
[95] Kleiminger Wilhelm, Kalyvianaki Evangelia, and Pietzuch Peter R.. 2011. Balancing load in stream processing with the cloud. In Proc. of IEEE ICDE’11. 16–21.Google Scholar
[96] Klimovic Ana, Wang Yawen, Stuedi Patrick, Trivedi Animesh, Pfefferle Jonas, and Kozyrakis Christos. 2018. Pocket: Elastic ephemeral storage for serverless analytics. In Proc. of USENIX OSDI’18. 427–444.Google Scholar
[97] Koliousis Alexandros, Weidlich Matthias, Fernandez Raul Castro, Wolf Alexander L., Costa Paolo, and Pietzuch Peter R.. 2016. SABER: Window-based hybrid stream processing for heterogeneous architectures. In Proc. of ACM SIGMOD’16. 555–569.Google Scholar
[98] Kombi Roland Kotto, Lumineau Nicolas, and Lamarre Philippe. 2017. A preventive auto-parallelization approach for elastic stream processing. In Proc. of IEEE ICDCS’17. 1532–1542.Google Scholar
[99] Kumbhare Alok G., Simmhan Yogesh, and Prasanna Viktor K.. 2014. PLAStiCC: Predictive look-ahead scheduling for continuous dataflows on clouds. In Proc. of IEEE/ACM CCGrid’14. 344–353.Google Scholar
[100] Kumbhare Alok G., Simmhan Yogesh L., Frincu Marc, and Prasanna Viktor K.. 2015. Reactive resource provisioning heuristics for dynamic dataflows on cloud infrastructure. IEEE Trans. Cloud Comput. 3, 2 (2015), 105–118.Google ScholarCross Ref
[101] Lakshmanan Geetika T., Li Ying, and Strom Robert E.. 2008. Placement strategies for internet-scale data stream systems. IEEE Internet Comput. 12, 6 (2008), 50–60.Google ScholarDigital Library
[102] Lakshmanan Geetika T. and Strom Robert E.. 2008. Biologically-inspired distributed middleware management for stream processing systems. In Middleware 2008.Lecture Notes in Computer Science, Vol. 5346. Springer, 223–242.Google Scholar
[103] Quoc Do Le, Beck Martin, Bhatotia Pramod, Chen Ruichuan, Fetzer Christof, and Strufe Thorsten. 2017a. PrivApprox: Privacy-preserving stream analytics. In Proc. of USENIX ATC’17. 659–672.Google Scholar
[104] Quoc Do Le, Chen Ruichuan, Bhatotia Pramod, Fetzer Christof, Hilt Volker, and Strufe Thorsten. 2017b. StreamApprox: Approximate computing for stream analytics. In Proc. of ACM/IFIP/USENIX MIDDLEWARE’17. ACM, New York, NY, 185–197.Google Scholar
[105] Lei Chuan and Rundensteiner Elke A.. 2014. Robust distributed query processing for streaming data. ACM Trans. Database Syst. 39, 2 (2014), Article 17, 45 pages.Google ScholarDigital Library
[106] Li Jack, Pu Calton, Chen Yuan, Gmach Daniel, and Milojicic Dejan S.. 2016. Enabling elastic stream processing in shared clusters. In Proc. of IEEE CLOUD’16. 108–115.Google Scholar
[107] Li Kejian, Liu Gang, and Lu Minhua. 2019. A holistic stream partitioning algorithm for distributed stream processing systems. In Proc. of PDCAT’19. IEEE, Los Alamitos, CA, 202–207.Google Scholar
[108] Li Teng, Xu Zhiyuan, Tang Jian, and Wang Yanzhi. 2018. Model-free control for distributed stream data processing using deep reinforcement learning. Proc. VLDB Endow. 11, 6 (2018), 705–718.Google ScholarDigital Library
[109] Liao Xiaofei, Huang Yu, Zheng Long, and Jin Hai. 2019. Efficient time-evolving stream processing at scale. IEEE Trans. Parallel Distrib. Syst. 30, 10 (2019), 2165–2178.Google ScholarCross Ref
[110] Liu Xunyun and Buyya Rajkumar. 2017. D-storm: Dynamic resource-efficient scheduling of stream processing applications. In Proc. of ICPADS’17. 485–492.Google Scholar
[111] Liu Xunyun and Buyya Rajkumar. 2020. Resource management and scheduling in distributed stream processing systems: A taxonomy, review, and future directions. ACM Comput. Surv. 53, 3 (2020), Article 50, 41 pages.Google Scholar
[112] Liu Xunyun, Dastjerdi Amir Vahid, Calheiros Rodrigo N., Qu Chenhao, and Buyya Rajkumar. 2018. A stepwise auto-profiling method for performance optimization of streaming applications. ACM Trans. Auton. Adapt. Syst. 12, 4 (2018), Article 24, 33 pages.Google ScholarDigital Library
[113] Lohrmann Björn, Janacik Peter, and Kao Odej. 2015. Elastic stream processing with latency guarantees. In Proc. of IEEE ICDCS’15. 399–410.Google Scholar
[114] Lohrmann Björn, Warneke Daniel, and Kao Odej. 2014. Nephele streaming: Stream processing under QoS constraints at scale. Clust. Comput. 17, 1 (2014), 61–78.Google ScholarDigital Library
[115] Lombardi Federico, Aniello Leonardo, Bonomi Silvia, and Querzoni Leonardo. 2018. Elastic symbiotic scaling of operators and resources in stream processing systems. IEEE Trans. Parallel Distrib. Syst. 29, 3 (2018), 572–585.Google ScholarCross Ref
[116] Luthra Manisha, Koldehofe Boris, Weisenburger Pascal, Salvaneschi Guido, and Arif Raheel. 2018. TCEP: Adapting to dynamic user environments by enabling transitions between operator placement mechanisms. In Proc. of ACM DEBS’18. 136–147.Google Scholar
[117] Madsen Kasper, Zhou Yongluan, and Cao Jianneng. 2017. Integrative dynamic reconfiguration in a parallel stream processing engine. In Proc. of IEEE ICDE’17. 227–230.Google Scholar
[118] Madsen Kasper, Zhou Yongluan, and Su Li. 2016. Enorm: Efficient window-based computation in large-scale distributed stream processing systems. In Proc. of ACM DEBS’16. 37–48.Google Scholar
[119] Mai Luo, Zeng Kai, Potharaju Rahul, Xu Le, Suh Steve, Venkataraman Shivaram, Paolo Costa, et al. 2018. Chi: A scalable and programmable control plane for distributed stream processing systems. Proc. VLDB Endow. 11, 10 (2018), 1303–1316.Google ScholarDigital Library
[120] Marangozova-Martin Vania, Palma Noël De, and El-Rheddane Ahmed. 2019. Multi-level elasticity for data stream processing. IEEE Trans. Parallel Distrib. Syst. 30, 10 (2019), 2326–2337.Google ScholarCross Ref
[121] Mei Yuan, Cheng Luwei, Talwar Vanish, Levin Michael Y., Jacques-Silva Gabriela, Nikhil Simha, Anirban Banerjee, et al. 2020. Turbine: Facebook’s service management platform for stream processing. In Proc. of IEEE ICDE’20. 1591–1602.Google Scholar
[122] Mencagli Gabriele. 2016. A game-theoretic approach for elastic distributed data stream processing. ACM Trans. Auton. Adapt. Syst. 11, 2 (2016), Article 13, 34 pages.Google ScholarDigital Library
[123] Mencagli Gabriele, Torquati Massimo, and Danelutto Marco. 2018. Elastic-PPQ: A two-level autonomic system for spatial preference query processing over dynamic data streams. Future Gener. Comput. Syst. 79 (2018), 862–877.Google ScholarCross Ref
[124] Mencagli Gabriele, Torquati Massimo, Danelutto Marco, and Matteis Tiziano De. 2017. Parallel continuous preference queries over out-of-order and bursty data streams. IEEE Trans. Parallel Distrib. Syst. 28, 9 (2017), 2608–2624.Google ScholarDigital Library
[125] Monte Bonaventura Del, Zeuch Steffen, Rabl Tilmann, and Markl Volker. 2020. Rhino: Efficient management of very large distributed state for stream processing engines. In Proc. of ACM SIGMOD’20. ACM, New York, NY, 2471–2486.Google Scholar
[126] Mu Weimin, Jin Zongze, Wang Junwei, Zhu Weilin, and Wang Weiping. 2019. BGElasor: Elastic-scaling framework for distributed streaming processing with deep neural network. In Network and Parallel Computing. Lecture Notes in Computer Science, Vol. 11783. Springer, 120–131.Google Scholar
[127] Najdataei Hannaneh, Nikolakopoulos Yiannis, Papatriantafilou Marina, Tsigas Philippas, and Gulisano Vincenzo. 2019. STRETCH: Scalable and elastic deterministic streaming analysis with virtual shared-nothing parallelism. In Proc. of ACM DEBS’19. 7–18.Google Scholar
[128] Nardelli Matteo, Cardellini Valeria, Grassi Vincenzo, and Presti Francesco Lo. 2019. Efficient operator placement for distributed data stream processing applications. IEEE Trans. Parallel Distrib. Syst. 30, 8 (2019), 1753–1767.Google ScholarCross Ref
[129] Nastic Stefan, Rausch Thomas, Scekic Ognjen, Dustdar Schahram, Gusev Marjan, Bojana Koteska, Magdalena Kostoska, Boro Jakimovski, Sasko Ristov, and Radu Prodan. 2017. A serverless real-time data analytics platform for edge computing. IEEE Internet Comput. 21, 4 (2017), 64–71.Google ScholarDigital Library
[130] Ni Xiang, Schneider Scott, Pavuluri Raju, Kaus Jonathan, and Wu Kun-Lung. 2019. Automating multi-level performance elastic components for IBM streams. In Proc. of ACM/IFIP Middleware’19. ACM, New York, NY, 163–175.Google Scholar
[131] O’Keeffe Dan, Salonidis Theodoros, and Pietzuch Peter R.. 2018. Frontier: Resilient edge processing for the Internet of Things. Proc. VLDB Endow. 11, 10 (2018), 1178–1191.Google ScholarDigital Library
[132] Ottenwälder Beate, Koldehofe Boris, Rothermel Kurt, Hong Kirak, Lillethun David J., and Ramachandran Umakishore. 2014. MCEP: A mobility-aware complex event processing system. ACM Trans. Internet Technol. 14, 1 (2014), Article 6, 24 pages.Google ScholarDigital Library
[133] Palyvos-Giannas Dimitris, Mencagli Gabriele, Papatriantafilou Marina, and Gulisano Vincenzo. 2021. Lachesis: A middleware for customizing OS scheduling of stream processing queries. In Proc. of ACM Middleware’21. 365–378.Google Scholar
[134] Papaemmanouil Olga, Çetintemel Ugur, and Jannotti John. 2009. Supporting generic cost models for wide-area stream processing. In Proc. of IEEE ICDE’09. 1084–1095.Google Scholar
[135] Park Heejin, Zhai Shuang, Lu Long, and Lin Felix X.. 2019. Streambox-TZ: Secure stream analytics at the edge with trustzone. In Proc. of USENIX ATC’19. 537–554.Google Scholar
[136] Pham Thao N., Chrysanthis Panos K., and Labrinidis Alexandros. 2016. Avoiding class warfare: Managing continuous queries with differentiated classes of service. VLDB J. 25, 2 (2016), 197–221.Google ScholarDigital Library
[137] Pham Thao N., Katsipoulakis Nikos R., Chrysanthis Panos K., and Labrinidis Alexandros. 2017. Uninterruptible migration of continuous queries without operator state migration. ACM SIGMOD Rec. 46, 3 (2017), 17–22.Google ScholarDigital Library
[138] Pietzuch Peter R., Ledlie Jonathan, Shneidman Jeffrey, Roussopoulos Mema, Welsh Matt, and Seltzer Margo I.. 2006. Network-aware operator placement for stream-processing systems. In Proc. of IEEE ICDE’06. 49–60.Google Scholar
[139] Qin Cui, Eichelberger Holger, and Schmid Klaus. 2019. Enactment of adaptation in data stream processing with latency implications—A systematic literature review. Inf. Softw. Technol. 111 (2019), 1–21.Google ScholarDigital Library
[140] Rahimzadeh Parisa, Lee Jinsung, Im Youngbin, Mau Siun-Chuon, Lee Eric C., Bradford O. Smith, Fatemah Al-Duoli, Carlee Joe-Wong, and Sangtae Ha. 2020. SPARCLE: Stream processing applications over dispersed computing networks. In Proc. of IEEE ICDCS’20. 1067–1078.Google Scholar
[141] Ravindra Sajith, Dayarathna Miyuru, and Jayasena Sanath. 2017. Latency aware elastic switching-based stream processing over compressed data streams. In Proc. of ACM/SPEC ICPE’17. 91–102.Google Scholar
[142] Repantis Thomas, Gu Xiaohui, and Kalogeraki Vana. 2009. QoS-aware shared component composition for distributed stream processing systems. IEEE Trans. Parallel Distrib. Syst. 20, 7 (2009), 968–982.Google ScholarDigital Library
[143] Rivetti Nicolo, Anceaume Emmanuelle, Busnel Yann, Querzoni Leonardo, and Sericola Bruno. 2016. Online scheduling for shuffle grouping in distributed stream processing systems. In Proc. of ACM/IFIP/USENIX Middleware’16.Google Scholar
[144] Rizou Stamatia, Dürr Frank, and Rothermel Kurt. 2010. Solving the multi-operator placement problem in large-scale operator networks. In Proc. of IEEE ICCCN’10. 1–6.Google Scholar
[145] Röger Henriette and Mayer Ruben. 2019. A comprehensive survey on parallelization and elasticity in stream processing. ACM Comput. Surv. 52, 2 (2019), Article 36, 37 pages.Google Scholar
[146] Runsewe Olubisi and Samaan Nancy. 2017. Cloud resource scaling for big data streaming applications using a layered multi-dimensional hidden Markov model. In Proc. of IEEE/ACM CCGRID’17. 848–857.Google Scholar
[147] Russo Gabriele Russo, Cardellini Valeria, Casale Giuliano, and Presti Francesco Lo. 2021. MEAD: Model-based vertical auto-scaling for data stream processing. In Proc. of IEEE/ACM CCGRID’21. 314–323.Google Scholar
[148] Russo Gabriele Russo, Cardellini Valeria, and Presti Francesco Lo. 2019. Reinforcement learning based policies for elastic stream processing on heterogeneous resources. In Proc. of ACM DEBS’19. 31–42.Google Scholar
[149] Russo Gabriele Russo, Cardellini Valeria, Presti Francesco Lo, and Nardelli Matteo. 2021. Towards a security-aware deployment of data streaming applications in fog computing. In Fog/Edge Computing For Security, Privacy, and Applications. Springer, 355–385.Google Scholar
[150] Sajjad Hooman P., Danniswara Ken, Al-Shishtawy Ahmad, and Vlassov Vladimir. 2016. SpanEdge: Towards unifying stream processing over central and near-the-edge data centers. In Proc. of IEEE/ACM SEC’16. 168–178.Google Scholar
[151] Salaht Farah Aït, Desprez Frédéric, and Lebre Adrien. 2020. An overview of service placement problem in fog and edge computing. ACM Comput. Surv. 53, 3 (2020), Article 65, 35 pages.Google Scholar
[152] Satzger Benjamin, Hummer Waldemar, Leitner Philipp, and Dustdar Schahram. 2011. ESC: Towards an elastic stream computing platform for the cloud. In Proc. of IEEE CLOUD’11. 348–355.Google Scholar
[153] Saurez Enrique, Hong Kirak, Lillethun Dave, Ramachandran Umakishore, and Ottenwälder Beate. 2016. Incremental deployment and migration of geo-distributed situation awareness applications in the fog. In Proc. of ACM DEBS’16. 258–269.Google Scholar
[154] Schneider Scott, Andrade Henrique, Gedik Bugra, Biem Alain, and Wu Kun-Lung. 2009. Elastic scaling of data parallel operators in stream processing. In Proc. of IEEE IPDPS’09. 1–12.Google Scholar
[155] Schneider Scott, Wolf Joel L., Hildrum Kirsten, Khandekar Rohit, and Wu Kun-Lung. 2016. Dynamic load balancing for ordered data-parallel regions in distributed streaming systems. In Proc. of ACM/IFIP/USENIX Middleware’16. ACM, New York, NY, Article 21, 14 pages.Google Scholar
[156] Schneider Scott and Wu Kun-Lung. 2017. Low-synchronization, mostly lock-free, elastic scheduling for streaming runtimes. In Proc. of ACM SIGPLAN PLDI’17. 648–661.Google Scholar
[157] Shah M. A., Hellerstein J. M., Chandrasekaran Sirish, and Franklin M. J.. 2003. Flux: An adaptive partitioning operator for continuous query systems. In Proc. of ICDE’03. IEEE, Los Alamitos, CA, 25–36.Google Scholar
[158] Sharaf Mohamed A., Chrysanthis Panos K., Labrinidis Alexandros, and Pruhs Kirk. 2008. Algorithms and metrics for processing multiple heterogeneous continuous queries. ACM Trans. Database Syst. 33, 1 (2008), Article 5, 44 pages.Google ScholarDigital Library
[159] Shukla Anshu and Simmhan Yogesh. 2018. Toward reliable and rapid elasticity for streaming dataflows on clouds. In Proc. of IEEE ICDCS’18. 1096–1106.Google Scholar
[160] Veith Alexandre da Silva, Souza Felipe R. de, Assunção Marcos D. de, Lefèvre Laurent, and Anjos Julio C. Santos dos. 2019. Multi-objective reinforcement learning for reconfiguring data stream analytics on edge computing. In Proc. of ICPP’19. ACM, New York, NY, Article 106, 10 pages.Google Scholar
[161] Singh Rayman Preet, Kumarasubramanian Bharath, Maheshwari Prateek, and Shetty Samarth. 2020. Auto-sizing for stream processing applications at LinkedIn. In Proc. of USENIX HotCloud’20.Google Scholar
[162] Slo Ahmad, Bhowmik Sukanya, and Rothermel Kurt. 2019. eSPICE: Probabilistic load shedding from input event streams in complex event processing. In Proc. of ACM/IFIP Middleware’19. ACM, New York, NY, 215–227.Google Scholar
[163] Slo Ahmad, Bhowmik Sukanya, and Rothermel Kurt. 2020. State-aware load shedding from input event streams in complex event processing. IEEE Trans. Big Data.To appear.Google Scholar
[164] Stonebraker Michael, Çetintemel Uǧur, and Zdonik Stan. 2005. The 8 requirements of real-time stream processing. ACM SIGMOD Rec. 34, 4 (2005), 42–47.Google ScholarDigital Library
[165] Sun Dawei, Gao Shang, Liu Xunyun, You Xindong, and Buyya Rajkumar. 2020. Dynamic redirection of real-time data streams for elastic stream computing. Future Gener. Comput. Syst. 112 (2020), 193–208.Google ScholarCross Ref
[166] Sun Dawei, Zhang Guangyan, Yang Songlin, Zheng Weimin, Khan Samee Ullah, and Li Keqin. 2015. Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments. Inf. Sci. 319 (2015), 92–112.Google ScholarDigital Library
[167] Tantalaki Nicoleta, Souravlas Stavros, and Roumeliotis Manos. 2020. A review on big data real-time stream processing and its scheduling techniques. Int. J. Parallel Emergent Distributed Syst. 35, 5 (2020), 571–601.Google ScholarCross Ref
[168] Tatbul Nesime, Çetintemel Uğur, Zdonik Stan, Cherniack Mitch, and Stonebraker Michael. 2003. Load shedding in a data stream manager. In Proc. of VLDB’03. 309–320.Google Scholar
[169] Tatbul Nesime, Çetintemel Uǧur, and Zdonik Stanley B.. 2007. Staying FIT: Efficient load shedding techniques for distributed stream processing. In Proc. of VLDB’07. ACM, New York, NY, 159–170.Google Scholar
[170] Tiwari Abhishek, Ramprasad Brian, Mortazavi Seyed H., Gabel Moshe, and Lara Eyal de. 2019. Reconfigurable streaming for the mobile edge. In Proc. of HotMobile’19. ACM, New York, NY, 153–158.Google Scholar
[171] To Quoc-Cuong, Soto Juan, and Markl Volker. 2018. A survey of state management in big data processing systems. VLDB J. 27, 6 (2018), 847–872.Google ScholarDigital Library
[172] Tolosana-Calasanz Rafael, Montes Javier Diaz, Rana Omer F., and Parashar Manish. 2017. Feedback-control and queueing theory-based resource management for streaming applications. IEEE Trans. Parallel Distrib. Syst. 28, 4 (2017), 1061–1075.Google ScholarDigital Library
[173] Tran Geoffrey Phi C., Walters John Paul, and Crago Stephen P.. 2018. Reducing tail latencies while improving resiliency to timing errors for stream processing workloads. In Proc. of IEEE/ACM UCC’18. 194–203.Google Scholar
[174] Tucker Peter A., Maier David, Sheard Tim, and Fegaras Leonidas. 2003. Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng. 15, 3 (2003), 555–568.Google ScholarDigital Library
[175] Tudoran Radu, Nano Olivier, Santos Ivo, Costan Alexandru, Soncu Hakan, Luc Bouge, and Gabriel Antoniu. 2014. JetStream: Enabling high performance event streaming across cloud data-centers. In Proc. of ACM DEBS’14. 23–34.Google Scholar
[176] Veen Jan Sipke van der, Waaij Bram van der, Lazovik Elena, Wijbrandi Wilco, and Meijer Robert J.. 2015. Dynamically scaling Apache Storm for the analysis of streaming data. In Proc. of IEEE BigDataService’15. 154–161.Google Scholar
[177] Venkataraman Shivaram, Panda Aurojit, Ousterhout Kay, Armbrust Michael, Ghodsi Ali, Michael J. Franklin, Benjamin Recht, and Ion Stoica. 2017. Drizzle: Fast and adaptable stream processing at scale. In Proc. of ACM SOSP’17. 374–389.Google Scholar
[178] Wang Ke, Floratou Avrilia, Agrawal Ashvin, and Musgrave Daniel. 2020. Spur: Mitigating slow instances in large-scale streaming pipelines. In Proc. of ACM SIGMOD’20. 2271–2285.Google Scholar
[179] Wang Li, Fu Tom Z. J., Ma Richard T. B., Winslett Marianne, and Zhang Zhenjie. 2019a. Elasticutor: Rapid elasticity for realtime stateful stream processing. In Proc. of ACM SIGMOD’19. 573–588.Google Scholar
[180] Wang Yidan, Tari Zahir, Farahabady Mohammad R. Hoseiny, and Zomaya Albert Y.. 2017. Model-based scheduling for stream processing systems. In Proc. of IEEE HPCC/SmartCity/DSS’17. 215–222.Google Scholar
[181] Wang Yidan, Tari Zahir, Huang Xiaoran, and Zomaya Albert Y.. 2019b. A network-aware and partition-based resource management scheme for data stream processing. In Proc. of ICPP’19. ACM, New York, NY, Article 20, 10 pages.Google Scholar
[182] Wei Xiaohui, Li Lina, Li Xiang, Wang Xingwang, Gao Shang, and Li Hongliang. 2019. Pec: Proactive elastic collaborative resource scheduling in data stream processing. IEEE Trans. Parallel Distrib. Syst. 30, 7 (2019), 1628–1642.Google ScholarDigital Library
[183] Wu Song, Hu Die, Ibrahim Shadi, Jin Hai, Xiao Jiang, Fei Chen, and Haikun Liu. 2019. When FPGA-accelerator meets stream data processing in the edge. In Proc. of IEEE ICDCS’19. 1818–1829.Google Scholar
[184] Wu Song, Liu Mi, Ibrahim Shadi, Jin Hai, Gu Lin, Chen Fei, and Liu Zhiyi. 2018. TurboStream: Towards low-latency data stream processing. In Proc. of IEEE ICDCS’18. 983–993.Google Scholar
[185] Xing Ying, Zdonik Stanley B., and Hwang Jeong-Hyon. 2005. Dynamic load distribution in the Borealis stream processor. In Proc. of IEEE ICDE’05. 791–802.Google Scholar
[186] Xu Jielong, Chen Zhenhua, Tang Jian, and Su Sen. 2014. T-storm: Traffic-aware online scheduling in storm. In Proc. of IEEE ICDCS’14. 535–544.Google Scholar
[187] Xu Jinlai, Palanisamy Balaji, Wang Qingyang, Ludwig Heiko, and Gopisetty Sandeep. 2022. Amnis: Optimized stream processing for edge computing. J. Parallel Distrib. Comput. 160 (2022), 49–64.Google ScholarDigital Library
[188] Xu Le, Peng Boyang, and Gupta Indranil. 2016. Stela: Enabling stream processing systems to scale-in and scale-out on-demand. In Proc. of IEEE IC2E’16. 22–31.Google Scholar
[189] Xu Le, Venkataraman Shivaram, Gupta Indranil, Mai Luo, and Potharaju Rahul. 2021. Move fast and meet deadlines: Fine-grained real-time stream processing with Cameo. In Proc. of USENIX NSDI’21. 389–405.Google Scholar
[190] Zacheilas Nikos, Kalogeraki Vana, Zygouras Nikolaos, Panagiotou Nikolaos, and Gunopulos Dimitrios. 2015. Elastic complex event processing exploiting prediction. In Proc. of IEEE Big Data’15. 213–222.Google Scholar
[191] Zacheilas Nikos, Zygouras Nikolas, Panagiotou Nikolaos, Kalogeraki Vana, and Gunopulos Dimitrios. 2016. Dynamic load balancing techniques for distributed complex event processing systems. In Distributed Applications and Interoperable Systems. Lecture Notes in Computer Science, Vol. 9687. Springer, 174–188.Google Scholar
[192] Zaharia Matei, Das Tathagata, Li Haoyuan, Hunter Timothy, Shenker Scott, and Stoica Ion. 2013. Discretized streams: Fault-tolerant streaming computation at scale. In Proc. of ACM SOSP’13. 423–438.Google Scholar
[193] Zamani Ali Reza, Balouek-Thomert Daniel, Villalobos Juan J., Rodero Ivan, and Parashar Manish. 2020. An edge-aware autonomic runtime for data streaming and in-transit processing. Future Gener. Comput. Syst. 110 (2020), 107–118.Google ScholarCross Ref
[194] Zeuch Steffen, Chaudhary Ankit, Monte Bonaventura Del, Gavriilidis Haralampos, Giouroukis Dimitrios, Philipp Grulich, Sebastian Bress, Jonas Traub, and Voker Markl. 2020. The NebulaStream platform for data and application management in the Internet of Things. In Proc. of CIDR’20.Google Scholar
[195] Zhang Ben, Jin Xin, Ratnasamy Sylvia, Wawrzynek John, and Lee Edward A.. 2018. AWStream: Adaptive wide-area streaming analytics. In Proc. of ACM SIGCOMM’18. 236–252.Google Scholar
[196] Zhang Quan, Song Yang, Routray Ramani, and Shi Weisong. 2016. Adaptive block and batch sizing for batched stream processing system. In Proc. of IEEE ICAC’16. 35–44.Google Scholar
[197] Zhang Shuhao, Zhang Feng, Wu Yingjun, He Bingsheng, and Johns Paul. 2019. Hardware-conscious stream processing: A survey. ACM SIGMOD Rec. 48, 4 (2019), 18–29.Google ScholarDigital Library
[198] Zhou Yongluan, Ooi Beng Chin, Tan Kian-Lee, and Wu Ji. 2006. Efficient dynamic operator placement in a locally distributed continuous query system. In On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE. Lecture Notes in Computer Science, Vol. 4275. Springer, 54–71.Google Scholar
[199] Zhou Yongluan, Wu Ji, and Leghari Ahmed Khan. 2013. Multi-query scheduling for time-critical data stream applications. In Proc. of SSDBM’13. ACM, New York, NY, Article 15, 12 pages.Google Scholar

Index Terms

Runtime Adaptation of Data Stream Processing Systems: The State of the Art

Recommendations

Generic windowing support for extensible stream processing systems

Stream processing applications process high volume, continuous feeds from live data sources, employ data-in-motion analytics to analyze these feeds, and produce near real-time insights with low latency. One of the fundamental characteristics of such ...
Read More
Towards Elastic and Sustainable Data Stream Processing on Edge Infrastructure
ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

Much of the data produced today is processed as it is generated by data stream processing systems. Although the cloud is often the target infrastructure for deploying data stream processing applications, resources located at the edges of the Internet ...
Read More
A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium
DEBS '17: Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems

Against the backdrop of ever-growing data volumes and trends like the Internet of Things (IoT) or Industry 4.0, Data Stream Processing Systems (DSPSs) or data stream processing architectures in general receive a greater interest. Continuously analyzing ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Computing Surveys Volume 54, Issue 11s
January 2022
785 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3551650
Editor:
Albert Zomaya
University of Sydney, Australia
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 9 September 2022
- Online AM: 10 February 2022
- Accepted: 1 January 2022
- Revised: 1 December 2021
- Received: 1 April 2021
Published in csur Volume 54, Issue 11s

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Data stream processing
adaptation
resource management
Qualifiers
- survey
- Refereed
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 18
  Total Citations
  View Citations
- 2,520
  Total Downloads
- Downloads (Last 12 months)906
- Downloads (Last 6 weeks)92
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

View Full Text

HTML Format

View this article in HTML Format .

View HTML Format

Runtime Adaptation of Data Stream Processing Systems: The State of the Art

ACM Computing Surveys

Abstract

Supplemental Material

Available for Download

REFERENCES

Cited By

Index Terms

Recommendations

Generic windowing support for extensible stream processing systems

Towards Elastic and Sustainable Data Stream Processing on Edge Infrastructure

A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium