Scalable Online Analytics on Cloud Infrastructures

The need for low latency analysis of high velocity real time continuous data streams has led to the emergence of Stream Processing Systems (SPSs). Contemporary SPSs allow a stream processing application to be hosted on Cloud infrastructures and dynamically scaled so as to adapt to the fluctuating data rates. However, the run time scalability incorporated in these SPSs are in their early adaptations and are based on simple local/global threshold based controls. This work studies the issues with the local and global auto scaling techniques that may lead to performance inefficiencies in real time traffic analysis on Cloud platforms and presents an efficient hybrid auto scaling strategy StreamScale which addresses the identified issues. The proposed StreamScale auto-scaling algorithm accounts for the gaps in the local/global scaling approaches and effectively identifies (de)parallelization opportunities in stream processing applications for maintaining QoS at reduced costs. Simulation based experimental evaluation on representative stream application topologies indicate that the proposed StreamScale auto-scaling algorithm exhibits better performance in comparison to both local and global auto-scaling approaches.