Yannis Sfakianakis
ABSTRACT
Server consolidation via virtualization is an essential technique for improving infrastructure cost in modern datacenters. From the viewpoint of datacenter operators, consolidation offers compelling advantages by reducing the number of physical servers, and reducing operational costs such as energy consumption. However, performance interference between co-located workloads can be crippling. Conservatively, and at significant cost, datacenter operators are forced to keep physical servers at low utilization levels (typically below 20%), to minimize adverse performance interactions.
In this paper, we focus on addressing the issue of performance interference on a virtualized server operating at high utilization levels. In our work, we find that eliminating interference in the I/O path is critical for achieving good performance on consolidated servers. We present Vanguard, a device driver stack that implements a full I/O path in the Linux kernel that provisions competing workloads with dedicated resources. We focus on two key resources: in-memory buffers for the filesystem, and space on SSD devices that serve as a transparent cache for block devices. Our approach effectively mitigates performance interference, for several mixes of transactional, streaming, and analytical processing workloads. We find that with our approach a server can run more workloads close to their nominal performance level as compared to the unmodified Linux I/O path, by careful allocation of I/O path resources to each workload. At excessive load levels, i.e. when the aggregate load exceeds the capabilities of the server, our approach can still provide isolated slices of the I/O path for a subset of the co-located workloads yielding at least 50% of their nominal performance. In addition, Vanguard is shown to be 2.5x more efficient in terms of service resource usage for a 4-workload mix, taking into account utilization and power consumption. With an I/O-heavy mix 6-workload mix, Vanguard is 8x more efficient than the unmodified baseline Linux system.
- The Apache HTTP Server. http://httpd.apache.org.Google Scholar
- H. Akkan, M. Lang, and L. M. Liebrock. Stepping towards noiseless linux environment. In Proceedings of the 2nd International Workshop on Runtime and Operating Systems for Supercomputers, ROSS, pages 7:1--7:7, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1460-2. URL http://doi.acm.org/10.1145/2318916.2318925. Google ScholarDigital Library
- S. F. Altschul, W. Gish, W. Miller, E. W. Myers, and D. J. Lipman. Basic Local Alignment Search Tool. Journal of Molecular Biology, 215:403--410, 1990.Google ScholarCross Ref
- I. Ari, M. Gottwals, D. Henze, and H. P. Laboratories. Performance boosting and workload isolation in storage area networks with sancache. In IEEE/NASA Conference on Mass Storage Systems and Technologies (MSST), 2006.Google Scholar
- J. Axboe. Flexible I/O Tester. https://github.com/axboe.Google Scholar
- G. Banga, P. Druschel, and J. C. Mogul. Resource containers: a new facility for resource management in server systems. In Proceedings of the third symposium on Operating systems design and implementation, OSDI, pages 45--58, Berkeley, CA, USA, 1999. USENIX Association. ISBN 1-880446-39-1. Google ScholarDigital Library
- S. Bansal and D. S. Modha. Car: Clock with adaptive replacement. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, FAST, pages 187--200, Berkeley, CA, USA, 2004. USENIX Association. URL http://dl.acm.org/citation.cfm?id=1096673.1096699. Google ScholarDigital Library
- G. Chen, W. He, J. Liu, S. Nath, L. Rigas, L. Xiao, and F. Zhao. Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services. In NSDI, volume 8, pages 337--350, 2008. Google ScholarDigital Library
- C. Delimitrou and C. Kozyrakis. Paragon: QoS-Aware Scheduling for Heterogeneous Datacenters. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), March 2013. Google ScholarDigital Library
- X. Fan, W.-D. Weber, and L. A. Barroso. Power Provisioning for a Warehouse-sized Computer. In Proceedings of the 34th Annual International Symposium on Computer Architecture (ISCA), 2007. Google ScholarDigital Library
- Y. Kanemasa, Q. Wang, J. Li, M. Matsubara, and C. Pu. Revisiting Performance Interference among Consolidated n-Tier Applications: Sharing is Better than Isolation. In Proceedings of the IEEE International Conference on Services Computing, 2013. Google ScholarDigital Library
- Y. Klonatos, T. Makatos, M. Marazakis, M. Flouris, and A. Bilas. Azor: Using two-level block selection to improve SSD-based I/O caches. In Proc. IEEE International Conference on Networking, Architecture, and Storage (NAS), 2011. Google ScholarDigital Library
- Y. Koh, R. Knauuerhase, P. Brett, M. Bowman, Z. Wen, and C. Pu. An Analysis of Performance Interference Effects in Virtual Environments. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software, 2007.Google ScholarCross Ref
- J. Leverich and C. Kozyrakis. Reconciling High Server Utilization and Sub-millisecond Quality-of-service. In Proceedings of the 9th European Conference on Computer Systems (EuroSys), 2014. Google ScholarDigital Library
- S. Malkwowski, Y. Kanemasa, H. Chen, M. Yamamoto, Q. Wang, D. Jayasinghe, C. Pu, and M. Kawaba. Challenges and Opportunities in Consolidation at High Resource utilization: Non-monotonic Response Time Variations in n-Tier Applications. In Proceedings of the IEEE 5th International Conference on Cloud Computing, 2012. Google ScholarDigital Library
- S. Mavridis, Y. Sfakianakis, A. Papagiannis, M. Marazakis, and A. Bilas. Jericho: Achieving scalability through optimal data placement on multicore systems. In 30th Symposium on Mass Storage Systems and Technologies (MSST), 2014.Google ScholarCross Ref
- N. Megiddo and D. S. Modha. Arc: A self-tuning, low overhead replacement cache. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies, FAST, pages 115--130, Berkeley, CA, USA, 2003. USENIX Association. URL http://dl.acm.org/citation.cfm?id=1090694.1090708. Google ScholarDigital Library
- R. Nathuji, A. Kansal, and A. Ghaffarkhah. Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds. In EuroSys: Proceedings of the 5th ACM SIGOPS/EuroSys European Conference on Computer Systems, 2010. Google ScholarDigital Library
- D. Novakovic, N. Vasic, S. Novakovic, D. Kostic, and R. Bianchini. DeepDive: Transparently Identifying and Managing Performance Interference in Virtualized Environments. In Proceedings of the 2013 USENIX Annual Technical Conference, June 2013. Google ScholarDigital Library
- Paul Menage. cgroups features, including cpusets and memory controller. In Linux Kernel cgroups Documentation. The Linux Kernel Archives, 2006.Google Scholar
- D. Shue, M. J. Freedman, and A. Shaikh. Performance Isolation and Fairness for Multi-Tenant Cloud Storage. In Proceedings of the 10th Operating Systems Design and Implementation (OSDI), 2012. Google ScholarDigital Library
- B. Snyder. Server Virtualization has Stalled, Despite the Hype. http://www.infoworld.com/t/server-virtualization/what-you-missed-server-virtualization-has-stalled-despite-the-hype-901, 2010.Google Scholar
- D. G. Sullivan and M. I. Seltzer. Isolation with flexibility: a resource management framework for central servers. In Proceedings of the annual conference on USENIX Annual Technical Conference, ATEC, pages 337--350, Berkeley, CA, USA, 2000. USENIX Association. URL http://dl.acm.org/citation.cfm?id=1267724.1267751. Google ScholarDigital Library
- "Transaction Processing Performance Council". TPC benchmark specifications. http://www.tpc.org.Google Scholar
- M. Wachs, M. Abd-El-Malek, E. Thereska, and G. R. Ganger. Argon: performance insulation for shared storage servers. In Proceedings of the 5th USENIX conference on File and Storage Technologies, FAST, pages 5--5, Berkeley, CA, USA, 2007. USENIX Association. URL http://dl.acm.org/citation.cfm?id=1267903.1267908. Google ScholarDigital Library
- X. Zhang, E. Tune, R. Hagmann, R. Jnagal, V. Gokhale, and J. Wilkes. CPI2: CPU performance isolation for shared compute clusters. In EuroSys: Proceedings of the 8th ACM European Conference on Computer Systems, April 2013. Google ScholarDigital Library
- D. Zheng, R. Burns, and A. S. Szalay. Toward millions of file system iops on low-cost, commodity hardware. In Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 69--71. ACM, 2013. Google ScholarDigital Library
Index Terms
- Vanguard: Increasing Server Efficiency via Workload Isolation in the Storage I/O Path
Recommendations
Performance Evaluation of Hypervisors for Cloud Computing
The virtualization of IT infrastructure enables consolidation and pooling of IT resources so they are shared over diverse applications to offset the limitation of shrinking resources and growing business needs. Virtualization provides a logical ...
Virtio network paravirtualization driver
One of the techniques used to improve I/O performance of virtual machines is paravirtualization. Paravirtualized devices are intended to reduce the performance overhead on full virtualization where all hardware devices are emulated. The interface of a ...
Live gang migration of virtual machines
HPDC '11: Proceedings of the 20th international symposium on High performance distributed computingThis paper addresses the problem of simultaneously migrating a group of co-located and live virtual machines (VMs), i.e, VMs executing on the same physical machine. We refer to such a mass simultaneous migration of active VMs as "live gang migration". ...
Comments