Sisu Xi
Chenyang Lu
Insup Lee
ABSTRACT
Recent years have witnessed two major trends in the development of complex real-time embedded systems. First, to reduce cost and enhance flexibility, multiple systems are sharing common computing platforms via virtualization technology, instead of being deployed separately on physically isolated hosts. Second, multicore processors are increasingly being used in real-time systems. The integration of real-time systems as virtual machines (VMs) atop common multicore platforms raises significant new research challenges in meeting the real-time performance requirements of multiple systems. This paper advances the state of the art in real-time virtualization by designing and implementing RT-Xen 2.0, a new real-time multicore VM scheduling framework in the popular Xen virtual machine monitor (VMM). RT-Xen 2.0 realizes a suite of real-time VM scheduling policies spanning the design space. We implement both global and partitioned VM schedulers; each scheduler can be configured to support dynamic or static priorities and to run VMs as periodic or deferrable servers. We present a comprehensive experimental evaluation that provides important insights into real-time scheduling on virtualized multicore platforms: (1) both global and partitioned VM scheduling can be implemented in the VMM at moderate overhead; (2) at the VMM level, while compositional scheduling theory shows partitioned EDF (pEDF) is better than global EDF (gEDF) in providing schedulability guarantees, in our experiments their performance is reversed in terms of the fraction of workloads that meet their deadlines on virtualized multi-core platforms; (3) at the guest OS level, pEDF requests a smaller total VCPU bandwidth than gEDF based on compositional scheduling analysis, and therefore using pEDF at the guest OS level leads to more schedulable workloads in our experiments; (4) a combination of pEDF in the guest OS and gEDF in the VMM -- configured with deferrable server -- leads to the highest fraction of schedulable task sets compared to other real-time VM scheduling policies; and (5) on a platform with a shared last-level cache, the benefits of global scheduling outweigh the cache penalty incurred by VM migration.
- COQOS Operating System. http://www.opensynergy.com/en/Products/COQOS/.Google Scholar
- Credit Scheduler. http://wiki.xen.org/wiki/Credit_Scheduler.Google Scholar
- Fiasco micro-kernel. http://os.inf.tu-dresden.de/fiasco/.Google Scholar
- INTEGRITY Multivisor. http://www.ghs.com/products/rtos/integrity_virtualization.html.Google Scholar
- Intel 64 and IA-32 Architectures Optimization Reference Manual. http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf.Google Scholar
- Intel Ivy Bridge 7-Zip LZMA Benchmark Results. http://www.7-cpu.com/cpu/IvyBridge.html.Google Scholar
- Kernel Based Virtual Machine. http://www.linux-kvm.org.Google Scholar
- LITMUS-RT. http://www.litmus-rt.org/.Google Scholar
- The Xen Project's Hypervisor for the ARM architecture. http://www.xenproject.org/developers/teams/arm-hypervisor.html.Google Scholar
- T. Aswathanarayana, D. Niehaus, V. Subramonian, and C. Gill. Design and Performance of Configurable Endsystem Scheduling Mechanisms. In RTAS, 2005. Google ScholarDigital Library
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the Art of Virtualization. ACM SIGOPS Operating Systems Review, 2003. Google ScholarDigital Library
- S. Barush and B. Brandenburg. Multiprocessor feasibility analysis of recurrent task systems with specified processor affinities. In RTSS, 2013. Google ScholarDigital Library
- A. Bastoni, B. B. Brandenburg, and J. H. Anderson. An Empirical Comparison of Global, and Clustered Multiprocessor EDF Schedulers. In RTSS. IEEE, 2010. Google ScholarDigital Library
- M. Behnam, I. Shin, T. Nolte, and M. Nolin. SIRAP: A Synchronization Protocol for Hierarchical Resource Sharing in Real-Time Open Systems. In EMSOFT, 2007. Google ScholarDigital Library
- E. Bini, G. Buttazzo, and M. Bertogna. The Multi Supply Function Abstraction for Multiprocessors. In RTCSA. IEEE, 2009. Google ScholarDigital Library
- B. B. Brandenburg and J. H. Anderson. On the Implementation of Global Real-Time Schedulers. In RTSS. IEEE, 2009. Google ScholarDigital Library
- F. Bruns, S. Traboulsi, D. Szczesny, E. Gonzalez, Y. Xu, and A. Bilgic. An Evaluation of Microkernel-Based Virtualization for Embedded Real-Time Systems. In ECRTS, 2010. Google ScholarDigital Library
- G. C. Buttazzo. Rate monotonic vs. edf: judgment day. Real-Time Syst., 29(1):5--26, Jan. 2005. Google ScholarDigital Library
- J. M. Calandrino, H. Leontyev, A. Block, U. C. Devi, and J. H. Anderson. LITMUS-RT: A Testbed for Empirically Comparing Real-Time Multiprocessor Schedulers. In RTSS, 2006. Google ScholarDigital Library
- F. Checconi, T. Cucinotta, D. Faggioli, and G. Lipari. Hierarchical Multiprocessor CPU Reservations for the Linux Kernel. OSPERT, 2009.Google Scholar
- A. Crespo, I. Ripoll, and M. Masmano. Partitioned Embedded Architecture Based on Hypervisor: the XtratuM Approach. In EDCC, 2010. Google ScholarDigital Library
- T. Cucinotta, G. Anastasi, and L. Abeni. Respecting Temporal Constraints in Virtualised Services. In COMPSAC, 2009. Google ScholarDigital Library
- T. Cucinotta, F. Checconi, G. Kousiouris, D. Kyriazis, T. Varvarigou, A. Mazzetti, Z. Zlatev, J. Papay, M. Boniface, S. Berger, et al. Virtualised e-learning with real-time guarantees on the irmos platform. In Service-Oriented Computing and Applications (SOCA). IEEE, 2010.Google Scholar
- M. Danish, Y. Li, and R. West. Virtual-CPU Scheduling in the Quest Operating System. In RTAS, 2011. Google ScholarDigital Library
- Z. Deng and J. Liu. Scheduling Real-Time Applications in an Open Environment. In RTSS, 1997. Google ScholarDigital Library
- A. Easwaran, M. Anand, and I. Lee. Compositional Analysis Framework Using EDP Resource Models. In RTSS, 2007. Google ScholarDigital Library
- A. Easwaran, I. Shin, and I. Lee. Optimal virtual cluster-based multiprocessor scheduling. Real-Time Syst., 43(1):25--59, Sept. 2009. Google ScholarDigital Library
- A. Easwaran, I. Shin, and I. Lee. Optimal Virtual Cluster-Based Multiprocessor Scheduling. Real-Time Systems, 2009. Google ScholarDigital Library
- S. Govindan, A. R. Nath, A. Das, B. Urgaonkar, and A. Sivasubramaniam. Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms. In VEE, 2007. Google ScholarDigital Library
- G. Gracioli, A. A. Fröhlich, R. Pellizzoni, and S. Fischmeister. Implementation and evaluation of global and partitioned scheduling in a real-time os. Real-Time Systems, 2013. Google ScholarDigital Library
- S. Groesbrink, L. Almeida, M. de Sousa, and S. M. Petters. Towards certifiable adaptive reservations for hypervisor-based virtualization. In RTAS, 2014.Google ScholarCross Ref
- D. Gupta, R. Gardner, and L. Cherkasova. Xenmon: Qos Monitoring and Performance Profiling Tool. Hewlett-Packard Labs, Tech. Rep. HPL-2005-187, 2005.Google Scholar
- H. Härtig, M. Völp, and M. Hähnel. The case for practical multi-resource and multi-level scheduling based on energy/utility. In RTCSA, 2013.Google ScholarCross Ref
- A. Lackorzyński, A. Warg, M. Völp, and H. Härtig. Flattening hierarchical scheduling. In EMSOFT. ACM, 2012.Google ScholarDigital Library
- J. Lee, S. Xi, S. Chen, L. T. X. Phan, C. Gill, I. Lee, C. Lu, and O. Sokolsky. Realizing Compositional Scheduling Through Virtualization. In RTAS. IEEE, 2012. Google ScholarDigital Library
- M. Lee, A. Krishnakumar, P. Krishnan, N. Singh, and S. Yajnik. Supporting Soft Real-Time Tasks in the Xen Hypervisor. In ACM Sigplan Notices, 2010. Google ScholarDigital Library
- H. Leontyev and J. H. Anderson. A Hierarchical Multiprocessor Bandwidth Reservation Scheme with Timing Guarantees. Real-Time Systems, 2009. Google ScholarDigital Library
- B. Lin and P. A. Dinda. Vsched: Mixing batch and interactive virtual machines using periodic real-time scheduling. In Proceedings of the 2005 ACM/IEEE conference on Supercomputing, 2005. Google ScholarDigital Library
- G. Lipari and E. Bini. A Framework for Hierarchical Scheduling on Multiprocessors: From Application Requirements to Run-Time Allocation. In RTSS. IEEE, 2010. Google ScholarDigital Library
- W. Lunniss, S. Altmeyer, C. Maiza, and R. I. Davis. Integrating Cache Related Pre-emption Delay Analysis into EDF Scheduling. In RTAS, 2013. Google ScholarDigital Library
- J. Regehr and J. Stankovic. HLS: A Framework for Composing Soft Real-Time Schedulers. In RTSS, 2001. Google ScholarDigital Library
- S. Altmeyer, R. I. Davis, and C. Maiza. Improved Cache Related Pre-emption Delay Aware Response Time Analysis for Fixed Priority Pre-emptive Systems. Real-Time Systems, 2012. Google ScholarDigital Library
- I. Shin and I. Lee. Compositional Real-Time Scheduling Framework. In RTSS, 2004. Google ScholarDigital Library
- Y. Wang and K. Lin. The Implementation of Hierarchical Schedulers in the RED-Linux Scheduling Framework. In ECRTS, 2000. Google ScholarDigital Library
- S. Xi, J. Wilson, C. Lu, and C. Gill. RT-Xen: Towards Real-Time Hypervisor Scheduling in Xen. In EMSOFT. ACM, 2011. Google ScholarDigital Library
- M. Xu, L. T. X. Phan, I. Lee, and O. Sokolsky. Compositional analysis under global deadline monotonic. Technical report, University of Pennsylvania, 2013.Google Scholar
- M. Xu, L. T. X. Phan, I. Lee, O. Sokolsky, S. Xi, C. Lu, and C. D. Gill. Cache-Aware Compositional Analysis of Real-Time Multicore Virtualization Platforms. In RTSS, 2013. Google ScholarDigital Library
- J. Yang, H. Kim, S. Park, C. Hong, and I. Shin. Implementation of compositional scheduling framework on virtualization. SIGBED Rev, 2011. Google ScholarDigital Library
- S. Yoo, K.-H. Kwak, J.-H. Jo, and C. Yoo. Toward under-millisecond I/O latency in Xen-ARM. In Proceedings of the Second Asia-Pacific Workshop on Systems, page 14. ACM, 2011. Google ScholarDigital Library
- H. Yun, G. Yao, R. Pellizzoni, M. Caccamo, and L. Sha. MemGuard: Memory Bandwidth Reservation System for Efficient Performance Isolation in Multi-core Platforms. In RTAS, 2013.Google Scholar
- W. Zhang, S. Rajasekaran, T. Wood, and M. Zhu. MIMP: Deadline and Interference Aware Scheduling of Hadoop Virtual Machines. In IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2014.Google Scholar
Index Terms
- Real-time multi-core virtual machine scheduling in xen
Recommendations
Fast Live Cloning of Virtual Machine Based on Xen
HPCC '09: Proceedings of the 2009 11th IEEE International Conference on High Performance Computing and CommunicationsVirtual Machine (VM) cloning is to create a replica of a source virtual machine (parent virtual machine); the replica, also called child virtual machine, owns exactly the same executing status as parent virtual machine. Fast live cloning guarantees that,...
Diagnosing performance overheads in the xen virtual machine environment
VEE '05: Proceedings of the 1st ACM/USENIX international conference on Virtual execution environmentsVirtual Machine (VM) environments (e.g., VMware and Xen) are experiencing a resurgence of interest for diverse uses including server consolidation and shared hosting. An application's performance in a virtual machine environment can differ markedly from ...
Xen and Co.: Communication-Aware CPU Management in Consolidated Xen-Based Hosting Platforms
Recent advances in software and architectural support for server virtualization have created interest in using this technology in the design of consolidated hosting platforms. Since virtualization enables easier and faster application migration as well ...
Comments