Justin Talbot
Christos Kozyrakis
ABSTRACT
This paper describes our rewrite of Phoenix, a MapReduce framework for shared-memory CMPs and SMPs. Despite successfully demonstrating the applicability of a MapReduce-style pipeline to shared-memory machines, Phoenix has a number of limitations; its uniform intermediate storage of key-value pairs, inefficient combiner implementation, and poor task overhead amortization fail to efficiently support a wide range of MapReduce applications, encouraging users to manually circumvent the framework. We describe an alternative implementation, Phoenix++, that provides a modular, flexible pipeline that can be easily adapted by the user to the characteristics of a particular workload. Compared to Phoenix, this new approach achieves a 4.7-fold performance improvement and increased scalability, while allowing users to write simple, strict MapReduce code.
- R. Chen, H. Chen, and B. Zang. Tiled-MapReduce: Optimizing resource usages of data-parallel applications on multicore with tiling. In Proc. of the 19th Int'l Conference on Parallel Architectures and Compilation Techniques, pages 523--534, 2010. Google Scholar
Digital Library
- J. O. Coplien. Curiously recurring template patterns. C++ Rep., 7:24--27, February 1995. Google ScholarDigital Library
- J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proc. of the 6th Symposium on Operating Systems Design & Implementation, 2004. Google ScholarDigital Library
- B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang. Mars: a MapReduce framework on graphics processors. In Proc. of the 17th Int'l Conference on Parallel Architectures and Compilation Techniques, pages 260--269, 2008. Google ScholarDigital Library
- C. Henderson. Boost.MapReduce. http://www.craighenderson.co.uk/mapreduce.Google Scholar
- Intel Corporation. Threading Building Blocks. http://www.threadingbuildingblocks.org.Google Scholar
- W. Jiang, V. T. Ravi, and G. Agrawal. A Map-Reduce system with an alternate API for multi-core environments. In Proc. of the 10th IEEE/ACM Int'l Symposium on Cluster, Cloud, and Grid Computing, pages 84--93, 2010. Google ScholarDigital Library
- Y. Mao, R. Morris, and F. Kaashoek. Optimizing MapReduce for multicore architectures. Technical Report MIT-CSAIL-TR-2010-020, 2010.Google Scholar
- C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating MapReduce for multi-core and multiprocessor systems. In Proc. of the 13th Int'l Symposium on High Performance Computer Architecture, pages 13--24, 2007. Google ScholarDigital Library
- Stanford University. The Phoenix system for MapReduce programming. http://mapreduce.stanford.edu.Google Scholar
- The Apache Software Foundation. Hadoop. http://hadoop.apache.org.Google Scholar
- R. M. Yoo, A. Romano, and C. Kozyrakis. Phoenix rebirth: Scalable MapReduce on a large-scale shared-memory system. In Proc. of the 2009 IEEE Int'l Symposium on Workload Characterization, pages 198--207, 2009. Google ScholarDigital Library
Index Terms
- Phoenix++: modular MapReduce for shared-memory systems
Recommendations
HPDA: A hybrid parity-based disk array for enhanced performance and reliability
Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that ...
RAPID-Cache-A Reliable and Inexpensive Write Cache for High Performance Storage Systems
Modern high performance disk systems make extensive use of nonvolatile RAM (NVRAM) write caches. A single-copy NVRAM cache creates a single point of failure while a dual-copy NVRAM cache is very expensive because of the high cost of NVRAM. This paper ...
MyCassandra: a cloud storage supporting both read heavy and write heavy workloads
SYSTOR '12: Proceedings of the 5th Annual International Systems and Storage ConferenceA cloud storage with persistence shows solid performance only with a read heavy or write heavy workload. There is a trade-off between the read-optimized and write-optimized design of a cloud storage. This is dominated by its storage engine, which is a ...
Comments