Sungpack Hong
Hassan Chafi
ABSTRACT
Large-scale graph analysis has recently been drawing lots of attention from both industry and academia. Although there are already several frameworks designed for scalable graph analysis, e.g. Giraph [1], all these frameworks adopt non-traditional programming models and APIs. This can significantly lower the productivity of the framework user. This paper discusses the feasibility of using an intuitive Domain-Specific Language (DSL) for graph analysis. Specifically, we use a compiler to translate Green-Marl [5] programs into an equivalent Giraph application, automatically bridging between very different programming models. We observe that the DSL programs are concise and intuitive, and that the compiler generated Giraph implementations exhibit performance on par with that of hand-written ones. However, the DSL compilation cannot but fail if the algorithm is fundamentally not compatible with the target framework. Overall, we believe that the DSL-based approach will provide great productivity benefits once it matures.
- Apache Giraph Project. http://giraph.apache.org.Google Scholar
- Green-Marl DSL. http://github.com/stanford-ppl/Green-Marl.Google Scholar
- Stanford network analysis library. http://snap.stanford.edu/snap.Google Scholar
- Apache Hadoop. http://hadoop.apache.org/.Google Scholar
- S. Hong, H. Chafi, E. Sedlar, and K. Olukotun. Green-Marl: A DSL for Easy and Ffficient Graph Analysis. In ASPLOS, 2012. Google ScholarDigital Library
- S. Hong, S. Salihoglu, J. Widom, and K. Olukotun. Tech Report: Compiling Green-Marl into GPS. http://ppl.stanford.edu/papers/tr_gm_gps.pdf.Google Scholar
- H. Kwak, C. Lee, H. Park, and S. Moon. What is Twitter, a social network or a news media? WWW '10, 2010. Google ScholarDigital Library
- J. Leskovec and C. Faloutsos. Sampling from large graphs. In KDD, 2006. Google ScholarDigital Library
- Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. Hellerstein. Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud. Proceedings of the VLDB Endowment, 5(8):716--727, 2012. Google ScholarDigital Library
- G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A System for Large-scale Graph Processing. In SIGMOD '10. ACM. Google ScholarDigital Library
- C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A Not-so-foreign Language for Data Processing. In SIGMOD. ACM, 2008. Google ScholarDigital Library
- L. Page. Method for node ranking in a linked database, Sept. 4 2001. US Patent 6,285,999.Google Scholar
- S. Salihoglu and J. Widom. GPS: Graph Processing System. http://infolab.stanford.edu/gps.Google Scholar
- S. Suri and S. Vassilvitskii. Counting triangles and the curse of the last reducer. In WWW. ACM, 2011. Google ScholarDigital Library
- A. Thusoo, J. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive: A Warehousing Solution Over a Map-Reduce Framework. Proceedings of the VLDB Endowment, 2(2):1626--1629, 2009. Google ScholarDigital Library
- Trinity. http://research.microsoft.com/en-us/projects/trinity/default.aspx.Google Scholar
- L. Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8):103--111, 1990. Google ScholarDigital Library
Recommendations
Declaratively defining domain-specific language debuggers
GCPE '11Tool support is vital to the effectiveness of domain-specific languages. With language workbenches, domain-specific languages and their tool support can be generated from a combined, high-level specification. This paper shows how such a specification ...
Creating and using domain-specific language features
GlobalDSL '13: Proceedings of the First Workshop on the Globalization of Domain Specific LanguagesThe value that domain-specific languages provide to their users is the domain-specific language features they contain. These features provide notations from the domain of interest, as well as domain-specific analysis and optimizations. But domain-...
Declaratively defining domain-specific language debuggers
GPCE '11: Proceedings of the 10th ACM international conference on Generative programming and component engineeringTool support is vital to the effectiveness of domain-specific languages. With language workbenches, domain-specific languages and their tool support can be generated from a combined, high-level specification. This paper shows how such a specification ...
Comments