Louai Alarabi
Mohamed F. Mokbel
Abstract
This demo presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop injects spatio-temporal awareness in the Hadoop base code, which results in achieving order(s) of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. The key idea behind ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System (HDFS). A real system prototype of ST-Hadoop, running on a local cluster of 24 machines, is demonstrated with two big-spatio-temporal datasets of Twitter and NYC Taxi data, each of around one billion records.
- A. Eldawy and M. F. Mokbel. Pigeon: A spatial mapreduce language. In ICDE, pages 1242--1245, 2014.Google Scholar
Cross Ref
- A. Eldawy and M. F. Mokbel. SpatialHadoop: A MapReduce Framework for Spatial Data. In ICDE, pages 1352--1363, 2015.Google ScholarCross Ref
- A. Eldawy, M. F. Mokbel, S. Alharthi, A. Alzaidy, K. Tarek, and S. Ghani. SHAHED: A MapReduce-based System for Querying and Visualizing Spatio-temporal Satellite Data. In ICDE, pages 1585--1596, 2015.Google ScholarCross Ref
- Z. Li, F. Hu, J. L. Schnase, D. Q. Duffy, T. Lee, M. K. Bowen, and C. Yang. A spatiotemporal indexing approach for efficient processing of big array-based climate data with mapreduce. International Journal of Geographical Information Science, pages 17--35, 2017.Google Scholar
- Q. Ma, B. Yang, W. Qian, and A. Zhou. Query Processing of Massive Trajectory Data Based on MapReduce. In CLOUDDB, pages 9--16, 2009. Google ScholarDigital Library
- Land Process Distributed Active Archive Center, Mar. 2015. https://lpdaac.usgs.gov/about.Google Scholar
- Data from NASA's Missions, Research, and Activities, 2017. http://www.nasa.gov/open/data.html.Google Scholar
- Data from NYC Taxi and Limosuine Commission, 2017. http://www.nyc.gov/html/tlc/.Google Scholar
- http://st-hadoop.cs.umn.edu/.Google Scholar
- H. Tan, W. Luo, and L. M. Ni. Clost: a hadoop-based storage system for big spatio-temporal data analytics. In CIKM, pages 2139--2143, 2012. Google ScholarDigital Library
- Twitter. The About webpage., 2017. https://about.twitter.com/company.Google Scholar
Index Terms
- A demonstration of ST-hadoop: a MapReduce framework for big spatio-temporal data
Recommendations
ST-Hadoop: a MapReduce framework for spatio-temporal data
This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside ...
Comments