Skip to main content
Top

2018 | OriginalPaper | Chapter

A Hybrid Index Model for Efficient Spatio-Temporal Search in HBase

Authors : Chengyuan Zhang, Lei Zhu, Jun Long, Shuangqiao Lin, Zhan Yang, Wenti Huang

Published in: Trends and Applications in Knowledge Discovery and Data Mining

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

With advances in geo-positioning technologies and geo-locat-ion services, there are a rapidly growing massive amount of spatio-tempor-al data collected in many applications such as location-aware devices and wireless communication, in which an object is described by its spatial location and its timestamp. Consequently, the study of spatio-temporal search which explores both geo-location information and temporal information of the data has attracted significant concern from research organizations and commercial communities. This work study the problem of spatio-temporal k-nearest neighbors search (STkNNS), which is fundamental in the spatial temporal queries. Based on HBase, a novel index structure is proposed, called Hybrid Spatio-Temporal HBase Index (HSTI for short), which is carefully designed and takes both spatial and temporal information into consideration to effectively reduce the search space. Based on HSTI, an efficient algorithm is developed to deal with spatio-temporal k-nearest neighbors search. Comprehensive experiments on real and synthetic data clearly show that HSTI is three to five times faster than the state-of-the-art technique.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang, Y., Lin, X., Wu, L., Zhang, W.: Effective multi-query expansions: collaborative deep networks for robust landmark retrieval. IEEE Trans. Image Process. 26(3), 1393–1404 (2017)MathSciNetCrossRef Wang, Y., Lin, X., Wu, L., Zhang, W.: Effective multi-query expansions: collaborative deep networks for robust landmark retrieval. IEEE Trans. Image Process. 26(3), 1393–1404 (2017)MathSciNetCrossRef
2.
go back to reference Wu, L., Huang, X., Zhang, C., Shepherd, J., Wang, Y.: An efficient framework of Bregman divergence optimization for co-ranking images and tags in a heterogeneous network. Multimed. Tools Appl. 74(15), 5635–5660 (2015)CrossRef Wu, L., Huang, X., Zhang, C., Shepherd, J., Wang, Y.: An efficient framework of Bregman divergence optimization for co-ranking images and tags in a heterogeneous network. Multimed. Tools Appl. 74(15), 5635–5660 (2015)CrossRef
3.
go back to reference Wang, Y., Zhang, W., Wu, L., Lin, X., Fang, M., Pan, S.: Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2153–2159 (2016) Wang, Y., Zhang, W., Wu, L., Lin, X., Fang, M., Pan, S.: Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2153–2159 (2016)
4.
go back to reference Wang, Y., Lin, X., Zhang, Q.: Towards metric fusion on multi-view data: a cross-view based graph random walk approach. In: 22nd ACM International Conference on Information and Knowledge Management, CIKM 2013, San Francisco, CA, USA, 27 October–1 November 2013, pp. 805–810 (2013) Wang, Y., Lin, X., Zhang, Q.: Towards metric fusion on multi-view data: a cross-view based graph random walk approach. In: 22nd ACM International Conference on Information and Knowledge Management, CIKM 2013, San Francisco, CA, USA, 27 October–1 November 2013, pp. 805–810 (2013)
5.
go back to reference Zhang, C., Zhang, Y., Zhang, W., Lin, X.: Inverted linear quadtree: efficient top K spatial keyword search. IEEE Trans. Knowl. Data Eng. 28(7), 1706–1721 (2016)CrossRef Zhang, C., Zhang, Y., Zhang, W., Lin, X.: Inverted linear quadtree: efficient top K spatial keyword search. IEEE Trans. Knowl. Data Eng. 28(7), 1706–1721 (2016)CrossRef
6.
go back to reference Wu, L., Wang, Y., Li, X., Gao, J.: What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recognit. 76, 727–738 (2018)CrossRef Wu, L., Wang, Y., Li, X., Gao, J.: What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recognit. 76, 727–738 (2018)CrossRef
7.
go back to reference Wu, L., Wang, Y., Ge, Z., Hu, Q., Li, X.: Structured deep hashing with convolutional neural networks for fast person re-identification. Comput. Vis. Image Underst. 167, 63–73 (2018)CrossRef Wu, L., Wang, Y., Ge, Z., Hu, Q., Li, X.: Structured deep hashing with convolutional neural networks for fast person re-identification. Comput. Vis. Image Underst. 167, 63–73 (2018)CrossRef
8.
go back to reference Liu, A., Liu, X., Long, J.: A trust-based adaptive probability marking and storage traceback scheme for wsns. Sensors 16(4), 451 (2016)MathSciNetCrossRef Liu, A., Liu, X., Long, J.: A trust-based adaptive probability marking and storage traceback scheme for wsns. Sensors 16(4), 451 (2016)MathSciNetCrossRef
9.
go back to reference Wu, L., Wang, Y., Gao, J., Li, X.: Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recognit. 73, 275–288 (2018)CrossRef Wu, L., Wang, Y., Gao, J., Li, X.: Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recognit. 73, 275–288 (2018)CrossRef
10.
go back to reference Wang, Y., Zhang, W., Wu, L., Lin, X., Zhao, X.: Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 57–70 (2017)CrossRef Wang, Y., Zhang, W., Wu, L., Lin, X., Zhao, X.: Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 57–70 (2017)CrossRef
11.
go back to reference Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q., Huang, X.: Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans. Image Process. 24(11), 3939–3949 (2015)MathSciNetCrossRef Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q., Huang, X.: Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans. Image Process. 24(11), 3939–3949 (2015)MathSciNetCrossRef
12.
go back to reference Wang, Y., Wu, L., Lin, X., Gao, J.: Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 29(10), 4833–4843 (2018)CrossRef Wang, Y., Wu, L., Lin, X., Gao, J.: Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 29(10), 4833–4843 (2018)CrossRef
13.
go back to reference Wang, Y., Wu, L.: Beyond low-rank representations: orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw. 103, 1–8 (2018)CrossRef Wang, Y., Wu, L.: Beyond low-rank representations: orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw. 103, 1–8 (2018)CrossRef
14.
go back to reference Wu, L., Wang, Y., Li, X., et al.: Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans. Cybern. PP(99), 1–12 (2018) Wu, L., Wang, Y., Li, X., et al.: Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans. Cybern. PP(99), 1–12 (2018)
15.
go back to reference Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: SIGMOD 1984, Proceedings of Annual Meeting, Boston, Massachusetts, 18–21 June 1984, pp. 47–57 (1984) Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: SIGMOD 1984, Proceedings of Annual Meeting, Boston, Massachusetts, 18–21 June 1984, pp. 47–57 (1984)
16.
go back to reference Beckmann, N., Kriegel, H., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data, Atlantic City, NJ, 23–25 May 1990, pp. 322–331 (1990) Beckmann, N., Kriegel, H., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data, Atlantic City, NJ, 23–25 May 1990, pp. 322–331 (1990)
17.
go back to reference Finkel, R.A., Bentley, J.L.: Quad trees: a data structure for retrieval on composite keys. Acta Inf. 4, 1–9 (1974)CrossRef Finkel, R.A., Bentley, J.L.: Quad trees: a data structure for retrieval on composite keys. Acta Inf. 4, 1–9 (1974)CrossRef
18.
go back to reference Brown, R.A.: Building a balanced k-d tree in O(kn log n) time. CoRR abs/1410.5420 (2014) Brown, R.A.: Building a balanced k-d tree in O(kn log n) time. CoRR abs/1410.5420 (2014)
19.
go back to reference Fox, A.D., Eichelberger, C.N., Hughes, J.N., Lyon, S.: Spatio-temporal indexing in non-relational distributed databases. In: Proceedings of the 2013 IEEE International Conference on Big Data, Santa Clara, CA, USA, 6–9 October 2013, pp. 291–299 (2013) Fox, A.D., Eichelberger, C.N., Hughes, J.N., Lyon, S.: Spatio-temporal indexing in non-relational distributed databases. In: Proceedings of the 2013 IEEE International Conference on Big Data, Santa Clara, CA, USA, 6–9 October 2013, pp. 291–299 (2013)
20.
go back to reference Eldawy, A., Mokbel, M.F.: A demonstration of spatialhadoop: an efficient mapreduce framework for spatial data. PVLDB 6(12), 1230–1233 (2013) Eldawy, A., Mokbel, M.F.: A demonstration of spatialhadoop: an efficient mapreduce framework for spatial data. PVLDB 6(12), 1230–1233 (2013)
21.
go back to reference Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q.: Exploiting correlation consensus: towards subspace clustering for multi-modal data. In: Proceedings of the ACM International Conference on Multimedia, MM 2014, Orlando, FL, USA, 03–07 November 2014, pp. 981–984 (2014) Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q.: Exploiting correlation consensus: towards subspace clustering for multi-modal data. In: Proceedings of the ACM International Conference on Multimedia, MM 2014, Orlando, FL, USA, 03–07 November 2014, pp. 981–984 (2014)
22.
go back to reference Aji, A., et al.: Hadoop-GIS: a high performance spatial data warehousing system over mapreduce. PVLDB 6(11), 1009–1020 (2013) Aji, A., et al.: Hadoop-GIS: a high performance spatial data warehousing system over mapreduce. PVLDB 6(11), 1009–1020 (2013)
24.
go back to reference Nishimura, S., Das, S., Agrawal, D., El Abbadi, A.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware services. In: 12th IEEE International Conference on Mobile Data Management, MDM 2011, Luleå, Sweden, 6–9 June 2011, vol. 1, pp. 7–16 (2011) Nishimura, S., Das, S., Agrawal, D., El Abbadi, A.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware services. In: 12th IEEE International Conference on Mobile Data Management, MDM 2011, Luleå, Sweden, 6–9 June 2011, vol. 1, pp. 7–16 (2011)
25.
go back to reference Morton, M.G.: A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing. International Business Machines Company, New York (1966) Morton, M.G.: A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing. International Business Machines Company, New York (1966)
26.
go back to reference Hsu, Y., Pan, Y., Wei, L., Peng, W., Lee, W.: Key formulation schemes for spatial index in cloud data managements. In: 13th IEEE International Conference on Mobile Data Management, MDM 2012, Bengaluru, India, 23–26 July 2012, pp. 21–26 (2012) Hsu, Y., Pan, Y., Wei, L., Peng, W., Lee, W.: Key formulation schemes for spatial index in cloud data managements. In: 13th IEEE International Conference on Mobile Data Management, MDM 2012, Bengaluru, India, 23–26 July 2012, pp. 21–26 (2012)
27.
go back to reference Zhang, N., Zheng, G., Chen, H., Chen, J., Chen, X.: HBaseSpatial: a scalable spatial data storage based on HBase. In: 13th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2014, Beijing, China, 24–26 September 2014, pp. 644–651 (2014) Zhang, N., Zheng, G., Chen, H., Chen, J., Chen, X.: HBaseSpatial: a scalable spatial data storage based on HBase. In: 13th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2014, Beijing, China, 24–26 September 2014, pp. 644–651 (2014)
28.
go back to reference Chen, X.Y., Zhang, C., Ge, B., Xiao, W.D.: Efficient historical query in HBase for spatio-temporal decision support. Int. J. Comput., Commun. Control. 11(5), 613–630 (2016)CrossRef Chen, X.Y., Zhang, C., Ge, B., Xiao, W.D.: Efficient historical query in HBase for spatio-temporal decision support. Int. J. Comput., Commun. Control. 11(5), 613–630 (2016)CrossRef
29.
go back to reference Roussopoulos, N., Kelley, S., Vincent, F.: Nearest neighbor queries. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, 22–25 May 1995, pp. 71–79 (1995) Roussopoulos, N., Kelley, S., Vincent, F.: Nearest neighbor queries. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, 22–25 May 1995, pp. 71–79 (1995)
30.
go back to reference Zheng, Y., Xie, X., Ma, W.: GeoLife: a collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull. 33(2), 32–39 (2010) Zheng, Y., Xie, X., Ma, W.: GeoLife: a collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull. 33(2), 32–39 (2010)
31.
go back to reference Yuan, J., Zheng, Y., Xie, X., Sun, G.: Driving with knowledge from the physical world. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, 21–24 August 2011, pp. 316–324 (2011) Yuan, J., Zheng, Y., Xie, X., Sun, G.: Driving with knowledge from the physical world. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, 21–24 August 2011, pp. 316–324 (2011)
32.
go back to reference Yuan, J., et al.: T-Drive: driving directions based on taxi trajectories. In: Proceedings of 18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, ACM-GIS 2010, San Jose, CA, USA, 3–5 November 2010, pp. 99–108 (2010) Yuan, J., et al.: T-Drive: driving directions based on taxi trajectories. In: Proceedings of 18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, ACM-GIS 2010, San Jose, CA, USA, 3–5 November 2010, pp. 99–108 (2010)
Metadata
Title
A Hybrid Index Model for Efficient Spatio-Temporal Search in HBase
Authors
Chengyuan Zhang
Lei Zhu
Jun Long
Shuangqiao Lin
Zhan Yang
Wenti Huang
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-030-04503-6_9

Premium Partner