Skip to main content
Top
Published in: Wireless Personal Communications 4/2018

03-02-2018

Information Intelligent Management System Based on Hadoop

Authors: Zhenguo Zhou, Zhenggang Huo

Published in: Wireless Personal Communications | Issue 4/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In order to explore the information management, the cloud computing technology is applied to the field of geographic information system, and remote sensing data storage and management system based on Hadoop is studied and realized. The main function of this system includes that the remote sensing data storage module provides the remote sensing data download function for data administrator, supports HTTP protocol and FTP protocol multi-threaded distributed HTTP download. The parallel constructing algorithm of remote sensing image of Pyramid based on Map Reduce is realized by the module, and layered cutting and block storage of massive remote sensing data are carried out. The GDAL open source library suitable for fast read raster data is used and it provides data resource for remote sensing data parallel cutting. In addition, the Geo Web Cache open source tile map service middleware is adopted and HBase is introduced as the storage support of tiles, which can deal with a large number of users’ visit, including loading and drag of maps. The system test is carried out to verify the effectiveness and practicability of the method proposed. The test results can show that the remote sensing data storage and management system based on Hadoop can effectively handle remote sensing data and improve the user’s experience. It is concluded that the information management system has high effectiveness and good practicability.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Liroz-Gistau, M., Akbarinia, R., Agrawal, D., & Valduriez, P. (2016). Fp-Hadoop: Efficient processing of skewed mapreduce jobs. Information Systems, 60, 69–84.CrossRef Liroz-Gistau, M., Akbarinia, R., Agrawal, D., & Valduriez, P. (2016). Fp-Hadoop: Efficient processing of skewed mapreduce jobs. Information Systems, 60, 69–84.CrossRef
2.
go back to reference He, H., Du, Z., Zhang, W., & Chen, A. (2016). Optimization strategy of Hadoop small file storage for big data in healthcare. Journal of Supercomputing, 72(10), 1–12.CrossRef He, H., Du, Z., Zhang, W., & Chen, A. (2016). Optimization strategy of Hadoop small file storage for big data in healthcare. Journal of Supercomputing, 72(10), 1–12.CrossRef
3.
go back to reference Park, D., Wang, J., & Kee, Y. S. (2016). In-storage computing for Hadoop MapReduce framework: Challenges and possibilities. IEEE Transactions on Computers, PP(99), 1.CrossRef Park, D., Wang, J., & Kee, Y. S. (2016). In-storage computing for Hadoop MapReduce framework: Challenges and possibilities. IEEE Transactions on Computers, PP(99), 1.CrossRef
4.
go back to reference Magana-Zook, S., Gaylord, J. M., Knapp, D. R., Dodge, D. A., & Ruppert, S. D. (2016). Large-scale seismic waveform quality metric calculation using Hadoop. Computers & Geosciences, 94, 18–30.CrossRef Magana-Zook, S., Gaylord, J. M., Knapp, D. R., Dodge, D. A., & Ruppert, S. D. (2016). Large-scale seismic waveform quality metric calculation using Hadoop. Computers & Geosciences, 94, 18–30.CrossRef
5.
go back to reference Li, Z., & Shen, H. (2017). Measuring scale-up and scale-out Hadoop with remote and local file systems and selecting the best platform. IEEE Transactions on Parallel and Distributed Systems, PP(99), 3201–3214.CrossRef Li, Z., & Shen, H. (2017). Measuring scale-up and scale-out Hadoop with remote and local file systems and selecting the best platform. IEEE Transactions on Parallel and Distributed Systems, PP(99), 3201–3214.CrossRef
6.
go back to reference Hodor, P., Chawla, A., Clark, A., & Neal, L. (2016). Cl-dash: Rapid configuration and deployment of Hadoop clusters for bioinformatics research in the cloud. Bioinformatics, 32(2), 301–303. Hodor, P., Chawla, A., Clark, A., & Neal, L. (2016). Cl-dash: Rapid configuration and deployment of Hadoop clusters for bioinformatics research in the cloud. Bioinformatics, 32(2), 301–303.
7.
go back to reference Um, J. H., Lee, S., Kim, T. H., Jeong, C. H., Song, S. K., & Jung, H. (2016). Distributed RDF store for efficient searching billions of triples based on Hadoop. Journal of Supercomputing, 72(5), 1825–1840.CrossRef Um, J. H., Lee, S., Kim, T. H., Jeong, C. H., Song, S. K., & Jung, H. (2016). Distributed RDF store for efficient searching billions of triples based on Hadoop. Journal of Supercomputing, 72(5), 1825–1840.CrossRef
8.
go back to reference Li, C., Chen, T., He, Q., Zhu, Y., & Li, K. (2016). Mruninovo: An efficient tool for de novo peptide sequencing utilizing the Hadoop distributed computing framework. Bioinformatics, 33(6), 944. Li, C., Chen, T., He, Q., Zhu, Y., & Li, K. (2016). Mruninovo: An efficient tool for de novo peptide sequencing utilizing the Hadoop distributed computing framework. Bioinformatics, 33(6), 944.
9.
go back to reference Ferraro, P. U., Roscigno, G., Cattaneo, G., & Giancarlo, R. (2017). Fastdoop: A versatile and efficient library for the input of FASTA and FASTQ files for MapReduce Hadoop bioinformatics applications. Bioinformatics, 33(10), 1575. Ferraro, P. U., Roscigno, G., Cattaneo, G., & Giancarlo, R. (2017). Fastdoop: A versatile and efficient library for the input of FASTA and FASTQ files for MapReduce Hadoop bioinformatics applications. Bioinformatics, 33(10), 1575.
10.
go back to reference Fu, X., Gao, Y., Luo, B., Du, X., & Guizani, M. (2017). Security threats to Hadoop: Data leakage attacks and investigation. IEEE Network, PP(99), 12–16. Fu, X., Gao, Y., Luo, B., Du, X., & Guizani, M. (2017). Security threats to Hadoop: Data leakage attacks and investigation. IEEE Network, PP(99), 12–16.
11.
go back to reference Cai, X., Li, F., Li, P., Ju, L., & Jia, Z. (2017). SLA-aware energy-efficient scheduling scheme for Hadoop YARN. Journal of Supercomputing, 73(8), 3526–3546.CrossRef Cai, X., Li, F., Li, P., Ju, L., & Jia, Z. (2017). SLA-aware energy-efficient scheduling scheme for Hadoop YARN. Journal of Supercomputing, 73(8), 3526–3546.CrossRef
12.
go back to reference Nguyen, M. C., Won, H., Son, S., Gil, M. S., & Moon, Y. S. (2017). Prefetching-based metadata management in advanced multitenant Hadoop. Journal of Supercomputing, 73(2), 1–21. Nguyen, M. C., Won, H., Son, S., Gil, M. S., & Moon, Y. S. (2017). Prefetching-based metadata management in advanced multitenant Hadoop. Journal of Supercomputing, 73(2), 1–21.
Metadata
Title
Information Intelligent Management System Based on Hadoop
Authors
Zhenguo Zhou
Zhenggang Huo
Publication date
03-02-2018
Publisher
Springer US
Published in
Wireless Personal Communications / Issue 4/2018
Print ISSN: 0929-6212
Electronic ISSN: 1572-834X
DOI
https://doi.org/10.1007/s11277-018-5411-4

Other articles of this Issue 4/2018

Wireless Personal Communications 4/2018 Go to the issue