Skip to main content
Erschienen in: Wireless Personal Communications 4/2018

03.02.2018

Information Intelligent Management System Based on Hadoop

verfasst von: Zhenguo Zhou, Zhenggang Huo

Erschienen in: Wireless Personal Communications | Ausgabe 4/2018

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In order to explore the information management, the cloud computing technology is applied to the field of geographic information system, and remote sensing data storage and management system based on Hadoop is studied and realized. The main function of this system includes that the remote sensing data storage module provides the remote sensing data download function for data administrator, supports HTTP protocol and FTP protocol multi-threaded distributed HTTP download. The parallel constructing algorithm of remote sensing image of Pyramid based on Map Reduce is realized by the module, and layered cutting and block storage of massive remote sensing data are carried out. The GDAL open source library suitable for fast read raster data is used and it provides data resource for remote sensing data parallel cutting. In addition, the Geo Web Cache open source tile map service middleware is adopted and HBase is introduced as the storage support of tiles, which can deal with a large number of users’ visit, including loading and drag of maps. The system test is carried out to verify the effectiveness and practicability of the method proposed. The test results can show that the remote sensing data storage and management system based on Hadoop can effectively handle remote sensing data and improve the user’s experience. It is concluded that the information management system has high effectiveness and good practicability.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Liroz-Gistau, M., Akbarinia, R., Agrawal, D., & Valduriez, P. (2016). Fp-Hadoop: Efficient processing of skewed mapreduce jobs. Information Systems, 60, 69–84.CrossRef Liroz-Gistau, M., Akbarinia, R., Agrawal, D., & Valduriez, P. (2016). Fp-Hadoop: Efficient processing of skewed mapreduce jobs. Information Systems, 60, 69–84.CrossRef
2.
Zurück zum Zitat He, H., Du, Z., Zhang, W., & Chen, A. (2016). Optimization strategy of Hadoop small file storage for big data in healthcare. Journal of Supercomputing, 72(10), 1–12.CrossRef He, H., Du, Z., Zhang, W., & Chen, A. (2016). Optimization strategy of Hadoop small file storage for big data in healthcare. Journal of Supercomputing, 72(10), 1–12.CrossRef
3.
Zurück zum Zitat Park, D., Wang, J., & Kee, Y. S. (2016). In-storage computing for Hadoop MapReduce framework: Challenges and possibilities. IEEE Transactions on Computers, PP(99), 1.CrossRef Park, D., Wang, J., & Kee, Y. S. (2016). In-storage computing for Hadoop MapReduce framework: Challenges and possibilities. IEEE Transactions on Computers, PP(99), 1.CrossRef
4.
Zurück zum Zitat Magana-Zook, S., Gaylord, J. M., Knapp, D. R., Dodge, D. A., & Ruppert, S. D. (2016). Large-scale seismic waveform quality metric calculation using Hadoop. Computers & Geosciences, 94, 18–30.CrossRef Magana-Zook, S., Gaylord, J. M., Knapp, D. R., Dodge, D. A., & Ruppert, S. D. (2016). Large-scale seismic waveform quality metric calculation using Hadoop. Computers & Geosciences, 94, 18–30.CrossRef
5.
Zurück zum Zitat Li, Z., & Shen, H. (2017). Measuring scale-up and scale-out Hadoop with remote and local file systems and selecting the best platform. IEEE Transactions on Parallel and Distributed Systems, PP(99), 3201–3214.CrossRef Li, Z., & Shen, H. (2017). Measuring scale-up and scale-out Hadoop with remote and local file systems and selecting the best platform. IEEE Transactions on Parallel and Distributed Systems, PP(99), 3201–3214.CrossRef
6.
Zurück zum Zitat Hodor, P., Chawla, A., Clark, A., & Neal, L. (2016). Cl-dash: Rapid configuration and deployment of Hadoop clusters for bioinformatics research in the cloud. Bioinformatics, 32(2), 301–303. Hodor, P., Chawla, A., Clark, A., & Neal, L. (2016). Cl-dash: Rapid configuration and deployment of Hadoop clusters for bioinformatics research in the cloud. Bioinformatics, 32(2), 301–303.
7.
Zurück zum Zitat Um, J. H., Lee, S., Kim, T. H., Jeong, C. H., Song, S. K., & Jung, H. (2016). Distributed RDF store for efficient searching billions of triples based on Hadoop. Journal of Supercomputing, 72(5), 1825–1840.CrossRef Um, J. H., Lee, S., Kim, T. H., Jeong, C. H., Song, S. K., & Jung, H. (2016). Distributed RDF store for efficient searching billions of triples based on Hadoop. Journal of Supercomputing, 72(5), 1825–1840.CrossRef
8.
Zurück zum Zitat Li, C., Chen, T., He, Q., Zhu, Y., & Li, K. (2016). Mruninovo: An efficient tool for de novo peptide sequencing utilizing the Hadoop distributed computing framework. Bioinformatics, 33(6), 944. Li, C., Chen, T., He, Q., Zhu, Y., & Li, K. (2016). Mruninovo: An efficient tool for de novo peptide sequencing utilizing the Hadoop distributed computing framework. Bioinformatics, 33(6), 944.
9.
Zurück zum Zitat Ferraro, P. U., Roscigno, G., Cattaneo, G., & Giancarlo, R. (2017). Fastdoop: A versatile and efficient library for the input of FASTA and FASTQ files for MapReduce Hadoop bioinformatics applications. Bioinformatics, 33(10), 1575. Ferraro, P. U., Roscigno, G., Cattaneo, G., & Giancarlo, R. (2017). Fastdoop: A versatile and efficient library for the input of FASTA and FASTQ files for MapReduce Hadoop bioinformatics applications. Bioinformatics, 33(10), 1575.
10.
Zurück zum Zitat Fu, X., Gao, Y., Luo, B., Du, X., & Guizani, M. (2017). Security threats to Hadoop: Data leakage attacks and investigation. IEEE Network, PP(99), 12–16. Fu, X., Gao, Y., Luo, B., Du, X., & Guizani, M. (2017). Security threats to Hadoop: Data leakage attacks and investigation. IEEE Network, PP(99), 12–16.
11.
Zurück zum Zitat Cai, X., Li, F., Li, P., Ju, L., & Jia, Z. (2017). SLA-aware energy-efficient scheduling scheme for Hadoop YARN. Journal of Supercomputing, 73(8), 3526–3546.CrossRef Cai, X., Li, F., Li, P., Ju, L., & Jia, Z. (2017). SLA-aware energy-efficient scheduling scheme for Hadoop YARN. Journal of Supercomputing, 73(8), 3526–3546.CrossRef
12.
Zurück zum Zitat Nguyen, M. C., Won, H., Son, S., Gil, M. S., & Moon, Y. S. (2017). Prefetching-based metadata management in advanced multitenant Hadoop. Journal of Supercomputing, 73(2), 1–21. Nguyen, M. C., Won, H., Son, S., Gil, M. S., & Moon, Y. S. (2017). Prefetching-based metadata management in advanced multitenant Hadoop. Journal of Supercomputing, 73(2), 1–21.
Metadaten
Titel
Information Intelligent Management System Based on Hadoop
verfasst von
Zhenguo Zhou
Zhenggang Huo
Publikationsdatum
03.02.2018
Verlag
Springer US
Erschienen in
Wireless Personal Communications / Ausgabe 4/2018
Print ISSN: 0929-6212
Elektronische ISSN: 1572-834X
DOI
https://doi.org/10.1007/s11277-018-5411-4

Weitere Artikel der Ausgabe 4/2018

Wireless Personal Communications 4/2018 Zur Ausgabe

Neuer Inhalt