Skip to main content

2018 | OriginalPaper | Buchkapitel

Research on a New Density Clustering Algorithm Based on MapReduce

verfasst von : Yun Wu, Zhixiong Zhang

Erschienen in: Geo-Spatial Knowledge and Intelligence

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The empirical solution parameters for the Density-Based Spatial Clustering of Applications with Noise(DBSCAN) resulted in poor Clustering effect and low execution efficiency, An adaptive DBSCAN algorithm based on genetic algorithm and MapReduce programming framework is proposed. The genetic algorithm (minPts) and scanning radius size (Eps) optimized intensive interval threshold, at the same time, combined with the similarities and differences of data sets using the Hadoop cluster parallel computing ability of two specifications, the data is reasonable of serialization, finally realizes the adaptive parallel clustering efficiently. Experimental results show that the improved algorithm (GA) - DBSCANMR when dealing with the data set of magnitude 3 times execution efficiency is improved DBSCAN algorithm, clustering quality improved by 10%, and this trend increases as the amount of data, provides a more precise threshold DBSCAN algorithm to determine the implementation of the method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Chen, G., Liu, B.Q., Wu, Y.: An adaptive DBSCAN algorithm based on gaussian distribution. Microelectron. Comput. 30(03), 27–30 (2013) Chen, G., Liu, B.Q., Wu, Y.: An adaptive DBSCAN algorithm based on gaussian distribution. Microelectron. Comput. 30(03), 27–30 (2013)
2.
Zurück zum Zitat Feng, Z., Qian, X., Zhao, N.: Greedy DBSCAN: a DBSCAN improvement algorithm for multi-density clustering. Comput. Appl. Res. (9) (2016) Feng, Z., Qian, X., Zhao, N.: Greedy DBSCAN: a DBSCAN improvement algorithm for multi-density clustering. Comput. Appl. Res. (9) (2016)
3.
Zurück zum Zitat Ester, M., Kriegel, H.P., Sander, J., et al.: A Density-based algorithm for discovering clusters in large spatial databases with noise. pp. 226–231 (2008) Ester, M., Kriegel, H.P., Sander, J., et al.: A Density-based algorithm for discovering clusters in large spatial databases with noise. pp. 226–231 (2008)
4.
Zurück zum Zitat Wang, W.T., Wu, Y.L., Tang, C.Y., et al.: Adaptive density-based spatial clustering of applications with noise (DBSCAN) according to data. In: International Conference on Machine Learning and Cybernetics, IEEE (2015) Wang, W.T., Wu, Y.L., Tang, C.Y., et al.: Adaptive density-based spatial clustering of applications with noise (DBSCAN) according to data. In: International Conference on Machine Learning and Cybernetics, IEEE (2015)
5.
Zurück zum Zitat Uncu, O., Gruver, W.A., Kotak, D.B., et al.: GRIDBSCAN: grid density-based spatial clustering of applications with noise. vol. 4, pp. 2976–2981 (2006) Uncu, O., Gruver, W.A., Kotak, D.B., et al.: GRIDBSCAN: grid density-based spatial clustering of applications with noise. vol. 4, pp. 2976–2981 (2006)
6.
Zurück zum Zitat Shufen, L., Dongxue, M., Xiaoyan, W.: Study on DBSCAN Algorithm Based on Grid Element. J. Jilin Univ. Eng. Sci. 44(4), 1135–1139 (2014) Shufen, L., Dongxue, M., Xiaoyan, W.: Study on DBSCAN Algorithm Based on Grid Element. J. Jilin Univ. Eng. Sci. 44(4), 1135–1139 (2014)
7.
Zurück zum Zitat Luo, Q.: DBSCAN algorithm based on cloud computing. Wuhan University of Technology (2013) Luo, Q.: DBSCAN algorithm based on cloud computing. Wuhan University of Technology (2013)
8.
Zurück zum Zitat Yang, Y.: Study on adaptive density clustering algorithm based on MapReduce. Tianjin University (2013) Yang, Y.: Study on adaptive density clustering algorithm based on MapReduce. Tianjin University (2013)
9.
Zurück zum Zitat He, Y., Tan, H., Luo, W., et al.: MR-DBSCAN: an efficient parallel density-based clustering algorithm using MapReduce. In: IEEE 17th International Conference on Parallel and Distributed Systems, IEEE Computer Society 2011, pp. 473–480 (2011) He, Y., Tan, H., Luo, W., et al.: MR-DBSCAN: an efficient parallel density-based clustering algorithm using MapReduce. In: IEEE 17th International Conference on Parallel and Distributed Systems, IEEE Computer Society 2011, pp. 473–480 (2011)
10.
Zurück zum Zitat Lei, L., Nie, R., Wang, J., et al.: Improved DBSCAN algorithm based on MapReduce. Comput. Sci. (s2) 396–399 (2015) Lei, L., Nie, R., Wang, J., et al.: Improved DBSCAN algorithm based on MapReduce. Comput. Sci. (s2) 396–399 (2015)
11.
Zurück zum Zitat Li, L., Xi, Y.: Research on clustering algorithm and its parallelization strategy. In: International Conference on Computational and Information Sciences, pp. 325–328 (2011) Li, L., Xi, Y.: Research on clustering algorithm and its parallelization strategy. In: International Conference on Computational and Information Sciences, pp. 325–328 (2011)
12.
Zurück zum Zitat Design, application and application of data clustering algorithm based on MapReduce. J. East China Inst. Technol Design, application and application of data clustering algorithm based on MapReduce. J. East China Inst. Technol
13.
Zurück zum Zitat Song, J., Xu, S., Guo, C., et al.: A task distribution algorithm for optimizing energy consumption of MapReduce system. J. Comput. 39(2), 323–338 (2016)MathSciNet Song, J., Xu, S., Guo, C., et al.: A task distribution algorithm for optimizing energy consumption of MapReduce system. J. Comput. 39(2), 323–338 (2016)MathSciNet
14.
Zurück zum Zitat Wang, Z., Chen, Q., Li, Z., et al.: MapReduce data equalization method based on incremental partition strategy. J. Comput. (1) 19–35 (2016) Wang, Z., Chen, Q., Li, Z., et al.: MapReduce data equalization method based on incremental partition strategy. J. Comput. (1) 19–35 (2016)
15.
Zurück zum Zitat Yaling, X., Jifu, Z., Xiao, Q.: Data placement strategy in MapReduce cluster environment. J. Softw. (8), 2056–2073 (2015) Yaling, X., Jifu, Z., Xiao, Q.: Data placement strategy in MapReduce cluster environment. J. Softw. (8), 2056–2073 (2015)
Metadaten
Titel
Research on a New Density Clustering Algorithm Based on MapReduce
verfasst von
Yun Wu
Zhixiong Zhang
Copyright-Jahr
2018
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-13-0896-3_55