Skip to main content
Erschienen in: Cluster Computing 1/2024

10.01.2023

File block multi-replica management technology in cloud storage

verfasst von: Qinlu He, Fan Zhang, Genqing Bian, Weiqi Zhang, Zhen Li, Zhimin Yu, Hao Feng

Erschienen in: Cluster Computing | Ausgabe 1/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

With the rapid development of cloud computing technology and the broad application of the Internet, more and more user data are stored in the cloud storage system online. Data copy technology is an effective technical means to manage various data in cloud storage systems, which has the advantages of improving data access speed and enhancing data availability in cloud storage systems. Since a data copy management strategy has an important impact on the performance of cloud storage systems, how to design a data copy management strategy is of great significance to achieving high-quality cloud storage management. Firstly, this paper analyzes the HDFS random copy placement strategy and its shortcomings. Secondly, proposes a copy placement strategy based on evaluation value and load balancing. Thirdly, gives the evaluation function of rack and node by considering multiple evaluation indicators such as node disk space load capacity, node memory utilization, node CPU utilization, rack load rate and network distance between racks. Fourthly, finds suitable nodes through computer shelf evaluation values and node evaluation values when placing replicas. Finally, through simulation experiments, the experimental results of the HDFS random replica placement strategy and the evaluation results and load balancing-based replica placement strategy are compared and analyzed.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Tao, M., Ota, K., Dong, M.: DSARP: dependable scheduling with active replica placement for workflow applications in cloud computing. IEEE Trans. Cloud Comput. 4, 8 (2020) Tao, M., Ota, K., Dong, M.: DSARP: dependable scheduling with active replica placement for workflow applications in cloud computing. IEEE Trans. Cloud Comput. 4, 8 (2020)
2.
Zurück zum Zitat Mansouri, N., Javidi, N., Zade, N.: Hierarchical data replication strategy to improve performance in cloud computing. Front. Comput. Sci. 15(2), 17 (2021)CrossRef Mansouri, N., Javidi, N., Zade, N.: Hierarchical data replication strategy to improve performance in cloud computing. Front. Comput. Sci. 15(2), 17 (2021)CrossRef
3.
Zurück zum Zitat Al-Ramahi, N.M., Odeh, M., Alrabie, Z., et al.: The TOEQCC framework for sustainable adoption of cloud computing at higher education institutions in the kingdom of Jordan. Sustainability 14(19), 12744 (2022)CrossRef Al-Ramahi, N.M., Odeh, M., Alrabie, Z., et al.: The TOEQCC framework for sustainable adoption of cloud computing at higher education institutions in the kingdom of Jordan. Sustainability 14(19), 12744 (2022)CrossRef
4.
Zurück zum Zitat Bhatta, D., Mashayekhy, L.: A bifactor approximation algorithm for cloudlet placement in edge computing. IEEE Trans. Parallel Distrib. Syst: A Publ. IEEE Comput. Soc. 33(8), 1787–1798 (2022)CrossRef Bhatta, D., Mashayekhy, L.: A bifactor approximation algorithm for cloudlet placement in edge computing. IEEE Trans. Parallel Distrib. Syst: A Publ. IEEE Comput. Soc. 33(8), 1787–1798 (2022)CrossRef
5.
Zurück zum Zitat Cui, L., Zhang, J., Yue, L., et al.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. Serv. Comput. IEEE Trans. 11(4), 727–739 (2018)CrossRef Cui, L., Zhang, J., Yue, L., et al.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. Serv. Comput. IEEE Trans. 11(4), 727–739 (2018)CrossRef
6.
Zurück zum Zitat Jiaojiao, Wu., Yanping, Li., Tianyin, W., et al.: CPDA: a confidentiality-preserving deduplication cloud storage with public cloud auditing. IEEE Access 7, 160482–160497 (2019)CrossRef Jiaojiao, Wu., Yanping, Li., Tianyin, W., et al.: CPDA: a confidentiality-preserving deduplication cloud storage with public cloud auditing. IEEE Access 7, 160482–160497 (2019)CrossRef
7.
Zurück zum Zitat Guangyan, Z., Guiyong, Wu., Shupeng, W., et al.: Ca Co: an efficient Cauchy coding approach for cloud storage systems. IEEE Trans. Comput. 65(2), 435–477 (2016)MathSciNetCrossRef Guangyan, Z., Guiyong, Wu., Shupeng, W., et al.: Ca Co: an efficient Cauchy coding approach for cloud storage systems. IEEE Trans. Comput. 65(2), 435–477 (2016)MathSciNetCrossRef
8.
Zurück zum Zitat He, L., Qian, Z. Shang, F.: A novel predicted replication strategy in cloud storage. J. Supercomput. 76 4838–4856 (2020)CrossRef He, L., Qian, Z. Shang, F.: A novel predicted replication strategy in cloud storage. J. Supercomput. 76 4838–4856 (2020)CrossRef
9.
Zurück zum Zitat H Xu G Wang L Luo M Lei 2018 "The Design of Reliability Simulation of Cloud System in the Cloudsim," 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China 215–219. H Xu G Wang L Luo M Lei 2018 "The Design of Reliability Simulation of Cloud System in the Cloudsim," 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China 215–219.
10.
Zurück zum Zitat Lizhen, C., Junhua, Z., Lingxi, Y., et al.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. IEEE Trans. Serv. Comput. 11(4), 727–739 (2015) Lizhen, C., Junhua, Z., Lingxi, Y., et al.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. IEEE Trans. Serv. Comput. 11(4), 727–739 (2015)
11.
Zurück zum Zitat Souravlas, S., Sifaleras, A.: Binary-tree based estimation of file requests for efficient data replication. IEEE Trans. Parallel Distrib. Syst. 28(7), 1839–1852 (2017)CrossRef Souravlas, S., Sifaleras, A.: Binary-tree based estimation of file requests for efficient data replication. IEEE Trans. Parallel Distrib. Syst. 28(7), 1839–1852 (2017)CrossRef
12.
Zurück zum Zitat Yuhua, L., Haiying, S.: EAFR: an energy-efficient adaptive file replication system in data-intensive clusters. IEEE Trans. Parallel Distrib. Syst. 28(4), 1017–1030 (2017)CrossRef Yuhua, L., Haiying, S.: EAFR: an energy-efficient adaptive file replication system in data-intensive clusters. IEEE Trans. Parallel Distrib. Syst. 28(4), 1017–1030 (2017)CrossRef
13.
Zurück zum Zitat He, Q., Bian, G., Zhang, W., et al.: Research on routing strategy in cluster deduplication system. IEEE Access 9, 135485–135495 (2021)CrossRef He, Q., Bian, G., Zhang, W., et al.: Research on routing strategy in cluster deduplication system. IEEE Access 9, 135485–135495 (2021)CrossRef
14.
Zurück zum Zitat Ting, Y., Haibo, P., Wei, Li., et al.: An energy-efficient storage strategy for cloud datacenters based on variable K-coverage of a hypergraph. IEEE Trans. Parallel Distrib. Syst. 28(12), 3344–3355 (2017)CrossRef Ting, Y., Haibo, P., Wei, Li., et al.: An energy-efficient storage strategy for cloud datacenters based on variable K-coverage of a hypergraph. IEEE Trans. Parallel Distrib. Syst. 28(12), 3344–3355 (2017)CrossRef
16.
Zurück zum Zitat Juipin, Y.: Elastic load balancing using self-adaptive replication management[J]. IEEE Access 5(99), 7495–7504 (2017) Juipin, Y.: Elastic load balancing using self-adaptive replication management[J]. IEEE Access 5(99), 7495–7504 (2017)
17.
Zurück zum Zitat Chitra, D.D., Rhymend, U.V.: Load balancing in cloud computing environment using improved weighted round robin algorithm for nonpreemptive dependent tasks. Scientific World J. 2016, 1–14 (2016) Chitra, D.D., Rhymend, U.V.: Load balancing in cloud computing environment using improved weighted round robin algorithm for nonpreemptive dependent tasks. Scientific World J. 2016, 1–14 (2016)
18.
Zurück zum Zitat Junfeng, T., Weiping, Li.: Pheromone-based genetic algorithm adaptive selection algorithm in cloud storage. Int. J. Grid Distrib. Comput. 9(6), 269–278 (2016)CrossRef Junfeng, T., Weiping, Li.: Pheromone-based genetic algorithm adaptive selection algorithm in cloud storage. Int. J. Grid Distrib. Comput. 9(6), 269–278 (2016)CrossRef
19.
Zurück zum Zitat Huang, C.: Analysis on application of wavelet neural network in wind electricity power prediction. Appl. Mech. Mater. 7, 3764–3769 (2022)MathSciNet Huang, C.: Analysis on application of wavelet neural network in wind electricity power prediction. Appl. Mech. Mater. 7, 3764–3769 (2022)MathSciNet
21.
Zurück zum Zitat Pengden, Li., Xiaofan, Y.: On dynamic recovery of cloud storage system under advanced persistent threats. IEEE Access 7, 103556–103569 (2019)CrossRef Pengden, Li., Xiaofan, Y.: On dynamic recovery of cloud storage system under advanced persistent threats. IEEE Access 7, 103556–103569 (2019)CrossRef
22.
Zurück zum Zitat Wang, B., Lv, B., Song, Y.: A hybrid genetic algorithm with integer coding for task offloading in edge-cloud cooperative computing. IAENG Int. J. Comput. Sci. 49(2), 503–510 (2022) Wang, B., Lv, B., Song, Y.: A hybrid genetic algorithm with integer coding for task offloading in edge-cloud cooperative computing. IAENG Int. J. Comput. Sci. 49(2), 503–510 (2022)
23.
Zurück zum Zitat Jing, W., Zhiyuan, Y., Kuanching, Li., et al.: Local codes with cooperative repair in distributed storage of cyber-physical-social systems[J]. IEEE Access 8, 38622–38632 (2020)CrossRef Jing, W., Zhiyuan, Y., Kuanching, Li., et al.: Local codes with cooperative repair in distributed storage of cyber-physical-social systems[J]. IEEE Access 8, 38622–38632 (2020)CrossRef
24.
Zurück zum Zitat Mansouri, N., Javidi, M.M.: A review of data replication based on meta-heuristics approach in cloud computing and data grid. Soft Comput. 24(11), 1–28 (2020) Mansouri, N., Javidi, M.M.: A review of data replication based on meta-heuristics approach in cloud computing and data grid. Soft Comput. 24(11), 1–28 (2020)
25.
Zurück zum Zitat Yan, W., Yuankai, G.: Forecasting method of stock market volatility in time series data based on mixed model of ARIMA and XGBoost. China Commun. 17(3), 205–221 (2020)CrossRef Yan, W., Yuankai, G.: Forecasting method of stock market volatility in time series data based on mixed model of ARIMA and XGBoost. China Commun. 17(3), 205–221 (2020)CrossRef
26.
Zurück zum Zitat Park, H., Lee, D., Moon, J.: LDPC code design for distributed storage: balancing repair bandwidth, reliability and storage overhead. IEEE Trans. Commun. 66(2), 507–520 (2018)CrossRef Park, H., Lee, D., Moon, J.: LDPC code design for distributed storage: balancing repair bandwidth, reliability and storage overhead. IEEE Trans. Commun. 66(2), 507–520 (2018)CrossRef
27.
Zurück zum Zitat Scheid, E.J., Rodrigues, B.B., Granville, L.Z., Enabling Dynamic, S.L.A., Smart, C.-B., Contracts, et al.: IFIP/IEEE symposium on integrated network and service management (IM). Arlington 2019, 53–61 (2019) Scheid, E.J., Rodrigues, B.B., Granville, L.Z., Enabling Dynamic, S.L.A., Smart, C.-B., Contracts, et al.: IFIP/IEEE symposium on integrated network and service management (IM). Arlington 2019, 53–61 (2019)
28.
Zurück zum Zitat Xu, X., Yang, C., Shao, J.: Data replica placement mechanism for open heterogeneous storage systems. Procedia Comput. Sci. 109, 18–25 (2017)CrossRef Xu, X., Yang, C., Shao, J.: Data replica placement mechanism for open heterogeneous storage systems. Procedia Comput. Sci. 109, 18–25 (2017)CrossRef
29.
Zurück zum Zitat Song, P.C., Pan, J.S., Chu, S.C.: A parallel compact cuckoo search algorithm for three-dimensional path planning. Appl. Soft Comput. 94, 106443 (2022)CrossRef Song, P.C., Pan, J.S., Chu, S.C.: A parallel compact cuckoo search algorithm for three-dimensional path planning. Appl. Soft Comput. 94, 106443 (2022)CrossRef
30.
Zurück zum Zitat Saaty, T.L.: Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 1(1), 83–98 (2008) Saaty, T.L.: Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 1(1), 83–98 (2008)
31.
Zurück zum Zitat Calheiros, R.N., Ranjan, R., Beloglazov, A., et al.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)CrossRef Calheiros, R.N., Ranjan, R., Beloglazov, A., et al.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)CrossRef
32.
Zurück zum Zitat Wang, R., Peng, L.: Topology-aware congestion control algorithm in data center network. J. Comput. Appl. 36(9), 2357 (2016) Wang, R., Peng, L.: Topology-aware congestion control algorithm in data center network. J. Comput. Appl. 36(9), 2357 (2016)
Metadaten
Titel
File block multi-replica management technology in cloud storage
verfasst von
Qinlu He
Fan Zhang
Genqing Bian
Weiqi Zhang
Zhen Li
Zhimin Yu
Hao Feng
Publikationsdatum
10.01.2023
Verlag
Springer US
Erschienen in
Cluster Computing / Ausgabe 1/2024
Print ISSN: 1386-7857
Elektronische ISSN: 1573-7543
DOI
https://doi.org/10.1007/s10586-022-03952-1

Weitere Artikel der Ausgabe 1/2024

Cluster Computing 1/2024 Zur Ausgabe

Premium Partner