Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Arabian Journal for Science and Engineering
Published in: Arabian Journal for Science and Engineering 11/2019

29-06-2019 | Research Article - Computer Engineering and Computer Science

Storage Node Allocation Methods for Erasure Code-based Cloud Storage Systems

Authors: Ojus Thomas Lee, G. J. Akash, S. D. Madhu Kumar, Priya Chandran

Published in: Arabian Journal for Science and Engineering | Issue 11/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Erasure codes, in the recent past, have emerged as an alternative to data replication-based systems for storing big data. Efficient choice of code and data nodes from the numerous available storage nodes is the key to the performance of any storage system. This paper presents the Storage Node Allocation Problem for selecting the suitable set of nodes for holding data and code blocks by representing the storage systems as a complete bipartite graph. Additionally, the paper formally proves that the problem is NP-hard and proposes approximate solutions using greedy, ant colony optimization and clustering-based methods. The solutions accomplish efficient choice of storage nodes by utilizing parameters like bandwidth availability, distance between the nodes, computational load and disk space availability.
previous article Cooling Computer Chips with Cascaded and Non-cascaded Thermoelectric Devices
next article Brain MR Imaging Tumor Detection Using Monogenic Signal Analysis-Based Invariant Texture Descriptors

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Footnotes
1
The number of storage nodes in the data center in our consideration is \(>= 10,000\), which we virtually partition as \(m >= 5000\) and \(y >= 5000\). The number of storage nodes to be allocated depends on the code RS(x, y) and as an example for RS(10,8), the values \(x = 10\) and \(y = 8\).
 
Literature
1.
Lee, O.T., Kumar, S.D.M., Chandran, P.: Erasure coded storage systems for cloud storage; challenges and opportunities. In: 2016 International Conference on Data Science and Engineering (ICDSE), Cochin, India, pp. 1–7 (2016)
2.
3.
Plank, J.S.: Erasure codes for storage systems: a brief primer. Usenix Mag. 38(6), 44–50 (2013)MathSciNet
4.
5.
6.
Li, R., Li, X., Lee, P.P., Huang, Q.: Repair pipelining for erasure-coded storage. In: 2017 \(\{\)USENIX\(\}\) Annual Technical Conference (\(\{\)USENIX\(\}\) \(\{\)ATC\(\}\) 17), pp. 567–579 (2017)
7.
Wang, J., Wang, T., Luo, Y., Shum, K.W.: Capacity of distributed storage systems with clusters and separate nodes. arXiv preprint arXiv:​1901.​03000 (2019)
8.
Abebe, M., Daudjee, K., Glasbergen, B., Tian, Y.: Ec-store: Bridging the gap between storage and latency in distributed erasure coded systems. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 255–266. IEEE (2018)
9.
10.
Noori, M.; Ardakani, M.: Allocation for heterogeneous storage nodes. IEEE Commun. Lett. 19(12), 2102–2105 (2015)CrossRef
11.
Akhlaghi, S.; Kiani, A.; Ghanavati, M.R.: Cost-bandwidth tradeoff in distributed storage systems. Comput. Commun. 33(17), 2105–2115 (2010)CrossRef
12.
13.
Kao, Y.H., Dimakis, A.G., Leong, D., Ho, T.: Distributed storage allocations and a hypergraph conjecture of erdős. In: IEEE International Symposium on Information Theory Proceedings (ISIT), Istanbul, Turkey, pp. 902–906. IEEE (2013)
14.
Kumar, A.; Tandon, R.; Clancy, T.C.: On the latency and energy efficiency of distributed storage systems. IEEE Trans. Cloud Comput. 5(2), 221–233 (2017)CrossRef
15.
Liao, X.; Qin, Z.; Ding, L.: Data embedding in digital images using critical functions. Signal Process. Image Commun. 58, 146–156 (2017)CrossRef
16.
Liao, X.; Shu, C.: Reversible data hiding in encrypted images based on absolute mean difference of multiple neighboring pixels. J. Vis. Commun. Image Represent. 28, 21–27 (2015)CrossRef
17.
Yu, Q., Shum, K.W., Sung, C.W.: Minimization of storage cost in distributed storage systems with repair consideration. In: Global Telecommunications Conference (GLOBECOM), Houston, Texas, USA., pp. 1–5. IEEE (2011)
18.
Bhattacharya, H., Chattopadhyay, S., Chattopadhyay, M., Banerjee, A.: A novel intelligent modeling of storage and bandwidth constraints in distributed storage allocation. In: International Conference on Computational Intelligence, Communications, and Business Analytics, pp. 336–346. Springer (2017)
19.
Yousafzai, A.; Gani, A.; Noor, R.M.; Sookhak, M.; Talebian, H.; Shiraz, M.; Khan, M.K.: Cloud resource allocation schemes: review, taxonomy, and opportunities. Knowl. Inf. Syst. 50(2), 347–381 (2017)CrossRef
20.
Skourletopoulos, G., Mavromoustakis, C.X., Mastorakis, G., Sahalos, J.N., Batalla, J.M., Dobre, C.: Cost-benefit analysis game for efficient storage allocation in cloud-centric internet of things systems: a game theoretic perspective. In: 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), pp. 1149–1154. IEEE (2017)
21.
Rashmi, K.V.; Shah, N.B.; Gu, D.; Kuang, H.; Borthakur, D.; Ramchandran, K.: A hitchhiker’s guide to fast and efficient data reconstruction in erasure-coded data centers. ACM SIGCOMM Comput. Commun. Rev. 44(4), 331–342 (2015)CrossRef
22.
Ojus, O.T.L., Akash, A.G., Kumar, S.M.K.M., Priya, P.C.: A method for storage node allocation in erasure code based storage systems. In: 2017 IEEE 3rd International Conference on Collaboration and Internet Computing (CIC), pp. 449–454. IEEE (2017)
23.
Akash, G.J., Lee, O.T., Kumar, S.M., Chandran, P., Cuzzocrea, A.: Rapid: a fast data update protocol in erasure coded storage systems for big data. In: Storage Systems for Big Data., 2nd IEEE/ACM International Workshop on Distributed Big Data Management(DBDM), Madrid, Spain (2017)
24.
Xia, M., Saxena, M., Blaum, M., Pease, D.A.: A tale of two erasure codes in hdfs. In: 13th USENIX Conference on File and Storage Technologies (FAST 15), Santa Clara, CA, USA, pp. 213–226. USENIX Association (2015)
25.
Blum, C.: Ant colony optimization: introduction and recent trends. Phys. Life Rev. 2(4), 353–373 (2005)CrossRef
26.
Stützle, T.; Hoos, H.H.: Max-min ant system. Future Gener. Comput. Syst. 16(8), 889–914 (2000)CrossRefMATH
27.
Benson, T., Akella, A., Maltz, D.A.: Network traffic characteristics of data centers in the wild. In: Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, Melbourne, Australia, pp. 267–280. ACM (2010)
28.
Shabeera, T.; Kumar, S.M.; Salam, S.M.; Krishnan, K.M.: Optimizing vm allocation and data placement for data-intensive applications in cloud using aco metaheuristic algorithm. Eng. Sci. Technol. Int. J. 20(2), 616–628 (2017)CrossRef
29.
Huang, J.; Liang, X.; Qin, X.; Xie, P.; Xie, C.: Scale-rs: an efficient scaling scheme for rs-coded storage clusters. IEEE Trans. Parallel Distrib. Syst. 26(6), 1704–1717 (2015)CrossRef
Metadata
Title
Storage Node Allocation Methods for Erasure Code-based Cloud Storage Systems
Authors
Ojus Thomas Lee
G. J. Akash
S. D. Madhu Kumar
Priya Chandran
Publication date
29-06-2019
Publisher
Springer Berlin Heidelberg
Published in
Arabian Journal for Science and Engineering / Issue 11/2019
Print ISSN: 2193-567X
Electronic ISSN: 2191-4281
DOI
https://doi.org/10.1007/s13369-019-03983-8

Other articles of this Issue 11/2019

Arabian Journal for Science and Engineering 11/2019 Go to the issue

Research Article - Computer Engineering and Computer Science

A New Heuristic Clustering Algorithm Based on RSU for Internet of Vehicles

Research Article - Computer Engineering and Computer Science

UFC: A Unified POI Recommendation Framework

Research Article - Computer Engineering and Computer Science

Hybrid Filter–Wrapper Feature Selection Method for Sentiment Classification

Research Article - Computer Engineering and Computer Science

Performance Optimization of Cognitive Decision Engine for CR-Based IoTs Using Various Parameter-Less Meta-Heuristic Techniques

Research Article--Computer Engineering and Computer Science

Microarray Filtering-Based Fuzzy C-Means Clustering and Classification in Genomic Signal Processing

Research Article - Computer Engineering and Computer Science

Brain MR Imaging Tumor Detection Using Monogenic Signal Analysis-Based Invariant Texture Descriptors

Premium Partners

    Image Credits