Skip to main content
Erschienen in: Cluster Computing 2/2020

11.05.2019

Classified enhancement model for big data storage reliability based on Boolean satisfiability problem

verfasst von: Hong Huang, Latifur Khan, Shaohua Zhou

Erschienen in: Cluster Computing | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Disk reliability is a serious problem in the big data foundation environment. Although the reliability of disk drives has greatly improved over the past few years, they are still the most vulnerable core components in the server. If they fail, the result can be catastrophic: it can take some days to recover data, sometimes data lost forever. These are unacceptable for some important data. XOR parity is a typical method to generate reliability syndrome, thus improving the reliability of the data. In practice, we find that the data is still likely to be lost. In most storage systems reliability improvements are achieved through the allocation of additional disks in Redundant Arrays of Independent Disks (RAID), which will increase the hardware costs, thus it will be very difficult for cost-constrained environments. Therefore, how to improve the data integrity without raising the hardware cost has aroused much interest of big data researchers. This challenge is when creating non-traditional RAID geometries, care must be taken to respect data dependence relationships to ensure that the new RAID strategy improves reliability, which is a NP-hard problem. In this paper, we present an approach for characterizing these challenges using high-dimension variants of the n-queens problem that enables performable solutions via the SAT solver MiniSAT, and use the greedy algorithm to analyze the queen’s attack domain, as a basis for reliability syndrome generation. A large number of experiments show that the approach proposed in this paper is feasible in software-defined data centers and the performance of the algorithm can meet the current requirements of the big data environment.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
3.
Zurück zum Zitat Zhao, H., Xu, Y., Xiang, L.: Scaling up of E-Msr codes based distributed storage systems with fixed number of redundancy nodes. Int. J. Distrib. Parallel Syst. 3(5), 1 (2012)CrossRef Zhao, H., Xu, Y., Xiang, L.: Scaling up of E-Msr codes based distributed storage systems with fixed number of redundancy nodes. Int. J. Distrib. Parallel Syst. 3(5), 1 (2012)CrossRef
4.
Zurück zum Zitat Rozier, E.W.D., Sanders, W.H.: A framework for efficient evaluation of the fault tolerance of deduplicated storage systems. In: Proceedings of the IEEE/IFIP International Conference on Dependable Systems & Networks. IEEE Computer Society (2012) Rozier, E.W.D., Sanders, W.H.: A framework for efficient evaluation of the fault tolerance of deduplicated storage systems. In: Proceedings of the IEEE/IFIP International Conference on Dependable Systems & Networks. IEEE Computer Society (2012)
5.
Zurück zum Zitat Bayram, U., Divine, D., Zhou, P., et al.: Improving reliability with dynamic syndrome allocation in intelligent software defined data centers. In: Proceedings of the 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, pp. 219–230 (2015) Bayram, U., Divine, D., Zhou, P., et al.: Improving reliability with dynamic syndrome allocation in intelligent software defined data centers. In: Proceedings of the 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, pp. 219–230 (2015)
6.
Zurück zum Zitat Bayram, U., Rozier, K.Y., Rozier, E.W.D.: Characterizing data dependence constraints for dynamic reliability using N-queens attack domains. In: Proceedings of the International Conference on Quantitative Evaluation of Systems. Springer, Cham, pp. 211–227 (2015) Bayram, U., Rozier, K.Y., Rozier, E.W.D.: Characterizing data dependence constraints for dynamic reliability using N-queens attack domains. In: Proceedings of the International Conference on Quantitative Evaluation of Systems. Springer, Cham, pp. 211–227 (2015)
7.
Zurück zum Zitat Liu, X., Fan, L., Wang, L., et al.: Multiobjective reliable cloud storage with its particle swarm optimization algorithm. Math. Probl. Eng. 2016, 14 (2016)MathSciNetMATH Liu, X., Fan, L., Wang, L., et al.: Multiobjective reliable cloud storage with its particle swarm optimization algorithm. Math. Probl. Eng. 2016, 14 (2016)MathSciNetMATH
8.
Zurück zum Zitat Chen, P.M., Lee, E.K., Gibson, G.A., et al.: RAID: high-performance, reliable secondary storage. ACM Comput. Surv. (CSUR) 26(2), 145–185 (1994)CrossRef Chen, P.M., Lee, E.K., Gibson, G.A., et al.: RAID: high-performance, reliable secondary storage. ACM Comput. Surv. (CSUR) 26(2), 145–185 (1994)CrossRef
9.
Zurück zum Zitat Corbett, P., English, B., Goel, A., et al.: Row-diagonal parity for double disk failure correction. In: Proceedings of the 3rd USENIX Conference on File and Storage Technologies. USENIX Association Berkeley, CA, USA, 1–14 (2004) Corbett, P., English, B., Goel, A., et al.: Row-diagonal parity for double disk failure correction. In: Proceedings of the 3rd USENIX Conference on File and Storage Technologies. USENIX Association Berkeley, CA, USA, 1–14 (2004)
10.
Zurück zum Zitat Schroeder, B., Gibson, G.A.: Disk failures in the real world: what does an mttf of 1,000,000 hours mean to you? FAST 7(1), 1–16 (2007) Schroeder, B., Gibson, G.A.: Disk failures in the real world: what does an mttf of 1,000,000 hours mean to you? FAST 7(1), 1–16 (2007)
11.
Zurück zum Zitat Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., et al.: Xoring elephants: novel erasure codes for big data. Proc. VLDB Endow. 6(5), 325–336 (2013)CrossRef Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., et al.: Xoring elephants: novel erasure codes for big data. Proc. VLDB Endow. 6(5), 325–336 (2013)CrossRef
12.
Zurück zum Zitat Li, T., Mehta, A., Yang, P.: Security Analysis of Email systems. In: Proceedings of the 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud). IEEE, pp. 91–96 (2017) Li, T., Mehta, A., Yang, P.: Security Analysis of Email systems. In: Proceedings of the 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud). IEEE, pp. 91–96 (2017)
13.
Zurück zum Zitat Turner, V., Gantz, J.F., Reinsel, D.: The digital universe of opportunities: rich data and the increasing value of the internet of things. IDC Anal. Fut. 16, 1–10 (2014) Turner, V., Gantz, J.F., Reinsel, D.: The digital universe of opportunities: rich data and the increasing value of the internet of things. IDC Anal. Fut. 16, 1–10 (2014)
14.
Zurück zum Zitat Utard, G., Vernois, A.: Data durability in peer to peer storage systems. In: CCGrid 2004 IEEE International Symposium on Cluster Computing and the Grid, 2004. IEEE, pp. 90-97 (2004) Utard, G., Vernois, A.: Data durability in peer to peer storage systems. In: CCGrid 2004 IEEE International Symposium on Cluster Computing and the Grid, 2004. IEEE, pp. 90-97 (2004)
15.
Zurück zum Zitat Rozier, E.W.D., Zhou, P., Divine, D.: Building intelligence for software defined data centers: modeling usage patterns. In: International Systems & Storage Conference (2013) Rozier, E.W.D., Zhou, P., Divine, D.: Building intelligence for software defined data centers: modeling usage patterns. In: International Systems & Storage Conference (2013)
16.
Zurück zum Zitat Wu, X., Xu, Y., Yuen, C., et al.: A tag encoding scheme against pollution attack to linear network coding. IEEE Trans. Parallel Distrib. Syst. 25(1), 33–42 (2014)CrossRef Wu, X., Xu, Y., Yuen, C., et al.: A tag encoding scheme against pollution attack to linear network coding. IEEE Trans. Parallel Distrib. Syst. 25(1), 33–42 (2014)CrossRef
18.
Zurück zum Zitat Gong, W., Zhou, X.: A survey of SAT solver. In: AIP Conference Proceedings, vol. 1836, No. 1, p. 020059. AIP Publishing (2017) Gong, W., Zhou, X.: A survey of SAT solver. In: AIP Conference Proceedings, vol. 1836, No. 1, p. 020059. AIP Publishing (2017)
19.
Zurück zum Zitat Rozier, E.W.D., Sanders, W.H., Zhou, P., et al.: Modeling the fault tolerance consequences of deduplication. In: Reliable Distributed Systems. IEEE (2011) Rozier, E.W.D., Sanders, W.H., Zhou, P., et al.: Modeling the fault tolerance consequences of deduplication. In: Reliable Distributed Systems. IEEE (2011)
20.
Zurück zum Zitat Bell, J., Stevens, B.: A survey of known results and research areas for n-queens. Discrete Math. 309(1), 1–31 (2009)MathSciNetCrossRef Bell, J., Stevens, B.: A survey of known results and research areas for n-queens. Discrete Math. 309(1), 1–31 (2009)MathSciNetCrossRef
22.
Zurück zum Zitat Rozier, E.W., Rozier, K.Y.: SMT-driven intelligent storage for big data. In: Proceedings of the Ninth International Workshop on Constraints in Formal Verification (CFV 2015), Austin, Texas, USA (2015) Rozier, E.W., Rozier, K.Y.: SMT-driven intelligent storage for big data. In: Proceedings of the Ninth International Workshop on Constraints in Formal Verification (CFV 2015), Austin, Texas, USA (2015)
23.
Zurück zum Zitat Huang, C., Li, J., Chen, M.: On optimizing XOR-based codes for fault-tolerant storage applications. In: Information Theory Workshop, 2007. ITW’07. IEEE. IEEE, pp. 218–223 (2007) Huang, C., Li, J., Chen, M.: On optimizing XOR-based codes for fault-tolerant storage applications. In: Information Theory Workshop, 2007. ITW’07. IEEE. IEEE, pp. 218–223 (2007)
24.
Zurück zum Zitat Schwarz, S.J.T., Long, D.D.E., Paris, J.F.: Reliability of disk arrays with double parity. In: Proceedings of the 2013 IEEE 19th Pacific Rim International Symposium on Dependable Computing (PRDC). IEEE, pp. 108–117 (2013) Schwarz, S.J.T., Long, D.D.E., Paris, J.F.: Reliability of disk arrays with double parity. In: Proceedings of the 2013 IEEE 19th Pacific Rim International Symposium on Dependable Computing (PRDC). IEEE, pp. 108–117 (2013)
25.
Zurück zum Zitat Zhu, Y., Lee, P.P.C., Hu, Y., et al.: On the speedup of single-disk failure recovery in xor-coded storage systems: Theory and practice. In: Proceedings of the 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST). IEEE, pp. 1–12 (2012) Zhu, Y., Lee, P.P.C., Hu, Y., et al.: On the speedup of single-disk failure recovery in xor-coded storage systems: Theory and practice. In: Proceedings of the 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST). IEEE, pp. 1–12 (2012)
26.
Zurück zum Zitat Keedwell, A.D., Dnes, J.: Latin squares and their applications. Elsevier, Amsterdam (2015) Keedwell, A.D., Dnes, J.: Latin squares and their applications. Elsevier, Amsterdam (2015)
27.
Zurück zum Zitat Gutirreznaranjo, M.A., Martnezdelamor, M.A., Prezhurtado, I., et al.: Solving the N-queens puzzle with P systems. Rosa M Gutirrez Escudero 1, 99–210 (2012) Gutirreznaranjo, M.A., Martnezdelamor, M.A., Prezhurtado, I., et al.: Solving the N-queens puzzle with P systems. Rosa M Gutirrez Escudero 1, 99–210 (2012)
28.
Zurück zum Zitat Pris, J.F., Long, D.D.E., Litwin, W.: Three-dimensional redundancy codes for archival storage. In: Proceedings of the 2013 IEEE 21st International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), IEEE. pp. 328–332 (2013) Pris, J.F., Long, D.D.E., Litwin, W.: Three-dimensional redundancy codes for archival storage. In: Proceedings of the 2013 IEEE 21st International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), IEEE. pp. 328–332 (2013)
Metadaten
Titel
Classified enhancement model for big data storage reliability based on Boolean satisfiability problem
verfasst von
Hong Huang
Latifur Khan
Shaohua Zhou
Publikationsdatum
11.05.2019
Verlag
Springer US
Erschienen in
Cluster Computing / Ausgabe 2/2020
Print ISSN: 1386-7857
Elektronische ISSN: 1573-7543
DOI
https://doi.org/10.1007/s10586-019-02941-1

Weitere Artikel der Ausgabe 2/2020

Cluster Computing 2/2020 Zur Ausgabe