Skip to main content

2017 | OriginalPaper | Buchkapitel

SD2DS-Based Datastore for Large Files

verfasst von : Adam Krechowicz, Arkadiusz Chrobot, Stanisław Deniziak, Grzegorz Łukawski

Erschienen in: Proceedings of the 2015 Federated Conference on Software Development and Object Technologies

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We are introducing Scalable Distributed Two-Layer Datastore. The system that is an efficient solution while storing relatively big multimedia files. In the article we are focusing on storing high-resolution photos. We are introducing some of the key implementation concepts as well as the careful evaluation. We are comparing our solution with two of the most recognizable data storing systems: MongoDB and Memcached.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Sadalage, P.J., Fowler, M.: NoSQL distilled: a brief guide to the emerging world of polyglot persistence. Pearson Education, London (2012) Sadalage, P.J., Fowler, M.: NoSQL distilled: a brief guide to the emerging world of polyglot persistence. Pearson Education, London (2012)
4.
Zurück zum Zitat Sapiecha, K., Łukawski, G.: Scalable distributed two–layer data structures (SD2DS). IJDST 4, 15–30 (2013) Sapiecha, K., Łukawski, G.: Scalable distributed two–layer data structures (SD2DS). IJDST 4, 15–30 (2013)
5.
Zurück zum Zitat Krechowicz, A., Deniziak, S., Bedla, M., Chrobot, A., Łukawski, G.: Scalable distributed two-layer block based datastore. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 302–311. Springer, Heidelberg (2016). doi:10.1007/978-3-319-32149-3_29 CrossRef Krechowicz, A., Deniziak, S., Bedla, M., Chrobot, A., Łukawski, G.: Scalable distributed two-layer block based datastore. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 302–311. Springer, Heidelberg (2016). doi:10.​1007/​978-3-319-32149-3_​29 CrossRef
6.
Zurück zum Zitat Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. (TOCS) 26(2), 4 (2008)CrossRef Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. (TOCS) 26(2), 4 (2008)CrossRef
7.
Zurück zum Zitat DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. In: ACM SIGOPS Operating Systems Review, vol. 41, no. 6, pp. 205–220. ACM (2007) DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. In: ACM SIGOPS Operating Systems Review, vol. 41, no. 6, pp. 205–220. ACM (2007)
9.
Zurück zum Zitat Bronson, N., Amsden, Z., Cabrera, G., Chakka, P., Dimov, P., Ding, H., Ferris, J., Giardullo, A., Kulkarni, S., Li, H.C., et al.: Tao: facebook’s distributed data store for the social graph. In: USENIX Annual Technical Conference, pp. 49–60 (2013) Bronson, N., Amsden, Z., Cabrera, G., Chakka, P., Dimov, P., Ding, H., Ferris, J., Giardullo, A., Kulkarni, S., Li, H.C., et al.: Tao: facebook’s distributed data store for the social graph. In: USENIX Annual Technical Conference, pp. 49–60 (2013)
12.
Zurück zum Zitat Carra, D., Michiardi, P.: Memory partitioning in memcached: an experimental performance analysis. In: ICC 2014, IEEE International Conference on Communications, 10–14 June 2014, Sydney, Australia, June 2014. http://wwweurecom.fr/publication/4320 Carra, D., Michiardi, P.: Memory partitioning in memcached: an experimental performance analysis. In: ICC 2014, IEEE International Conference on Communications, 10–14 June 2014, Sydney, Australia, June 2014. http://​wwweurecom.​fr/​publication/​4320
15.
Zurück zum Zitat Krechowicz, A., Deniziak, S., Łukawski, G., Bedla, M.: Preserving data consistency in scalable distributed two layer data structures. Beyond Databases, Architectures and Structures. Communications in Computer and Information Science, vol. 521, pp. 126–135. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18422-7_11 Krechowicz, A., Deniziak, S., Łukawski, G., Bedla, M.: Preserving data consistency in scalable distributed two layer data structures. Beyond Databases, Architectures and Structures. Communications in Computer and Information Science, vol. 521, pp. 126–135. Springer, Heidelberg (2015). doi:10.​1007/​978-3-319-18422-7_​11
17.
Zurück zum Zitat Litwin, W., Neimat, M.-A., Schneider, D.: RP*: a family of order preserving scalable distributed data structures. In: Proceedings of the Twentieth International Conference on Very Large Databases, Santiago, Chile, pp. 342–353 (1994). citeseer.ist.psu.edu/736278.html Litwin, W., Neimat, M.-A., Schneider, D.: RP*: a family of order preserving scalable distributed data structures. In: Proceedings of the Twentieth International Conference on Very Large Databases, Santiago, Chile, pp. 342–353 (1994). citeseer.​ist.​psu.​edu/​736278.​html
18.
Zurück zum Zitat Litwin, W., Neimat, M.-A., Schneider, D.: Rp*: a family of order preserving scalable distributed data structures. In: VLDB, vol. 94, pp. 12–15 (1994) Litwin, W., Neimat, M.-A., Schneider, D.: Rp*: a family of order preserving scalable distributed data structures. In: VLDB, vol. 94, pp. 12–15 (1994)
19.
Zurück zum Zitat Litwin, W.: Linear hashing: a new tool for file and table addressing. In: VLDB 1980: Proceedings of the Sixth International Conference on Very Large Data Bases. VLDB Endowment, pp. 212–223 (1980) Litwin, W.: Linear hashing: a new tool for file and table addressing. In: VLDB 1980: Proceedings of the Sixth International Conference on Very Large Data Bases. VLDB Endowment, pp. 212–223 (1980)
20.
Zurück zum Zitat Litwin, W.: Trie hashing. In: Proceedings of the 1981 ACM SIGMOD International Conference on Management of Data, pp. 19–29. ACM (1981) Litwin, W.: Trie hashing. In: Proceedings of the 1981 ACM SIGMOD International Conference on Management of Data, pp. 19–29. ACM (1981)
21.
Zurück zum Zitat Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 143–154. ACM (2010) Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 143–154. ACM (2010)
22.
Zurück zum Zitat Fei-Fei, R.F.L., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR 2004, IEEE Workshop on Generative-Model Based Vision (2004) Fei-Fei, R.F.L., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR 2004, IEEE Workshop on Generative-Model Based Vision (2004)
24.
Zurück zum Zitat Łukawski, G., Sapiecha, K.: Fault tolerant record placement for decentralized SDDS LH*. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 312–320. Springer, Heidelberg (2008). doi:10.1007/978-3-540-68111-3_33 CrossRef Łukawski, G., Sapiecha, K.: Fault tolerant record placement for decentralized SDDS LH*. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 312–320. Springer, Heidelberg (2008). doi:10.​1007/​978-3-540-68111-3_​33 CrossRef
25.
Zurück zum Zitat Janowski, L., Kozowski, P., Baran, R., Romaniak, P., Gowacz, A., Rusc, T.: Quality assessment for a visual and automatic license plate recognition. Multimedia Tools Appl. 68(1), 23–40 (2014)CrossRef Janowski, L., Kozowski, P., Baran, R., Romaniak, P., Gowacz, A., Rusc, T.: Quality assessment for a visual and automatic license plate recognition. Multimedia Tools Appl. 68(1), 23–40 (2014)CrossRef
26.
Zurück zum Zitat Deniziak, R., Bak, S., Czarnecki, R.: Synthesis of real-time cloud applications for internet of things. Turk. J. Electr. Eng. Comput. Sci. 7719, 35–49 (2013) Deniziak, R., Bak, S., Czarnecki, R.: Synthesis of real-time cloud applications for internet of things. Turk. J. Electr. Eng. Comput. Sci. 7719, 35–49 (2013)
Metadaten
Titel
SD2DS-Based Datastore for Large Files
verfasst von
Adam Krechowicz
Arkadiusz Chrobot
Stanisław Deniziak
Grzegorz Łukawski
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-46535-7_13