Top

Published in:

2018 | OriginalPaper | Chapter

Entropy Aware Adaptive Compression for SQL Column Stores

Authors : K. T. Sridhar, Jimson Johnson

Published in: Beyond Databases, Architectures and Structures. Facing the Challenges of Data Proliferation and Growing Variety

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

With the advent of SQL column stores, compression has gained renewed interest and drawn considerable attention from both academia and industry. Unlike row stores, column stores use lightweight compression methods and, generally, compression granularity is at entire column level. In this paper we outline and explore an alternative compression strategy for column stores that works at a different granularity and adapts itself to data, on-the-fly, using a compression planner. The approach yields good compression ratios, facilitates compression during bulk data load and also mitigates some issues that arise from having to maintain global meta-data on compression. We describe its implementation in analytics database dbX, a cloud agnostic, columnar MPP SQL product and present experimental results.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter EvOLAP Graph – Evolution and OLAP-Aware Graph Data Model

next chapter SIMD Acceleration for Main-Memory Index Structures – A Survey

http://storage.googleapis.com/books/ngrams/books/datasetsv2.html.

Abadi, D., Boncz, P.A., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5(3), 197–280 (2013)CrossRef

Abadi, D.J., Madden, S., Ferreira, M.: Integrating compression and execution in column-oriented database systems. In: SIGMOD 2006, pp. 671–682. ACM (2006)

Abadi, D.J., Madden, S., Hachem, N.: Column-stores vs. row-stores: how different are they really? In: SIGMOD 2008, pp. 967–980. ACM (2008)

Baklarz, G.: DB2 compression estimation tool. Technical report, IBM Corporation, Canada, October 2016

Binnig, C., Hildenbrand, S., Färber, F.: Dictionary-based order-preserving string compression for main memory column stores. In: SIGMOD 2009, pp. 283–296. ACM (2009)

Copeland, G.P., Khoshafian, S.: A decomposition storage model. In: SIGMOD, pp. 268–279. ACM (1985)

Damme, P., Habich, D., Hildebrandt, J., Lehner, W.: Lightweight data compression algorithms: an experimental survey. In: Proceedings of 20th EDBT, pp. 72–83 (2017)

Flajolet, P., Fusy, E., Gandouet, O., Mennier, F.: HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm. In: Analysis of Algorithms 2007 (AofA07), pp. 127–146 (2007)

Goldstein, J., Ramakrishnan, R., Shaft, U.: Compressing relations and indexes. In: Proceedings of 14th ICDE, pp. 370–379. IEEE (1998)

10.

Graefe, G., Shapiro, L.: Data compression and database performance. In: Proceedings of ACM/IEEE-CS Symposium on Applied Computing, pp. 22–27. IEEE (1991)

11.

Hovestadt, M., Kao, O., Kliem, A., Warneke, D.: Evaluating adaptive compression to mitigate the effects of shared I/O in clouds. In: 25th IEEE IPDPS, pp. 1042–1051. IEEE (2011)

12.

Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 35(1), 40–45 (2012)

13.

Iyer, B.R., Wilhite, D.: Data compression support in databases. In: Proceedings of 20th VLDB, pp. 695–704 (1994)

14.

Krintz, C., Sucu, S.: Adaptive on-the-fly compression. IEEE Trans. Parallel Distrib. Syst. 17(1), 15–24 (2006)CrossRef

15.

Krueger, J., Grund, M., Tinnefeld, C., Plattner, H., Zeier, A., Faerber, F.: Optimizing write performance for read optimized databases. In: Kitagawa, H., Ishikawa, Y., Li, Q., Watanabe, C. (eds.) DASFAA 2010. LNCS, vol. 5982, pp. 291–305. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12098-5_23CrossRef

16.

Lang, H., Mühlbauer, T., Funke, F., Boncz, P.A., Neumann, T., Kemper, A.: Data blocks: hybrid OLTP and OLAP on compressed storage using both vectorization and compilation. In: SIGMOD 2016, pp. 311–326. ACM (2016)

17.

Lelewer, D.A., Hirschberg, D.S.: Data compression. ACM Comput. Surv. 19(3), 261–296 (1987)CrossRef

18.

Poess, M., Potapov, D.: Data compression in oracle. In: Proceedings of 29th VLDB, pp. 761–770 (2003)

19.

Raman, V., et al.: DB2 with BLU acceleration: so much more than just a column store. PVLDB 6(11), 1080–1091 (2013)

20.

Sridhar, K.T., Sakkeer, M.A.: Optimizing database load and extract for big data era. In: Bhowmick, S.S., Dyreson, C.E., Jensen, C.S., Lee, M.L., Muliantara, A., Thalheim, B. (eds.) DASFAA 2014. LNCS, vol. 8422, pp. 503–512. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-05813-9_34CrossRef

21.

Stonebraker, M., et al.: C-store: a column-oriented DBMS. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 553–564. VLDB Endowment (2005)

22.

TeraData: compression types supported by teradata database, release 16.0 (2016). https://info.teradata.com

23.

Viglas, S.: Just-in-time compilation for SQL query processing. PVLDB 6(11), 1190–1191 (2013)

24.

Wust, J., Krüger, J., Gund, M., Hartmann, U., Plattner, H.: Sparse dictionaries for in-memory column stores. In: Proceedings of 4th DBKDA, pp. 25–33. IARIA (2012)

25.

Zukowski, M., Héman, S., Nes, N., Boncz, P.A.: Super-scalar RAM-CPU cache compression. In: Proceedings of 22nd ICDE, pp. 59–71. IEEE (2006)

Title: Entropy Aware Adaptive Compression for SQL Column Stores
Authors: K. T. Sridhar
Jimson Johnson
Publisher: Springer International Publishing
Book: Beyond Databases, Architectures and Structures. Facing the Challenges of Data Proliferation and Growing Variety
Print ISBN: 978-3-319-99986-9

Electronic ISBN: 978-3-319-99987-6

Copyright Year: 2018
DOI: https://doi.org/10.1007/978-3-319-99987-6_7

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner