Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Computing and Visualization in Science
Erschienen in: Computing and Visualization in Science 5-6/2018

09.11.2018 | Original Article

Multilevel techniques for compression and reduction of scientific data—the univariate case

verfasst von: Mark Ainsworth, Ozan Tugluk, Ben Whitney, Scott Klasky

Erschienen in: Computing and Visualization in Science | Ausgabe 5-6/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We present a multilevel technique for the compression and reduction of univariate data and give an optimal complexity algorithm for its implementation. A hierarchical scheme offers the flexibility to produce multiple levels of partial decompression of the data so that each user can work with a reduced representation that requires minimal storage whilst achieving the required level of tolerance. The algorithm is applied to the case of turbulence modelling in which the datasets are traditionally not only extremely large but inherently non-smooth and, as such, rather resistant to compression. We decompress the data for a range of relative errors, carry out the usual analysis procedures for turbulent data, and compare the results of the analysis on the reduced datasets to the results that would be obtained on the full dataset. The results obtained demonstrate the promise of multilevel compression techniques for the reduction of data arising from large scale simulations of complex phenomena such as turbulence modelling.
Vorheriger Artikel Finite elements for scalar convection-dominated equations and incompressible flow problems: a never ending story?
Nächster Artikel A nonlinear eigenmode solver for linear viscoelastic structures

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
Ainsworth, M., Klasky, S., Whitney, B.: Compression using lossless decimation: analysis and application. SIAM J. Sci. Comput. 39(4), B732–B757 (2017)MathSciNetCrossRefMATH
2.
Austin, W., Ballard, G., Kolda, T. G.: Parallel tensor compression for large-scale scientific data. In: 2016 IEEE international parallel and distributed processing symposium (IPDPS), pp. 912–922, May 2016
3.
4.
Bautista, G., Leonardo, A., Cappello, F.: Improving floating point compression through binary masks. In: 2013 IEEE international conference on big data, pp. 326–331, October 2013
5.
Bornemann, F., Yserentant, H.: A basic norm equivalence for the theory of multilevel methods. Numer. Math. 64(1), 455–476 (1993)MathSciNetCrossRefMATH
6.
Burtscher, M., Hari, M., Annie, Y., Farbod, H.: Real-time synthesis of compression algorithms for scientific data. In: SC ‘16: proceedings of the international conference for high performance computing, networking, storage and analysis, IEEE, pp. 264–275, November 2016
7.
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Series in Telecommunications, 1st edn. Wiley, New York (1991)CrossRefMATH
8.
9.
Daubechies, I.: The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inf. Theory 36(5), 961–1005 (1990)MathSciNetCrossRefMATH
10.
Di, S., Cappello, F.: Fast error-bounded lossy HPC data compression with SZ. In: 2016 IEEE 30th international parallel and distributed processing symposium, IEEE, Chicago, IL, USA, pp. 730–739, May 2016
11.
Donoho, D.L., Vetterli, M., DeVore, R.A., Daubechies, I.: Data compression and harmonic analysis. IEEE Trans. Inf. Theory 44(6), 2435–2476 (1998)MathSciNetCrossRefMATH
12.
Edmunds, D.E., Triebel, H.: Function Spaces, Entropy Numbers, Differential Operators, 1st edn. Cambridge University Press, Cambridge (1996)CrossRefMATH
13.
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)MATH
14.
Grgic, S., Kers, K., Grgic, M.: Image compression using wavelets. In: Proceedings of the IEEE international symposium on industrial electronics, 1999. ISIE ‘99, vol. 1, pp. 99–104 (1999)
15.
Griebel, M., Oswald, P.: Stable splittings of Hilbert spaces of functions of infinitely many variables. J. Complex. 41, 126–151 (2017)MathSciNetCrossRefMATH
16.
Ibarria, L., Lindstrom, P., Rossignac, J., Szymczak, A.: Out-of-core compression and decompression of large n-dimensional scalar fields. Comput. Graph. Forum 22(3), 343–348 (2003)CrossRef
17.
Johns Hopkins Turbulence Databases. Forced isotropic turbulence dataset description, October 2017. Last update: 10/19/2017 5:55:14 PM. Accessed 01 Feb 2018
18.
Kolmogorov, A.: The local structure of turbulence in incompressible viscous fluid for very large Reynolds’ numbers. Akademiia Nauk SSSR Doklady 30, 301–305 (1941)MathSciNet
19.
Lakshminarasimhan, S., Shah, N., Ethier, S., Klasky, S., Latham, R., Ross, R., Samatova, N. F.: Compressing the incompressible with ISABELA: in-situ reduction of spatio-temporal data. In: Emmanuel J., Raymond N., Jean R. (eds) Euro-Par 2011: Parallel Processing Workshops, Lecture Notes in Computer Science, Bordeaux, France, Springer, Berlin, Heidelberg, vol. 6852, pp. 366–379, August 2011
20.
Li, Y., Perlman, E., Wan, M., Yang, Y., Meneveau, C., Burns, R., Chen, S., Szalay, A., Eyink, G.: A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence. J. Turbul. 9, N31 (2008)CrossRefMATH
21.
Lindstrom, P.: Fixed-rate compressed floating-point arrays. IEEE Trans. Vis. Comput. Graph. 20(12), 2674–2683 (2014)CrossRef
22.
Lindstrom, P., Isenburg, M.: Fast and efficient compression of floating-point data. IEEE Trans. Vis. Comput. Graph. 12(5), 1245–1250 (2006)CrossRef
23.
Marcellin, M. W., Gormish, M. J., Bilgin, A., Boliek, M. P.: An overview of JPEG-2000. In: Proceedings DCC 2000. Data compression conference, pp. 523–541 (2000)
24.
Oswald, P.: Multilevel Finite Element Approximation. Theory and Applications. Teubner Skripten zur Numerik. B. G. Teubner, Stuttgart (1994)CrossRefMATH
25.
Perlman, E., Burns, R., Li, Y., Meneveau, C.: Data exploration of turbulence simulations using a database cluster. In: Proceedings of the 2007 ACM/IEEE conference on supercomputing, ACM, Reno, NV, USA, vol. 23, November 2007
26.
Salomon, D.: Data Compression: The Complete Reference, 4th edn. Springer, London (2007)MATH
27.
Schendel, E. R., Jin, Y., Shah, N., Chen, J., Chang, C. S., Ku, S.-H., Ethier, S., Klasky, S., Latham, R., Ross, R., Samatova, N. F.: ISOBAR preconditioner for effective and high-throughput lossless data compression. In: 2012 IEEE 28th international conference on data engineering, pp. 138–149, April 2012
28.
Schneider, K., Farge, M., Pellegrino, G., Rogers, M.M.: Coherent vertex simulation of three-dimensional turbulent mixing layers using orthogonal wavelets. J. Fluid Mech. 534, 39–66 (2005)MathSciNetCrossRefMATH
29.
Shah, N., Schendel, E. R., Lakshminarasimhan, S., Pendse, S. V., Rogers, T., Samatova, N. F.: Improving I/O throughput with PRIMACY: preconditioning ID-mapper for compressing incompressibility. In: 2012 IEEE international conference on cluster computing, pp. 209–219, September 2012
30.
Strengert, M., Magallón, M., Weiskopf, D., Guthe, S., Ertl, T.: Hierarchical visualization and compression of large volume datasets using GPU clusters. EGPGV, pp. 41–48 (2004)
31.
Wallace, G. K.: The JPEG still picture compression standard. IEEE Trans. Consum. Electron. 38(1), xviii–xxxiv (1992)
Metadaten
Titel
Multilevel techniques for compression and reduction of scientific data—the univariate case
verfasst von
Mark Ainsworth
Ozan Tugluk
Ben Whitney
Scott Klasky
Publikationsdatum
09.11.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
Computing and Visualization in Science / Ausgabe 5-6/2018
Print ISSN: 1432-9360
Elektronische ISSN: 1433-0369
DOI
https://doi.org/10.1007/s00791-018-00303-9

Weitere Artikel der Ausgabe 5-6/2018

Computing and Visualization in Science 5-6/2018 Zur Ausgabe

EditorialNotes

Preface

Special Issue FEM Symposium 2017

Finite elements for scalar convection-dominated equations and incompressible flow problems: a never ending story?

Special Issue FEM Symposium 2017

A massively parallel Eikonal solver on unstructured meshes

Original Article

A nonlinear eigenmode solver for linear viscoelastic structures

Special Issue FEM Symposium 2017

Numerical methods for fractional diffusion

Premium Partner

    Bildnachweise