Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Parallel Programming
Published in: International Journal of Parallel Programming 6/2016

01-12-2016

Czip: A Fast Lossless Compression Algorithm for Climate Data

Authors: Xiaomeng Huang, Yufang Ni, Dexun Chen, Songbin Liu, Haohuan Fu, Guangwen Yang

Published in: International Journal of Parallel Programming | Issue 6/2016

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Climate data have been dramatically increasing in volume in recent years. This huge volume of climate data poses considerable challenges for data storage, archiving and sharing. In this paper, we propose a lossless compression algorithm for climate data, named czip. We efficiently eliminate data redundancy through several new methods, including adaptive prediction, eXclusive OR differencing, multiway compression and static regions. To utilize the multiple cores available on modern computers, czip is implemented in parallel. Experimental results show that czip can achieve outstanding compression ratios as well as deflating and inflating throughputs; czip can achieve 800 MB/s deflating throughputs and over 2600 MB/s inflating throughputs on a server with 16 cores.
previous article Transparent Speculative Parallelization of Discrete Event Simulation Applications Using Global Variables
next article Combining Data and Computation Distribution Directives for Hybrid Parallel Programming : A Transformation System

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Overpeck, J.T., Meehl, G.A., Bony, S., Easterling, D.R.: Climate data challenges in the 21 st century. Science (Washington) 331(6018), 700–702 (2011)CrossRef
2.
3.
4.
5.
6.
7.
Isenburg, M., Lindstrom, P., Snoeyink, J.: Lossless compression of predicted floating-point geometry. IEEE Trans. Inf. Theory 37(8), 869–877 (2005)MATH
8.
Burtscher, M., Ratanaworabhan, P.: FPC: a high-speed compressor for double-precision floating-point data. IEEE Trans. Comput. 58(1), 18–31 (2009)MathSciNetCrossRef
9.
C. 120.0-G-2: Lossless data compression. In: Report Concerning Space Data System Standards. Green Book (Issue 2) (2006)
10.
Lindstrom, P., Isenburg, M.: Fast and efficient compression of floating-point data. IEEE Trans. Comput. 12(5), 1245–1250 (2006)
11.
Ibarria, L., Lindstrom, P., Rossignac, J., Szymczak, A.: Out-of-core compression and decompression of large n-dimensional scalar fields. Comput. Graph. Forum 22(3), 343–348 (2003)CrossRef
12.
Wheeler, D., Burrows, M.: A block-sorting lossless data compression algorithm. Digital Systems Research Center Report, vol. 124 (1994)
13.
14.
Yeh, P.-S., Xia-Serafino, W., Miles, L., Kobler, B., Menasce, D.: Implementation of ccsds lossless data compression in hdf. In: Earth Science Technology Conference (2002)
15.
O’Neil, M.A., Burtscher, M.: Floating-point data compression at 75 gb/s on a gpu. In: Proceedings of the Fourth Workshop on General Purpose Processing on Graphics Processing Units, p. 7. ACM (2011)
16.
Sanchez, V., Nasiopoulos, P., Abugharbieh, R.: Lossless compression of 4d medical images using h. 264/avc. In: 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 2. pp. II–II, IEEE (2006)
17.
Woodring, J., Mniszewski, S., Brislawn, C., DeMarle, D., Ahrens, J.: Revisiting wavelet compression for large-scale climate data using jpeg, 2000 and ensuring data precision. In: 2011 IEEE Symposium Large Data Analysis and Visualization (LDAV), pp. 31–38 (2011)
18.
Ma, K.-L., Shen, H.-W.: Compression and accelerated rendering of time-varying volume data. In: Proceedings of the 2000 International Computer Symposium-Workshop on Computer Graphics and Virtual Reality, pp. 82–89 (2000)
19.
Fout, N., Ma, K.-L., Ahrens, J.: Time-varying, multivariate volume data reduction. In: Proceedings of the 2005 ACM Symposium on Applied Computing. ACM, pp. 1224–1230 (2005)
20.
Fout, N., Ma, K.-L.: An adaptive prediction-based approach to lossless compression of floating-point volume data. IEEE Trans. Comput. 18(12), 2295–2304 (2012)
21.
Engelson, V., Fritzson, D., Fritzson, P.: Lossless compression of high-volume numerical data from simulations. In: Data Compression Conference. Citeseer (2000)
22.
Robinson, T.: Simple Lossless and Near-Lossless Waveform Compression. Cambridge University Engineering Department, Cambridge (1995)
23.
Hans, M., Schafer, R.W.: Lossless compression of digital audio. IEEE Trans. Comput. 18(4), 21–32 (2001)
24.
Taylor, K., Stouffer, R., Meehl, G.: An overview of CMIP5 and the experiment design. IEEE Trans. Comput. 93(4), 485 (2012)
25.
26.
27.
28.
Songbin, L., Xiaomeng, H., Haohuan, F.: Data reduction analysis for climate data sets. In: 10th IFIP International Conference on Network and Parallel Computing (2013)
29.
Rice, R.F.: Practical universal noiseless coding. In: 23rd Annual Technical Symposium. International Society for Optics and Photonics, pp. 247–267 (1979)
30.
31.
32.
Metadata
Title
Czip: A Fast Lossless Compression Algorithm for Climate Data
Authors
Xiaomeng Huang
Yufang Ni
Dexun Chen
Songbin Liu
Haohuan Fu
Guangwen Yang
Publication date
01-12-2016
Publisher
Springer US
Published in
International Journal of Parallel Programming / Issue 6/2016
Print ISSN: 0885-7458
Electronic ISSN: 1573-7640
DOI
https://doi.org/10.1007/s10766-016-0403-z

Other articles of this Issue 6/2016

International Journal of Parallel Programming 6/2016 Go to the issue

OriginalPaper

A Cross-ISA Kernelized High-Performance Parallel Emulator

Editorial

Preface to the Special Issue on Sequential Code Parallelization

OriginalPaper

Towards Scalable Java HPC with Hybrid and Native Communication Devices in MPJ Express

OriginalPaper

Purge-Rehab: Eager Software Transactional Memory with High Performance Under Contention

OriginalPaper

Parallel Implementations of the Cooperative Particle Swarm Optimization on Many-core and Multi-core Architectures

OriginalPaper

Automatic Parallelization: Executing Sequential Programs on a Task-Based Parallel Runtime

Premium Partner

    Image Credits