WaveRange: wavelet-based data compression for three-dimensional numerical simulations on regular grids

We restrict our attention to data sampled on single- or multi-block grids with each block using three-dimensional Cartesian indexing, as shown in Fig. 2. Numerical methods that involve such kinds of topology are common in HPC for the ease and efficiency of data management. Atmospheric flow simulations of the global scale can be performed using a yin–yang grid that consists of two overlapping blocks. Regional and urban simulations can incorporate geometrical representation of landscape features and buildings by using immersed boundary approaches (Lundquist et al. 2010; Matsuda et al. 2018) that effectively reduce the computational domain to a rectangular box.

Let \(\{f_{i_x,i_y,i_z}\}\) be a three-dimensional scalar field sampled on a grid \(\{x_{i_x,i_y,i_z}, y_{i_x,i_y,i_z}, z_{i_x,i_y,i_z}\}\), where \(i_x=1,...,n_x\); \(i_y=1,...,n_y\); \(i_z=1,...,n_z\). This grid may be curvilinear. However, the positions of the grid nodes in the physical space are not required by the data compression algorithm. Only the array of values \(f_{i_x,i_y,i_z}\) and the number of grid points in each direction \(n_x\), \(n_y\) and \(n_z\) constitute the input data. If the grid is multi-block, either each block can be treated independently by the compression algorithm, or the blocks can be merged in one array if their dimensions match. Therefore, in the following discussion we will refer to the rectilinear grid schematically shown in Fig. 2, without loss of generality.

The wavelet transform produces an array of real values \(\{F_{i_x,i_y,i_z}\}\) with the same number of elements as in the original array \(\{f_{i_x,i_y,i_z}\}\), i.e., containing \(n_x \times n_y \times n_z\) elements in total. We use a three-dimensional multi-resolution transform based on the biorthogonal Cohen–Daubechies–Feauveau 9/7 (CDF9/7) wavelet, expressed in terms of lifting steps (Daubechies and Sweldens 1998; Getreuer 2013). These wavelets have proved efficient for image compression; therefore, they are likely to be efficient for scientific data having similar locally correlated spatial structure. Numerical experiments with sample CFD velocity data confirmed that this transform yields higher compression ratio than using lower-order biorthogonal wavelets or orthogonal wavelets such as Haar, Daubechies (D4...D20) and Symlets (Sym4...Sym10). Previous work by Li et al. (2015) also showed that the CDF9/7 wavelet performed better than the Haar wavelet for scientific data compression, albeit using thresholding of the wavelet coefficients rather than entropy coding.

Basically, the transform consists of a finite sequence of filtering steps, called lifting steps, applied to one-dimensional (1D) signal. Let \({g_i}\), \(i=1,...,m\) be a 1D array of real values. One level of the forward transform consists in calculating the values of approximation coefficients \({s_j}\), \(j=1,...,{\left\lceil m/2\right\rceil }\) and detail coefficients \({d_j}\), \(j=1,...,\left\lfloor m/2\right\rfloor\) (\({\left\lceil \cdot \right\rceil }\) and \(\left\lfloor \cdot \right\rfloor\) are the ceiling and the floor functions, respectively) using lifting stepswhere \(\alpha =-1.5861343420693648\), \(\beta =-0.0529801185718856\), \(\gamma =0.8829110755411875\), \(\delta =0.4435068520511142\) and \(\zeta =1.1496043988602418\) (Daubechies and Sweldens 1998; Getreuer 2013). For boundary handling, coefficients \(s_{{\left\lceil m/2\right\rceil }+1}^{(0)}\), \(s_{{\left\lceil m/2\right\rceil }+1}^{(1)}\), \(d_{0}^{(1)}\) and \(d_{0}^{(2)}\) are defined and set identical to zero. If m is odd, then the missing element \(d_{{\left\lceil m/2\right\rceil }}^{(0)}\), which is necessary for calculating \(d_{{\left\lceil m/2\right\rceil }}^{(1)}\) and \(s_{{\left\lceil m/2\right\rceil }}^{(1)}\) subsequently, is determined using extrapolationThe output of (1) is an array of size m in which the first \({\left\lceil m/2\right\rceil }\) elements contain the approximation coefficients \(s_j\) and the last \(\left\lfloor m/2\right\rfloor\) elements contain the detail coefficients \(d_j\).

The inverse transform admits coefficients \(s_j\) and \(d_j\) at input, and resolves the lifting steps (1) in the reverse order to produce \(g_j\) at the output. More specifically, \(s_l^{(2)}\) and \(d_l^{(2)}\) are determined from the last two lines, then the third to last equation is solved with respect to \(s_l^{(1)}\) and so on.

The three-dimensional transform is constructed by applying the above one-dimensional transform sequentially in the three directions of the three-dimensional data array; see Appendix A for further explanation. We found that repeating the process 4 times is practically sufficient to reach the maximum compression ratio, because, after 4 levels of the transform, the number of approximation coefficients becomes as small as 1/4096 of the total number of elements in the dataset. We therefore always set \(L=4\) in WaveRange. This means that, as long as \(n_x \ge 15\), \(n_y \ge 15\) and \(n_z \ge 15\), the same number (four) of 1D transform levels is realized in all three directions, even in those cases when there is a dramatic difference between \(n_x\), \(n_y\) and \(n_z\). Note that the transform is in place, i.e., \(\{f_{i_x,i_y,i_z}\}\) and \(\{F_{i_x,i_y,i_z}\}\) physically share the same memory by being stored in the same array.

An example field and its transform are displayed in Fig. 3. On the left, the original field in physical space, \(\{f_{i_x,i_y,i_z}\}\), is displayed. In this example, it contains point values of the velocity component \(u_x\) in a turbulent wake flow, which is described in Section C. The magnitude of the point data values is shown on a logarithmic color scale. On the right, the output of Algorithm 2 is shown, in which the approximation coefficients and the detail coefficients on all levels are packed in one three-dimensional array \(\{F_{i_x,i_y,i_z}\}\). Seven-eighth of its elements correspond to the smallest scale detail coefficients. They are all small in magnitude: Most of them are below the visibility threshold of the selected color scale, and only few large ones appear in the boundary layer. On the next and subsequent levels, details in the turbulent wake become increasingly larger in magnitude. The approximation coefficients occupy the upper left corner of the domain. They are small in number and large in magnitude. Note that the transform is near lossless (in the terminology of Li et al. 2018) such that all values of \(\{f_{i_x,i_y,i_z}\}\) can be calculated from \(\{F_{i_x,i_y,i_z}\}\) with floating-point round-off accuracy.

From this point on, \(\{F_{i_x,i_y,i_z}\}\) is treated as a one-dimensional array of real values \(F_i\), \(i=1,...,N\), where \(N=n_x \times n_y \times n_z\). An example graphical illustration of quantization is shown in Fig. 4. Quantization represents each element \(F_i\) as a set of one-byte numbers (i.e., integer numbers ranging from 0 to 255), as required for the subsequent entropy coding. In general, entropy coders are not limited to 256 size alphabet, but this size is convenient as it corresponds to the char type, native in C language. A double-precision floating-point variable \(F_i\) can be losslessly quantized using eight one-byte numbers. However, in applications related to the numerical simulation of turbulent flows, quantization with some data loss has to be accepted in order to achieve the desired high compression ratio.

Lossy compression is achieved by \(J < 8\) one-byte integer numbers \(Q_{i}^{\langle j \rangle }\) per variable \(F_i\), indexed with a superscript \(\langle j \rangle\), with common offsets \(F_{\min }^{\langle j \rangle }\) independent of i, using the following approximation:with the approximation error no greater than \(\delta ^{\langle J \rangle }\). The latter is controlled by the relative tolerance \(\varepsilon\), which is a user-specified parameter of the compression routine. Note that \(\varepsilon\) can be regarded as the desired \(L^\infty\) error in f in physical space, normalized with \(\max {|f|}\), but \(\delta ^{\langle J \rangle }\) in (3) sets the absolute error in \(F_i\) in wavelet space. To relate \(\delta ^{\langle J \rangle }\) with \(\varepsilon\), we definewhere the maximum is taken over all elements, and \(\eta\) is a constant coefficient. By trial and error, we have found that the value \(\eta = 1.75\) guarantees that \(||f-\check{f}||_\infty / \max {|f|} \approx \varepsilon\) as long as we use four levels of the wavelet transform. Quantization adds random noise with amplitude \(\varepsilon _F\) to the wavelet coefficients F, i.e., \(\max |F-\check{F}|=\varepsilon _F\). The pointwise error in the physical space, \(f-\check{f}\), is a weighted sum of \(F-\check{F}\), with the weight determined by the lifting coefficients and by the length of the filter. From this consideration, it may be possible to evaluate \(\eta\) analytically, but the empirical value of 1.75 proves acceptable in all test cases considered in this paper, see D. We then assign \(\delta ^{\langle J \rangle } = \varepsilon _F\). The values of \(Q_i^{\langle j \rangle }\), \(\delta ^{\langle j \rangle }\), \(F^{\langle j \rangle }_{\min }\) and J are determined as follows from Algorithm 1, where we use square brackets to denote the nearest integer. In the example, \(J=3\) bit planes are required to represent the floating-point data with the desired accuracy \(\varepsilon\). The amount of memory required to store one bit plane is 8 times less than for the original dataset. Thus, after quantization, the data size is J/8 times the original size.

Entropy coding is a technique allowing to reduce the quantity of storage required to hold a message without any loss of information. The message can be any sequence of characters drawn from some selected alphabet. The implementation that we use in the present work is based on an alphabet of 256-bit symbols, i.e., integers from 0 to 255. Conceptually, entropy coding consists in representing frequently occurring subsequences with few bits and rarely occurring ones with many bits. Shannon’s coding theorem serves the entropy of a message as a theoretical bound to possible lossless compression (Shannon 1948).

The entropy coding method that we employ in our present work is the range coding (Martin 1979). Our current implementation uses an open-source coder rngcod13 developed by Schindler (1999). The size of the encoded files produced by this coder is within a fraction of per cent from the theoretical bound, and the operation speed is faster compared to other similar methods such as arithmetic coding.

The input message consists of the bit planes \(Q_{i}^{\langle j \rangle }\) obtained during the quantization step. These data arrays are divided in blocks of 60,000 elements. The frequencies of each value in a block are counted and copied to the output stream of the coder. The data block is then encoded using the calculated frequencies. Given a stream of 256-bit symbols \(Q_{i}^{\langle j \rangle }\) and their frequencies, the coder produces a shorter stream of bits to represent these symbols. The output stream is written in a file, appended with metadata necessary for decoding.

The output file size depends on the contents of the input message \(Q_{i}^{\langle j \rangle }\). It can be less than 1% if only a few elements of \(Q_{i}^{\langle j \rangle }\) are nonzero. It can be above \(80\%\) if the input message resembles white noise. Practical situations in between these two extremes are considered in Sect. 3.

The homogeneous isotropic turbulence (HIT) dataset considered here is similar to the velocity fields analyzed by Onishi et al. (2013). It was obtained by integrating the incompressible Navier–Stokes equations in a \(2\pi\)-periodic cube box, in dimensionless units, using a fourth-order finite-difference scheme, second-order Runge–Kutta time marching and HSMAC velocity–pressure coupling (Onishi et al. 2011). The flow domain was discretized using a uniform Cartesian grid consisting of \(n^3 = 512^3\) staggered grid points. Power input was supplied using external isotropic forcing to achieve a statistically stationary state. A snapshot velocity field (u,v,w) was stored in double precision, each component occupying 1 GB of disk space.

The dataset used in the present study is visualized in Fig. 5a using an iso-surface of the vorticity magnitude, and the energy spectrum is shown in Fig. 5b. Blue color corresponds to the original data. The three velocity components are in the range \([-4.08, 4.16]\), \([-5.20, 4.82]\) and \([-4.31, 4.74]\), respectively. The turbulence kinetic energy is equal to \(K=1.44\). With the dissipation rate \(\epsilon =0.262\), the Kolmogorov length scale is equal to \(\eta =(\nu ^3/\epsilon )^{1/4}=0.00845\), such that \(k_\mathrm{max}\eta = 2.16\), where \(k_\mathrm{max}=n/2\). The Taylor microscale \(\lambda =\sqrt{10 \nu K / \epsilon }=0.246\) yields the Reynolds number \(Re_\lambda =\sqrt{2K/3} \lambda / \nu =218\). Homogeneous isotropic turbulence presents a challenge for the data compression. Turbulent flow fills the entire domain with fluctuations at all scales, the smallest scale being of the same order as the discretization grid step (see Fig. 5b).

Before analyzing the compression performance in this case, let us discuss the error control properties. Among the variety of physically motivated error metrics we choose \(L^\infty\), which is intuitive and perhaps the most stringent. The \(L^\infty\) norm is the most sensitive norm to large local errors. Scientific computing commonly relies on numerical evaluation of derivatives (using, e.g., finite-difference schemes). Therefore, even if only one data point value contains a large error, this error is likely to propagate in space and in time in the subsequent numerical simulation or post-processing. Other, problem specific metrics will be discussed later on. Figure 6a shows the relative \(L^\infty\) error of the velocity field reconstructed from compressed data. The compression algorithm takes the tolerance \(\varepsilon\) as a control parameter. This allows to plot the reconstruction error as a function of \(\varepsilon\). Each velocity component is scaled by its maximum absolute value, yieldingwhere f stands for one of the components (u, v or w) of the original velocity field, and \(\check{f}\) is its reconstruction from the compressed data. The maxima and the minima are calculated over all grid points. In Fig. 6a, the dash–dot diagonal line visualizes the identity relationship between \(\varepsilon\) and \(e_\infty\). The actual data for all three velocity components closely follows this trend, with the discrepancy only becoming noticeable when \(\varepsilon\) is smaller than the accumulated round-off error, and for large \(\varepsilon\) when the discrete nature of quantization becomes apparent as there remain only few nonzero detail coefficients. For all \(\varepsilon \in [10^{-14},10^{-3}]\), the difference between \(\varepsilon\) and \(e_\infty\) is less than 15%, i.e., the desired error control is successfully achieved.

Figure 6b shows a plot of the relative compressed file sizeversus \(e_\infty\), where \(s(f)=1\) GB is the disk space required to store a \(512^3\) array of double-precision values, and \(s(\varphi )\) is the respective compressed file size in GB. The dash–dot diagonal line in this plot represents the storage requirement for a \(512^3\) array of hypothetical custom-precision values, which is 100% in the case of double precision, 50% for the single precision and 0 for the full loss of information, and all intermediate values are linearly interpolated. This line sets a reference it terms of the compression achievable by simply quantizing the floating-point array elements with tolerance \(\varepsilon\) and using smaller, but constant number of bits per element. Its slope is equal to 6.39% of storage per one decimal order of magnitude of accuracy.

The point markers connected by solid lines show the amount disk space actually used by the compressed velocity field files and the respective reconstruction error. When the accuracy is in the range between \(10^{-14}\) and \(10^{-3}\), it is well approximated with a fittingwhich is shown with a dotted magenta line. The negative intercept can be explained as follows. When \(\varepsilon\) (and, consequently, \(e_\infty\)) is large, only a few significant bits are sufficient to represent the field (u, v, w) with the desired accuracy. Hence, the wavelet transform after quantization becomes sparse, and it is very efficiently compressed by entropy coding. In the intermediate range of \(\varepsilon\), increasing accuracy by one order of magnitude comes at a cost of \(5.2\%\) increased storage. Least significant bit planes of the wavelet coefficients are not sparse, they are noise-like, and their lossless compression ratio by entropy coding asymptotically tends to a fairly low value typical of noise, \(0.0639/0.052 \approx 1.23\), in the hypothetical limit of \(\varepsilon \rightarrow 0\). However, for \(\varepsilon <10^{-14}\), accumulation of round-off errors becomes significant, \(\varSigma\) sharply increases to 76%, while \(e_\infty\) saturates at \(5 \times 10^{-15}\). Round-off error could be avoided by switching to integer wavelet transform ensuring perfect reconstruction. Extrapolation of the linear trend suggests that this approach may be superior than, for example, direct application of LZMA encoding, as shown by crosses in Fig. 6b. It is also noteworthy that compression with \(\varepsilon =10^{-8}\) allows to reduce the data storage by a factor of 3, which is significantly better than using the native single-precision floating-point format. Compression with \(\varepsilon =10^{-6}\) increases this ratio to 5. On the other hand, from the figure it is apparent that multi-fold reduction of the file size, which is our objective, entails data loss.

Let us proceed with an example from seismology. Similarly to the analysis in the previous section, we compress an example dataset with a given \(\varepsilon\), measure the compressed file size, then reconstruct the fields from the compressed format and measure the \(L^\infty\) error \(e_\infty\). The example dataset is a restart file for a synthetic seismogram computation of a large earthquake using SPECFEM3D, a spectral element code based on a realistic fully three-dimensional Earth model (Komatitsch et al. 2013). In the simulation, the domain is split in as many slices as the number of processors used. Data that correspond to different slices are stored in separate files, and in this example, we only process one of them. The file contains 31 single-precision arrays, but only the first 9 are large: the displacement, velocity and acceleration of the crust and mantle (\(3 \times 19904633\) elements each), of the inner core (\(3 \times 1270129\) elements each) and of the outer core (2577633 elements each). Only these arrays are compressed, while the remaining 22 small arrays containing in total 5250 single-precision numbers are directly copied in the end of the compressed file. Note that, although the grid is three-dimensional, the spectral element solver packs the variables in one-dimensional arrays. This packing preserves spatial localization; therefore, wavelet compression remains efficient. Figure 7 shows the relative compressed file size \(\varSigma _1=s(\varphi )/s(f_1)\) versus \(e_\infty\), where \(s(f_1)=757\) MB is the original single-precision file size. The result is close to the trend obtained in the previous sections for the turbulent fluid flow, the latter shown with a dotted line.

In this section, we evaluate the compression of restart files produced in a global weather simulation using the Multi-Scale Simulator for the Geoenvironment (MSSG), which is a coupled nonhydrostatic atmosphere–ocean–land model developed at the Center for Earth Information Science and Technology, Japan Agency for Marine-Earth Science and Technology (JAMSTEC) (Takahashi et al. 2013). Its atmospheric component includes a large eddy simulation (LES) model for the turbulent atmospheric boundary layer and a six-category bulk cloud micro-physics model (Onishi and Takahashi 2012). Longwave and shortwave radiation transfer is taken into account using the Model simulation radiation transfer code version 10 (MstranX) (Sekiguchi and Nakajima 2008). In the global weather simulation mode, MSSG uses a yin–yang grid system in order to relax Courant–Friedrichs–Lewy condition on the sphere. Higher-order space and time discretization schemes are employed.

In a study by Nakano et al. (2017), nine tropical cyclones were simulated within the framework of the Global 7km mesh nonhydrostatic Model Intercomparison Project for improving TYphoon forecast (TYMIP-G7). The main objective of that project was to understand and statistically quantify the advantages of high-resolution global atmospheric models toward the improvement of TC track and intensity forecasts. Thus, in MSSG, each horizontal computational domain covered \(4056 \times 1352\) grids in the yin–yang latitude–longitude grid system. The average horizontal grid spacing was 7 km. The vertical level comprised 55 vertical layers with a top height of 40 km and the lowermost vertical layer at 75 m. It was recognized that, when requiring the output data for every 1 or 3 h over 5-day periods be stored for analyses, the total volume of storage summed up to a huge amount.

Here, we use the initial data of July 29, 2014 at 12:00 UTC. The dataset is stored in multiple files. They include a text header file that contains descriptive parameters of the dataset such as its size, hydrodynamic field identifiers and domain decomposition parameters. The hydrodynamic field variables are stored in 1024 binary files, each containing all \(n_f=12\) fields within the same sub-domain. Thus, each sub-domain contains \(n_f \times 254 \times 43 \times 55\) grid point data in double precision, totaling to 55 MB of data in one file. First 512 of these files belong to the Yin grid and the remaining 512 to the Yang grid.

Either the compression can be performed on each data subset file independently or, alternatively, the subsets can be merged before applying the wavelet transform. It is noticed in Sect. 3.1 that the compression ratio has a tendency to increase with the data size. Hence, all subsets of each hydrodynamic field are concatenated as parts of a three-dimensional array of size \(4064 \times 2752 \times 55\). The fields are processed sequentially requiring 4.6 GB of RAM for the input array plus up to 6.4 GB for the encoded output data and for temporary arrays.

First, let us quantify the compression performance of this restart dataset. Figure 8a shows that the relative error \(e_\infty\) varies almost identically to the threshold \(\varepsilon\), where \(e_\infty\) is calculated as the maximum relative error over all data points of all fields. It saturates at the level of \(10^{-14}\) due to round-off. The compressed file size, shown in Fig. 8b, is significantly smaller than in the previously considered cases of turbulent incompressible velocity data. The linear fitshown with a green dotted line, has a greater offset from the dash–dot diagonal and a less steep slope compared with the HIT fit (7), which is shown with a magenta dotted line. The global weather simulation is more complex than the incompressible Navier–Stokes, the fields contained in the restart files are heterogeneous and have sparser wavelet transform that compresses more efficiently.

To measure the effect of lossy restart data compression on the simulation accuracy, the following protocol was implemented.We focus on typhoon Halong. Figure 9a–c shows the typhoon core trajectory, minimum pressure in the core and the wind speed, respectively. The best track from observation is shown using the black color, and the result of the original simulation is shown using the red color. The typhoon trajectory is predicted well during the first three days, after that it deviates more to the north during the simulation. The predicted pressure drop and the increase in the wind speed are slightly advanced in time compared with the observation data, but the maximum wind speed is evaluated accurately.

The results of the restarts with \(\varepsilon =10^{-16}\), \(10^{-6}\) and \(10^{-4}\) are shown with different colors. All of them visually coincide with the original result during the first 24 h, after which the discrepancy grows large enough to be visible, but it remains in all cases significantly smaller than the difference with respect to the observation.

For \(\varepsilon = 10^{-3}\) or larger, it was found impossible to restart the simulation because of numerical instability. In fact, the onset of such instability is already noticeable in the case of \(\varepsilon = 10^{-4}\), as the wind speed becomes slightly different in the very beginning of that simulation. We investigate further on this effect by plotting the difference between the restart and the original results on a logarithmic scale. We consider the \(L^2\) error norm of the wind speed, obtained by summation over all grid points in latitude–longitude square window \(\varOmega\) of size \(10^\circ \times 10^\circ\) centered on the typhoon as predicted in the original simulation,where \(U_\mathrm{restart}\) and \(U_\mathrm{original}\) is the velocity magnitude in the restart simulation and in the original simulation, respectively. For all simulations with \(\varepsilon \le 10^{-5}\), the error increases polynomially as the power \(\approx 2\) of the physical time after the restart point, until saturation after about 72 h. This trend arises from the nonlinear dynamics of the system and it shows no distinguishable deterministic relation with \(\varepsilon\) as long as the latter is sufficiently small. For \(\varepsilon = 10^{-4}\), the error increases rapidly during the first time iterations, but then it decreases and ultimately follows the same trend as described previously. It is apparent that larger values of \(\varepsilon\) entail faster initial error growth and, ultimately, numerical divergence that cannot be accommodated by the physical model (Fig. 10).

Figure 8 shows that successful restarts belong to the intermediate regime (where \(\varSigma\) evolves according to equation (8)) and to the high-accuracy tail of the compression diagram, such that the compressed file size is greater than 2.5, i.e., the compression ratio is less than 40. The low-accuracy end of the diagram showing the file size of less than 1% can be practical for the data archiving only if it is not intended as input for restarting the simulation. Taking into consideration these opposing requirements of simulation reliability and data storage efficiency, it is advisable to compress the restart data of global weather simulations with the relative tolerance of \(\varepsilon =10^{-6}\).

In a study by Matsuda et al. (2018), a tree-crown-resolving large eddy simulation coupled with a three-dimensional radiative transfer model was applied to an urban area around the Tokyo Bay. Source terms that represent contributions of the ground surface, buildings, tree crowns and anthropogenic heat were integrated within the MSSG model in order to perform urban-scale simulations. In particular, tree crowns were taken into account using the volumetric radiosity method. The landscape was set based on geographic information system (GIS) data from the Tokyo Metropolitan Government. The initial and side-boundary atmospheric conditions were imposed by the linear interpolation of the mesoscale data provided by the Japan Meteorological Agency. The computational domain was a rectangular box discretized with uniform grid step (5 m) in two perpendicular horizontal directions and slightly stretched in the vertical direction (from 5 m near the sea level to 15 m in the upper layers). We consider restart data for a simulation using \(N = 2500 \times 2800 \times 151\) grid points. Only the hydrodynamic fields are compressed since all other restart data use much less disk space. The domain is decomposed in equal Cartesian blocks of \(50 \times 25 \times 151\) points. These datasets are stored in 5600 files, each containing 21 hydrodynamic fields. The files occupy 166 GB of disk space in total.

As shown in Fig. 11, data compression dramatically reduces the storage requirement. We have compared two approaches. The first (‘united file’) is to read the data from all sub-domains and concatenate in one array per field, then transform and encode each field and write all encoded data in one binary file. This is the same procedure as used in Sect. 3.3. The second approach (‘divided file’) consists in processing each sub-domain independently, producing 5600 compressed binary files. Since it does not require any communication between sub-domains, processing can be executed in parallel with ideal speedup. The parallel speedup comes at a price of larger compressed file size, by \(\approx 2 \%\) of the original data size. The sub-domain files are smaller than the united file; therefore, their compression ratio is overall lower, as explained in Sect. 3.1. In addition, part of the difference is due to the sub-domain data being normalized with the respective local maximum absolute value instead of using the global maximum. The difference may be insignificant when considering the high-accuracy end of the plot, e.g., for \(\varepsilon =10^{-14}\), \(\varSigma\) increases from 21 to 23%. However, when the tolerance is set to \(\varepsilon =10^{-6}\), the variation of the compressed file size from 6 to 8% of the original restart file size can be considered as relatively large. Using MPI communication for parallel wavelet transform and range coding, it may be possible to achieve better trade-off between parallel speedup and compression ratio.

We follow similar procedure as in Sect. 3.3 to evaluate the effect of lossy compression upon restart. Taking restart files from a previous simulation as initial data, three new simulations have been performed for the period of 16:00-16:10 JST (Japan Standard Time) on August 11, 2007. With the wind speed of about 2 m s\(^{-1}\), the ten-minute time span of these simulations yields the characteristic advection length of 1.2 km, which is less than the domain size but an order of magnitude greater the size of a typical building. The first of the simulations resumes from the original files, while the second and the third resume from compressed data with \(\varepsilon =10^{-6}\) and \(10^{-12}\), respectively. We use the united compressed file format that introduces no artifacts at the boundaries between sub-domains. In the following discussion, we analyze the output data written on disk in the end of these simulations.

Table 1 shows the residual relative error in the \(L^\infty\) and \(L^2\) norms, respectively, calculated asandwhere f denotes a hydrodynamic field obtained in the reference simulation that resumed from the original restart data, and \(\check{f}(\varepsilon )\) stands for the respective field computed starting from the compressed data. To simplify the notation, f and \(\check{f}(\varepsilon )\) are treated as one-dimensional arrays.

The relative \(L^\infty\) error is of order 100% in both cases for most of the field variables, except for pressure fluctuation that reaches 14% and for the base density and pressure, both of which are constant in time and therefore remain of the same order as \(\varepsilon\) or less. The relative \(L^2\) error is, in general, two orders of magnitude smaller than the respective \(L^\infty\) error, which means that, in most part of the domain, the pointwise residual error is much smaller than the respective peak values. Interestingly, the error tends to be smaller for \(\varepsilon =10^{-6}\) than for \(\varepsilon =10^{-12}\), this may be explained by greater artificial dissipation introduced at \(\varepsilon =10^{-6}\) that slows down the convection of coherent structures.

Comparing the present results with the global numerical simulation described in Sect. 3.3, one of the key differences is in the spatial resolution. In this building-resolving simulation, the eddy turnover time of the smallest wake vortices is of order of seconds. Therefore, after 10 min (i.e., by the end of the simulation), the small structures de-correlate, producing large pointwise error.

To gain better insight, let us consider a horizontal plane at 20 m altitude above the sea level. Figure 12 shows the velocity magnitude distribution U in different cases. The top row panels (a), (b) and (c) correspond to the result at 16:10 JST of the simulation resumed from the original restart data. Panel (a) displays the entire slice while (b) and (c) zoom on selected sub-domains. The result of a restart with \(\varepsilon =10^{-6}\) is shown in Fig. 12d–f. Large-scale structures are essentially the same as in Fig. 12a–b. However, a careful comparison between panels (f) and (c) reveals significant differences on a smaller scale. To focus on such small-scale discrepancy, we calculate the difference between the velocity fields obtained with and without compression, \(|\varDelta U| = |U(\varepsilon =10^{-6}) - U(\varepsilon =0)|\), and display it in Fig. 12g–i on an exaggerated color scale. The darker tone of panels (g) and (h) suggests that \(|\varDelta U|\) is generally much smaller than U. There are, however, many bright spots around the buildings that mark the small-scale differences in the wake. A zoom on one of these spots displayed in Fig. 12i reveals that, locally, \(|\varDelta U|\) is of the same order of magnitude as U, in agreement with the global \(L^\infty\) error evaluations shown in Table 1. In addition, the error is spatially organized in a pattern characteristic of mixing layers. The vorticity plots in Fig. 13a–c show that \(\omega _z\) is small in the bulk of the fast current (lower bottom corner of the sub-domain), but many strong small-scale vortices are present in the mixing layer as well as in the slow current around the buildings. Although these small vortices show qualitatively similar arrangement in Fig. 13a as in 13b, the exact position differs by as much as the core size. Consequently, the error \(|\varDelta \omega _z| = |\omega _z(\varepsilon =10^{-6}) - \omega _z(\varepsilon =0)|\) is a superposition of strong well-localized peaks. It is worth mentioning that a restart with \(\varepsilon =10^{-12}\) has led to very similar results. This is expected as small-scale vortices shed from the buildings evolve rapidly and chaotically. Exact deterministic repetition of such simulation requires that the initial data be exact. On the other hand, the initial error has negligible effect on a kilometer scale.

We used Intel Advisor 2019 Update 4 (build 594873) for the performance analysis. WaveRange was built using gcc 7.4.0 with –O2 level optimization and AVX2 support, under the Ubuntu 18.04.1 operating system. The x-velocity component of the HIT dataset was read from a file in the FluSI HDF5 format, compressed with \(\epsilon =10^{-6}\) and written on disk also using HDF5. Then it was decompressed. The compression and the decompression programs have been profiled separately. Therefore, the results are presented in two columns in Table 2 and in two panels in Fig. 14.

In the compression procedure, the wavelet transform is the least time-consuming step. This can be explained by the relatively large fraction of vector operations in it: there are 12 vector loops and 12 scalar kernels shown in Fig. 14 with the red and the blue circles, respectively. Two of them achieve the L1 bandwidth bound. The need for strided memory access is the main factor that limits further optimization of the transform (these loops are marked with blue circles situated below the DRAM bandwidth line). In contrast, only 1 of the 8 range coder kernels has been vectorized, and the most time-consuming one is the function that encodes a symbol using frequencies. This explains why range coding is the most time-consuming step. Quantization and other parts of the program count 2 vector loops and 4 scalar kernels. In the quantization process, type conversion from double to char presents difficulties for the automatic optimization. The overall time in vectorized loops amounts to 11% of the total CPU time.

Decompression with WaveRange takes longer time than compression. Although the inverse wavelet transform has 15 vector and 6 scalar kernels, its execution takes longer than the forward transform used in the compression. Range decoding is by far the most expensive part, as it amounts to 61% of the decompression CPU time. Only 1 of the 10 range coder kernels has been vectorized. The most time-consuming function is the one that calculates cumulative frequency of the next symbol. Dequantization is relatively cheap: it only takes 13% of the total CPU time. The overall vector time ratio is 12%, which is nearly the same as in the compression program.

Baselines in terms of lossless compression achievable with general-purpose utilities are provided in Appendix E, showing the smallest relative compressed file size of 81.2%. This means that many-fold data reduction cannot be achieved without loss of information in this example.

The performance of different lossy compression methods, including WaveRange, is displayed in Fig. 15, which shows \(\varSigma\), \(\theta _c\) and \(\theta _d\) as functions of the \(L^\infty\) error norm. Three other lossy methods have been evaluated. Scale–offset is an HDF5 filter that performs a scale and offset operation, then truncates the result to a minimum number of bits. Scaling is performed by multiplication of each data value by 10 to the power of a given scale factor, which can be adjusted in order to reach the desired compressed file size or accuracy. SZ-1.4 (Tao et al. 2017) and ZFP (Diffenderfer et al. 2019) are two state-of-the-art scientific data compressors. The former is based on the Lorenzo predictor and the latter uses a custom transform that operates multi-dimensional blocks of small size. Both are implemented as HDF5 filters and both can operate in a fixed error mode. To obtain the plots in Fig. 15, we followed the same procedure as described in the previous sections: the HIT x-velocity data file was compressed, the file size was measured, then it was decompressed and the \(L^\infty\) error was measured. In addition, the execution time was measured for the compression and for the decompression separately.

Let us first discuss the compression performance in terms of \(\varSigma\). The scale–offset compression generally produces larger files than WaveRange. Note that this method is equivalent to quantization (3) in the limit of large \(L^\infty\) error. ZPF produces slightly larger files than WaveRange, except when the set tolerance is smaller than \(10^{-14}\) and the \(L^\infty\) error of WaveRange increases significantly due to round-off. The performance of SZ-1.4 varies largely depending on the control parameter configuration. In our test, we optimized it for the maximum compression ratio at a given error magnitude. Thus, the maximum quantization interval number was increased as the error decreased. With this setting, SZ-1.4 could produce smaller files than WaveRange, but only for the relative error greater than \(10^{-8}\). SZ-1.4 switched to lossless mode when the relative error smaller than \(10^{-10}\) was requested.

Considering the throughput \(\theta _c\) and \(\theta _d\), ZFP was generally the fastest in our tests, although it was outperformed by SZ-1.4 in certain cases by a small amount. WaveRange was 3 times slower than ZFP for the compression and 5 times slower for the decompression. This is not surprising, considering that the transform used in ZFP was optimized for the maximum throughput at the cost of a certain decrease in the compression ratio. Interestingly, despite the relatively high algorithmic complexity, WaveRange compression was faster than the scale–offset compression.