Big Data Reduction Methods: A Survey

Network (also known as graph) theory is playing a primary role in reduction of high-dimensional unstructured big data into low-dimensional structured data [39]. However, the extraction of topological structures (networks) from big data is quite challenging due to the heterogeneity and complex data structures. The authors in [40] proposed network theory-based approach to extract the topological and dynamical network properties from big data. The topological networks are constructed by establishing and evaluating relationships (links) among different data points. The statistical node analysis of the networks is performed for optimization and big data reduction [41]. The optimized networks are represented as small-world networks, free-scale networks, and random networks and are ranked on the basis of statistical parameters, namely mean, standard deviation, and variance. Mathematically, scale-free networks are formally represented as given in Eq. 1 using the main parameter as shown in Eq. 2. where \( p_{k} \) represents fraction of nodes having \( k \) degree and parameter \( \gamma \) having range of \( 2 < \gamma < 3 \).

Similarly, formal representation of random networks is presented in Eq. 3 using the main parameter as shown in Eq. 4. where \( p \) is the probability distribution of edges between any two nodes, \( n \) shows the number of nodes and \( z \) is calculated as \( z = \left( {n - 1} \right)p \). The mathematical representation of small-world networks is performed using Eq. 5 with main parameter as shown in Eq. 6. where \( n \) represents the nodes in network and \( d \) is the distance between two randomly chosen nodes in the network.

Traditionally, causality analysis is performed to establish the connection between different nodes. However, the extraction of topological networks from unstructured data is hard due to high dependency and one-way relationship on the links. The alternate to causality analysis is the assessment of influence among different nodes and establishing the connection between them on the basis of their co-occurrence. Although the technique does not ensure that nodes in the network can influence each other, the assessment of their co-occurrence could become a strong relationship parameter to link different nodes. Preferential attachment property enables influence assessment [41]. The property exhibits that newly established connections are most probably made with highly connected nodes in the topological networks. In addition, the identification of influential nodes and effective assessment of their co-occurrence can significantly reduce big data. Mathematically, the co-occurrence-based influence measure \( \Delta \left( {v_{i1} ,v_{il} } \right) \) between two nodes \( v_{i1} \) and \( v_{il} \) is represented as shown in Eq. 7.where \( W_{{p_{i} }} = \frac{{w\left( {v_{i1} ,v_{i2} } \right)}}{{{ deg }\left( {v_{i1} } \right)}} \) and \( p\left( {v_{i1} ,v_{il} } \right) \) is the shortest path between \( v_{i1} \) and \( v_{il} \) and \( 0 \le \Delta \left( {v_{i1} ,v_{il} } \right) \le 1 \).

Aside from the influence-based assessment, similarity graph and collapsing simplicial complexes in network structures are used for topological network reduction in big datasets. Similarity graph is used to model the relationship among different datasets on the basis of their similarity matrix [17]. Further optimization of similarity graph is performed by merging every vertex with the maximal similarity clique (MSC), where a similarity clique (SC) is the collection of nodes with negligible distance. The notion is to convert a SC into MSC by adding an adjacent vertex that violates the properties of the SC. The graph is reduced when every MSC in the similarity graph is merged into a single node and the number of nodes and their edges are reduced in turn. Although the MSC is an efficient scheme to reduce topological networks, finding all of the MSCs in the similarity graph is quite challenging and computationally inefficient.

Moreover, the simplicial complexes are the multi-dimensional algebraic representations of large datasets [18]. The simplicial complex approach is applied over persistent homology, which is a method to compute topological features of large spaces at different levels of spatial resolutions. The persistent homology algorithm, as an alternate of hierarchical clustering, takes nested sequences of simplicial complexes and returns the summary of topological features at all levels. The concept of strong collapse that is keeping relevance information about all of the nodes in the simplicial complex is used to reduce the big data. The strong collapse is also useful for parallel and distributed computations, but the increasing computational complexities of maintaining relevance information of all of the nodes in complex prove to be the bottleneck. The concept of selective collapse algorithm is introduced to reduce the computational complexity of processing persistent homology. The proposed framework keeps computation traces of collapsing the complex using persistence algorithm proposed in [42]. In addition, the strong collapses are represented by forest that facilitates in easy processing of persistence across all nodes of the complex. Although the selective collapse algorithm is useful for the big data reduction, the empirical evidences are still lacking in the literature.

The reduced-size datasets are easy to handle in terms of processing and in-network data movement inside the big data storage systems. Compression-based methods are suitable candidates for data reduction in terms of size by preserving the whole data streams. Although computationally inefficient and involving decompression overhead, the methods allow to preserve the entire datasets in the original form. Numerous techniques for big data compression are proposed in the literature, including spatiotemporal compression, gzip, anamorphic stretch transform (AST), compressed sensing, parallel compression, sketching, and adaptive compression, to name a few [43‐45]. Table 1 presents the summary of these methods.

Big data reduction in cloud environments is quite challenging due to multiple levels of virtualization and heterogeneity in the underlying cloud infrastructure. A spatiotemporal technique for data compression on big graph data in the cloud generates reduced datasets [45]. The technique performs online clustering of streaming data by correlating similarities in their time series to share workload in the clusters. In addition, it performs temporal compression on each network node to reduce the overall data. The proposed technique effectively meets the data processing quality and acceptable fidelity loss of the most of the application requirements. On the other hand, wireless sensor networks (WSNs) are generating large data streams at massive scales. The spatiotemporal data compression algorithm proposed in [46] ensures efficient communication, transmission, and storage of data in WSNs-based big data environment. The proposed approach not only reduces the size of transmitted data but also ensures prolonged network lifetime. The algorithm measures the correlation degree of sensed data, which determines the content of the data to be transmitted.

The authors in [44] proposed an efficient big data reduction scheme for the IP-activity dataset in social science. The techniques are based on compression method to utilize the standalone computer machines instead of large-scale distributed systems, such as Hadoop and big table. The authors used eight core processors from a 32 core AMD Opteron Processor 5356 machine of 2.3 GHZ speed and 20 GB RAM. The 9 TB of loose text files was converted into a binary format for readability in social science settings. The methodology is based on three steps for: (1) information representation, (2) parallel processing, and (3) compression and storage. Big data reduction is performed in second and third steps. First, each of the data files is processed and converted into a corresponding HDFS file; then, map-reduce approach is used to link each individual HDFS file with the corresponding geographical locations and aggregate in lateral stages. In map-phase, the files are linked, whereas in the reduce stage, all of the resultant HDFS files are converted into a single HDFS file. Finally, the gzip algorithm was used with the compression level of five over the optimal data chunk size of 100,000 × 1. The results exhibited a significant amount of data reduction from 9.08 TB (raw data) to 3.07 TB (HDFS converted data) and 0.50 TB (compressed data). Although the proposed approach contributed significantly, it is only useful for the given dataset. Therefore, online data analysis for big data reduction remains challenging in streaming big data environments.

The anamorphic stretch transform (AST) stretches the sharp features more strongly as compared to the coarse features of the [47]. The AST could be applied to both the analogue (for digitization) and digital signals (for compression). The technique is primarily based on self-adaptive stretch where more samples are associated with sharp features and fewer samples are associated with redundant coarse features. The strength of the AST is its ability to enhance the utility of limited samples as well as reducing the overall size of the data. The results demonstrated that 56-fold data could be compressed using the AST equations (see Eq. 8 and Eq. 9).where \( \left( {\omega + \omega_{\text{m}} } \right) \) and \( \omega_{\text{m}} \) represent carrier and modulation frequencies, respectively. The \( \varphi \left( \omega \right) \) is an auto-correlation function of the AST with embedded kernel that represents a frequency-dependent phase operation. The compression or expansion of time, bandwidth product is dependent on the correct selection of \( \varphi \left( \omega \right) \). The structured modulation distribution function, \( S_{\text{M}} \), is used to select \( \varphi \left( \omega \right) \).where \( t \) represents the time variable.

The storage of big data and its complete processing for anomaly detection raises the issues of privacy due to exposure of each data point. The authors proposed compressed sensing also known as compressed sampling technique for compressible and/or sparse signals that project a high-dimensional data in low-dimensional space using random measurement matrix [48]. The method is selected because compressive sensing theory enables to address the limitations of Nyquist sampling [49] theory and can perform data acquisition and compression simultaneously. This strength of the compressed sensing theory enables to detect anomalies in big data streams. Mathematically, for a signal, \( x \in R^{N} \) compressed sensing can be represented using Eq. 10.where \( \emptyset \) shows the \( N \times N \) orthonormal transform basis and \( \theta \) is used as the expansion coefficient vector under the orthonormal basis. If \( x \in R^{K} \) sparse signal where \( K \ne 0 \) in vector \( \theta \) and \( K < N \), the signal \( x \) can be collected with a small set of non-adaptive and linear measurements (see Eq. 11).where \( \varPsi \) is a \( M \times N \) random measurement matrix and \( M < N \). Here, (\( \emptyset \), \( \varPsi \)) represents a pair of orthobases that follow the incoherence restriction.

Compressive sensing theory is useful because low-dimensional space fully represents high-dimensional data. The results showed that the proposed algorithm produced satisfactory results with and without compressed sensing. The compression was applied by the ratio of 1:3 and 1:5, and the experiments for human detection were performed from behind the brick and gypsum walls.

The authors in [50] proposed a novel file format called multi-scale electrophysiology format (MEF) to manage massively generated electrophysiology data. The block-wise lossy compression algorithm (RED encoding) is used in addition to the cyclic redundancy check (CRC), encryption, and block index structure of the MEF. The RED encoding ensures high lossless compression rate and higher computational speed and is also able to adopt with statistical variation in the raw data, which is very important for non-stationary ECG signals. The experimental results showed that for 32 KHZ recordings with each block of 32,556 samples acquired in one second, the MEF obtained reasonably better compression rate. The authors recommend recording at least 2000 samples in each block for the maximum performance gain.

The amount of data generated in vehicular ad hoc networks is massive due to on-board monitoring of vehicles and relevant spatiotemporal data acquisition. The authors in [46] utilized sketching algorithm called count-min sketch algorithm to compress vehicular movement data and achieved compact communication. The algorithm maps frequency counters to compressed vectors using hash tables. Although the main focus of the proposed study is on privacy preservation, data reduction is also performed significantly.

The large-scale scientific simulations generate a huge amount of data that widens the gap between the I/O capacity and computation abilities of high-end computing (HEC) machines [51]. This bottleneck for data analysis raises the need for in situ analytics where simulation data are processed prior to the I/O. Although feasible, the in situ analysis of peta-scale data incurs computation overhead in the HEC machines. The authors proposed adaptive compression service for in situ analytics middle-ware to effectively utilize available bandwidth and to optimize the performance of the HEC during end-to-end data transfer. Experimental results with gyro-kinetic simulation (GKW) on 80-node 1280 core cluster machine show that the compression ratio and available computational power are two main factors to achieve the maximum compression. The authors in [43] further proposed a framework called FlexAnalytics and profiled three compression algorithms called lossy [52], bzip2 [53], and gzip [54]. The experimental results show that all three compressions are not useful for optimized data transfer with bandwidth more than 264.19 Mb/s. This bottleneck imposes the challenge of compression/decompression time reduction to cope with the I/O needs of HEC.

Although compression-based data reduction methods are feasible for big data reduction, the processing overhead of de-compression introduces latency which lowers the performance of analytics algorithms in real-time environments. Moreover, the additional computations consume more cloud resources and increase the overall operational costs of big data systems. However, the techniques enable to store big data efficiently without significantly losing the original data in both the lossy and the lossless compression-based methods.

Data redundancy is the key issue for data analysis in big data environments. Three main reasons for data redundancy are: (1) addition of nodes, (2) expansion of datasets, and (3) data replication. The addition of a single virtual machine (VM) brings around 97% more redundancy, and the growth in large datasets comes with 47% redundant data points [13]. In addition, the storage mechanism for maximum data availability (also called data replication) brings 100% redundancy at the cluster level. Therefore, effective data deduplication and redundancy elimination methods can cope with the challenge of redundancy. The workload analysis shows that the 3× higher throughput improves performance about 45% but in some extreme cases the performance degrades up to 161%. The energy overhead of deduplication is 7%; however, the overall energy saved by processing deduplicated data is 43%. The performance is degraded to 5%, whereas energy overhead is 6% for pure solid state drive (SSD) environments. However, in hybrid environment the system’s performance is improved up to 17%.

Cluster deduplication is a generalized big data reduction scheme for disk-based cluster backup systems. The redundant data stored on multiple disks and partitions are a serious challenge for big data processing systems. The deduplication techniques allow to handle different data chunks (partitions) using hash functions to lower intra-node and inter-node communication overheads. In addition, these methods improve the storage efficiency by eliminating redundant data from multiple nodes. Large-scale cluster deduplication schemes face challenge of information-island (only server-level deduplication is possible due to the communication overhead) where data routing is the key issue. Another major challenge is disk-chunk-index-lookup (keeping duplicated chunk indexes of large datasets creates memory overheads), which degrades the performance of backup clients due to frequently random I/O for lookup and indexing.

Data deduplication schemes are based on either locality or similarity of data in the cluster. Locality-based approaches (stateful or stateless routing schemes) work on the location of duplicated data and perform optimization [14]. The major issue with the locality-based approach is the communication overhead of transferring similar data to same nodes. On the other hand, similarity-based schemes distribute the similar data to the same nodes across the cluster and reduce communication burden [56]. Although the schemes solve the problem of communication overhead, they prove ineffective for inter-node data deduplication system. To cope with challenges of communication overhead and ineffectiveness in inter-node deduplication systems, some hybrid techniques are also proposed in the recent literature. For example, SiLo [15] and \( \Sigma \)-Dedupe [12] used both the similarity- and locality-based techniques where SiLo addressed only the challenge of inter-node deduplication while \( \Sigma \)-Dedupe creates the balance between high deduplication and scalability across all of the nodes in the cluster. Although the cluster-level deduplication is effective for big data reduction, new deduplication methods are required to improve energy efficiency and resource awareness in large-scale data centers. The evaluation of \( \Sigma \)-Dedupe is performed in terms of efficiency analysis and normalized deduplication ratio using Eqs. 12 and 13. The duplication efficiency (DE) is measured using Eq. 12 as follows:where physical and logical size of datasets are denoted by \( P \) and \( L \), respectively, and \( T \) represents the processing time for deduplication. In addition, \( {\text{DT}} \) represents deduplication throughput and \( {\text{DR}} \) represents the deduplication ratio in the overall data. Similarly, the normalized effective deduplication ratio (NEDR) is used to measure the cluster-wide deduplication and storage imbalances collectively (see Eq. 13)

In Eq. 13, the \( {\text{CDR}} \) represents the cluster-level deduplication ratio and the \( {\text{SDR}} \) denotes single-node-level deduplication ratio. In addition, \( \alpha \) represents the average usage of storage while \( \sigma \) shows the standard deviation of cluster-wide storage usage.

The massive amount of data movement in data centers increases the computational and communicational burdens. The exploitation of in-network data processing techniques can reduce the aforementioned complexities. The authors of [57] proposed an in-network data processing technique for bandwidth reduction by customizing routing algorithms, eliminating network redundancy (by caching frequent packets), and reducing on-path data. The proposed technique performs partial data reduction and significantly improved throughput for in-network query processing.

In contrast, mobile users in the same locality or with the same habits generate similar data points causing a huge amount of redundant data in participatory big data environments. In addition, the absence of spatiotemporal correlation among sensory data in mobile opportunistic networks is also a great challenge. The authors in [58] proposed a cooperative sensing and data forwarding framework for mobile opportunistic networks where sampling redundancy is eliminated to save energy consumption. The authors proposed two data forwarding protocols [epidemic routing with fusion and binary spray and wait with fusion (BSWF)] by leveraging data fusion. The essence of the research is the intelligent fusion of sensory data to eliminate redundancy. The simulation results revealed that proposed method can remove 93% of redundancy in the data as compared to non-cooperative methods.

The issue of data duplication or redundancy has been addressed by researchers in different environments at different levels (mobile, cluster, cloud, and data center). Therefore, the selection of best method depends upon the application models. For example, in mobile opportunistic networks and mobile crowd sensing environments, the data redundancy elimination methods are best suited when they are deployed in mobile devices. Similarly, for scientific and highly correlated data deduplication is best suitable when it is performed at cluster, data center, and cloud level.

Data preprocessing is the second important phase of big data processing, and it must be preprocessed before storage at large-scale infrastructures [19]. This approach helps in big data reduction and also extracts the meta-data for further processing. The authors argue that primary approaches for data preprocessing are based on semantic analysis (using ontologies) or linked data structures (such as Google knowledge graph). However, this literature review uncovers few other techniques, such as low memory pre-filters for streaming data, URL filtration method, and map-reduce implementation of 2D peak detection methods in the big genomic data.

Low memory pre-filters are used for preprocessing genomic sequencing streaming data. The algorithm runs in a single pass and gives improved performance for error correction and lossy compression in data streams. In addition, the algorithm extracts the subsets of data streams using sketch-based techniques and applies pre-filtering algorithm for lossy compression and error correction. The algorithm first constructs the Bruijn graph, and the subsets are extracted using locus-specific graph analysis technique [59]. The massive data redundancy is handled using the \( k \)-mers median, and subsequently, digital normalization is employed as the data reduction technique. The authors argued that 95% of the data can be removed in the normal sequencing sample and the percentage reaches 98% of high-coverage single sequencing data. The results show that memory requirement for proposed algorithm is reduced from 3 TB to 300 GB of RAM.

Wearable sensors generate multi-dimensional, nonlinear, dynamic data streams with weak correlation between data points. The authors in [60] used locality-sensitive bloom filter to enhance the efficiency of instance-based learning for front-end data preprocessing near the sensing elements. The technique enables the filtration and communication of only the relevant and meaningful information to reduce computational and communication burden. The authors discussed the big healthcare data system for elderly patients and developed a prototype of the proposed solution. The architecture of the system is based upon a wearable sensor with bluetooth low energy (BLE) interface and can communicate with mobile application and/or PC to establish a personal area network (PAN). The mobile application processes the data and recognizes the state of the user. The sensor data and user states are further transmitted to a remote big data server through TCP/UDP ports. The compression algorithms are applied to incoming data streams, and resultant compressed files remain 10% of the actual data streams.

An application of big data reduction is the filtration of malicious URLs in Internet security applications. The authors in [21] proposed two feature reduction techniques that extract the lexical features and the descriptive features and then combine their results. The lexical features extract the structure of the URLs. However, the issue with lexical features is that malicious URL addresses have constantly changing behavior to abstain from malware detection software. The descriptive features are extracted to track and preserve different states of the same URL to label it as malicious. The authors selected passive-aggressive (for dense feature extraction) and confidence weighted algorithms (for sparse feature extraction) as the online learning algorithms and trained their models with extracted features [61, 62]. The prediction results of the filtration technique demonstrate around 75% data reduction with approximately 90% retention rate (inverse of data loss).

The analysis of large-scale proteomics data, which is the protein-level representation of big genomic data, requires massive computational resources to study different protein properties, such as expressions, changes in protein structures, and the interaction with other proteins. The protein molecules are too large to be identified by spectrometer and therefore are broken into smaller fragments called peptides. The mass spectrometer outputs the graphical output where each spectrum of data points is shown using Gaussian curves for peptide identification. The preprocessing step of proteomics data analysis is the identification of curves also called the 2D peaks. Each of the samples submitted to the spectrometer takes around 100 min to 4 h for complete analysis. During the passage of peptides, the spectrometer takes snapshots of spectrum every second where each peptide remains visible for several spectrums. The authors proposed a map-reduce implementations for proteomics data analysis where 2D peaks are picked at map level and further analyzed at reduce level [63]. The data reduction takes place at map level by applying preprocessing techniques for decoding the arrays, noise removal, and management of the overlapping peaks in the spectrum. Experimental results show that the given map-reduce implementation completes the data analysis in 22 min.

Recently light detection and ranging (LiDAR) technology enabled the generation of big 3D spatial data [64]. A cloud computing-based LiDAR processing system (CLiPS) processes big 3D spatial data effectively. The CLiPS uses several preprocessing techniques for data reduction to deal with large size of data. The data reduction is performed using a vertex decimation approach to provide a user’s preferred parameters to reduce the big data. The results show the advantage of cloud computing technologies over the conventional systems comparing performance and time consumption.

The literature review of these techniques reveals that data preprocessing techniques are highly dependent on the nature of big data and also encourage further investigation of the underlying problem. Therefore, these techniques could not be generalized for all types of big data streams.

Big data reduction is mainly considered to be the dimension reduction problem because the massive collection of big data streams introduces the ‘curse of dimensionality’ with millions of features (variables and dimensions) that increases the storage and computational complexity of big data systems [5]. A wide range of dimension reduction methods are proposed in the existing literature. The methods are based on clustering, map-reduce implementations of existing dimension reduction methods, feature selection techniques, and fuzzy logic implementations. Table 2 presents the summary of the above-mentioned methods.

The dynamic quantum clustering (DQC) enables powerful visualization of high-dimensional big data [8]. It outlines subsets of the data on the basis of density among all of the correlated variables in high-dimensional feature space. The DQC is scalable to very large systems due to its support for highly distributed data in parallel environments. The DQC is based on quantum mechanics techniques from physics. It works by constructing a potential proxy function to estimate the density of data points. The function named as parzen estimator, \( \emptyset \left( {\vec{x}} \right) \), is applied over \( n \)-dimensional feature space, and it is the sum of Gaussian functions centered at each data point, \( \vec{x} \) (see Eq. 14). The DQC next defines a vector function that satisfies the Schrodinger equation (see Eq. 15). Afterward, the DQC computes Gaussians functions from subsets using Hamiltonian operator defined in the potential function and multiplies the results by quantum-time evolution operator \( e^{ - i\delta tH} \) (where \( \delta t \) is set as small, \( i \) is the \( i \)th iteration, and \( H \) is the Hamiltonian distance). The DQC then computes the new center of each Gaussian and iterates the whole procedure. The results show that large and complex datasets could be analyzed using the DQC without any prior assumption about the number of clusters or using any expert information. In addition, the DQC could be applied to noisy data to identify and eliminate unimportant features to produce better results. This data reduction strategy makes DQC useful for big data analysis.

The potential function \( V\left( {\vec{x}} \right) \) can be defined over the same \( n \)-dimensional feature space and defined as the function for which \( \varphi \left( {\vec{x}} \right) \) satisfies the time-independent Schrodinger equation.

Conventional dimensionality reduction algorithms that use Gaussian maximum likelihood estimator could not handle the datasets with over 20,000 variables. The BIGQuic addresses the issue by applying a parallel divide-and-conquer strategy that can be applied up to 1-million variables in the feature space for dimensionality reduction [9]. The results show that the proposed algorithm is highly scalable and faster than the existing algorithms, such as Glasso and ALM [65, 66].

Knowledge discovery from high-dimensional big social image datasets is quite challenging. The authors proposed a new framework called twister which is a map-reduce implementation of k-means algorithm for dimensionality reduction [67]. The authors proposed a topology-aware pipeline-based method to accelerate broadcasting and to overcome the limitations of existing massively parallel infrastructure (MPI) implementations. In addition, the performance of the system was improved using local reduction techniques. This technique reduces local data before shuffling. The amount of data reduced is estimated using Eq. 16.

Normally, online learning techniques take the full feature set as the input, which is quite challenging when dealing with high-dimensional features space. The authors proposed an online feature selection (OFS) approach where the online learners only work on small and fixed-length feature sets. However, the selection of active features for accurate prediction is a key issue in the approaches presented in [68]. The authors investigated sparsity regularization and truncation techniques and proposed a new algorithm called the OFS. The results showed that the OFS outperformed RAND and \( {\text{PE}}_{\text{trun}} \) algorithms for UCI datasets and it works best in online learning mode as compared to batch-mode learning.

The corsets are the small set of points that represent the larger population of data and maintain the actual properties of overall population. These properties vary by nature of knowledge discovery algorithms. For example, the corsets representing first k-components maps with first k-components in the big data. Similarly, the corsets containing k-clusters with radius r approximate the big data and obtain the k-clusters with same r. In this way, the authors [11] applied corsets to reduce the big data into small and manageable size, which reduces the overall data complexity. The authors mapped corsets with k-means, principal component analysis (PCA) and projective clustering algorithms deployed with massively parallel streaming data [69, 70]. The big data is reduced in such a way that high dimensions of input space do not affect the cardinalities of corsets. In addition, the corsets are merged by maintaining the property that union of two corsets represents the reduced set of union of two big datasets. The experimental results showed that corsets are suitable to address NP-hard problems in massively parallel and streaming data environments where big data complexity is reduced by application of data processing algorithms on small datasets that are approximated from big data. In addition, the corsets are paradigm shift in big data analysis where the focus of research remains on big data reduction instead of improving the computational efficiency of existing algorithms.

Medical big data comes across several issues regarding extraction of structures, storage of massive data streams, and uncovering the useful knowledge patterns. Research shows that fuzzy classification methods are good choice to cope with the above-mentioned issues. Recently, the authors of [71] presented linguistic hedges fuzzy classifier with selected features (LHNFCSFs) to reduce dimensions, select features, and perform classification operations. The integration of linguistic hedges in adaptive neural-fuzzy classifier enhances the accuracy. The LHNFCSF reduces the feature space effectively and enhances the performance of the classifier by removing unimportant, noisy, or redundant features. The results depict that the LHNFCSF addresses the medical big data issues by reducing the dimensions of large datasets and speeding up the learning process and improves the classification performance.

Tensors are multi-dimensional representations of data elements with at least one extra dimension as compared to matrices. The increasing numbers of elements demand more computational power to process the tensors. Tensor processing works fine with small tensors. However, processing large tensors is a challenging task [10]. Tensor decomposition (TD) schemes are used to extract small but representative tensors from large tensors [72]. Three widely used TD strategies include canonical polyadic decomposition (CPD), tucker decomposition, and tensor trains (TT). The TD schemes represent the large tensors linked with their small representations. These decomposition schemes reduce the high dimensionality in big datasets and establish the interconnection among tensors to form tensor networks (TNs). These TNs enable to further reduce the data size by using optimization-based algorithms to find factor matrices and optimize using linear and nonlinear least square methods. The case studies show that tensor decomposition strategies could be used to alleviate/eliminate dimensionality in large scientific computing datasets and have many potential applications for feature extraction, cluster analysis, classification, data fusion, anomaly detection, pattern recognition, integration, time-series analysis, predictive modeling, multi-way component analysis, and regression.

The feature hashing (FH) method reduces feature dimensionality by randomly assigning each feature in the actual space to a new dimension in a lower-dimensional space [73]. This is done by simply hashing the ID of the original features. Usually, all dimensional reduction techniques degrade the data quality. However, most of them preserve the geometric qualities of the data. Alternately, the FH does not preserve the data quality. Research shows that the degradation of data quality is so minimal that its benefits are outweighed by the cost. The FH scales linearly with simple preprocessing and preservation of data sparsity, if exists. The scalability property of the FH makes it a natural choice for million (or even billion) feature datasets. For example, the FH method applied to email spam filtering shows its power when applied upon sparse and streaming data with real-time requirements of mass customization. The results show that the feature set is reduced from one billion to one million features.

Big data streams enter with episodic information and create high-dimensional feature spaces. Normally, the feature extraction methods need whole data in the memory that increases the computational complexity of big data systems and degrades the performance of classifiers. The incremental feature extraction methods are the best choice to handle such issues [74]. Incremental partial least squares (IPLSs) is a variant of the partial least squares method that effectively reduces dimensions from large-scale streaming data and improves the classification accuracy. The proposed algorithm works in two-stage feature extraction process. First, the IPLS adopts the target function to update the historical means and to extract the leading projection direction. Second, the IPLS calculates the rest of projection directions that are based on the equivalence between the Krylov sequence and the partial least square vectors [75]. The comparison of the IPLS was performed with incremental PCA algorithm, incremental inter-class scatter method, and incremental maximum margin criterion technique. The results revealed that the IPLS showed improved performance in terms of accuracy and computational efficiency.

Systems of systems (SoS)—case study The integration of heterogeneous and independent operating computing systems to collectively maximize the performance as compared to the individual settings leads toward the SoS [76]. Nowadays, SoS is contributing to generate big data and raises the need for data reduction. Few examples of statistical and computational intelligence tools for data reduction in SoS include the PCA, clustering, fuzzy-logic, neuro-computing, and evolutionary computing, such as genetic algorithms, and Bayesian networks. The authors applied data reduction methods at different stages of analyzing photovoltaic data that were collected from different sources. The original dataset contained 250 variables, which is highly dimensional and is not practical due to limitations of execution time and memory constraints on a desktop computer. Two approaches for data reduction at this stage were considered: (1) labeling the interesting attributes by domain expert and (2) development of an adaptive learning algorithm for automatic attribute selection. The authors employed the first approach for data reduction. The authors further cleaned-up the data and removed all invalid data points from the dataset. For example, solar irradiance in night hours generates data points with negative values, therefore not feasible for contribution in the study. After removing the invalid data, the data points containing very low values for global horizontal irradiance (GHI), direct normal irradiance (DNI), and direct horizontal irradiance (DHI) are removed to create more crispy data for further analysis.

The cleaned data are further fed into two nonparametric model generation tools, namely the fuzzy inference system generator and back-propagation neural network training tools using MATLAB fuzzy logic toolbox and the neural network toolbox. The initial evaluation of both of the tools revealed that the input variables should be further reduced for performance maximization in terms of execution time and memory consumption. The authors expanded the nonlinear data by using additional variables that in turn increased the performance of the training model but also increased time and space complexity. Therefore, the PCA is applied for dimension reduction to compress the data without significant information loss. After application of the PCA, further dimension reduction was performed using genetic algorithm (GA). First, the data were reduced using the GA on the full set of initial data and remaining data were expanded nonlinearly. Finally, the expanded dataset is used to train a neural network to assess the overall effectiveness of the GA. In practice, the time- and computation-related constraints were limited to the selection of training data to first 1000 samples. The first iteration of GA took initially 244 samples and reduced it to 74. The results showed that the GA reduced the number of attributes up to 70%.

Recently, several DM and ML methods have also been proposed for big data reduction. The methods are either applied to reduce data immediately after its acquisition or customized to address some specific problems. For example, the authors [78] proposed a context-aware big data forwarding scheme using distributed wearable sensors. The scheme is based on hidden markov model (HMM) to enable context-aware features in the distributed sensor model and forwards only relevant information when there is some significant change in the data [78]. The proposed scheme reduces the communication and storage overhead in big data environment. The authors compared the result of proposed locality-sensitive hash (LSH)-based method with linear perception (LP)-based dimensionality reduction algorithm and argued on the effectiveness of the proposed scheme as compared to the LP-based dimensionality reduction methods.

The problem of mining uncertain big data due to existential probabilities becomes worse and requires huge efforts and computational power to explore the incrementally growing uncertain search space. Therefore, the search space is needed to be reduced for uncovering the maximum certain and useful patterns. The authors of [79] proposed a map-reduce algorithm to reduce the search space and mine frequent patterns from uncertain big data. The algorithm facilitates the users to confine their search space by setting some succinct anti-monotone (SAM) constraints for data analysis and subsequently mines the uncertain big data to uncover frequent patterns that satisfy the user-specified SAM constraints. The input of the algorithm is uncertain big data, user-specified minimum support (minsup) threshold, and the SAM constraints. Two sets of map-reduce implementations are used to uncover singleton and non-singleton frequent patterns. Experimental results show that the user-specified SAM (termed as selectivity) is directly proportional to multiple parameters which are derived from algorithm’s runtime, the pairs returned by map function, the pairs sorted and shuffled by reduce function, and the required constraints checks.

Artificial intelligence methods, for example, artificial neural networks (ANNs) have also potential for big data reduction and compression. The authors in [80] proposed a self-organized Kohonen network-based method to reduce big hydrographic data acquired from the deep seas. The proposed system first converts the raw data into ‘swath’ files using a combination of four filters: (1) limit filter, (2) amplitude filter, (3) along track filter, and (4) across track filter. The appropriate combinations of the filters ensure the optimal dataset size. Despite filtering, the sample size is still large to be considered as big data. The self-organized Kohonen networks are trained and optimized using filtered hydrographic data to cluster the incoming data streams. The experimental results exhibited the feasibility of self-organized Kohonen networks for big hydrographic data for further analysis.

In addition to conventional machine learning algorithms, based on shallow learning models, deep learning is creating space as an option for big data reduction methods [81]. Deep learning models are hierarchical representations of supervised and unsupervised classification methods that are best suitable for large-scale, high-dimensional big data streams [22]. Deep learning models become computationally inefficient with the increase in big data complexity. However, the availability of MPIs (clusters/clouds) can address the aforementioned issue. Conventionally, deep learning models work at multiple layers with different granularities of information and predictive abilities. Two well-established deep learning architectures are deep belief networks (DBNs) and convolutional neural networks (CNNs). The DBN learning models are developed in two stages: (1) the initial models are developed using unsupervised learning methods with unlabeled data streams and (2) the models are fine-tuned using supervised learning methods and labeled data streams. The typical architecture of the DBN in Fig. 2 shows the multilayer representation of the deep learning model. The architecture is based on an input and output layer with multiple intermediate hidden layers. The output of each \( \left( {n - 1} \right) \)th layer becomes the input of the nth layer in the architecture. In addition, the learning models are fine-tuned using back-propagation methods to support generative performance and judicial power of the DBNs. Although the CNNs are based on learning models, they differ from the DBNs. The CNNs layers are either the convolutional layers to support the convolution of several input filters or subsampling layers to reduce the size of output variable from previous layers. Effective utilization of these deep learning models in conjunction with MPIs can significantly reduce big data streams.

Deep learning models are inherently computationally complex and require many-core CPUs and large-scale computational infrastructures. Some recent learning approaches for such large-volume, complex data include locally connected networks [82, 83], improved optimizers, and deep stacking networks (DSNs). The authors of [84] proposed the hybrid deep learning model, called DisBelief, to address the issue of high dimensionality in big data streams. Disbelief utilizes a large-scale cluster to parallelize both the data and the learning models using synchronization, message passing, and multi-threading techniques. The DisBelief model first achieves parallelism by partitioning large-scale networks into small blocks that are mapped to a single node and then achieves data parallelism using two separate distribution optimization procedures called stochastic gradient descent (SGD) for online optimization and sandblaster for batch optimization. Although feasible for big data reduction, the deep learning models are resource hungry and require MPIs based on clusters of CPUs or GPUs. Therefore, there is a need to develop optimized deep learning strategies to achieve resource efficiency and reduce communication burdens inside the MPIs.

The wide spectrum view of the proposed methods for big data reduction uncovers the fact that the research on big data reduction methods is being carried out at several levels of big data architecture and in different forms.