The curse of indecomposable aggregates for big data exploratory analysis with a case for frequent pattern cubes

Exploratory big data analytics requires the interaction delays to be kept at minimum. Although data cubes help this goal by pre-calculating the measures of interest, some aggregations are not decomposable and require runtime scans through the cube data which will cause the response time to exceed the real-time interaction limits. One of such costly aggregations is the calculation of the frequent patterns over data cube partitions. The existing inefficient merge-and-count approach used for solving this problem is not feasible in the world of big data. In this paper, an efficient approach is proposed for mining frequent patterns from cube data accompanied by a formal overview of decomposable and indecomposable data aggregates. A new concept of semi-decomposable aggregates is introduced that sits in between these two extremes. With the case of frequent pattern mining problem, we show that sometimes indecomposable aggregates are in fact semi-decomposable and exploratory data analysis can still be realized for them. The proposed FPCubes algorithm shows promising experimental results for aggregating frequent patterns which can help exploratory frequent itemset analysis on real-world multidimensional big datasets.