Liangzhe Li
ABSTRACT
A key factor of measuring database performance is query response time, which is dominated by I/O time. Database partitioning is among techniques that can help users reduce the I/O time significantly. However, how to efficiently partition tables in a database is not an easy problem, especially when we want to have this partitioning task done automatically by the system itself. This paper introduces an algorithm called Self-Managing Online Partitioner for Databases (SMOPD) in vertical partitioning based on closed item sets mining from a query set and system statistic information mined from system statistic views. This algorithm can dynamically monitor the database performance using user-configured parameters and automatically detect the performance trend so that it can decide when to perform a re-partitioning action without feedback from DBAs. This algorithm can free DBAs from the heavy tasks of keeping monitoring the system and struggling against the large statistic tables. The paper also presents the experimental results evaluating the performance of the algorithm using the TPC-H benchmark.
- Jindal, A., and Dittrich, J., Relax and Let the Database do the Partitioning Online. In BIRTE, 2011.Google Scholar
- Rao, J., Zhang, C., Megiddo, N., and Lohman, G. M., Automating Physical Database Design in a Parallel Database. In SIGMOD, page 558--569, 2002. Google ScholarDigital Library
- Guinepain, S. and Gruenwald, L., Using Cluster Computing to Support Automatic and Dynamic Database Clustering, IWAPT, 2008.Google ScholarCross Ref
- Li, L., and Gruenwald, L., Autonomous Database Partitioning Using Data Mining on Single Computers and Cluster Computers, IDEAS12, August 2012. Google ScholarDigital Library
- Pasquier, N., Bastidem, Y., Taouil, R. and Lakhal, L., Efficient Mining of Association Rules Using Closed Item set Lattices, Information Systems, Vol. 24, No. 1, 1999. Google ScholarDigital Library
- Frank, E. G., Procedures for Detecting Outlying Observations in Samples, Technometrics, Vol. 11, No. 1, pp. 1--21, February 1969.Google ScholarCross Ref
- McCormick, W. T. Schweitzer P. J., and White T. W., Problem Decomposition and Data Reorganization by A Clustering Technique, Operation Research, Vol. 20, No. 5, September 1972.Google Scholar
- Navathe, S., Ceri, S., Wierhold, G. and Dou, J., Vertical Partitioning Algorithms for Database Design, ACM Transactions on Database Systems, Vol. 9, No. 4, December 1984. Google ScholarDigital Library
- Wesley W. Chu and I. Ieong, A Transaction-Based Approach to Vertical Partitioning for Relational Database Systems, IEEE Transactions on Software Engineering, Vol. 19, No. 8, August 1993. Google ScholarDigital Library
- Horowitz, E. and Sahni, S., Fundamentals of Computer Algorithms, Rockville, MD: Computer Science Press, 1978. Google ScholarDigital Library
- Navathe, S. and Ra M., Vertical Partitioning for Database Design: A Graph Algorithm, ACM SIGMOD International Conference on Management of Data, 1989. Google ScholarDigital Library
- Berthuet R., Cours de Statistiques, CUST, Clermont-Ferrand, France, 1994.Google Scholar
- Papadomanolakis, S., Dash, D. and Ailamaki, A., Efficient Use of the Query Optimizer for Automated Physical Design, VLDB 2007, Proceedings of the 33rd International Conference Very Large Databases, September 2007. Google ScholarDigital Library
- Rodriguez, L. and Li, X., A Dynamic Vertical Partitioning Approach for Distributed Database System, Systems, Man, and Cybernetics (SMC), IEEE International Conference 2011.Google Scholar
- Abuelyaman, E., S., An Optimized Scheme for Vertical Partitioning of a Distributed Database, IJCSNS International Journal of Computer Science and Network Security, Vol. 8, No. 1, 2008.Google Scholar
- Navathe, S., Ceri, S., Wierhold, G. and Dou, J., Vertical Partitioning Algorithms for Database Design, ACM Transactions on Database Systems, Vol. 9, No. 4, December 1984. Google ScholarDigital Library
- Duan S., Thummala V., and Babu S., Tuning Database Configuration Parameters with Ituned, Proc. VLDB Endow., vol. 2, pp. 1246--1257, August 2009. Google ScholarDigital Library
- Rodd, S. F., and Kulkrani, U. P., Adaptive Tuning Algorithm for Performance tuning of Database Management System", International Journal of Computer Science and Information Security, Vol. 8, No. 1, April 2010.Google Scholar
- Schnaitter, K., Abiteboul, S., Milo, T., and Polyzotis, N., On-line Index Selection for Shifting Workloads. In International Workshop on Self-Managing Database Systems, pages 459--468, 2007. Google ScholarDigital Library
- http://www.tpc.org.Google Scholar
- Curino C., et al., Schism: a Workload-Driven Approach to Database Replication and Partitioning. In VLDB, 2010. Google ScholarDigital Library
- Schnaitter, K., and Polyzotis, N., Semi-Automatic Index Tuning: Keeping DBAs in The Loop. PVLDB, 5(5): 478--489, 2012. Google ScholarDigital Library
- Agrawal S., Chu E., and Narasayya V., Automatic Physical Design Tuning: Work-load as a Sequence. In SIGMOD, 2006. Google ScholarDigital Library
- Agrawal, S., et al. Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design. In SIGMOD, 2004. Google ScholarDigital Library
- Bruno, N., and Chaudhuri, S., Constrained Physical Design Tuning. PVLDB, 2008. Google ScholarDigital Library
Recommendations
NoSQL databases: MongoDB vs cassandra
C3S2E '13: Proceedings of the International C* Conference on Computer Science and Software EngineeringIn the past, relational databases were used in a large scope of applications due to their rich set of features, query capabilities and transaction management. However, they are not able to store and process big data effectively and are not very ...
Comments