Justin DeBrabant
Michael Stonebraker
Abstract
The traditional wisdom for building disk-based relational database management systems (DBMS) is to organize data in heavily-encoded blocks stored on disk, with a main memory block cache. In order to improve performance given high disk latency, these systems use a multi-threaded architecture with dynamic record-level locking that allows multiple transactions to access the database at the same time. Previous research has shown that this results in substantial overhead for on-line transaction processing (OLTP) applications [15].
The next generation DBMSs seek to overcome these limitations with architecture based on main memory resident data. To overcome the restriction that all data fit in main memory, we propose a new technique, called anti-caching, where cold data is moved to disk in a transactionally-safe manner as the database grows in size. Because data initially resides in memory, an anti-caching architecture reverses the traditional storage hierarchy of disk-based systems. Main memory is now the primary storage device.
We implemented a prototype of our anti-caching proposal in a high-performance, main memory OLTP DBMS and performed a series of experiments across a range of database sizes, workload skews, and read/write mixes. We compared its performance with an open-source, disk-based DBMS optionally fronted by a distributed main memory cache. Our results show that for higher skewed workloads the anti-caching architecture has a performance advantage over either of the other architectures tested of up to 9× for a data size 8× larger than memory.
- eXtremeDB. http://www.mcobject.com.Google Scholar
- H-Store. http://hstore.cs.brown.edu.Google Scholar
- MemSQL. http://www.memsql.com.Google Scholar
- VoltDB. http://www.voltdb.com.Google Scholar
- Oracle TimesTen Products and Technologies. Technical report, February 2007.Google Scholar
- P. M. G. Apers, C. A. van den Berg, J. Flokstra, P. W. P. J. Grefen, M. L. Kersten, and A. N. Wilschut. PRISMA/DB: A parallel, main memory relational DBMS. IEEE Trans. on Knowl. and Data Eng., 4(6):541-554, 1992. Google Scholar
- J. Baulier, P. Bohannon, S. Gogate, S. Joshi, C. Gupta, A. Khivesera, H. F. Korth, P. McIlroy, J. Miller, P. P. S. Narayan, M. Nemeth, R. Rastogi, A. Silberschatz, and S. Sudarshan. Datablitz: A high performance main-memory storage manager. VLDB, pages 701-, 1998. Google Scholar
- Q. Chen, M. Hsu, and R. Wu. MemcacheSQL a scale-out sql cache engine. In Enabling Real-Time Business Intelligence, volume 126 of Lecture Notes in Business Information Processing, pages 23-37. 2012.Google Scholar
- B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with YCSB. In SoCC, pages 143-154, 2010. Google Scholar
- C. A. Curino, D. E. Difallah, A. Pavlo, and P. Cudre-Mauroux. Benchmarking OLTP/Web Databases in the Cloud: The OLTP-Bench Framework. CloudDB, pages 17-20, October 2012. Google Scholar
- D. J. DeWitt, R. H. Katz, F. Olken, L. D. Shapiro, M. R. Stonebraker, and D. Wood. Implementation techniques for main memory database systems. SIGMOD Rec., 14(2):1-8, 1984. Google Scholar
- C. Diaconu, C. Freedman, E. Ismert, P.-A. Larson, P. Mittal, R. Stonecipher, N. Verma, and M. Zwilling. Hekaton: SQL Server's Memory-Optimized OLTP Engine. In SIGMOD, pages 1-12, 2013. Google Scholar
- F. Färber, S. K. Cha, J. Primsch, C. Bornhövd, S. Sigg, and W. Lehner. Sap hana database: data management for modern business applications. SIGMOD Rec., 40(4):45-51, Jan. 2012. Google Scholar
- B. Fitzpatrick. Distributed Caching with Memcached. Linux J., 2004(124):5-, Aug. 2004. Google Scholar
- S. Harizopoulos, D. J. Abadi, S. Madden, and M. Stonebraker. OLTP through the looking glass, and what we found there. In SIGMOD, pages 981-992, 2008. Google Scholar
- P. Helland, H. Sammer, J. Lyon, R. Carr, P. Garrett, and A. Reuter. Group commit timers and high volume transaction systems. In Proceedings of the 2nd International Workshop on High Performance Transaction Systems, pages 301-329, 1989. Google Scholar
- H. V. Jagadish, D. F. Lieuwen, R. Rastogi, A. Silberschatz, and S. Sudarshan. Dalí: A high performance main memory storage manager. In VLDB, pages 48-59, 1994. Google Scholar
- R. Kallman, H. Kimura, J. Natkins, A. Pavlo, A. Rasin, S. Zdonik, E. P. C. Jones, S. Madden, M. Stonebraker, Y. Zhang, J. Hugg, and D. J. Abadi. H-Store: A High-Performance, Distributed Main Memory Transaction Processing System. Proc. VLDB Endow., 1(2):1496-1499, 2008. Google Scholar
- A. Kemper and T. Neumann. HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. ICDE, pages 195-206, 2011. Google Scholar
- J. J. Levandoski, P.-A. Larson, and R. Stoica. Identifying hot and cold data in main-memory databases. In ICDE, 2013.Google Scholar
- K. Li and J. F. Naughton. Multiprocessor main memory transaction processing. DPDS, pages 177-187, 1988. Google Scholar
- N. Malviya, A. Weisberg, S. Madden, and M. Stonebraker. Recovery algorithms for in-memory OLTP databases. In Submission, 2013.Google Scholar
- O. Mutlu, H. Kim, and Y. N. Patt. Techniques for efficient processing in runahead execution engines. ISCA, pages 370-381, 2005. Google Scholar
- M. A. Olson, K. Bostic, and M. Seltzer. Berkeley DB. USENIX ATEC, 1999. Google Scholar
- J. Ousterhout, P. Agrawal, D. Erickson, C. Kozyrakis, J. Leverich, D. Mazières, S. Mitra, A. Narayanan, G. Parulkar, M. Rosenblum, S. M. Rumble, E. Stratmann, and R. Stutsman. The case for ramclouds: scalable high-performance storage entirely in dram. SIGOPS Oper. Syst. Rev., 43(4):92-105, Jan. 2010. Google Scholar
- A. Pavlo, C. Curino, and S. Zdonik. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In SIGMOD, pages 61-72, 2012. Google Scholar
- D. R. K. Ports, A. T. Clements, I. Zhang, S. Madden, and B. Liskov. Transactional consistency and automatic management in an application data cache. OSDI'10, pages 1-15, 2010. Google Scholar
- R. Stoica and A. Ailamaki. Enabling efficient os paging for main-memory oltp databases. In DaMon, 2013. Google Scholar
- M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The end of an architectural era: (it's time for a complete rewrite). In VLDB, pages 1150-1160, 2007. Google Scholar
- M. Stonebraker and L. A. Rowe. The design of POSTGRES. SIGMOD, pages 340-355, 1986. Google Scholar
- T. Team. In-memory data management for consumer transactions the timesten approach. SIGMOD '99, pages 528-529, 1999. Google Scholar
- The Transaction Processing Council. TPC-C Benchmark (Revision 5.9.0). http://www.tpc.org/tpcc/, June 2007.Google Scholar
- A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, and D. J. Abadi. Calvin: fast distributed transactions for partitioned database systems. In SIGMOD, pages 1-12, 2012. Google Scholar
- A. Whitney, D. Shasha, and S. Apter. High Volume Transaction Processing Without Concurrency Control, Two Phase Commit, SQL or C++. In HPTS, 1997.Google Scholar
Index Terms
- Anti-caching: a new approach to database management system architecture
Recommendations
DCD—disk caching disk: a new approach for boosting I/O performance
ISCA '96: Proceedings of the 23rd annual international symposium on Computer architectureThis paper presents a novel disk storage architecture called DCD, Disk Caching Disk, for the purpose of optimizing I/O performance. The main idea of the DCD is to use a small log disk, referred to as cache-disk, as a secondary disk cache to optimize ...
Mid-tier caching: the TimesTen approach
SIGMOD '02: Proceedings of the 2002 ACM SIGMOD international conference on Management of dataTimesTen is an in-memory, application-tier data manager that delivers low response time and high throughput. Applications may create tables and manage them exclusively in TimesTen, and they may optionally cache frequently used subsets of a disk-based ...
Comments