Hasso Plattner
Abstract
Five years ago I proposed a common database approach for transaction processing and analytical systems using a columnar in-memory database, disputing the common belief that column stores are not suitable for transactional workloads. Today, the concept has been widely adopted in academia and industry and it is proven that it is feasible to run analytical queries on large data sets directly on a redundancy-free schema, eliminating the need to maintain pre-built aggregate tables during data entry transactions. The resulting reduction in transaction complexity leads to a dramatic simplification of data models and applications, redefining the way we build enterprise systems. First analyses of productive applications adopting this concept confirm that system architectures enabled by in-memory column stores are conceptually superior for business transaction processing compared to row-based approaches. Additionally, our analyses show a shift of enterprise workloads to even more read-oriented processing due to the elimination of updates of transaction-maintained aggregates.
- D. J. Abadi, S. R. Madden, and N. Hachem. Column-Stores vs. Row-Stores: How Different Are They Really? ACM, 2008.Google Scholar
Digital Library
- P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency control and recovery in database systems. Boston, MA, USA, 1986. Google ScholarDigital Library
- G. P. Copeland and S. N. Khoshafian. A decomposition storage model. SIGMOD, 1985. Google ScholarDigital Library
- M. Faust, D. Schwalb, J. Krüger, and H. Plattner. Fast lookups for in-memory column stores: Group-key indices, lookup and maintenance. In ADMS@VLDB, 2012.Google Scholar
- M. Grund, J. Krueger, H. Plattner, A. Zeier, P. Cudre-Mauroux, and S. Madden. HYRISE---A Main Memory Hybrid Storage Engine. VLDB, 2010. Google ScholarDigital Library
- S. Idreos, F. Groffen, N. Nes, S. Manegold, K. S. Mullender, and M. L. Kersten. Monetdb: Two decades of research in column-oriented database architectures. IEEE Data Eng. Bull., 2012.Google Scholar
- T. Karnagel, R. Dementiev, R. Rajwar, K. Lai, T. Legler, B. Schlegel, and W. Lehner. Improving in-memory database index performance with intel transactional synchronization extensions. HPCA, 2014.Google Scholar
- M. Kaufmann, P. Vagenas, P. M. Fischer, D. Kossmann, and F. Färber. Comprehensive and interactive temporal query processing with SAP HANA. VLDB, 2013. Google ScholarDigital Library
- A. Kemper and T. Neumann. HyPer: A hybrid OLTP&OLAP Main Memory Database System based on Virtual Memory Snapshots. ICDE, 2011. Google ScholarDigital Library
- J. Krüger, C. Kim, M. Grund, N. Satish, D. Schwalb, J. Chhugani, P. Dubey, H. Plattner, and A. Zeier. Fast updates on read-optimized databases using multi-core cpus. VLDB, 2011.Google ScholarDigital Library
- P.-A. Larson, S. Blanas, C. Diaconu, C. Freedman, J. M. Patel, and M. Zwilling. High-performance concurrency control mechanisms for main-memory databases. VLDB, 2011. Google ScholarDigital Library
- R. MacNicol and B. French. Sybase IQ Multiplex - Designed for Analytics. VLDB, 2004. Google ScholarDigital Library
- C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. TODS, 1998. Google ScholarDigital Library
- T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, and T. Neumann. ScyPer: elastic OLAP throughput on transactional data. DanaC, 2013.Google ScholarDigital Library
- S. Müller and H. Plattner. Aggregates caching in columnar in-memory databases. IMDM@VLDB, 2013.Google Scholar
- H. Plattner. A common database approach for oltp and olap using an in-memory column database. SIGMOD, 2009. Google ScholarDigital Library
- H. Plattner. SanssouciDB: An In-Memory Database for Processing Enterprise Workloads. BTW, 2011.Google Scholar
- D. Schwalb, M. Faust, and J. Krüger. Leveraging in-memory technology for interactive analyses of point-of-sales data. ICDEW, 2014.Google ScholarCross Ref
- V. Sikka, F. Färber, W. Lehner, S. K. Cha, T. Peh, and C. Bornhövd. Efficient Transaction Processing in SAP HANA Database: The End of a Column Store Myth. SIGMOD, 2012. Google ScholarDigital Library
- D. Ślȩzak, J. Wróblewski, and V. Eastwood. Brighthouse: an analytic data warehouse for ad-hoc queries. VLDB, 2008.Google Scholar
- M. Stonebraker, D. J. Abadi, A. Batkin, X. Chen, M. Cherniack, M. Ferreira, E. Lau, A. Lin, S. Madden, E. O'Neil, P. O'Neil, A. Rasin, N. Tran, and S. Zdonik. C-store: A column-oriented dbms. VLDB, 2005. Google ScholarDigital Library
- C. Tinnefeld, S. Müller, H. Kaltegärtner, and S. Hillig. Available-To-Promise on an In-Memory Column Store. BTW, 2011.Google Scholar
- Transaction Processing Performance Council (TPC). TPC-C Benchmark. http://www.tpc.org/tpcc/.Google Scholar
- T. Willhalm, N. Popovici, Y. Boshmaf, H. Plattner, A. Zeier, and J. Schaffner. SIMD-Scan: Ultra Fast in-Memory Table Scan Using on-Chip Vector Processing Units. VLDB, 2009. Google ScholarDigital Library
- J. Wust, J.-H. Boese, F. Renkes, S. Blessing, J. Krueger, and H. Plattner. Efficient logging for enterprise workloads on column-oriented in-memory databases. CIKM, 2012. Google ScholarDigital Library
- J. Wust, M. Grund, K. Höwelmeyer, D. Schwalb, and H. Plattner. Concurrent execution of mixed enterprise workloads on in-memory databases. DASFAA, 2014.Google ScholarCross Ref
Index Terms
- The impact of columnar in-memory databases on enterprise systems: implications of eliminating transaction-maintained aggregates
Recommendations
Assessing the Suitability of In-Memory Databases in an Enterprise Context
ES '15: Proceedings of the 2015 International Conference on Enterprise SystemsIt is still not fully clear if the increased query execution speed offered by in-memory databases unfolds its potential benefits over traditional disk-based databases in an enterprise context. This paper aims at comparing the performance of in-memory ...
Aggregation strategies for columnar in-memory databases in a mixed workload
PIKM '11: Proceedings of the 4th workshop on Workshop for Ph.D. students in information & knowledge managementThe recent trend towards analytics on operational data has led to an approach of reunifying online transactional processing and online analytical processing in one single database. The advent of columnar in-memory databases makes this viable and ...
Interoperable Data Migration between NoSQL Columnar Databases
EDOCW '14: Proceedings of the 2014 IEEE 18th International Enterprise Distributed Object Computing Conference Workshops and DemonstrationsNoSQL databases have emerged as the solution to handle large quantities of user-generated contents still guaranteeing fault tolerance, availability and scalability. Each NoSQL database offers differentiated properties and characteristics as well as ...
Comments