Hasso Plattner
ABSTRACT
When SQL and the relational data model were introduced 25 years ago as a general data management concept, enterprise software migrated quickly to this new technology. It is fair to say that SQL and the various implementations of RDBMSs became the backbone of enterprise systems. In those days. we believed that business planning, transaction processing and analytics should reside in one single system. Despite the incredible improvements in computer hardware, high-speed networks, display devices and the associated software, speed and flexibility remained an issue.
The nature of RDBMSs, being organized along rows, prohibited us from providing instant analytical insight and finally led to the introduction of so-called data warehouses. This paper will question some of the fundamentals of the OLAP and OLTP separation. Based on the analysis of real customer environments and experience in some prototype implementations, a new proposal for an enterprise data management concept will be presented.
In our proposal, the participants in enterprise applications, customers, orders, accounting documents, products, employees etc. will be modeled as objects and also stored and maintained as such. Despite that, the vast majority of business functions will operate on an in memory representation of their objects. Using the relational algebra and a column-based organization of data storage will allow us to revolutionize transactional applications while providing an optimal platform for analytical data processing. The unification of OLTP and OLAP workloads on a shared architecture and the reintegration of planning activities promise significant gains in application development while simplifying enterprise systems drastically.
The latest trends in computer technology -- e.g. blade architecture, multiple CPUs per blade with multiple cores per CPU allow for a significant parallelization of application processes. The organization of data in columns supports the parallel use of cores for filtering and aggregation. Elements of application logic can be implemented as highly efficient stored procedures operating on columns. The vast increase in main memory combined with improvements in L1--, L2--, L3--caching, together with the high data compression rate column storage will allow us to support substantial data volumes on one single blade. Distributing data across multiple blades using a shared nothing approach provides further scalability.
- D.J. Abadi, S. Madden, and M. Ferreira. Integrating Compression and Execution in Column-Oriented Database Systems. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Chicago, Illinois, USA, June 27-29, 2006, pages 671--682. ACM, 2006. Google Scholar
Digital Library
- S. Aulbach, T. Grust, D. Jacobs, A. Kemper, and J. Rittinger. Multi-Tenant Databases for Software as s Service: Schema--Mapping Techniques. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008, pages 1195--1206. ACM, 2008. Google ScholarDigital Library
- H. Berenson, P.A. Bernstein, J. Gray, J. Melton, E.J. O'Neil, and P.E. O'Neil. A Critique of ANSI SQL Isolation Levels. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, May 22-25, 1995, pages 1--10. ACM Press, 1995. Google ScholarDigital Library
- A. Bog, J. Krueger, and J. Schaner. A Composite Benchmark for Online Transaction Processing and Operational Reporting. In IEEE Symposium on Advanced Management of Information for Globalized Enterprises, 2008.Google ScholarCross Ref
- P. Boncz. Monet: A Next-Generation DBMS Kernel for Query-Intensive Applications. 2002. PhD Thesis, Universiteit van Amsterdam, Amsterdam, The Netherlands.Google Scholar
- P.A. Boncz, S. Manegold, and M.L. Kersten. Database Architecture Optimized for the New Bottleneck: Memory Access. In VLDB'99, Proceedings of 25th International Conference on Very Large Data Bases, September 7-10, 1999, Edinburgh, Scotland, UK, pages 54--65. Morgan Kaufmann, 1999. Google ScholarDigital Library
- S. Chaudhuri and U. Dayal. An Overview of Data Warehousing and OLAP Technology. SIGMOD Record, 26(1):65--74, 1997. Google ScholarDigital Library
- G.P. Copeland and S. Khoshafian. A Decomposition Storage Model. In Proceedings of the 1985 ACM SIGMOD International Conference on Management of Data, Austin, Texas, May 28-31, 1985, pages 268--279. ACM Press, 1985. Google ScholarDigital Library
- C.D. French. One Size Fits All -- Database Architectures Do Not Work for DDS. In Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, May 22-25, 1995, pages 449--450. ACM Press, 1995. Google ScholarDigital Library
- B. Gates. Information At Your Fingertips. Keynote address, Fall/COMDEX, Las Vegas, Nevada, November 1994.Google Scholar
- J. Gray. Tape is Dead. Disk is Tape. Flash is Disk, RAM Locality is King. Storage Guru Gong Show, Redmon, WA, 2006.Google Scholar
- J.L. Hennessy and D.A. Patterson. Computer Architecture -- A Quantitative Approach. Morgan Kaufmann, fourth edition, 2007. Google ScholarDigital Library
- W.H. Inmon. Building the Data Warehouse, 3rd Edition. John Wiley&Sons, Inc., New York, NY, USA, 2002. Google ScholarDigital Library
- G. Koch. Discovering Multi-Core: Extending the Benefits of Moore's Law. Technology@Intel, (7), 2005.Google Scholar
- D. Majumdar. A Quick Survey of MultiVersion Concurrency Algorithms, 2007. http://simpledbm.googlecode.com/~les/mvcc-survey-1.0.pdf.Google Scholar
- G.E. Moore. Cramming More Components Onto Integrated Circuits. Electronics, 38(8), 1965.Google Scholar
- V. Raman, G. Swart, L. Qiao, F. Reiss, V. Dialani, D. Kossmann, I. Narang, and R. Sidle. Constant-Time Query Processing. In Proceedings of the 24th International Conference on Data Engineering, ICDE 2008, April 7-12, 2008, Cancun, M´exico, pages 60--69. IEEE, 2008. Google ScholarDigital Library
- J. Schaner, A. Bog, J. Krueger, and A. Zeier. A Hybrid Row-Column OLTP Database Architecture for Operational Reporting. In Proceedings of the Second International Workshop on Business Intelligence for the Real-Time Enterprise, BIRTE 2008, in conjunction with VLDB'08, August 24, 2008, Auckland, New Zealand, 2008.Google Scholar
- M. Stonebraker. The Case for Shared Nothing. IEEE Database Engineering Bulletin, 9(1):4--9, 1986.Google Scholar
- M. Stonebraker, D.J. Abadi, A. Batkin, X. Chen, M. Cherniack, M. Ferreira, E. Lau, A. Lin, S. Madden, E.J. O'Neil, P.E. O'Neil, A. Rasin, N. Tran, and S. B. Zdonik. C-Store: A Column-oriented DBMS. In Proceedings of the 31st International Conference on Very Large Data Bases, Trondheim, Norway, August 30 -- September 2, 2005, pages 553--564. ACM, 2005. Google ScholarDigital Library
- M. Stonebraker, L.A. Rowe, and M. Hirohama. The Implementation of Postgres. IEEE Transactions on Knowledge and Data Engineering, 2(1):125--142, 1990. Google ScholarDigital Library
Index Terms
- A common database approach for OLTP and OLAP using an in-memory column database
Recommendations
Column-oriented database systems
Column-oriented database systems (column-stores) have attracted a lot of attention in the past few years. Column-stores, in a nutshell, store each database table column separately, with attribute values belonging to the same column stored contiguously, ...
Model Transformation From Object Relational Database to NoSQL Column Based Database
NISS '20: Proceedings of the 3rd International Conference on Networking, Information Systems & SecurityNoSQL databases play an important role in saving a huge amount of data. To benefit from the advantages of horizontal scalability and flexibility and with the fast data growing many companies are now replacing their traditional database management ...
Comments