Sudipto Das
Divyakant Agrawal
Amr El Abbadi
Abstract
A database management system (DBMS) serving a cloud platform must handle large numbers of application databases (or tenants) that are characterized by diverse schemas, varying footprints, and unpredictable load patterns. Scaling out using clusters of commodity servers and sharing resources among tenants (i.e., multitenancy) are important features of such systems. Moreover, when deployed on a pay-per-use infrastructure, minimizing the system's operating cost while ensuring good performance is also an important goal. Traditional DBMSs were not designed for such scenarios and hence do not possess the mentioned features critical for DBMSs in the cloud.
We present ElasTraS, which combines three design principles to build an elastically-scalable multitenant DBMS for transaction processing workloads. These design principles are gleaned from a careful analysis of the years of research in building scalable key-value stores and decades of research in high performance transaction processing systems. ElasTraS scales to thousands of tenants, effectively consolidates tenants with small footprints while scaling-out large tenants across multiple servers in a cluster. ElasTraS also supports low-latency multistep ACID transactions, is fault-tolerant, self-managing, and highly available to support mission critical applications. ElasTraS leverages Albatross, a low overhead on-demand live database migration technique, for elastic load balancing by adding more servers during high load and consolidating to fewer servers during usage troughs. This elastic scaling minimizes the operating cost and ensures good performance even in the presence of unpredictable changes to the workload.
We elucidate the design principles, explain the architecture, describe a prototype implementation, present the detailed design and implementation of Albatross, and experimentally evaluate the implementation using a variety of transaction processing workloads. On a cluster of 20 commodity servers, our prototype serves thousands of tenants and serves more than 1 billion transactions per day while migrating tenant databases with minimal overhead to allow lightweight elastic scaling. Using a cluster of 30 commodity servers, ElasTraS can scale-out a terabyte TPC-C database serving an aggregate throughput of approximately one quarter of a million TPC-C transactions per minute.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, ElasTraS: An elastic, scalable, and self-managing transactional database for the cloud
- Aguilera, M. K., Merchant, A., Shah, M., Veitch, A., and Karamanolis, C. 2007. Sinfonia: A new paradigm for building scalable distributed systems. In Proceedings of the 21st ACM Symposium on Operating System Principles. ACM, New York, NY, 159--174. Google Scholar
Digital Library
- Baker, J., Bond, C., et al. 2011. Megastore: Providing scalable, highly available storage for interactive services. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research. 223--234.Google Scholar
- Berenson, B., Bernstein, P., Gray, J., Melton, J., O'Neil, E., and O'Neil, P. 1995. A critique of ANSI SQL isolation levels. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 1--10. Google ScholarDigital Library
- Bernstein, P., Reid, C., and Das, S. 2011b. Hyder: A transactional record manager for shared Flash. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research. 9--20.Google Scholar
- Bernstein, P. A., Cseri, I., et al. 2011a. Adapting Microsoft SQL Server for cloud computing. In Proceedings of the 27th International Conference on Data Engineering. IEEE, 1255--1263. Google ScholarDigital Library
- Bernstein, P. A. and Newcomer, E. 2009. Principles of Transaction Processing, 2nd Ed. Morgan Kaufmann Publishers Inc., San Francisco, CA.Google Scholar
- Bradford, R., Kotsovinos, E., Feldmann, A., and Schiöberg, H. 2007. Live wide-area migration of virtual machines including local persistent state. In Proceedings of the 3rd International Conference on Virtual Execution Environments. ACM, New York, NY, 169--179. Google ScholarDigital Library
- Brantner, M., Florescu, D., Graf, D., Kossmann, D., and Kraska, T. 2008. Building a database on S3. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 251--264. Google ScholarDigital Library
- Chandrasekaran, C. and Bamford, R. 2003. Shared cache - The future of parallel databases. In Proceedings of ICDE. IEEE, 840--850.Google Scholar
- Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., and Gruber, R. E. 2006. Bigtable: A distributed storage system for structured data. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, 205--218. Google ScholarDigital Library
- Clark, C., Fraser, K., Hand, S., Hansen, J. G., Jul, E., Limpach, C., Pratt, I., and Warfield, A. 2005. Live migration of virtual machines. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation. USENIX Association, Berkeley, CA, 273--286. Google ScholarDigital Library
- Cooper, B. F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.-A., Puz, N., Weaver, D., and Yerneni, R. 2008. PNUTS: Yahoo!'s hosted data serving platform. Proc. VLDB Endow. 1, 2, 1277--1288. Google ScholarDigital Library
- Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., and Sears, R. 2010. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing. ACM, New York, NY, 143--154. Google ScholarDigital Library
- Curino, C., Jones, E., Popa, R., Malviya, N., Wu, E., Madden, S., Balakrishnan, H., and Zeldovich, N. 2011. Relational Cloud: A database service for the cloud. In Proceedings of the 5th Biennial Conference on Innovative Data Systems Research. 235--240.Google Scholar
- Curino, C., Jones, E., Zhang, Y., and Madden, S. 2010. Schism: a workload-driven approach to database replication and partitioning. Proc. VLDB Endow. 3, 1 48--57. Google ScholarDigital Library
- Das, S., Agrawal, D., and El Abbadi, A. 2009. ElasTraS: An elastic transactional data store in the cloud. In Proceedings of the 1st USENIX Workshop on Hot topics on Cloud Computing. USENIX Association, Berkeley, CA, 1--5. Google ScholarDigital Library
- Das, S., Agrawal, D., and El Abbadi, A. 2010. G-Store: A scalable data store for transactional multi key access in the cloud. In Proceedings of the 1st ACM Symposium on Cloud Computing. ACM, New York, NY, 163--174. Google ScholarDigital Library
- Das, S. Nishimura, S., Agrawal, D., and El Abbadi, A. 2011. Albatross: Lightweight elasticity in shared storage databases for the cloud using live data migration. Proc. VLDB Endow. 4, 8, 494--505. Google ScholarDigital Library
- DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. 2007. Dynamo: Amazon's highly available key-value store. In Proceedings of the 21st ACM Symposium on Operating System Principles. ACM, New York, NY, 205--220. Google ScholarDigital Library
- Elmore, A. J., Das, S., Agrawal, D., and El Abbadi, A. 2011. Zephyr: live migration in shared nothing databases for elastic cloud platforms. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 301--312. Google ScholarDigital Library
- Eswaran, K. P., Gray, J. N., Lorie, R. A., and Traiger, R. A. 1976. The notions of consistency and predicate locks in a database system. Comm. ACM 19, 11, 624--633. Google ScholarDigital Library
- Gray, J. 1978. Notes on data base operating systems. In Operating Systems, An Advanced Course, Springer, 393--481. Google ScholarDigital Library
- Gray, J. and Reuter, A. 1992. Transaction Processing: Concepts and Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA. Google ScholarDigital Library
- HDFS 2010. HDFS: A distributed file system that provides high throughput access to application data. http://hadoop.apache.org/hdfs/.Google Scholar
- Hunt, P., Konar, M., Junqueira, F. P., and Reed, B. 2010. ZooKeeper: wait-free coordination for internet-scale systems. In Proceedings of the USENIX Annual Technical Conference. USENIX Association, Berkeley, CA. Google ScholarDigital Library
- Jacobs, D. and Aulbach, S. 2007. Ruminations on multi-tenant databases. In Proceedings of Datenbanksysteme in Business, Technologie und Web. 514--521.Google Scholar
- Kung, H. T. and Robinson, J. T. 1981. On optimistic methods for concurrency control. ACM Trans. Data. Syst. 6, 2, 213--226. Google ScholarDigital Library
- Lamport, L. 1998. The part-time parliament. ACM Trans. Comput. Syst. 16, 2, 133--169. Google ScholarDigital Library
- Liu, H., Jin, H., Liao, X., Hu, L., and Yu, C. 2009. Live migration of virtual machine based on full system trace and replay. In Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing. ACM, New York, NY, 101--110. Google ScholarDigital Library
- Lomet, D. B., Fekete, A., Weikum, G., and Zwilling, M J. 2009. Unbundling Transaction Services in the Cloud. In Proceedings of the 4th Biennial Conference on Innovative Data Systems Research.Google Scholar
- Neuvonen, S., Wolski, A., Manner, M., and Raatikka, V. 2009. Telecommunication application transaction processing (TATP) benchmark description 1.0. http://tatpbenchmark. sourceforge.net/TATP Description.pdf.Google Scholar
- Patterson, P., Elmore, A. J., Nawab, F., Agrawal, D., and El Abbadi, A. 2012. Serializability, not serial: Concurrency control and availability in multi-datacenter datastores. Proc. VLDB Endow. 5, 11, 1459--1470. Google ScholarDigital Library
- Peng, D. and Dabek, F. 2010. Large-scale incremental processing using distributed transactions and notifications. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA. Google ScholarDigital Library
- Rao, L. 2010. One database to rule the cloud: salesforce debuts Database.com for the enterprise. http://goo.gl/0BxVq.Google Scholar
- Sockut, G. H. and Iyer, B. R. 2009. Online reorganization of databases. ACM Comput. Surv. 41, 3, 1--136. Google ScholarDigital Library
- Tankel, D. 2010. Scalability of the Hadoop distributed file system. http://yhoo.it/HDFS.Google Scholar
- Transaction Processing Performance Council. 2009. TPC-C benchmark (Version 5.10.1).Google Scholar
- Theimer, M. M., Lantz, K. A., and Cheriton, D. R. 1985. Preemptable remote execution facilities for the V-system. In Proceedings of the 10th ACM Symposium on Operating System Principles. ACM, New York, NY, 2--12. Google ScholarDigital Library
- Trushkowsky, B., Bodík, P., Fox, A., Franklin, M. J., Jordan, M. I., and Patterson, D. A. 2011. The SCADS director: Scaling a distributed storage system under stringent performance requirements. In Proceedings of the 9th USENIX Conference on File and Storage Technologies. USENIX Association, Berkeley, CA. Google ScholarDigital Library
- Vo, H. T., Chen, C., and Ooi, B. C. 2010. Towards elastic transactional cloud storage with range query support. Proc. VLDB Endow. 3, 1, 506--514. Google ScholarDigital Library
- von Eicken, T. 2008. Righscale blog: Animoto's Facebook scale-up. http://goo.gl/C7Bh.Google Scholar
- Weikum, G. and Vossen, G. 2001. Transactional Information Systems: Theory, Algorithms, and the Practice Of Concurrency Control and Recovery. Morgan Kaufmann Inc., San Francisco, CA. Google ScholarDigital Library
- Weissman, C. D. and Bobrowski, S. 2009. The design of the force.com multitenant Internet application development platform. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 889--896. Google ScholarDigital Library
- Yang, F., Shanmugasundaram, J., and Yerneni, R. 2009. A scalable data platform for a large number of small applications. In Proceedings of the 4th Biennial Conference on Innovative Data Systems Research.Google Scholar
Index Terms
- ElasTraS: An elastic, scalable, and self-managing transactional database for the cloud
Recommendations
Zephyr: live migration in shared nothing databases for elastic cloud platforms
SIGMOD '11: Proceedings of the 2011 ACM SIGMOD International Conference on Management of dataMultitenant data infrastructures for large cloud platforms hosting hundreds of thousands of applications face the challenge of serving applications characterized by small data footprint and unpredictable load patterns. When such a platform is built on ...
Migration of Multi-tier Applications to Infrastructure-as-a-Service Clouds: An Investigation Using Kernel-Based Virtual Machines
GRID '11: Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid ComputingTo investigate challenges of multi-tier application migration to Infrastructure-as-a-Service (IaaS) clouds we performed an experimental investigation by deploying a processor bound and input-output bound variant of the RUSLE2 erosion model to an IaaS ...
A Methodology for Tenant Migration in Legacy Shared-Table Multi-tenant Applications
Distributed Applications and Interoperable SystemsAbstractMulti-tenancy enables cost-effective SaaS through resource consolidation. Multiple customers, or tenants, are served by a single application instance, and isolation is enforced at the application level. Service load for different tenants can vary ...
Comments