Abstract
The availability of public computing resources in the cloud has revolutionized data analysis, but requesting cloud resources often involves complex decisions for consumers. Estimating the completion time and cost of a computation and requesting the appropriate cloud resources are challenging tasks even for an expert user. We propose a new market-based framework for pricing computational tasks in the cloud. Our framework introduces an agent between consumers and cloud providers. The agent takes data and computational tasks from users, estimates time and cost for evaluating the tasks, and returns to consumers contracts that specify the price and completion time. Our framework can be applied directly to existing cloud markets without altering the way cloud providers offer and price services. In addition, it simplifies cloud use for consumers by allowing them to compare contracts, rather than choose resources directly. We present design, analytical, and algorithmic contributions focusing on pricing computation contracts, analyzing their properties, and optimizing them in complex workflows. We conduct an experimental evaluation of our market framework over a real-world cloud service and demonstrate empirically that our market ensures three key properties: (a) that consumers benefit from using the market due to competitiveness among agents, (b) that agents have an incentive to price contracts fairly, and (c) that inaccuracies in estimates do not pose a significant risk to agents' profits. Finally, we present a fine-grained pricing mechanism for complex workflows and show that it can increase agent profits by more than an order of magnitude in some cases.
- A. Aboulnaga, Z. Wang, and Z. Y. Zhang. Packing the most onto your cloud. In CloudDB, pages 25--28, 2009. Google ScholarDigital Library
- O. Agmon Ben-Yehuda, M. Ben-Yehuda, A. Schuster, and D. Tsafrir. Deconstructing amazon ec2 spot instance pricing. TEAC, 1(3):16:1--16:20, 2013. Google ScholarDigital Library
- M. Ahmad, A. Aboulnaga, S. Babu, and K. Munagala. Interaction-aware scheduling of report-generation workloads. VLDBJ, 20(4):589--615, 2011. Google ScholarDigital Library
- M. Akdere, U. Çetintemel, M. Riondato, E. Upfal, and S. Zdonik. Learning-based query performance modeling and prediction. In ICDE, pages 390--401, 2012. Google ScholarDigital Library
- A. Al Daoud, S. Agarwal, and T. Alpcan. Brief announcement: Cloud computing games: Pricing services of large data centers. In DISC, pages 309--310, 2009. Google ScholarDigital Library
- A. Bossenbroek, A. Tirado-Ramos, and P. Sloot. Grid resource allocation by means of option contracts. ISJ, 3(1):49--64, 2009.Google Scholar
- R. Buyya. Market-oriented cloud computing: Vision, hype, and reality of delivering computing as the 5th utility. In CCGRID, pages 1--1, 2009. Google ScholarDigital Library
- Y. Chi, H. Hacígümüş, W.-P. Hsiung, and J. F. Naughton. Distribution-based query scheduling. PVLDB, 6(9):673--684, 2013. Google ScholarDigital Library
- B. Chun, P. Buonadonna, A. AuYoung, C. Ng, D. Parkes, J. Shneidman, A. Snoeren, and A. Vahdat. Mirage: A microeconomic resource allocation system for sensornet testbeds. In EmNetS, pages 19--28, 2005. Google ScholarDigital Library
- D. De Roure, C. Goble, and R. Stevens. The design and realisation of the virtual research environment for social sharing of workflows. FGCS, 25(5):561--567, 2009. Google ScholarDigital Library
- J. Duggan, O. Papaemmanouil, U. Çetintemel, and E. Upfal. Contender: A resource modeling approach for concurrent query performance prediction. In EDBT, pages 109--120, 2014.Google Scholar
- A. Floratou, J. M. Patel, W. Lang, and A. Halverson. When free is not really free: What does it cost to run a database workload in the cloud? In TPCTC, pages 163--179, 2012. Google ScholarDigital Library
- A. Ganapathi, H. Kuno, U. Dayal, J. L. Wiener, A. Fox, M. Jordan, and D. Patterson. Predicting multiple metrics for queries: Better decisions enabled by machine learning. In ICDE, pages 592--603, 2009. Google ScholarDigital Library
- S. Ganguly, W. Hasan, and R. Krishnamurthy. Query optimization for parallel execution. In SIGMOD, pages 9--18, 1992. Google ScholarDigital Library
- M. Hadji, W. Louati, and D. Zeghlache. Constrained pricing for cloud resource allocation. In NCA, pages 359--365, 2011. Google ScholarDigital Library
- H. Herodotou, F. Dong, and S. Babu. No one (cluster) size fits all: Automatic cluster sizing for data-intensive analytics. In SOCC, pages 18:1--18:14, 2011. Google ScholarDigital Library
- S. Ibrahim, B. He, and H. Jin. Towards pay-as-you-consume cloud computing. In SCC, pages 370--377, 2011. Google ScholarDigital Library
- C. Joe-Wong and S. Sen. Mathematical frameworks for pricing in the cloud: Revenue, fairness, and resource allocations. CoRR, abs/1212.0022, 2012.Google Scholar
- G. Juve, A. Chervenak, E. Deelman, S. Bharathi, G. Mehta, and K. Vahi. Characterizing and profiling scientific workflows. FGCS, 29(3):682--692, 2013. Google ScholarDigital Library
- Z. Karampaglis, A. Gounaris, and Y. Manolopoulos. A bi-objective cost model for database queries in a multi-cloud environment. In MEDES, pages 19:109--19:116, 2014. Google ScholarDigital Library
- T. Kiefer, H. Schön, D. Habich, and W. Lehner. A query, a minute: Evaluating performance isolation in cloud databases. In TPCTC, pages 173--187, 2014.Google Scholar
- R. S. G. Lanzelotte, P. Valduriez, and M. Zaït. On the effectiveness of optimization search strategies for parallel execution spaces. In VLDB, pages 493--504, 1993. Google ScholarDigital Library
- J. Li, A. C. König, V. Narasayya, and S. Chaudhuri. Robust estimation of resource consumption for sql queries using statistical techniques. PVLDB, 5(11):1555--1566, 2012. Google ScholarDigital Library
- Y. Li, E. Lo, M. L. Yiu, and W. Xu. Query optimization over cloud data market. In EDBT, pages 229--240, 2015.Google Scholar
- Z. Liu, H. Hacigümüş, H. J. Moon, Y. Chi, and W.-P. Hsiung. Pmax: Tenant placement in multitenant databases for profit maximization. In EDBT, pages 442--453, 2013. Google ScholarDigital Library
- D. Lorenz and A. Orda. Qos routing in networks with uncertain parameters. TON, 6(6):768--778, 1998. Google ScholarDigital Library
- R. Marcus and O. Papaemmanouil. Wisedb: A learning-based workload management advisor for cloud databases. PVLDB, 9(10):780--791, 2016. Google ScholarDigital Library
- M. Mihailescu and Y. M. Teo. Dynamic resource pricing on federated clouds. In CCGrid, pages 513--517, 2010. Google ScholarDigital Library
- N. Nisan, S. London, O. Regev, and N. Camiel. Globally distributed computation over the internet-the popcorn project. In ICDCS, pages 592--601, 1998. Google ScholarDigital Library
- J. Ortiz, V. T. de Almeida, and M. Balazinska. Changing the face of database cloud services with personalized service level agreements. In CIDR, 2015.Google Scholar
- J. Ortiz, B. Lee, and M. Balazinska. Perfenforce demonstration: Data analytics with performance guarantees. In SIGMOD, pages 2141--2144, 2016. Google ScholarDigital Library
- M. B. Qureshi, M. M. Dehnavi, N. Min-Allah, M. S. Qureshi, H. Hussain, I. Rentifis, N. Tziritas, T. Loukopoulos, S. U. Khan, C.-Z. Xu, and A. Y. Zomaya. Survey on grid resource allocation mechanisms. JGC, 12(2):399--441, 2014. Google ScholarDigital Library
- L. Ramakrishnan, J. S. Chase, D. Gannon, D. Nurmi, and R. Wolski. Deadline-sensitive workflow orchestration without explicit resource control. JPDC, 71(3):343--353, 2011. Google ScholarDigital Library
- E. Rasmusen. Games and information : an introduction to game theory. Blackwell Pub., 2007.Google Scholar
- J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz. Runtime measurements in the cloud: Observing, analyzing, and reducing variance. PVLDB, 3(1--2):460--471, 2010. Google ScholarDigital Library
- G. Stuer, K. Vanmechelen, and J. Broeckhove. A commodity market algorithm for pricing substitutable grid resources. FGCS, 23(5):688--701, 2007. Google ScholarDigital Library
- Z. Tan and J. R. Gurd. Market-based grid resource allocation using a stable continuous double auction. In GRID, pages 283--290, 2007. Google ScholarDigital Library
- M. Tanaka and Y. Murakami. Strategy-proof pricing for cloud service composition. TCC, PP(99):1--1, 2014.Google Scholar
- K. Vanmechelen and J. Broeckhove. A comparative analysis of single-unit vickrey auctions and commodity markets for realizing grid economies with dynamic pricing. In GECON, pages 98--111, 2007. Google ScholarDigital Library
- H. Varian. Intermediate Microeconomics: A Modern Approach. W.W. Norton & Company, 2010.Google Scholar
- R. Wang. Auctions versus posted-price selling. Am. Econ. Rev., 83(4), 1993.Google Scholar
- Y. Wang, A. Meliou, and G. Miklau. A consumer-centric market for database computation in the cloud. CoRR, abs/1609.02104, 2016.Google Scholar
- R. Wolski, J. S. Plank, J. Brevik, and T. Bryan. Analyzing market-based resource allocation strategies for the computational grid. IJHPCA, 15(3):258--281, 2001. Google ScholarDigital Library
- K. Wolstencroft, R. Haines, D. Fellows, A. R. Williams, D. Withers, S. Owen, S. Soiland-Reyes, I. Dunlop, A. Nenadic, P. Fisher, J. Bhagat, K. Belhajjame, F. Bacall, A. Hardisty, A. N. de la Hidalga, M. P. B. Vargas, S. Sufi, and C. A. Goble. The taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. NAR, 41(Webserver-Issue):557--561, 2013.Google Scholar
- W. Wu, X. Wu, H. Hacigümüş, and J. F. Naughton. Uncertainty aware query execution time prediction. PVLDB, 7(14):1857--1868, 2014. Google ScholarDigital Library
- P. Xiong, Y. Chi, S. Zhu, J. Tatemura, C. Pu, and H. HacigümüŞ. Activesla: A profit-oriented admission control framework for database-as-a-service providers. In SOCC, pages 15:1--15:14, 2011. Google ScholarDigital Library
- H. Xu and B. Li. Dynamic cloud pricing for revenue maximization. TCC, 1(2):158--171, 2013. Google ScholarDigital Library
- Z. Ye, A. Bouguettaya, and X. Zhou. Qos-aware cloud service composition based on economic models. In ICSOC, pages 111--126, 2012. Google ScholarDigital Library
- C. S. Yeo, S. Venugopal, X. Chu, and R. Buyya. Autonomic metered pricing for a utility computing service. FGCS, 26(8):1368--1380, 2010. Google ScholarDigital Library
- A. Zhou and B. He. Transformation-based monetary cost optimizations for workflows in the cloud. TCC, 2(1):85--98, 2014.Google Scholar
Recommendations
Cloud Market Maker
Cloud providers commonly incur heavy upfront set up costs which remain almost constant whether they serve a single or many customers. In order to generate a return on this investment, a suitable pricing strategy is required by providers. Established ...
Towards green cloud computing: Demand allocation and pricing policies for cloud service brokerage
BIG DATA '15: Proceedings of the 2015 IEEE International Conference on Big Data (Big Data)Functioning as an intermediary between tenants and cloud providers, cloud service brokerages (CSBs) can bring about great benefits to the cloud market. CSBs buy the cloud resources, i.e., servers, with lower prices from cloud providers and sell the ...
Cloud service engineering
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2Building on compute and storage virtualization, Cloud Computing provides scalable, network-centric, abstracted IT infrastructure, platforms, and applications as on-demand services that are billed by consumption. Cloud Service Engineering is the ...
Comments