Skip to main content
Top
Published in: Knowledge and Information Systems 12/2020

11-08-2020 | Regular Paper

SLA-driven resource re-allocation for SQL-like queries in the cloud

Authors: Mohamed Mehdi Kandi, Shaoyi Yin, Abdelkader Hameurlain

Published in: Knowledge and Information Systems | Issue 12/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Cloud computing has become a widely used environment for database querying. In this context, the goal of a query optimizer is to satisfy the needs of tenants and maximize the provider’s benefit. Resource allocation is an important step toward achieving this goal. Allocation methods are based on analytical formulas and statistics collected from a catalog to estimate the cost of various possible allocations and then choose the best one. However, the allocation initially chosen is not necessarily the optimal one because of the approximate nature of the analytical formulas and the fact that the catalog may not be up to date. To solve this problem, existing work was proposed to collect statistics during the execution of the query and then trigger a re-allocation if suboptimality is detected. However, these proposals consider that queries have the same level of priority. Unlike the existing work, we propose in this paper a method of statistics collector placement and resource re-allocation by taking into account that the cloud is a multi-tenant environment and queries have different services-level agreements. In the experimental section, we show that our method provides a better benefit for the provider compared to state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Footnotes
1
In practice, there may be an additional waiting time if other different queries share the same resources. If so, the waiting time must be included in \(ET_{stat}(q)\).
 
Literature
1.
go back to reference Agarwal S, Kandula S, Bruno N, Wu MC, Stoica I, Zhou J (2012) Reoptimizing data parallel computing. In: Presented as part of the 9th \(\{\)USENIX\(\}\) symposium on networked systems design and implementation (\(\{\)NSDI\(\}\) 12), pp 281–294 Agarwal S, Kandula S, Bruno N, Wu MC, Stoica I, Zhou J (2012) Reoptimizing data parallel computing. In: Presented as part of the 9th \(\{\)USENIX\(\}\) symposium on networked systems design and implementation (\(\{\)NSDI\(\}\) 12), pp 281–294
2.
go back to reference Armbrust M, Xin RS, Lian C, Huai Y, Liu D, Bradley JK, Meng X, Kaftan T, Franklin MJ, Ghodsi A et al (2015) Spark sql: Relational data processing in spark. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data. ACM, pp 1383–1394 Armbrust M, Xin RS, Lian C, Huai Y, Liu D, Bradley JK, Meng X, Kaftan T, Franklin MJ, Ghodsi A et al (2015) Spark sql: Relational data processing in spark. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data. ACM, pp 1383–1394
7.
go back to reference Bi J, Yuan H, Tan W, Zhou M, Fan Y, Zhang J, Li J (2015) Application-aware dynamic fine-grained resource provisioning in a virtualized cloud data center. IEEE Trans Autom Sci Eng 14(2):1172–1184CrossRef Bi J, Yuan H, Tan W, Zhou M, Fan Y, Zhang J, Li J (2015) Application-aware dynamic fine-grained resource provisioning in a virtualized cloud data center. IEEE Trans Autom Sci Eng 14(2):1172–1184CrossRef
8.
go back to reference Bi J, Yuan H, Tie M, Tan W (2015) Sla-based optimisation of virtualised resource for multi-tier web applications in cloud data centres. Enterp Inf Syst 9(7):743–767CrossRef Bi J, Yuan H, Tie M, Tan W (2015) Sla-based optimisation of virtualised resource for multi-tier web applications in cloud data centres. Enterp Inf Syst 9(7):743–767CrossRef
9.
go back to reference Bouganim L, Kapitskaia O, Valduriez P (1998) Memory-adaptive scheduling for large query execution. In: Proceedings of the seventh international conference on information and knowledge management. ACM, pp 105–115 Bouganim L, Kapitskaia O, Valduriez P (1998) Memory-adaptive scheduling for large query execution. In: Proceedings of the seventh international conference on information and knowledge management. ACM, pp 105–115
10.
go back to reference Bruno N, Jain S, Zhou J (2013) Continuous cloud-scale query optimization and processing. Proc VLDB Endow 6(11):961–972CrossRef Bruno N, Jain S, Zhou J (2013) Continuous cloud-scale query optimization and processing. Proc VLDB Endow 6(11):961–972CrossRef
11.
go back to reference Date CJ, Darwen H (1987) A guide to the SQL standard, vol 3. Addison-Wesley, New York Date CJ, Darwen H (1987) A guide to the SQL standard, vol 3. Addison-Wesley, New York
12.
go back to reference Floratou A, Minhas UF, Özcan F (2014) Sql-on-hadoop: full circle back to shared-nothing database architectures. Proc VLDB Endow 7(12):1295–1306CrossRef Floratou A, Minhas UF, Özcan F (2014) Sql-on-hadoop: full circle back to shared-nothing database architectures. Proc VLDB Endow 7(12):1295–1306CrossRef
13.
go back to reference Garcia-Molina H (2008) Database systems: the complete book. Pearson Education India, Noida Garcia-Molina H (2008) Database systems: the complete book. Pearson Education India, Noida
15.
go back to reference Ioannidis YE, Christodoulakis S (1991) On the propagation of errors in the size of join results, vol 20. ACM, New York Ioannidis YE, Christodoulakis S (1991) On the propagation of errors in the size of join results, vol 20. ACM, New York
16.
go back to reference Kabra N, DeWitt DJ (1998) Efficient mid-query re-optimization of sub-optimal query execution plans. ACM SIGMOD Record ACM 27:106–117CrossRef Kabra N, DeWitt DJ (1998) Efficient mid-query re-optimization of sub-optimal query execution plans. ACM SIGMOD Record ACM 27:106–117CrossRef
17.
go back to reference Kandi MM, Yin S, Hameurlain A (2018) An integer linear-programming based resource allocation method for sql-like queries in the cloud. In: Proceedings of the 33rd annual ACM symposium on applied computing. ACM, pp 161–166 Kandi MM, Yin S, Hameurlain A (2018) An integer linear-programming based resource allocation method for sql-like queries in the cloud. In: Proceedings of the 33rd annual ACM symposium on applied computing. ACM, pp 161–166
18.
go back to reference Kandi MM, Yin S, Hameurlain A (2019) Resource auto-scaling for sql-like queries in the cloud based on parallel reinforcement learning. Int J Grid Util Comput 10(6):654–671CrossRef Kandi MM, Yin S, Hameurlain A (2019) Resource auto-scaling for sql-like queries in the cloud based on parallel reinforcement learning. Int J Grid Util Comput 10(6):654–671CrossRef
19.
go back to reference Kllapi H, Sitaridi E, Tsangaris MM, Ioannidis Y (2011) Schedule optimization for data processing flows on the cloud. In: Proceedings of the 2011 ACM SIGMOD international conference on management of data. ACM, pp 289–300 Kllapi H, Sitaridi E, Tsangaris MM, Ioannidis Y (2011) Schedule optimization for data processing flows on the cloud. In: Proceedings of the 2011 ACM SIGMOD international conference on management of data. ACM, pp 289–300
20.
go back to reference Nag B, DeWitt DJ (1998) Memory allocation strategies for complex decision support queries. In: Proceedings of the seventh international conference on information and knowledge management (CIKM), Citeseer, vol 2. pp 116–123 Nag B, DeWitt DJ (1998) Memory allocation strategies for complex decision support queries. In: Proceedings of the seventh international conference on information and knowledge management (CIKM), Citeseer, vol 2. pp 116–123
25.
go back to reference Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Zhang N, Antony S, Liu H, Murthy R (2010) Hive-a petabyte scale data warehouse using hadoop. In: 2010 IEEE 26th international conference on data engineering (ICDE). IEEE, pp 996–1005 Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Zhang N, Antony S, Liu H, Murthy R (2010) Hive-a petabyte scale data warehouse using hadoop. In: 2010 IEEE 26th international conference on data engineering (ICDE). IEEE, pp 996–1005
27.
go back to reference Vavilapalli VK, Murthy AC, Douglas C, Agarwal S, Konar M, Evans R, Graves T, Lowe J, Shah H, Seth S et al (2013) Apache hadoop yarn: yet another resource negotiator. In: Proceedings of the 4th annual symposium on cloud computing. ACM, p 5 Vavilapalli VK, Murthy AC, Douglas C, Agarwal S, Konar M, Evans R, Graves T, Lowe J, Shah H, Seth S et al (2013) Apache hadoop yarn: yet another resource negotiator. In: Proceedings of the 4th annual symposium on cloud computing. ACM, p 5
28.
go back to reference Yin S, Hameurlain A, Morvan F (2018) SLA definition for multi-tenant dbms and its impact on query optimization. IEEE Trans Knowl Data Eng 30(11):2213–2226 Yin S, Hameurlain A, Morvan F (2018) SLA definition for multi-tenant dbms and its impact on query optimization. IEEE Trans Knowl Data Eng 30(11):2213–2226
29.
go back to reference Yu PS, Cornell DW (1993) Buffer management based on return on consumption in a multi-query environment. VLDB J 2(1):1–38CrossRef Yu PS, Cornell DW (1993) Buffer management based on return on consumption in a multi-query environment. VLDB J 2(1):1–38CrossRef
30.
go back to reference Yuan H, Bi J, Tan W, Li BH (2016) Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans Autom Sci Eng 14(1):337–348CrossRef Yuan H, Bi J, Tan W, Li BH (2016) Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans Autom Sci Eng 14(1):337–348CrossRef
31.
go back to reference Yuan H, Bi J, Zhou M, Ammari AC (2017) Time-aware multi-application task scheduling with guaranteed delay constraints in green data center. IEEE Trans Autom Sci Eng 15(3):1138–1151CrossRef Yuan H, Bi J, Zhou M, Ammari AC (2017) Time-aware multi-application task scheduling with guaranteed delay constraints in green data center. IEEE Trans Autom Sci Eng 15(3):1138–1151CrossRef
32.
go back to reference Yuan H, Bi J, Zhou M (2018) Spatial task scheduling for cost minimization in distributed green cloud data centers. IEEE Trans Autom Sci Eng 16(2):729–740CrossRef Yuan H, Bi J, Zhou M (2018) Spatial task scheduling for cost minimization in distributed green cloud data centers. IEEE Trans Autom Sci Eng 16(2):729–740CrossRef
Metadata
Title
SLA-driven resource re-allocation for SQL-like queries in the cloud
Authors
Mohamed Mehdi Kandi
Shaoyi Yin
Abdelkader Hameurlain
Publication date
11-08-2020
Publisher
Springer London
Published in
Knowledge and Information Systems / Issue 12/2020
Print ISSN: 0219-1377
Electronic ISSN: 0219-3116
DOI
https://doi.org/10.1007/s10115-020-01501-z

Other articles of this Issue 12/2020

Knowledge and Information Systems 12/2020 Go to the issue

Premium Partner