Skip to main content
Top
Published in: The Journal of Supercomputing 9/2023

03-02-2023

Dynamic two-side matching of tasks and resources in wide-area distributed computing environments

Authors: Yao Song, Liang Wang, Limin Xiao, Runnan Shen, Jinquan Wang, Chenhao Zhang

Published in: The Journal of Supercomputing | Issue 9/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Recently, wide-area distributed computing environments have become popular due to its huge resource capacities. In the wide-area distributed computing environment, matching between tasks and resources plays an important role in improving system performance. However, the geographically distribution of diverse resources complicates the matching problems, thus degrading the system performance. To achieve an efficient matching between task requirements and resource capacities, this study proposes a dynamic two-side matching of tasks and resources in wide-area distributed computing environments. First, the proposed method establishes the task requirement model and the resource capacity model using a uniform description based on the characteristic analysis to mitigate the impact of diversity and simplify the matching process. Then, a two-side matching degree metric is designed to comprehensively quantify the matching quality between the task requirements and resource capacities. Finally, a resource selection strategy is raised to guide the scheduling based on the matching degree. The experimental results indicate that compared with the state-of-the-art matching methods, the proposed method reduces the overall completion time and response delay by up to 35.60% and 29.28%, respectively.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Chen Q, Zheng Z, Hu C, Wang D, Liu F (2020) On-edge multi-task transfer learning: Model and practice with data-driven task allocation. IEEE Trans Parallel Distrib Syst 31:1357–1371CrossRef Chen Q, Zheng Z, Hu C, Wang D, Liu F (2020) On-edge multi-task transfer learning: Model and practice with data-driven task allocation. IEEE Trans Parallel Distrib Syst 31:1357–1371CrossRef
2.
go back to reference Cheng L, Wang Y, Liu Q, Epema DHJ, Liu C, Mao Y, Murphy J (2021) Network-aware locality scheduling for distributed data operators in data centers. IEEE Trans Parallel Distrib Syst 32:1494–1510CrossRef Cheng L, Wang Y, Liu Q, Epema DHJ, Liu C, Mao Y, Murphy J (2021) Network-aware locality scheduling for distributed data operators in data centers. IEEE Trans Parallel Distrib Syst 32:1494–1510CrossRef
3.
go back to reference Xu K, Lv L, Li T, Shen M, Wang H, Yang K (2020) Minimizing tardiness for data-intensive applications in heterogeneous systems: a matching theory perspective. IEEE Trans Parallel Distrib Syst 31:144–158CrossRef Xu K, Lv L, Li T, Shen M, Wang H, Yang K (2020) Minimizing tardiness for data-intensive applications in heterogeneous systems: a matching theory perspective. IEEE Trans Parallel Distrib Syst 31:144–158CrossRef
4.
go back to reference Li B, Yang Y, Su J, Liang Z, Wang S (2020) Two-sided matching decision-making model with hesitant fuzzy preference information for configuring cloud manufacturing tasks and resources. J Intell Manuf 25:1–15 Li B, Yang Y, Su J, Liang Z, Wang S (2020) Two-sided matching decision-making model with hesitant fuzzy preference information for configuring cloud manufacturing tasks and resources. J Intell Manuf 25:1–15
5.
go back to reference Zhao L, Yang Y, Munir A, Liu AX, Li Y, Qu W (2020) Optimizing geo-distributed data analytics with coordinated task scheduling and routing. IEEE Trans Parallel Distrib Syst 31:279–293CrossRef Zhao L, Yang Y, Munir A, Liu AX, Li Y, Qu W (2020) Optimizing geo-distributed data analytics with coordinated task scheduling and routing. IEEE Trans Parallel Distrib Syst 31:279–293CrossRef
6.
go back to reference Bi Y, Yuan H, Chu Y, Liu H (2019) Multilevel and intelligent rent-seeking and matching resource strategy and value creation of public service platform in big data environment. Computer Sci 46:42–49 Bi Y, Yuan H, Chu Y, Liu H (2019) Multilevel and intelligent rent-seeking and matching resource strategy and value creation of public service platform in big data environment. Computer Sci 46:42–49
8.
go back to reference Raveendran N, Zhang H, Song L, Wang L-C, Hong CS, Han Z (2022) Pricing and resource allocation optimization for iot fog computing and nfv: An epec and matching based perspective. IEEE Trans Mobile Comput 21:1349–1361CrossRef Raveendran N, Zhang H, Song L, Wang L-C, Hong CS, Han Z (2022) Pricing and resource allocation optimization for iot fog computing and nfv: An epec and matching based perspective. IEEE Trans Mobile Comput 21:1349–1361CrossRef
9.
go back to reference Feng W, Zheng J, Jiang W (2020) Joint pilot and data transmission power control and computing resource allocation algorithm for massive mimo-mec networks. IEEE Access 8:80801–80811CrossRef Feng W, Zheng J, Jiang W (2020) Joint pilot and data transmission power control and computing resource allocation algorithm for massive mimo-mec networks. IEEE Access 8:80801–80811CrossRef
10.
go back to reference Chen Y, Li Z, Yang B, Nai K, Li K (2020) A stackelberg game approach to multiple resources allocation and pricing in mobile edge computing. Fut Gener Comput Syst 108:273–287CrossRef Chen Y, Li Z, Yang B, Nai K, Li K (2020) A stackelberg game approach to multiple resources allocation and pricing in mobile edge computing. Fut Gener Comput Syst 108:273–287CrossRef
11.
go back to reference Wang Z, Zheng S, Ge Q, Li K (2020) Online offloading scheduling and resource allocation algorithms for vehicular edge computing system. IEEE Access 8:52428–52442CrossRef Wang Z, Zheng S, Ge Q, Li K (2020) Online offloading scheduling and resource allocation algorithms for vehicular edge computing system. IEEE Access 8:52428–52442CrossRef
12.
go back to reference Li C, Bai J, Jianhang T (2019) Joint optimization of data placement and scheduling for improving user experience in edge computing. J Parallel Distrib Comput 125:93–105CrossRef Li C, Bai J, Jianhang T (2019) Joint optimization of data placement and scheduling for improving user experience in edge computing. J Parallel Distrib Comput 125:93–105CrossRef
13.
go back to reference Wang W, Li B, Liang B, Li J (2016) Multi-resource fair sharing for datacenter jobs with placement constraints. SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, 1003–1014 Wang W, Li B, Liang B, Li J (2016) Multi-resource fair sharing for datacenter jobs with placement constraints. SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, 1003–1014
14.
go back to reference Naik VK, Garbacki P, Kummamuru K, Zhao Y (2006) On-line evolutionary resource matching for job scheduling in heterogeneous grid environments. 12th International Conference on Parallel and Distributed Systems - (ICPADS’06) 2, 6 Naik VK, Garbacki P, Kummamuru K, Zhao Y (2006) On-line evolutionary resource matching for job scheduling in heterogeneous grid environments. 12th International Conference on Parallel and Distributed Systems - (ICPADS’06) 2, 6
15.
go back to reference Leong SH, Parodi A, Kranzlmüller DA (2017) A robust reliable energy-aware urgent computing resource allocation for flash-flood ensemble forecasting on hpc infrastructures for decision support. Fut Gener Comput Syst 68:136–149CrossRef Leong SH, Parodi A, Kranzlmüller DA (2017) A robust reliable energy-aware urgent computing resource allocation for flash-flood ensemble forecasting on hpc infrastructures for decision support. Fut Gener Comput Syst 68:136–149CrossRef
16.
go back to reference Ma Y, Wang H, Xiong J, Diao J, Ma D (2020) Joint allocation on communication and computing resources for fog radio access networks. IEEE Access 8:108310–108323CrossRef Ma Y, Wang H, Xiong J, Diao J, Ma D (2020) Joint allocation on communication and computing resources for fog radio access networks. IEEE Access 8:108310–108323CrossRef
17.
go back to reference Yuan Y, Yang T, Hu Y, Feng H, Hu B (2021) Two-timescale resource allocation for cooperative d2d communication: A matching game approach. IEEE Trans Vehicular Technol 70:543–557CrossRef Yuan Y, Yang T, Hu Y, Feng H, Hu B (2021) Two-timescale resource allocation for cooperative d2d communication: A matching game approach. IEEE Trans Vehicular Technol 70:543–557CrossRef
18.
go back to reference Kumar R, Saad A, Grande RED (2021) Correct: Connection-oriented resource matching for vehicular clouds. ICC 2021 - IEEE International Conference on Communications, 1–6 Kumar R, Saad A, Grande RED (2021) Correct: Connection-oriented resource matching for vehicular clouds. ICC 2021 - IEEE International Conference on Communications, 1–6
19.
go back to reference Mao Y, Zhang J, Song SH, Letaief KB (2017) Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems. IEEE Trans Wireless Commun 16:5994–6009CrossRef Mao Y, Zhang J, Song SH, Letaief KB (2017) Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems. IEEE Trans Wireless Commun 16:5994–6009CrossRef
20.
go back to reference Tran TX, Pompili D (2019) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans Vehicular Technol 68:856–868CrossRef Tran TX, Pompili D (2019) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans Vehicular Technol 68:856–868CrossRef
21.
go back to reference Sun W, Liu J, Yue Y, Zhang H (2018) Double auction-based resource allocation for mobile edge computing in industrial internet of things. IEEE Trans Industr Inf 14:4692–4701CrossRef Sun W, Liu J, Yue Y, Zhang H (2018) Double auction-based resource allocation for mobile edge computing in industrial internet of things. IEEE Trans Industr Inf 14:4692–4701CrossRef
22.
go back to reference Yuan Y, Li Z, Liu Z, Yang Y, Guan X (2022) Double deep q-network based distributed resource matching algorithm for d2d communication. IEEE Trans Vehicular Technol 71:984–993CrossRef Yuan Y, Li Z, Liu Z, Yang Y, Guan X (2022) Double deep q-network based distributed resource matching algorithm for d2d communication. IEEE Trans Vehicular Technol 71:984–993CrossRef
23.
go back to reference Roth AE, Sotomayor M (1990) Two-sided matching: A study in game-theoretic modeling and analysis Roth AE, Sotomayor M (1990) Two-sided matching: A study in game-theoretic modeling and analysis
24.
go back to reference Mitzenmacher M (2001) The power of two choices in randomized load balancing. IEEE Trans Parallel Distrib Syst 12:1094–1104CrossRef Mitzenmacher M (2001) The power of two choices in randomized load balancing. IEEE Trans Parallel Distrib Syst 12:1094–1104CrossRef
25.
go back to reference Mukhopadhyay A, Mazumdar RR (2016) Analysis of randomized join-the-shortest-queue (jsq) schemes in large heterogeneous processor-sharing systems. IEEE Trans Control Netw Syst 3:116–126CrossRefMATHMathSciNet Mukhopadhyay A, Mazumdar RR (2016) Analysis of randomized join-the-shortest-queue (jsq) schemes in large heterogeneous processor-sharing systems. IEEE Trans Control Netw Syst 3:116–126CrossRefMATHMathSciNet
26.
go back to reference Ding D, Fan X, Luo S (2015) User-oriented cloud resource scheduling with feedback integration. J Supercomput 72:3114–3135CrossRef Ding D, Fan X, Luo S (2015) User-oriented cloud resource scheduling with feedback integration. J Supercomput 72:3114–3135CrossRef
27.
go back to reference Garg H, Arora R (2018) Dual hesitant fuzzy soft aggregation operators and their application in decision-making. Cognit Comput 10:769–789CrossRef Garg H, Arora R (2018) Dual hesitant fuzzy soft aggregation operators and their application in decision-making. Cognit Comput 10:769–789CrossRef
28.
go back to reference Chen L-T, Chen S (2022) Volunteer multi-person multi-task optimization dispatch method considering two-sided matching. Soft Comput 26:3837–3861CrossRef Chen L-T, Chen S (2022) Volunteer multi-person multi-task optimization dispatch method considering two-sided matching. Soft Comput 26:3837–3861CrossRef
29.
go back to reference Li C, Tang X (2020) On fault-tolerant bin packing for online resource allocation. IEEE Trans Parallel Distrib Syst 31:817–829CrossRef Li C, Tang X (2020) On fault-tolerant bin packing for online resource allocation. IEEE Trans Parallel Distrib Syst 31:817–829CrossRef
30.
go back to reference Casanova H, Legrand A, Quinson M (2008) Simgrid: A generic framework for large-scale distributed experiments. Tenth International Conference on Computer Modeling and Simulation (uksim 2008), 126–131 Casanova H, Legrand A, Quinson M (2008) Simgrid: A generic framework for large-scale distributed experiments. Tenth International Conference on Computer Modeling and Simulation (uksim 2008), 126–131
31.
go back to reference Feitelson DG, Tsafrir D, Krakov D (2014) Experience with using the parallel workloads archive. J Parallel Distrib Comput 74:2967–2982CrossRef Feitelson DG, Tsafrir D, Krakov D (2014) Experience with using the parallel workloads archive. J Parallel Distrib Comput 74:2967–2982CrossRef
Metadata
Title
Dynamic two-side matching of tasks and resources in wide-area distributed computing environments
Authors
Yao Song
Liang Wang
Limin Xiao
Runnan Shen
Jinquan Wang
Chenhao Zhang
Publication date
03-02-2023
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 9/2023
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-023-05056-y

Other articles of this Issue 9/2023

The Journal of Supercomputing 9/2023 Go to the issue

Premium Partner