Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

13-11-2020 | Issue 6/2021

The Journal of Supercomputing 6/2021

OKCM: improving parallel task scheduling in high-performance computing systems using online learning

Journal:
The Journal of Supercomputing > Issue 6/2021
Authors:
Jingbo Li, Xingjun Zhang, Li Han, Zeyu Ji, Xiaoshe Dong, Chenglong Hu
Important notes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Abstract

Task scheduling is becoming increasingly important in large-scale high-performance computing real-time systems as the parallel scale, number and types of task continue to increase. The prioritizing policies and backfilling mechanisms are the most useful practice for improving scheduling performance. In particular, these methods highly depend on the task running time prediction. Previous studies focused on improving the running time prediction accuracy, resulting in higher time overhead and deployment difficulties in real-time scheduling system. In this paper, an efficient running time prediction model, referred to as online learning and K-nearest neighbors (KNN)-based predictor with correction mechanism (OKCM), is proposed. OKCM updates in real time through online algorithm and is friendly to users with a small data accumulation by KNN-based predictor. To evaluate our model, a trace-driven simulator, named HPCsim, is designed and implemented. The experimental results demonstrated that OKCM can achieve higher prediction accuracy with a low overhead. Furthermore, OKCM can achieve significant scheduling performance improvement and can be used to enhance primary prioritizing and backfilling methods without being restricted by specific scheduling method.

Please log in to get access to this content

To get access to this content you need the following product:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 58.000 Bücher
  • über 300 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Testen Sie jetzt 30 Tage kostenlos.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 50.000 Bücher
  • über 380 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Umwelt
  • Maschinenbau + Werkstoffe




Testen Sie jetzt 30 Tage kostenlos.

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Umwelt
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 30 Tage kostenlos.

Literature
About this article

Other articles of this Issue 6/2021

The Journal of Supercomputing 6/2021 Go to the issue

Premium Partner

    Image Credits