Anzeige
Erschienen in:
2010 | OriginalPaper | Buchkapitel
FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads
verfasst von : Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, Sujay Parekh, Kun-Lung Wu, Andrey Balmin
Erschienen in: Middleware 2010
Verlag: Springer Berlin Heidelberg
Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.
Wählen Sie Textabschnitte aus um mit Künstlicher Intelligenz passenden Patente zu finden. powered by
Markieren Sie Textabschnitte, um KI-gestützt weitere passende Inhalte zu finden. powered by
Originally, MapReduce implementations such as
Hadoop
employed
First In First Out
(
fifo
) scheduling, but such simple schemes cause job starvation. The
Hadoop Fair Scheduler
(
hfs
) is a slot-based MapReduce scheme designed to ensure a degree of fairness among the jobs, by guaranteeing each job at least some minimum number of allocated slots. Our prime contribution in this paper is a different,
flexible
scheduling allocation scheme, known as
flex
. Our goal is to optimize any of a variety of standard scheduling theory metrics (response time, stretch, makespan and
Service Level Agreements
(
sla
s), among others) while ensuring the same minimum job slot guarantees as in
hfs
, and maximum job slot guarantees as well. The
flex
allocation scheduler can be regarded as an add-on module that works synergistically with
hfs
. We describe the mathematical basis for
flex
, and compare it with
fifo
and
hfs
in a variety of experiments.