PADS: Performance-Aware Dynamic Scheduling for Effective MapReduce Computation in Heterogeneous Clusters

  • Prince Hamandawana
  • , Ronnie Mativenga
  • , Se Jin Kwon
  • , Tae Sun Chung

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

A lot of previous works on Map-Reduce improved job completion performance through implementing additional instrumentation modules which collects system level information for making scheduling decisions. However the extra instrumentation may not scale well with increasing number of task-trackers. To this end, we design PADS, a lightweight scheduler which uses time prediction to schedule tasks without additional instrumentation modules. Results shows PADS improves performance by 6%, 12%, and 9% as compared to ESAMR, LA, and DDAS respectively.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages160-161
Number of pages2
ISBN (Electronic)9781538683194
DOIs
StatePublished - 29 Oct 2018
Event2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 - Belfast, United Kingdom
Duration: 10 Sep 201813 Sep 2018

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Volume2018-September
ISSN (Print)1552-5244

Conference

Conference2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
Country/TerritoryUnited Kingdom
CityBelfast
Period10/09/1813/09/18

Keywords

  • Hadoop
  • Heterogeneity
  • MapReduce
  • Scheduling

Fingerprint

Dive into the research topics of 'PADS: Performance-Aware Dynamic Scheduling for Effective MapReduce Computation in Heterogeneous Clusters'. Together they form a unique fingerprint.

Cite this