MapReduce is one among the famous processing model for huge scale information (Big Data)
processing in distributed computing. Since there may be a possibility of slot based MapReduce
framework (eg. Hadoop MRv1) displaying some poor execution as a result of its unoptimized resource
allocation. To venture on this, this paper finds and further streamlines the data distribution and resource
allocation from the following three key perspectives. To begin with, because of the pre-configuration of
the map slots and reduce slots which are not replaceable slots can be extremely under used. Since map
slots may be completely used while reduce slots are empty and the other way around, considering the slot
based model we set forth an option strategy called Dynamic Hadoop Slot Allocation. It unwinds the slot
allocation parameters to permit slots to be reallocated to map or reduce task assignments relying upon their
needs. Second the speculative execution can handle the straggler issue which sufficiently fit to enhance the
execution for a job however to determine the expense of cluster proficiency. In context of this we further
show Speculative Execution Performance Balancing so as to adjust the execution exchange between a
single job and a batch of jobs. Third, delay scheduling has indicated to enhance the information and data
locality at the fair cost. On the other hand we propose a method called Slot Pre Scheduling that can
enhance the data locality yet with no effect on cost. At last by melding all the strategies together we make
an orderly slot allocation framework called DynMR (Dynamic Map Reduce) which can enhance the
execution of MapReduce workloads significantly.
Real Time Impact Factor:
Pending
Author Name: Anil Sagar T, Ramakrishna V Moni
URL: View PDF
Keywords: MapReduce, DynMR, Delay Scheduler, Hadoop Fair Scheduler, Slot Allocation, Slot Pre Schedule
ISSN: 2394-2231
EISSN: 2394-2231
EOI/DOI:
Add Citation
Views: 1750