Thesis (M.Sc.) - Cairo University - Faculty of Computers and Information - Department of Computer Science

MapReduce is a framework and runtime environment for big data processing over distributed systems (e.g., cluster, cloud, and grids). MapReduce has become an effective framework for processing and analysis of huge data size in large systems. On the other hand, Hadoop represents one of the core frameworks based on Map/Reduce for Big Data analysis and processing. One of the critical issues in MapReduce is task failure which could increase the cost of the job and affect resource utilization. Currently, MapReduce fault tolerance mechanism is based on rescheduling failure tasks on other nodes to re- execute again. Therefore, task rescheduling affects resource utilization, as well as, execution time. In this thesis, a new Rollback-recovery model called Pessimistic Log-based rollback (PLR) is introduced to support MapReduce fault tolerance. According to the proposed PLR model, a logging process has introduced to enable rollback by recording the task, which is determinant in the log report when the failure occurs. When a task is failed, the proposed PLR model will reactivate the execution of this task starting from the last state before failing on the same node which optimistically can solve the MapReduce task failure problem. In the worst case, the task will be rescheduled into another node to be re-executed. The experimental results of the proposed PLR model show that MapReduce performance is improved in the case of failure by reducing the execution time by 35% approximately

Issued also as CD