Speaker: Wojciech Kazana (Google)

Title: From MapReduce to Pregel and Flume: a successful story of taming distributed computing.

Abstract: Over ten years have past since the introduction of MapReduce and while it is still at the very core of Google, the development has not stopped. The graph structure of Internet led to the creation of Pregel: a parallel graph processing system. The need for more and more complex MapReduce pipelines resulted in Flume: an efficient way to run parallel pipelines.

In this talk we will introduce the key ideas behind MapReduce: both the details of the original model and the technical difficulties that an efficient implementation has to face. We will then show why Pregel and Flume are not so obvious extensions and how we were able to make them work.