mapreducereducerscombiners

combiner and reducer can be different?


In many MapReduce programs, I see a reducer being used as a combiner as well. I know this is because of the specific nature of those programs. But I am wondering if they can be different.


Solution

  • Yes, a combiner can be different to the Reducer, although your Combiner will still be implementing the Reducer interface. Combiners can only be used in specific cases which are going to be job dependent. The Combiner will operate like a Reducer, but only on the subset of the Key/Values output from each Mapper.

    One constraint that your Combiner will have, unlike a Reducer, is that the input/output key and value types must match the output types of your Mapper.