I am new to Hadoop. I want to test my mapper part alone using MRUnit Test. I have tried a lot. But i dont know how to solve the following error-
"The method setMapper(Mapper) in the type MapDriver is not applicable for the arguments (Recommand.IdIndexMapper)". I am using Hadoop-1.2.1, Eclipse Juno,mrunit-1.0.0-hadoop1.jar,junit-4.11,mockito-all-1.9.5.jar. Bellow are my code,
My Mapper Class:
Class name: Recommand,
public static class IdIndexMapper extends MapReduceBase implements Mapper<LongWritable, Text, Text, Text>{
public void map(LongWritable key, Text val, OutputCollector<Text, Text> output,Reporter reporter)throws IOException{
String[] ids;
String ln=val.toString();
ids=ln.split("\t");
output.collect(new Text(ids[0]),new Text(ids[1]));
// System.out.println(ids[0]+" "+ids[1]);
}
My Testing Code:
package org.apache.hadoop.examples;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.classification.InterfaceAudience.Private;
import org.apache.hadoop.classification.InterfaceAudience.Public;
import org.apache.hadoop.examples.WordCount.IntSumReducer;
//import org.apache.hadoop.examples.WordCount.TokenizerMapper;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mrunit.mapreduce.MapDriver;
import org.junit.Before;
import org.junit.Test;
import org.myorg.Recommand.IdIndexMapper;
public class RecomTest {
MapDriver<LongWritable, Text, Text, Text> mapDriver;
@Before
public void setUp() throws Exception {
IdIndexMapper mapper=new IdIndexMapper();
mapper.configure(new JobConf());
mapDriver=new MapDriver<LongWritable, Text, Text,Text>();
mapDriver.setMapper(mapper);
}
@Test
public void testMapper() throws IOException {
final LongWritable inputKey = new LongWritable(0);
final Text inputValue = new Text("M1023 M1024,M1022,M1025");
final Text outputKey = new Text("M1023");
final Text outputValue = new Text("M1024,M1022,M1025");
mapDriver.withInput(inputKey, inputValue);
mapDriver.withOutput(outputKey, outputValue);
mapDriver.runTest();
}
}
Error I got is:
The method setMapper(Mapper) in the type MapDriver is not applicable for the arguments (Recommand.IdIndexMapper)
Can anyone help me to solve this please?
1.0.0 MapDriver.setMapper()
expects a org.apache.hadoop.mapred.Mapper
. You are using the older org.apache.hadoop.mapreduce.Mapper
. They are two different animals. If you want to use MRUnit 0.8.x you can use the old Mapper
. You can get 0.8.x here
EDIT
I see the real problem. The above is incorrect. There are two MapDrivers
- org.apache.hadoop.mrunit.mapreduce.MapDriver
(the one you're using) and org.apache.hadoop.mrunit.MapDriver
. You should be using the latter MapDriver
Also note that there are two different MRUnits. There's a hadoop1 and hadoop2 version. The latter MapDriver
(the one you need) is in the hadoop2. You can get the download here