I am studying Hadoop from the Definitive Guide book and tried to execute this piece of code which is resulting in the error.
Example from Chapter-5. Link to Github code:
source:
https://github.com/tomwhite/hadoop-book/blob/master/ch05/src/main/java/v1/MaxTemperatureMapper.java
public class MaxTemperatureMapper
extends Mapper<LongWritable, Text, Text, IntWritable> {
@Override
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
String year = line.substring(15, 19);
int airTemperature = Integer.parseInt(line.substring(87, 92));
context.write(new Text(year), new IntWritable(airTemperature));
}
}
public class MaxTemperatureMapperTest {
@Test
public void processesValidRecord() throws IOException, InterruptedException {
Text value = new Text("0043011990999991950051518004+68750+023550FM-12+0382" +
"99999V0203201N00261220001CN9999999N9-00111+99999999999");
new MapDriver<LongWritable, Text, Text, IntWritable>()
.withMapper(new MaxTemperatureMapper())
.withInputValue(value)
.withOutput(new Text("1950"), new IntWritable(-11))
.runTest();
}
The error I'm getting is the following:
java.lang.IllegalStateException: No input was provided
at org.apache.hadoop.mrunit.MapDriverBase.preRunChecks(MapDriverBase.java:286)
at org.apache.hadoop.mrunit.mapreduce.MapDriver.run(MapDriver.java:142)
at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:640)
at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:627)
at book.hadoopdefinitiveguide.chap5.examples.MaxTemperatureMapperTest.processesValidRecord(MaxTemperatureMapperTest.java:12)
This happens to be the first code I'm executing in hadoop and it is throwing this error. Any help is appreciated. Thanks in advance
As @Thomas Junblut pointed out, you need to specify a key along with the value.
But (assuming you're using mrunit 1.0 or higher), withInputKey|Value
is deprecated. You should instead be using withInput(K1 key, V1 val)
, where you specify bot the key and value in one method, instead of withInputValue(..).withInputKey(..)
. So you'd have something like this
new MapDriver<LongWritable, Text, Text, IntWritable>()
.withMapper(new MaxTemperatureMapper())
.withInput(new LongWritable(), value)
.withOutput(new Text("1950"), new IntWritable(-11))
.runTest();
The new LongWritable()
is just an arbitrary key.
EDIT
So after further testing, it's not a problem with your code (except for the deprecation, but that isn't the cause).
I ran the same test using the code I posted and got the exact same error. I tested with some old practice project I had. It seemed to be a problem with the classes not being built correctly. I created a whole new project, rewrote (copy-paste) the mapper class and created a new test case, saved everything, ran it, and it worked fine. Try to do this. BTW I'm on eclipse using the Eclipse Hadoop plugin to create the MR Project