So, I've successfully executed the SimpleShortestPathComputation on my computer via the script shown here:
#VARIABLES
user_dir=/user/hduser
jar=giraph-examples/target/giraph-examples-1.1.0-for-hadoop-2.4.0-jar-with-dependencies.jar
runner=org.apache.giraph.GiraphRunner
computation=org.apache.giraph.examples.SimpleShortestPathsComputation
informat=org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
outformat=org.apache.giraph.io.formats.IdWithValueTextOutputFormat
#DELETE PREVIOUS
bin/hdfs dfs -rm -r $user_dir/output/shortestpaths
#GIRAPH JOB
bin/hadoop jar $GIRAPH_HOME/$jar $runner -Dgiraph.yarn.task.heap.mb=3000 $computation -vif $informat -vip $user_dir/input/tiny_graph.txt -vof $outformat -op $user_dir/output/shortestpaths -w 1
Now the problem is, I'm trying to run my own job. It's actually a direct copy-paste from the SimpleShortestPathComputation class, only changed the package name and class name. Trying to run it with -libjars. Here's the full script:
#VARIABLES
user_dir=/user/hduser
jar=giraph-examples/target/giraph-examples-1.1.0-for-hadoop-2.4.0-jar-with-dependencies.jar
runner=org.apache.giraph.GiraphRunner
computation=org.apache.giraph.examples.SimpleShortestPathsComputation
informat=org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
outformat=org.apache.giraph.io.formats.IdWithValueTextOutputFormat
libjars=/usr/local/hadoop-2.4.0/lib/giraphtrials.jar,$GIRAPH_HOME/giraph-core.jar
#Setup class paths
export HADOOP_CLASSPATH=/usr/local/hadoop-2.4.0/lib/giraphtrials.jar:$GIRAPH_HOME/$jar:$HADOOP_CLASSPATH
#DELETE PREVIOUS
bin/hdfs dfs -rm -r $user_dir/output/shortestpaths
#GIRAPH JOB
bin/hadoop jar $GIRAPH_HOME/$jar $runner -libjars $libjars \
GiraphAlgs.GiraphPBFS -vif $informat -vip $user_dir/input/tiny_graph.txt \
-vof $outformat -op $user_dir/output/shortestpaths -w 1
As you can see, I've tried to use -libjars and HADOOP_CLASSPATH suggestion from this Stackoverflow question to make it work but unfortunately it still gives me a ClassNotFoundException. For better or worse, it doesn't throw it at me in the terminal anymore (it used to): terminal picture. As you can see, it only fails with a general container message now.
Unfortunately, it still gives me the Java.lang.ClassNotFoundException in the logs: log picture. Using Hadoop 2.4.0 and Giraph 1.1.0. I'm running out of ideas what might be wrong with my Giraph and starting to think if I should change careers.
You need to have the jar which contains the class GiraphAlgs.GiraphPBFS in the hadoop classpath.
Also, verify that your classpath is correct set by running $bin/hadoop classpath.
Once in hadoop 2.7 setting HADOOP_CLASSPATH variable didn't work, I had to copy the jar in the hadoop share lib directory: HADOOP_HOME/share/hadoop/mapreduce/lib.
