mavenhadoophadoop-streaming

Compiling Apache Hadoop Source in Eclipse


After about 4 tries I've managed to use git to checkout apache's Hadoop source code, issue a

mvn eclipse:eclipse

command and then import all of the projects into eclipse. So far this has been the most successful I have been. I am ALMOST there. I attempted to build and clean all projects and everything was going well. Now I only have 3 errors to get rid of. I would be EXTREMELY appreciate if anyone could help me with this. I have been trying to get this to work for about 2 days or so. Anyway, in Eclipse I have about 20 Hadoop projects. There are only two that have errors. One is called "hadoop-streaming" and the other is "hadoop-tools-dist."

The error says the following:

Project 'hadoop-streaming' is missing required source folder: 'C:/Users/User/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf'

The other errors are mainly related to this one. That's what Eclipse says anyway. I tried to copy the folder to where it was needed but that didn't clear up the error. Does anyone know how to fix this? If you need anything from me just ask.


Solution

  • I don't know how the Hadoop project looks like, but I suppose that it is using maven. So instead of using the

    mvn eclipse:eclipse
    

    I would suggest using the import functionality of eclipse -> import maven projects (even though sometimes works harder than one might expect). Related to your issue, this kind of error occurred to me whenever there was some kind of folder that is not added as source folder in eclipse (generated folders for source etc.). So, I would advise checking that the folder you mentioned is already there and if it is added as a source folder. If not, just mark it accordingly (source folder). Clean the project (Project > Clean...) and if need update the maven project(Right click on the parent project > Maven > Update project ...)

    Good luck!