springhadoopspring-dataspring-data-hadoop

How to simulate hdfs operations using spring data


I'm new to spring data-hadoop and would like to ask one general question. I have files in different format and would like to extract the useful content with Apache Tika and store as text files in HDFS. I've gone through the reference documentation of spring data-hadoop(http://docs.spring.io/spring-hadoop/docs/2.0.0.RELEASE/reference/html/store.html) but didn't understand how to do it. And I didn't find any other useful resources for this.

Is there any sample projects or sources for writing data to HDFS using spring data-hadoop ?


Solution

  • From Risberg's comment one useful example :-

    https://github.com/trisberg/springone-2015/tree/master/boot-ingest

    Another code snippet with TextFileWriter implementation of DataWriter interface :-

       //build naming strategy
                 ChainedFileNamingStrategy namingStrategy = 
          new ChainedFileNamingStrategy( 
                Arrays.asList(new FileNamingStrategy[] { 
                    new StaticFileNamingStrategy("document"),
                             new UuidFileNamingStrategy(someUUID),
                              new StaticFileNamingStrategy("txt", ".") }));
            //set the naming strategy 
                textFileWriter.setFileNamingStrategy(namingStrategy);
                textFileWriter.write("this is a test content");
           //flush and close the writer
                textFileWriter.flush();
                textFileWriter.close();