apache-camel

Apache camel: SFTP: files downloaded multiple times


I've defined the following camel route:

    RouteBuilder rb = new RouteBuilder() {
        @Override
        public void configure() throws Exception {
            from("sftp://myhost//path/to/files/")
            .to("log:loggingCategory?level=INFO")
            .to("file:///tmp/");
        }
    };

When I start the context with this route camel does connect and it downloads the files. My problem is that camel repeats downloading the same files until the context is shut down. Why does the FTP2 component do this and how can I stop it?

I've included version 2.10.4 of camel-core and camel-ftp via maven.


Solution

  • The Indempotent Consumer does the trick. Docs of the FTP2 component refer to the File2 component as "as all the options there also applies for this component". There is a parameter idempotent=true that activates usage of an LRUCache:

    Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.

    My complete source definition now looks like this:

    from("sftp://myhost//path/to/files/?username=user&password=secret&idempotent=true")