spring-cloud-streamspring-cloud-dataflow

How do I setup multiple directory configuration using Spring Cloud Dataflow pre-packaged SFTP source?


When using the pre-packaged SFTP source I've been unable to configure multiple directories. Does anyone have a clear example on how to set this up? Either with multiple factories or with a single factory and multiple directories.

The example supplied here just shows the following config without any explanation of how to define the one. and two. code:

sftp.supplier.factories.one.host=host1
sftp.supplier.factories.one.port=1234,
sftp.supplier.factories.one.username = user1,
sftp.supplier.factories.one.password = pass1,
...
sftp.supplier.factories.two.host=host2,
sftp.supplier.factories.two.port=2345,
sftp.supplier.factories.two.username = user2,
sftp.supplier.factories.two.password = pass2,
sftp.supplier.directories=one.sftpSource,two.sftpSecondSource,
sftp.supplier.max-fetch=1,
sftp.supplier.fair=true

I've tried several different approaches and none seem to work, I always get a fileNotFoundException from jsch, which I assume is because its just defaulting to / for the remote directory for each factory connector.


Solution

  • The configuration looks correct.

    Those sftp.supplier.factories.one props would create a ConnectionFactory under one key. Same for two. Those sftp.supplier.directories means dirs per factory key. So, your one supposes to have that sftpSource dir to pull. Same for two - sftpSecondSource remote directory to pull from that second server.