I'm trying to implement a Spring Integration Flow transferring files from a local file system to a remote file system using Spring Integration's SMB support.
My Spring Boot configuration is as follows:
@Configuration
public class LocalToSmbFileTransferConfiguration {
@Bean
public SmbSessionFactory smbSessionFactory() {
final var smbSessionFactory = new SmbSessionFactory();
smbSessionFactory.setHost("myHost");
smbSessionFactory.setPort(445);
smbSessionFactory.setDomain("myDomain");
smbSessionFactory.setUsername("myUsername");
smbSessionFactory.setPassword("myPassword");
smbSessionFactory.setShareAndDir("/myShareAndDir");
smbSessionFactory.setSmbMinVersion(DialectVersion.SMB210);
smbSessionFactory.setSmbMaxVersion(DialectVersion.SMB311);
return smbSessionFactory;
}
@Bean
public IntegrationFlow fileToSmbSyncFlow() {
return IntegrationFlow.from(
Files.inboundAdapter(new File("myLocalDirectory"))
.preventDuplicates(false),
e -> e.poller(Pollers.fixedDelay(1000).maxMessagesPerPoll(-1))
)
.transform(Files.toStringTransformer())
.handle(Smb.outboundAdapter(smbSessionFactory(), FileExistsMode.REPLACE)
.useTemporaryFileName(true)
.remoteDirectory("/remoteDirectory"),
e -> e.advice(originalFileDeletionAdvice())
)
.get();
}
@Bean
public ExpressionEvaluatingRequestHandlerAdvice originalFileDeletionAdvice() {
final var advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnSuccessExpressionString("headers['" + FileHeaders.ORIGINAL_FILE + "'].delete()");
advice.setOnFailureExpressionString("headers['" + FileHeaders.ORIGINAL_FILE + "'].delete()");
return advice;
}
}
The final result of the synchronization is as expected, i.e. all files in the local directory are synchronized to the remote directory and deleted in the local directory.
However, when there are multiple files in the local directory, I can observe a delay of about 6 to 7 seconds between each single file transfer in my application logs:
2025-02-20T15:57:59.194Z INFO 6496 --- [ scheduling-1] o.s.i.smb.session.SmbSessionFactory : SMB share init: myHost:445/myShareAndDir
2025-02-20T15:58:06.311Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Remote directory [/remoteDirectory/] exists in share [smb://myDomain;myUsername:******@myHost:445/myShareAndDir].
2025-02-20T15:58:06.329Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully wrote remote file [/remoteDirectory/file1.txt.writing].
2025-02-20T15:58:06.340Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully renamed remote resource [/remoteDirectory/file1.txt.writing] to [/remoteDirectory/file1.txt].
2025-02-20T15:58:06.346Z INFO 6496 --- [ scheduling-1] o.s.i.smb.session.SmbSessionFactory : SMB share init: myHost:445/myShareAndDir
2025-02-20T15:58:12.401Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Remote directory [/remoteDirectory/] exists in share [smb://myDomain;myUsername:******@myHost:445/myShareAndDir].
2025-02-20T15:58:12.420Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully wrote remote file [/remoteDirectory/file2.txt.writing].
2025-02-20T15:58:12.429Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully renamed remote resource [/remoteDirectory/file2.txt.writing] to [/remoteDirectory/file2.txt].
2025-02-20T15:58:12.436Z INFO 6496 --- [ scheduling-1] o.s.i.smb.session.SmbSessionFactory : SMB share init: myHost:445/myShareAndDir
2025-02-20T15:58:19.503Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Remote directory [/remoteDirectory/] exists in share [smb://myDomain;myUsername:******@myHost:445/myShareAndDir].
2025-02-20T15:58:19.520Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully wrote remote file [/remoteDirectory/file3.txt.writing].
2025-02-20T15:58:19.530Z INFO 6496 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully renamed remote resource [/remoteDirectory/file3.txt.writing] to [/remoteDirectory/file3.txt].
My expectation was a delay of about 1 s (or 1000 ms) between each file transfer.
I have no explanation for the observed behaviour. I would appreciate if someone could explain what is happening in my setup and how multiple files can be transferred without delay.
Update
The application logs for the upstream scenario (see above) show one SMB connection being created per file being transferred.
The following application logs for the downstream scenario show one SMB connection being created per poll. For further context, see my comment on the answer below.
2025-02-21T08:20:44.106Z INFO 8648 --- [ scheduling-1] o.s.i.smb.session.SmbSessionFactory : SMB share init: myHost:445/myShareAndDir
2025-02-21T08:20:51.355Z DEBUG 8648 --- [ scheduling-1] o.s.i.smb.session.SmbSessionFactory : SMB share initialized.
2025-02-21T08:20:51.378Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/] for path [/remoteImportDirectory/].
2025-02-21T08:20:51.378Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/] for path [/remoteImportDirectory].
2025-02-21T08:20:51.424Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/] for path [/remoteImportDirectory/].
2025-02-21T08:20:51.425Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/] for path [/remoteImportDirectory].
2025-02-21T08:20:51.512Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully listed 3 in [/remoteImportDirectory]: [smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file1.txt, smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file2.txt, smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file3.txt]
2025-02-21T08:20:51.550Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file1.txt] for path [/remoteImportDirectory/file1.txt].
2025-02-21T08:20:51.655Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully read resource [/remoteImportDirectory/file1.txt].
2025-02-21T08:20:51.680Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file1.txt] for path [/remoteImportDirectory/file1.txt].
2025-02-21T08:20:51.704Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully removed resource [/remoteImportDirectory/file1.txt].
2025-02-21T08:20:51.704Z DEBUG 8648 --- [ scheduling-1] o.s.i.s.i.SmbInboundFileSynchronizer : deleted remote file: /remoteImportDirectory/file1.txt
2025-02-21T08:20:51.728Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file2.txt] for path [/remoteImportDirectory/file2.txt].
2025-02-21T08:20:51.820Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully read resource [/remoteImportDirectory/file2.txt].
2025-02-21T08:20:51.848Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file2.txt] for path [/remoteImportDirectory/file2.txt].
2025-02-21T08:20:51.873Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully removed resource [/remoteImportDirectory/file2.txt].
2025-02-21T08:20:51.873Z DEBUG 8648 --- [ scheduling-1] o.s.i.s.i.SmbInboundFileSynchronizer : deleted remote file: /remoteImportDirectory/file2.txt
2025-02-21T08:20:51.899Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file3.txt] for path [/remoteImportDirectory/file3.txt].
2025-02-21T08:20:51.997Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully read resource [/remoteImportDirectory/file3.txt].
2025-02-21T08:20:52.020Z DEBUG 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Created new jcifs.smb.SmbFile[smb://myDomain;myUsername:******@myHost:445/remoteImportDirectory/file3.txt] for path [/remoteImportDirectory/file3.txt].
2025-02-21T08:20:52.046Z INFO 8648 --- [ scheduling-1] o.s.integration.smb.session.SmbSession : Successfully removed resource [/remoteImportDirectory/file3.txt].
2025-02-21T08:20:52.046Z DEBUG 8648 --- [ scheduling-1] o.s.i.s.i.SmbInboundFileSynchronizer : deleted remote file: /remoteImportDirectory/file3.txt
2025-02-21T08:20:52.094Z DEBUG 8648 --- [ scheduling-1] o.s.i.s.i.SmbInboundFileSynchronizer : 3 files transferred from '/remoteImportDirectory'
Spring Boot comes with a single thread for TaskScheduler by default: https://docs.spring.io/spring-boot/reference/features/task-execution-and-scheduling.html.
You may consider to increase its pool:
spring.task.scheduling.pool.size=10
Also, it is strange to see preventDuplicates(false). That means that the same file might be picked up by the next polling cycle. I understand that you delete those files, but that still might be an issue.
Any chances that you can share with us a simple project to reproduce on our side?
UPDATE
For the connection caching, consider to use a CachingSessionFactory wrapper around your SmbSessionFactory: https://docs.spring.io/spring-integration/reference/ftp/session-caching.html.
The doc is about FTP, but same applies for SMB as well.
Feel free to raise a GH issue, so we mention CachingSessionFactory in the SMB chapter, too: https://docs.spring.io/spring-integration/reference/smb.html