Error in downloading files from file server with Spring-Integration
Following the Spring guidelines, I was trying to use a SmbOutboundGateway from the spring-integration-smb library to download everything (whole structure, directory + files) at the remote file server specified by the SessionFactory.
Ref.
https://docs.spring.io/spring-integration/reference/smb.html#using-the-mget-command
https://docs.spring.io/spring-integration/reference/smb.html#outbound-gateway-java-dsl
However when I try to run the flow, it failed in between.
Source
import jcifs.DialectVersion;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.Gateway;
import org.springframework.integration.annotation.MessagingGateway;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway;
import org.springframework.integration.smb.dsl.Smb;
import org.springframework.integration.smb.dsl.SmbOutboundGatewaySpec;
import org.springframework.integration.smb.session.SmbSessionFactory;
import java.io.File;
@Configuration
public class SmbConfig {
@Bean
public SmbSessionFactory smbSessionFactory() {
SmbSessionFactory smbSession = new SmbSessionFactory();
smbSession.setHost("myhost");
smbSession.setPort(445);
smbSession.setDomain("mydomain");
smbSession.setUsername("myusername");
smbSession.setPassword("mypassword");
smbSession.setShareAndDir("pccommon/Patrick/");
smbSession.setSmbMinVersion(DialectVersion.SMB210);
smbSession.setSmbMaxVersion(DialectVersion.SMB311);
return smbSession;
}
@MessagingGateway
public interface MyGateway {
@Gateway(requestChannel = "testChannel")
void test(String path);
}
@Bean
public SmbOutboundGatewaySpec downloadFileGateway() {
return Smb.outboundGateway(smbSessionFactory(),
AbstractRemoteFileOutboundGateway.Command.MGET, "payload")
.options(AbstractRemoteFileOutboundGateway.Option.RECURSIVE)
.localDirectory(new File("" +
"/media/sf_linuxshare/Test"));
}
@Bean
public SmbOutboundGatewaySpec uploadFileGateway() {
return Smb.outboundGateway(smbSessionFactory(),
AbstractRemoteFileOutboundGateway.Command.MPUT, "'/'")
.options(AbstractRemoteFileOutboundGateway.Option.RECURSIVE)
.regexFileNameFilter(".*\\.xml$")
.remoteDirectoryExpression("'/uploaded'");
}
@Bean
public IntegrationFlow testFlow() {
return IntegrationFlow.from("testChannel")
.handle(downloadFileGateway())
.handle(uploadFileGateway())
// .channel(c -> c.queue("uploadFileOutputChannel"))
.get();
}
}
Objective
I was expecting that any folders and underlying files under the remote file server would be copied to local after the download flow, i.e. having the following as a result:
/media/sf_linuxshare/Test/Test_From_Remote/a.xml
/media/sf_linuxshare/Test/Test_From_Remote/b.xml
Before run
After run (actual result)
Observations
- It complained that the target files are not files
Caused by: java.io.IOException: [a.xml] is not a file.
2023-12-04 17:27:46.225 INFO 26615 --- [nio-8080-exec-2] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-12-04 17:27:46.225 INFO 26615 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2023-12-04 17:27:46.226 INFO 26615 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : Completed initialization in 1 ms
2023-12-04 17:27:46.354 INFO 26615 --- [nio-8080-exec-2] o.s.i.smb.session.SmbSessionFactory : SMB share init: hprifil0005.corp.auspost.local:445/pccommon/Patrick/
2023-12-04 17:27:53.008 WARN 26615 --- [nio-8080-exec-2] o.s.i.smb.outbound.SmbOutboundGateway : File name pattern must be '*' when using recursion
2023-12-04 17:27:53.069 INFO 26615 --- [nio-8080-exec-2] o.s.integration.smb.session.SmbSession : Successfully listed 1 in [null].
2023-12-04 17:27:53.141 INFO 26615 --- [nio-8080-exec-2] o.s.integration.smb.session.SmbSession : Successfully listed 2 in [./Test_From_Remote//].
2023-12-04 17:27:53.331 WARN 26615 --- [nio-8080-exec-2] a.c.a.c.c.e.CustomRestExceptionHandler : 0a8315f8-9547-4f44-969a-02f292be017f org.springframework.messaging.MessageHandlingException: Failure occurred while copying from remote to local directory, failedMessage=GenericMessage [payload=, headers={replyChannel=nullChannel, errorChannel=, id=026be840-5e21-32c1-c894-b6759f40c043, timestamp=1701671266279}]
org.springframework.messaging.MessageHandlingException: Failure occurred while copying from remote to local directory
at org.springframework.integration.support.utils.IntegrationUtils.wrapInHandlingExceptionIfNecessary(IntegrationUtils.java:191)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.get(AbstractRemoteFileOutboundGateway.java:1135)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.getRemoteFileForMget(AbstractRemoteFileOutboundGateway.java:1289)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.mGetWithRecursion(AbstractRemoteFileOutboundGateway.java:1251)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.mGet(AbstractRemoteFileOutboundGateway.java:1192)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.lambda$doMget$9(AbstractRemoteFileOutboundGateway.java:691)
at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:452)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.doMget(AbstractRemoteFileOutboundGateway.java:690)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.handleRequestMessage(AbstractRemoteFileOutboundGateway.java:586)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:136)
at org.springframework.integration.handler.AbstractMessageHandler.doHandleMessage(AbstractMessageHandler.java:105)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
at org.springframework.integration.channel.AbstractMessageChannel.sendInternal(AbstractMessageChannel.java:373)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:327)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:297)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:151)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:143)
at org.springframework.integration.gateway.MessagingGatewaySupport.send(MessagingGatewaySupport.java:467)
at org.springframework.integration.gateway.GatewayProxyFactoryBean.sendOrSendAndReceive(GatewayProxyFactoryBean.java:669)
at org.springframework.integration.gateway.GatewayProxyFactoryBean.invokeGatewayMethod(GatewayProxyFactoryBean.java:584)
at org.springframework.integration.gateway.GatewayProxyFactoryBean.doInvoke(GatewayProxyFactoryBean.java:550)
at org.springframework.integration.gateway.GatewayProxyFactoryBean.invoke(GatewayProxyFactoryBean.java:540)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:223)
at jdk.proxy3/jdk.proxy3.$Proxy78.test(Unknown Source)
at au.com.auspost.cbg.web.controller.HealthCheckController.mobilestarHealthCheck(HealthCheckController.java:134)
at au.com.auspost.cbg.web.controller.HealthCheckController.createHealthCheckText(HealthCheckController.java:84)
at au.com.auspost.cbg.web.controller.HealthCheckController.healthCheck(HealthCheckController.java:67)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:207)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:152)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1011)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903)
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:289)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.web.servlet.resource.ResourceUrlEncodingFilter.doFilter(ResourceUrlEncodingFilter.java:66)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:365)
at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:170)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82)
at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:166)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:738)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:341)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:894)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.IOException: [a.xml] is not a file.
at org.springframework.integration.smb.session.SmbSession.read(SmbSession.java:204)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.get(AbstractRemoteFileOutboundGateway.java:1124)
... 130 common frames omitted
2023-12-04 17:27:53.392 WARN 26615 --- [nio-8080-exec-2] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [org.springframework.messaging.MessageHandlingException: Failure occurred while copying from remote to local directory, failedMessage=GenericMessage [payload=, headers={replyChannel=nullChannel, errorChannel=, id=026be840-5e21-32c1-c894-b6759f40c043, timestamp=1701671266279}]]
2. It tried to rename the target files (it suffixed it with the containing folder name) which I am not sure why this was performed. i.e. it renamed the files to "a.xmlTest_From_Remote", "b.xmlTest_From_Remote"
Thanks in advance for any insights.
Solution
Confirmed as a bug: https://github.com/spring-projects/spring-integration/issues/8800.
Will be fixed in the next version.
As a workaround you can try to implement this MGET functionality yourself using SmbRemoteFileTemplate.
On the other hand you can try to use the whole remote directory name instead of empty input as you have right now. That's, essentially, how we have missed the problem with plain file name for SMB instead of expected with sub-directories when we fetch recursively.
When we have a this or that fix for the MGET, let's chat about MPUT part in a separate SO thread!
- Postgres / hibernate operator does not exist: text = bytea
- Spring boot not loading log4j2.xml
- How to verify 'Open xdg-open popup' in Chrome via Selenium Webdriver?
- Junit is throwing error with the com.fasterxml.jackson.databind.exc.InvalidDefinitionException
- @Repository not necessary when implementing JpaRepository?
- Implementing custom methods of Spring Data repository and exposing them through REST
- How to add custom method to Spring Data JPA
- How to use the Netbeans Profiler programmatically?
- Unknown entity: com.sun.proxy.$Proxy87
- What is an object graph in the Java garbage collector?
- Why does the paging library stop loading data after several calls?
- What is the fastest way to create a 2GB file containing random bytes in Java?
- SAXParseException; src-resolve: Cannot resolve the name '...' to a(n) 'type definition' component
- Is it good idea to use the DAO design pattern with spring Boot, instead of using repository design pattern provided by spring data jpa?
- Can i sum two String values without using Wrapper Class in java if yes than how ?
- How to debug the below flux code when expected log is missing?
- Implementing a simple turn-based game in Java using the wait-notify approach
- Why is Spring initialising my Aspect twice?
- AndroidManifest.xml - specified for property 'manifest' does not exist
- java.lang.RuntimeException: Duplicate class org.intellij.lang.annotations.Flow found in modules annotations-16.0.1.jar and annotations-java5-15.0.jar
- JFrame doesn't appear with ActionListener
- Escaping special character when generating an XML in Java
- Is it possible to override package path of Maven dependency?
- IntelliJ gradle add module dependency
- Paging Compile Issue : Not sure how to convert a Cursor to this method's return type
- Weird behaviour of Java MethodHandle.invokeExact
- How do I use a custom Serializer with Jackson?
- loadInitial method not getting called in PositionalDataSource<Item>
- How to use "=toString" in modify-overwrite-beta operation in input with objects?
- Ignore unused local variable warnings in VS Code