I have a Grails 3.1 application that uses Spring Boot version 1.3.7.RELEASE. It's deployed on AWS Elastic Beanstalk using the platform "Tomcat 9 with Corretto 8 running on 64bit Amazon Linux 2/4.9.1".
The application has several forms submitting data with multipart/form-data.
Suddenly, without any changes in the application code or configuration, POST requests from these forms started to fail with the FileCountLimitExceededException.
My question is:
How can I properly configure or fix this issue so multipart requests without files or with few files don't trigger this error?
And why is this error suddenly occurring in production on AWS Elastic Beanstalk?
StackTrace Error:
Jul 25 15:39:23 ip-172-31-4-71 server:
org.springframework.web.multipart.MultipartException: Could not parse
multipart servlet request; nested exception is
org.apache.tomcat.util.http.fileupload.impl.FileCountLimitExceededException:
attachment Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:111)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.<init>(StandardMultipartHttpServletRequest.java:85)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.multipart.support.StandardServletMultipartResolver.resolveMultipart(StandardServletMultipartResolver.java:76)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.DefaultUrlMappingInfo.getResolvedRequest(DefaultUrlMappingInfo.java:280)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.DefaultUrlMappingInfo.tryMultipartParams(DefaultUrlMappingInfo.java:270)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.DefaultUrlMappingInfo.checkDispatchAction(DefaultUrlMappingInfo.java:248)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.DefaultUrlMappingInfo.getActionName(DefaultUrlMappingInfo.java:229)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.AbstractUrlMappingInfo.populateParamsForMapping(AbstractUrlMappingInfo.java:98)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.AbstractUrlMappingInfo.configure(AbstractUrlMappingInfo.java:68)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.mvc.AbstractGrailsControllerUrlMappings$_collectControllerMappings_closure1.doCall(AbstractGrailsControllerUrlMappings.groovy:183)
Jul 25 15:39:23 ip-172-31-4-71 server: at
sun.reflect.GeneratedMethodAccessor351.invoke(Unknown Source) Jul 25
15:39:23 ip-172-31-4-71 server: at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jul 25 15:39:23 ip-172-31-4-71 server: at
java.lang.reflect.Method.invoke(Method.java:498) Jul 25 15:39:23
ip-172-31-4-71 server: at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
Jul 25 15:39:23 ip-172-31-4-71 server: at
groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) Jul 25
15:39:23 ip-172-31-4-71 server: at
org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
Jul 25 15:39:23 ip-172-31-4-71 server: at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024) Jul 25
15:39:23 ip-172-31-4-71 server: at
groovy.lang.Closure.call(Closure.java:414) Jul 25 15:39:23
ip-172-31-4-71 server: at groovy.lang.Closure.call(Closure.java:430)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3124)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3095)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.mvc.AbstractGrailsControllerUrlMappings.collectControllerMappings(AbstractGrailsControllerUrlMappings.groovy:178)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.mapping.mvc.GrailsControllerUrlMappings.matchAll(GrailsControllerUrlMappings.java:56)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.web.mapping.UrlMappingsHolder$matchAll.call(Unknown Source) Jul
25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.ReflectionUtils.matchAllUrlMappings(ReflectionUtils.groovy:210)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.access.intercept.AnnotationFilterInvocationDefinition.determineUrl(AnnotationFilterInvocationDefinition.groovy:109)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.access.intercept.AbstractFilterInvocationDefinition.getAttributes(AbstractFilterInvocationDefinition.groovy:74)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.access.intercept.AbstractSecurityInterceptor.beforeInvocation(AbstractSecurityInterceptor.java:196)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:123)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.UpdateRequestContextHolderExceptionTranslationFilter.doFilter(UpdateRequestContextHolderExceptionTranslationFilter.groovy:64)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.groovy:53)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:157)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:169)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
javax.servlet.FilterChain$doFilter.call(Unknown Source) Jul 25
15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.rest.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:143)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:205)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.groovy:62)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:91)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
grails.plugin.springsecurity.web.SecurityRequestHolderFilter.doFilter(SecurityRequestHolderFilter.groovy:58)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:75)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:67)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:120)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.boot.context.web.ErrorPageFilter.access$000(ErrorPageFilter.java:61)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.boot.context.web.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:95)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:113)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:762)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:656)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:397)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:935)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1826)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1189)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:658)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
Jul 25 15:39:23 ip-172-31-4-71 server: at
java.lang.Thread.run(Thread.java:750) Jul 25 15:39:23 ip-172-31-4-71
server: Caused by:
org.apache.tomcat.util.http.fileupload.impl.FileCountLimitExceededException:
attachment Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:459)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.connector.Request.parseParts(Request.java:2691)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.connector.Request.parseParameters(Request.java:3014)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.connector.Request.getParameter(Request.java:1141)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.apache.catalina.connector.RequestFacade.getParameter(RequestFacade.java:309)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.filters.HiddenHttpMethodFilter.getHttpMethodOverride(HiddenHttpMethodFilter.java:71)
Jul 25 15:39:23 ip-172-31-4-71 server: at
org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:60)
AWS Upgrading to Tomcat 9.0.71 or later is what caused this to occur: https://tomcat.apache.org/security-9.html#Fixed_in_Apache_Tomcat_9.0.71
You will want to increase these settings in application.yml
spring.servlet.multipart.max-file-size=10MB
spring.servlet.multipart.max-request-size=100MB
spring.servlet.multipart.max-parts=50
In Grails 7, which uses Spring Boot 3.5.x the settings are:
server.tomcat.max-part-count
server.tomcat.max-part-header-size
If that does not work due to the older version of Grails and Spring Boot you are using, you will need to adjust the settings in tomcat's $CATALINA_HOME/conf/server.xml
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443"
maxPartCount="50" />