I've some very old PHP code that does a POST multipart/form-data request to a Tomcat server and it passes in it an XML string as one part and a file, as attachment, as second part.
For completeness, the code is listed below:
function SendTmsNewsToHost($host, $port, $dest_url, $tnixdata, $attachment) {
global $debug, $TMS_LineId;
$mime_boundary = md5(date('r', time()));
if ($port==0) $port = 8080;
$destination = "http://$host:$port/$dest_url";
$eol = "\r\n";
$tmsnews_xml = '--' . $mime_boundary . $eol;
$tmsnews_xml .= 'Content-Disposition: form-data; name="_tnix"' . $eol . 'Content-Type: text/xml' . $eol . $eol;
$tmsnews_xml .= $tnixdata . $eol;
$msg_length = strlen($tmsnews_xml);
$tmsnews_attachment = "";
$fstr="";
if($attachment!='') {
// MIME attachments type recognization, not necessary for DAM
/*
require('mime_type_lib.php');
$mime_type = get_file_mime_type($attachment);
*/
$base_name=basename($attachment);
$fpr = fopen($attachment, "rb");
$fsize=filesize($attachment);
$fstr = fread($fpr, $fsize);
fclose($fpr);
$tmsnews_attachment = '--' . $mime_boundary . $eol;
$tmsnews_attachment .= 'Content-Disposition: attachment; name="attachment"; filename="' . $base_name . '"' . $eol;
$file_length = strlen($fstr);
$tmsnews_attachment .= 'Content-Length: ' . $file_length . $eol;
$tmsnews_attachment .= 'Content-Transfer-Encoding: binary' . $eol . $eol;
// include attachment length
// (Everything coming after the first empty line (\r\n) -- including your boundary delimiters -- should be counted in the total length )
// but not first: + strlen($eol);
$msg_length += strlen($tmsnews_attachment) + $file_length ;
}
$content = $tmsnews_xml . ($attachment!=''? $tmsnews_attachment . $fstr . $eol: '') . "--$mime_boundary--". $eol . $eol;
$curl = curl_init($destination);
if (is_resource($curl) === true) {
# TODO content-length check
$realContentSize = strlen($content);
$contentSize = ($msg_length+strlen($eol . "--$mime_boundary--". $eol . $eol));
flog(FINFO, "content logging");
flog(FINFO, "content logging $realContentSize ".$realContentSize);
flog(FINFO, "content logging $contentSize ".$contentSize);
flog(FINFO, "content logging");
flog(FINFO, $content);
# TODO content-length check
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: multipart/form-data; boundary="' . $mime_boundary . '"',
'Host: ' . "$host:$port",
'Content-length: ' . ($msg_length+strlen($eol . "--$mime_boundary--". $eol . $eol)),
'Connection: Close'
));
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_VERBOSE, true);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, $content);
$response = curl_exec($curl);
curl_close($curl);
return $response;
}
On the Java side, there is a Servlet which was parsing the entire InputStream by hand and extracting the information from the payload.
This worked fine with Tomcat 9.x, but it doesn't work anymore with Tomcat 11.0.9.
In particular, I hit the following problems:
allowCasualMultipartParsing="true" in the Tomcat context of the app;To solve the second point, I tried to rework the existing code to use the capabilities offered by the Servlet API and I refactored it as follows:
// Already existing code
public MultipartFormData(final HttpServletRequest request, final long byteLimit) throws IOException {
String contentType = getContentType(request);
if (contentType == null || !contentType.equalsIgnoreCase(CONTENT_TYPE)) {
throw new IllegalArgumentException("invalid content type: " + contentType);
}
if (getBoundary(request) == null) {
throw new IllegalArgumentException("missing boundary");
}
try {
initFromStream(request, byteLimit);
} finally {
close();
}
}
// Already existing code
@Override
public void close() {
Enumeration<String> fields = getFileFormFields();
while (fields.hasMoreElements()) {
String field = fields.nextElement();
String[] localPaths = getLocalPathValues(field);
for (String localPath : localPaths) new File(localPath).delete();
}
fileInfos_.clear();
}
// Already existing code
private String getContentType(final HttpServletRequest request) {
String contentType = request.getContentType();
if (contentType == null) {
return null;
}
int semiColonIndex = contentType.indexOf(';');
if (semiColonIndex == -1) {
return contentType;
}
return contentType.substring(0, semiColonIndex);
}
// Already existing code
private byte[] getBoundary(final HttpServletRequest request) {
String contentType = request.getContentType();
if (contentType == null) {
return null;
}
int boundaryIndex = contentType.toLowerCase().indexOf("boundary");
if (boundaryIndex == -1) {
return null;
}
int startIndex = boundaryIndex + 9;
// fix : gequotete boundaries auch akzeptieren
char delimiter = ';';
if ((contentType.length() > startIndex)
&& (contentType.charAt(startIndex) == '"')) {
startIndex++;
delimiter = '"';
}
int endIndex = startIndex;
for (int i = startIndex; i < contentType.length()
&& contentType.charAt(i) != delimiter; ++i) {
++endIndex;
}
return contentType.substring(startIndex, endIndex).getBytes();
}
// Refactored code
private void initFromStream(final HttpServletRequest input, final long byteLimit) throws IOException {
long requestSize = input.getContentLengthLong();
if (requestSize == -1) {
try {
requestSize = input.getParts().stream().mapToLong(Part::getSize).sum();
} catch (ServletException e) {
throw new IOException(e);
}
}
if (requestSize > byteLimit) throw new ByteLimitExceededException("byte limit exceeded: " + byteLimit);
readParts(input);
}
// Already existing code
private void addParameter(final String name, final String value) {
List<String> values = parameters_.get(name);
if (values == null) {
values = new LinkedList<>();
}
values.add(value);
parameters_.put(name, values);
}
// Refactored code
private void readParts(final HttpServletRequest input) throws IOException {
try {
for (Part part : input.getParts()) {
if (part.getSubmittedFileName() == null) {
addParameter(part.getName(), readValue(part));
} else {
addFileInfo(part.getName(), new FileInfo(readData(part), part.getSubmittedFileName(), part.getContentType()));
}
}
} catch (ServletException ex) {
throw new IOException(ex);
}
}
// Refactored code
private String readValue(Part part) throws IOException {
StringBuilder builder = new StringBuilder();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(part.getInputStream()))) {
String line;
while ((line = reader.readLine()) != null) builder.append(line).append(lineSeparator());
}
return builder.toString().strip();
}
// Refactored code
private String readData(Part part) throws IOException {
File file = File.createTempFile("multipartdatabean", null);
file.deleteOnExit();
try (BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(file));
BufferedInputStream in = new BufferedInputStream(part.getInputStream())) {
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = in.read(buffer)) != -1) {
out.write(buffer, 0, bytesRead);
}
}
return file.getPath();
}
// Already existing code
private void addFileInfo(final String name, final FileInfo info) {
List<FileInfo> infos = fileInfos_.get(name);
if (infos == null) {
infos = new LinkedList<>();
}
infos.add(info);
fileInfos_.put(name, infos);
}
Result: no errors anymore, but only the XML gets indeed processed. The attachment is simply lost.
I tried then by removing allowCasualMultipartParsing="true" and adding an explicit multipart configuration for the Servlet in the web.xml like
<servlet>
<servlet-name>NewsQueryService</servlet-name>
<servlet-class>foo.NSPortal</servlet-class>
<init-param>
<param-name>log4j-init-file</param-name>
<param-value>WEB-INF/etc/log4j.properties</param-value>
</init-param>
<init-param>
<param-name>log4j-read-intervall</param-name>
<param-value>10000</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
<multipart-config>
<location>/home/nsc/tmp</location>
<max-file-size>104857600</max-file-size>
<max-request-size>418018841</max-request-size>
<file-size-threshold>10485760</file-size-threshold>
</multipart-config>
</servlet>
but I got the same result.
I tried then to rework the implementation by using the Apache Commons FileUpload library and the examples in here, but I got the same result.
Also, since I've Spring Web 6.x available, I tried using their MultipartFilter in the web.xml like:
<filter>
<filter-name>multipart</filter-name>
<filter-class>org.springframework.web.multipart.support.MultipartFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>multipart</filter-name>
<servlet-name>NewsQueryService</servlet-name>
</filter-mapping>
and reworked the implementation above as follows:
public MultipartFormData(final HttpServletRequest request, final long byteLimit) throws IOException {
if(!(request instanceof MultipartHttpServletRequest multipartRequest)) throw new IllegalArgumentException("Not a multipart request");
long totalFileSizeBytes = 0;
for(Entry<String, MultipartFile> file : multipartRequest.getFileMap().entrySet()) {
MultipartFile multipartFile = file.getValue();
totalFileSizeBytes += multipartFile.getSize();
if (totalFileSizeBytes > byteLimit) throw new ByteLimitExceededException("byte limits exceeded");
File osFile = multipartFile.getResource().getFile();
addFileInfo(file.getKey(), new FileInfo(osFile.getPath(), osFile.getName(), multipartFile.getContentType()));
}
Enumeration<String> parameterNames = multipartRequest.getParameterNames();
while (parameterNames.hasMoreElements()) {
String parameterName = parameterNames.nextElement();
String[] parameterValues = multipartRequest.getParameterValues(parameterName);
for (String value : parameterValues) {
addParameter(parameterName, value);
}
}
}
but I got once more the same outcome.
Finally, looking at their filter implementation, I tried to mimic it for the limits I need as shown below:
public MultipartFormData(final HttpServletRequest request, final long byteLimit) throws IOException {
try {
long totalFileSizeBytes = 0;
for(Part part : request.getParts()) {
totalFileSizeBytes += part.getSize();
if (totalFileSizeBytes > byteLimit) throw new ByteLimitExceededException("byte limits exceeded");
String headerValue = part.getHeader(CONTENT_DISPOSITION);
ContentDisposition disposition = parse(headerValue);
String filename = disposition.getFilename();
if (filename == null) {
addParameter(part.getName(), readValue(part));
} else {
addFileInfo(part.getName(), new FileInfo(readData(part), filename, part.getContentType()));
}
part.delete();
}
} catch (ServletException ex) {
throw new IOException(ex);
}
}
but also this, in the end, produced the same outcome.
Does somebody have any idea?
The root cause is PHP client sends Content-Disposition: attachment instead of Content-Disposition: form-data for file parts.
Here's how you can fix the code:
// Change this line:
$tmsnews_attachment .= 'Content-Disposition: attachment; name="attachment"; filename="' . $base_name . '"' . $eol;
// To this:
$tmsnews_attachment .= 'Content-Disposition: form-data; name="attachment"; filename="' . $base_name . '"' . $eol;
Also remove the invalid Content-Length header from individual parts:
// Remove this line entirely:
$tmsnews_attachment .= 'Content-Length: ' . $file_length . $eol;
This way tomcat 11 will properly parse both XML and file parts.
If you can't modify the PHP client, As an alternative, implement a custom lenient multipart parser on the Java side, but fixing the client is the proper solution.