MalformedStreamException: Stream ended unexpectedly

Question

Given:

byteString is

-----------------------------149742642616556
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain

test
-----------------------------149742642616556--

Then this code (not optimized):

Pattern pattern = Pattern.compile(BOUNDARY_PATTERN); // "(?m)\\A-+\\d+$"
Matcher matcher = pattern.matcher(byteString);
String boundary = null;
while (matcher.find()) {
    boundary = matcher.group();
    contentType = "multipart/form-data; boundary=" + boundary;
}
LOG.info("Content Type = " + contentType);

@SuppressWarnings("deprecation")
org.apache.commons.fileupload.MultipartStream multipartStream =
        new org.apache.commons.fileupload.MultipartStream(new ByteArrayInputStream(byteString.getBytes()), boundary.getBytes());
ByteArrayOutputStream bos = new ByteArrayOutputStream();
multipartStream.readBodyData(bos); // throw error
byte[] byteBody = bos.toByteArray();

Throws this error:

org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:1005)
    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:903)
    at java.io.InputStream.read(InputStream.java:101)
    at org.apache.commons.fileupload.util.Streams.copy(Streams.java:100)
    at org.apache.commons.fileupload.util.Streams.copy(Streams.java:70)
    at org.apache.commons.fileupload.MultipartStream.readBodyData(MultipartStream.java:593)

What could be possibly wrong here? I would appreciate a help here.

Answer 1

The issue seems to be due to a bad end of line and the way the boundary is retrieved. According to a RFC2046 quote taken from a SO answer:

The Content-Type field for multipart entities requires one parameter, "boundary". The boundary delimiter line is then defined as a line consisting entirely of two hyphen characters ("-", decimal value 45) followed by the boundary parameter value from the Content-Type header field, optional linear whitespace, and a terminating CRLF.

The problem lies precisely on two points: the end of line type and the two hyphens preceding the boundary parameter value.

End of lines

Since your code doesn't show accurately the value of byteString, I tried both LF (\n) and CRLF (\r\n) end of lines to see what will happen.

It appears the issue is reproduced when a bad end of line - i.e. not CRLF - is right before the last boundary, as shown below:

String byteString=
    "-----------------------------149742642616556\r\n" +
    "Content-Disposition: form-data; name=\"file\"; filename=\"test.txt\"\r\n" +
    "Content-Type: text/plain; charset=UTF-8\r\n" +
    "\r\n" +
    "test\r\n" + // <-- only \n here lead to a MalformedStreamException
    "-----------------------------149742642616556--\r\n";

It sounds like the MultipartStream fails to parse the begin of the boundary, since it doesn't catch a right end of line (CRLF) on the previous line. So, I you used LF terminators, you should replace them by CRLF ones.

Boundary format

The RFC tells that a boundary delimiter is two hyphens + boundary parameter + CRLF. Your regexp doesn't catch only the boundary parameter value, it also includes the two hyphens. So I replaced this part:

// capturing group = boundary parameter value
String regexp="(?m)\\A--(-*\\d+)$";
// [...]
while (matcher.find()) {
    boundary = matcher.group(1);
    // [...]
}

Working code

Runnable as a MCVE

The code you'll find below can be run in a console without Tomcat. Only commons-fileupload-1.3.3-bin.tar.gz and commons-io-2.6-bin.tar.gz are needed.

To view what's parsed by the MultipartStream, I temporarily replaced bos by System.out in the readBodyData() call (as told in the comments).

• To compile:

  javac Test.java -classpath ./commons-fileupload-1.3.3-bin/commons-fileupload-1.3.3.jar 
  

• To run:

  java -classpath ./commons-fileupload-1.3.3-bin/commons-fileupload-1.3.3.jar:./commons-io-2.6/commons-io-2.6.jar:. Test
  

The code itself

import java.util.regex.*;
import java.io.*;
import org.apache.commons.fileupload.*;

public class Test {
    public final static void main(String[] argv) {
    String byteString=
        "-----------------------------149742642616556\r\n" +
        "Content-Disposition: form-data; name=\"file\"; filename=\"test.txt\"\r\n" +
        "Content-Type: text/plain; charset=UTF-8\r\n" +
        "\r\n" +
        "test\r\n" + // <-- only \n here lead to a MalformedStreamException
        "-----------------------------149742642616556--\r\n";

    String regexp="(?m)\\A--(-*\\d+)$"; // edited regexp to catch the right boundary

    Pattern pattern = Pattern.compile(regexp);
    Matcher matcher = pattern.matcher(byteString);
    String boundary = null;
    String contentType=null;
    while (matcher.find()) {
        boundary = matcher.group(1);
        contentType = "multipart/form-data; boundary=\"" + boundary + "\"";
    }

    System.out.println("boundary = \"" + boundary + "\"");

    @SuppressWarnings("deprecation")
        org.apache.commons.fileupload.MultipartStream multipartStream =
        new org.apache.commons.fileupload.MultipartStream
        (new ByteArrayInputStream(byteString.getBytes()), boundary.getBytes());
     ByteArrayOutputStream bos = new ByteArrayOutputStream();

    try {
        // Use the commented line instead the following one
        // To see what the multipartStream is reading (for debug)
        // multipartStream.readBodyData(System.out);
        multipartStream.readBodyData(bos);
    } catch (MultipartStream.MalformedStreamException e) {
        System.out.println("Malformed Exception " + e.getMessage());
    } catch (IOException e) {
        System.out.println(e.getMessage());
    }
    byte[] byteBody = bos.toByteArray();

    // Displaying the body read
    for(byte c : byteBody) {
        System.out.format("%c", c);
    }
    System.out.println();
    }
}

Output:

boundary = "---------------------------149742642616556"
-----------------------------149742642616556
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=UTF-8

test