I have multipart/form-data
that I am posting to an express endpoint /data/upload
, form markup below:
form(enctype="multipart/form-data", action="/data/upload", method="post")
input(type="file", name="data")
I'm using busboy
to read the file stream, which is working fine. From there, I want to send the stream again as multipart/form-data
to a second Java backend, using the request
npm module. JS client/Java server code below:
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
var reqBody = {
url: server.baseURL + 'api/data',
headers: {
'Connection': 'keep-alive',
'Content-Type': 'multipart/form-data'
},
formData: {
file: fileStream
}
};
request.post(reqBody, function (err, r, body) {
// Do rendering stuff, handle callback
});
});
Java endpoint (api/data)
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
public void addData(FormDataMultiPart formDataMultiPart) {
// Handle multipart data here
}
I don't think I'm sending the file correctly as multipart/form-data
here... but I'm having a hard time figuring out how to essentially pipe the stream from busboy
directly to request
without reading/writing from a temp file on the client-side. Any ideas?
Java stack trace:
Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server has received a request on thread qtp1631904921-24
3 > POST http://localhost:8080/api/data
3 > Connection: keep-alive
3 > Content-Length: 199
3 > Content-Type: multipart/form-data; boundary=--------------------------331473417509479560313628
3 > Host: localhost:8080
Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server responded with a response on thread qtp1631904921-24
3 < 400
17:07:13.003 [qtp1631904921-24] WARN org.eclipse.jetty.http.HttpParser parseNext - bad HTTP parsed: 400 No URI for HttpChannelOverHttp@425137da{r=1,c=false,a=IDLE,uri=null}
Rahat's recommended change:
31 var reqBody = {
32 url: server.baseURL + 'data',
33 headers: {
34 'Connection': 'keep-alive',
35 'Content-Type': 'multipart/form-data'
36 }
37 };
38
39 req.pipe(req.busboy.pipe(request.post(reqBody)));
Threw error:
Error: Cannot pipe. Not readable.
at Busboy.Writable.pipe (_stream_writable.js:154:22)
The problem here is that you need to provide 'Content-Length' for the multipart upload manually, because request
(and underlying form-data
) can't figure it out by themselves.
So request sends invalid Content-Length: 199 (the same for any incoming file size), which breaks the java multipart parser.
There are multiple workarounds:
1) Use incoming request 'Content-Length'
request.post({
url: server.baseURL + 'api/data',
formData: {
file: {
value: fileStream,
options: {
knownLength: req.headers['content-length']
}
}
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
This will produce a bit incorrect request though, because incoming length includes other upload fields and boundaries, but busboy was able to parse it w/o any complaints
2) Wait until file is completely buffered by the node app then send it to java
var concat = require('concat-stream')
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
fileStream.pipe(concat(function (fileBuffer) {
request.post({
url: server.baseURL + 'api/data',
formData: {
file: fileBuffer
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
}))
})
This will increase app memory consumption, so you needed to be careful and consider using busboy limits
3) Buffer file to disk before uploading (just for the reference)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With