How to forward a multipart/form-data POST request in Node to another service

Question

I need to send a multipart/form-data POST (xliff file) from the client to my Node.js server, and then capture the data in Node.js and forward that POST to another Java service.

I've used both multer and express-fileupload to parse the form-data stream and capture a Buffer of the xliff in Node.js and both gave me the file with its content as a buffer just fine.

However, I cannot seem to re-create a FormData object in the Node layer to forward the POST to the Java service.

I continue to get the error message "Connection terminated parsing multipart data" or just no response at all form the Java service.

I've Also attempted to use the tmp library to create a temporary file locally to write the buffer and then try to FormData('file', fs.createReadStream(<path>)), but that didn't seem to work for me either... though I'm not sure I was doing it correctly.

Using the exact same doPOST request directly form the Browser works fine, but once I try to capture the call in the Node layer and then forward the POST to the Java service, it doesn't work for me anymore.

.

const multer = require('multer');
const upload = multer();

router.post('/', upload.any(), (req, res) => {
  const { headers, files } = req;

  console.log('--------------- files:', files[0]); // object with buffer, etc.

  const XMLString = files[0].buffer.toString('utf8'); // xml string of the xliff

  const formFile = new FormData();
  formFile.append('file', XMLString);

  console.log('--------------- formFile:', formFile); // FormData object with a key of _streams: [<xml string with boundaries>, [Function: bound ]]

  headers['Content-Type'] = 'multipart/form-data';
  const url = 'some/url/to/Java/service'

  doPOST(url, formFile, {}, headers)
    .catch((error) => {
      const { status, data } = error.response;
      res.status(status).send(data);
    })
    .then(({ data }) => {
      res.send(data);
    });
});

Answer 1

You can directly pass the buffer to your form data, but then you also need to specify the filename parameter.

const multer = require('multer');
const upload = multer();

router.post('/', upload.any(), (req, res) => {
  const { headers, files } = req;
  const { buffer, originalname: filename } = files[0];

  const formFile = new FormData();
  formFile.append('file', buffer, { filename });

  headers['Content-Type'] = 'multipart/form-data';
  const url = 'some/url/to/Java/service'

  doPOST(url, formFile, {}, headers)
    .catch((error) => {
      const { status, data } = error.response;
      res.status(status).send(data);
    })
    .then(({ data }) => {
      res.send(data);
    });
});

Answer 2

This is a pseudo code example of the solution I came up with in NodeJS. I used a similar solution in ApolloGQL, but it applies all the same to ExpressJS. Hence, my example is in a pattern more akin to ExpressJS.

In the below example shows how to pass a JSON object as well as passing a file buffer into the FormData before sending it off.

const FormData = require('form-data'); // version ^3.0.0

router.post('/', async (req, res) => {
  const { body } = req;
  const { stringContentOfSomeFile } = body;
  
  // create formData for your request:
  const thisForm = new FormData();

  // passing a JSON object:
  // must declare "contentType: application/json" to avoid 415 status response from some systems:
  const someJson = JSON.stringify({  key: 'value', otherKey: 'otherValue' });
  thisForm.append('data', someJson, { contentType: 'application/json' });
  
  // passing a file buffer:
  const fileBuffer = Buffer.from(stringContentOfSomeFile, 'utf-8');
  thisForm.append('form_field_name_here', fileBuffer, 'file_name_here');

  const response = await axios.post('/path/to/endpoint', thisForm, {
    // must getHeaders() from "formData" to define the boundaries of the appended data:
    headers: { ...thisForm.getHeaders() },
  });

  // do whatever you need with the response:
  res.send(response);
});