Sending a CSV file from browser to nodejs server

Question

I am trying to send a csv file which is uploaded by the user, from browser to nodejs server for processing (file is over 50 mb, so the page becomes unresponsive). I'm using XMLHttpRequest for this purpose. I cannot find a solution to this. Any help is appreciated.

Javascript code

 var csv = document.getElementById('inputFile').files[0];
 var request = new XMLHttpRequest();
 request.open("POST", "/handleFile", true);
 request.setRequestHeader("Content-type", "text/csv");
 request.onreadystatechange = function() {
   if (request.readyState === XMLHttpRequest.DONE && request.status === 200) {
     console.log("yey");
   }
 }

 request.send(csv);

NodeJS server

 var express = require('express')
 var app = express()
 var bodyparser = require('body-parser')

 app.post('/handleFile', function(req, res) {
   console.log(req.body); // getting {} empty object here....
   console.log(req);

   var csv = req.body;
   var lines = csv.split("\n");
   var result = [];
   var headers = lines[0].split("\t");

   for (var i = 1; i < lines.length; i++) {
     var obj = {};
     var currentline = lines[i].split("\t");

     for (var j = 0; j < headers.length; j++) {
       obj[headers[j]] = currentline[j];
     }

     result.push(obj);
   }

   fileData = result;
 });

What did I do wrong? Is the XMLHttpRequest used incorrectly? or there is some other thing that i did not understand ? why is there no data in req.body even though its a post request. Or is there any other way to send a csv/text file to nodejs server from front end.

This question is not a duplicate because, the body-parser i.e. the middleware responsible for parsing the req.body does not handle text/csv and multipart/form-data . The above link is not the correct solution.

Answer 1

So, after looking around, I found that the problem was not my XMLHttpRequest. The request was received by the server just fine, but the body-parser could not parse the text/csv and multipart/form-data content-type. Here is the step by step answer to this problem.

1. In the client/browser-end whenever you are sending a large file to the server, convert it into multipart/form-data . It is the correct way of sending a text/csv/anyfile to the server.

   var csv=document.getElementById('inputFile').files[0];
   var formData=new FormData();
   formData.append("uploadCsv",csv);
   var request = new XMLHttpRequest();
   
    //here you can set the request header to set the content type, this can be avoided.
    //The browser sets the setRequestHeader and other headers by default based on the formData that is being passed in the request.
    request.setRequestHeader("Content-type", "multipart/form-data"); //----(*)
    request.open("POST","/handleFile", true);
   request.onreadystatechange = function (){
       if(request.readyState === XMLHttpRequest.DONE && request.status === 200) {
       console.log("yey");
       }
   }
   
   request.send(formData);
   

So, this is how you'll send your http request to the nodejs server.

2. On Node js server: Normally for application/json or any other request type the body-parser would have worked fine. But for large data and files i.e. multipart/form-data body-parser cannot parse the req.body. Thus it will give req.body as {} (empty object). Read about body-parser here.

So for these content-type you can use other middleware for handleling the request. Some are multer,multiparty,busboy etc. I used multer. Here is the code snipet.

    //EXPRESS
    var express = require('express')
    var app = express()

    var config=require('./config.js');
    //multer
    var multer  = require('multer');
    var upload = multer();
    app.post('/handleFile',upload.single('uploadCsv'), function(req, res, next) {
          // req.file is the `uploadCsv` file 
          // req.body will hold the text fields, if there were any 
          console.log(req.file);
          // the buffer here containes your file data in a byte array 
          var csv=req.file.buffer.toString('utf8');
     });

NOTE: This will still give you an error in nodejs server. hint: It has something to do with the line (*). Try removing it and see what happens. Google the rest ;)