Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I fetch a Readable Stream then convert to JSON client side?

I'm hoping to use a Google Sheets CSV as a data source. I'd like to fetch the CSV data on the client, then convert into JSON.

I'm able to fetch the CSV, which returns a ReadableStream. However, I'm not sure how to correctly read that response and convert into JSON.

I've found an npm package which should help convert the CSV data to JSON, but having a little bit of a time working with the stream.

Example: https://jsfiddle.net/21jeq1h5/3/

Can anyone point me in the right direction to use the ReadableStream?

like image 512
Vinnie James Avatar asked May 13 '18 21:05

Vinnie James


People also ask

How do I get json from fetch response?

To get JSON from the server using the Fetch API, you need to use the JavaScript fetch() method. Then you need to call the response. json() method to get the JSON.

How do you read data from a ReadableStream?

This involves two methods — ReadableStream. pipeThrough() , which pipes a readable stream through a writer/reader pair to transform one data format into another, and ReadableStream. pipeTo() , which pipes a readable stream to a writer acting as an end point for the pipe chain.

How do you read from a ReadableStream in node JS?

read() Method. The readable. read() method is an inbuilt application programming interface of Stream module which is used to read the data out of the internal buffer. It returns data as a buffer object if no encoding is being specified or if the stream is working in object mode.

What is difference between Axios and fetch?

Differences between Axios and Fetch:Axios has url in request object. Fetch has no url in request object. Axios is a stand-alone third party package that can be easily installed. Fetch is built into most modern browsers; no installation is required as such.


1 Answers

Since CSV is simply text, the solution is the use the response.text() method of the fetch() API.

https://developer.mozilla.org/en-US/docs/Web/API/Body/text

Once the text is onboard, it is as simple as parsing the CSV out of the file. If you want objects as an output it is imperative the headers are included in the CSV (which yours are).

I've included the code snippet below. It won't run on SO because SO sets the origin to null on AJAX requests. So I've also included a link to a working codepen solution.

fetch('https://docs.google.com/spreadsheets/d/e/KEY&single=true&output=csv')
  .then(response => response.text())
  .then(transform);

function transform(str) {
  let data = str.split('\n').map(i=>i.split(','));
  let headers = data.shift();
  let output = data.map(d=>{obj = {};headers.map((h,i)=>obj[headers[i]] = d[i]);return obj;});
  console.log(output);
}

Pen

https://codepen.io/randycasburn/pen/xjzzvW?editors=0012

Edit

I should add that if you truly want this in a JSON string (per your question), you can run

json = JSON.stringify(output);
like image 95
Randy Casburn Avatar answered Sep 30 '22 14:09

Randy Casburn