Web App - saving to a local file a very large stream of data

Question

I'm wondering if there's some proper way (internet protocol, browser API, etc.), for a web app in a modern browser, to get a stream of large data from a Web Server (in my case, nodejs, but the question is focused on client side), and to save it to one big file in the client's file system. The assumption is that the amount of data is too large to be entirely buffered in the App's assigned memory, so I guess somehow the process should append chunks of data from the stream directly to a file.

The data is basically tens of thousands of db records, that should be written to a format such as csv.

Hope I'm making any sense here.

Answer 1

You asked if there was a proper way. No, I'm afraid the technology that does it is badly supported and brand new. However, if you are brave enough to try anyway this is what you have.

1. Getting a stream of data from Web Server

If you don't have to support IE then you can use the Fetch API. See this question. If you do have to support IE then you could use a websocket to push down little bits at a time.

2. Writing a stream of data to the disk

StreamSaver.js Is a project that tries to accomplish this using the ReadableStream API, however it currently only works in Opera and Chrome.