Axios request with chunked response stream from Node

Question

I have a client app in React, and a server in Node (with Express). At server side, I have an endpoint like the following (is not the real endpoint, just an idea of what i'm doing):

function endpoint(req, res) {
   res.writeHead(200, {
        'Content-Type': 'text/plain',
        'Transfer-Encoding': 'chunked'
      });

   for(x < 1000){
      res.write(some_string + '\n');
      wait(a_couple_of_seconds); // just to make process slower for testing purposes
   }

   res.end();
}

This is working perfect, i mean, when I call this endpoint, I receive the whole stream with all the 1.000 rows.

The thing is that I cannot manage to get this data by chunks (for each 'write' or a bunch of 'writes') in order to show that on the frontend as soon as i'm receiving them..(think of a table that shows the rows as soon as i get them from the endpoint call)

In the frontend I'm using Axios to call the API with the following code:

async function getDataFromStream(_data): Promise<any> {
    const { data, headers } = await Axios({
        url: `http://the.api.url/endpoint`,
        method: 'GET',
        responseType: 'stream',
        timeout: 0,
    });

    // this next line doesn't work. it says that 'on' is not a function
    data.on('data', chunk => console.log('chunk', chunk)); 
    // data has actually the whole response data (all the rows)

    return Promise.resolve();
}

The thing is that the Axios call returns the whole data object after the 'res.end()' on the server is called, but I need to get data as soon as the server will start sending the chunks with the rows (on each res.write or whenever the server thinks is ready to send some bunch of chunks).

I have also tried not to use an await and get the value of the promise at the 'then()' of the axios call but it is the same behavior, the 'data' value comes with all the 'writes' together once the server does the 'res.end()'

So, what I doing wrong here ? maybe this is not possible with Axios or Node and I should use something like websockets to solve it. Any help will be very appreciate it because I read a lot but couldn't get a working solution yet.

Answer 1

For anyone interested in this, what I ended up doing is the following:

At the Client side, I used the Axios onDownloadProgress handler that allows handling of progress events for downloads.

So, I implemented something like this:

function getDataFromStream(_data): Promise<any> {
    return Axios({
        url: `http://the.api.url/endpoint`,
        method: 'GET',
        onDownloadProgress: progressEvent => {
           const dataChunk = progressEvent.currentTarget.response;
           // dataChunk contains the data that have been obtained so far (the whole data so far).. 
           // So here we do whatever we want with this partial data.. 
           // In my case I'm storing that on a redux store that is used to 
           // render a table, so now, table rows are rendered as soon as 
           // they are obtained from the endpoint.
        }


    }).then(({ data }) => Promise.resolve(data));    

}