`Error: Evaluation failed: Object` and `Error: Protocol error (Network.getResponseBody): Request content was evicted from inspector cache`

Question

• I am trying to download .ts files of different videos using Puppeteer.
• I do this by keeping track of all the .ts requests (which I invoke myself) and then saving/appending all their responses.
• However, sometimes UnhandledPromiseRejectionWarning: Error: Evaluation failed: Object occurs, which points to page.evaluate, out of no where. Although, I suspect this mostly happens when lots of .ts files are getting processed.
• Also, similarly, whenever there is a large file with lots of .ts, UnhandledPromiseRejectionWarning: Error: Protocol error (Network.getResponseBody): Request content was evicted from inspector cache occurs.

Any suggestions on how I could -

1. Solve the page.evaluate issue.

2. Clear the inspector cache when I am done with particular requests/data and make Puppeteer save a large number of .ts files.

/* This whole code is inside an async function */

/*    .... code for finding array ts ....      */

let requests = [], prs = [];

page.on('request', req => {
    if(req.url().endsWith('.ts')){
        requests.push(req);
    }
});

/* 
    ts[i].ts contains all the links to a stream/video's .ts files
    ts[i].filename contains the name of that video
*/

for (i = 0; i < ts.length; i++) {
    for(const ele of ts[i].ts){ 
        // Not writing await is on purpose; for saving all the pending promises
        let x = page.evaluate(async (link, file) => {

            // jQuery is included in the page itself
            return $.ajax({
                url : link,
                headers : {"custom" : file}
            });
        }, ele, ts[i].filename);

        prs.push(x);
    }

    // This makes request.response() be filled
    await Promise.all(prs);

    for(let req of requests){
        let res = await req.response().buffer();
        let fname = req.headers().customHeader;

        // Data written to relevant file
        fs.appendFileSync(fname, res);  
    }

    prs = [];
    requests = [];
}

Answer 1

There is an issue about the cache error in puppeteer repo: #1599

There's a limitation on the resource content size that devtools keep in memory. It's 10Mb for resource, and there's also a total of 100Mb limit to store all resources

There's experimental support in the protocol to amend these limits: Network.enable.

And here's how to try to enable working with larger files:

await page._client.send('Network.enable', {
  maxResourceBufferSize: 1024 * 1204 * 100,
  maxTotalBufferSize: 1024 * 1204 * 200,
})

Maybe this will help in your case.