How to wait for all downloads to complete with Puppeteer?

Question

I have a small web scraping application that downloads multiple files from a web application where the URLs require visting the page.

It works fine if I keep the browser instance alive in between runs, but I want to close the instance in between runs. When I call browser.close() my downloads are stopped because the chrome instance is closed before the downloads have finished.

Does puppeteer provide a way to check if downloads are still active, and wait for them to complete? I've tried page.waitForNavigation({ waitUntil: "networkidle0" }) and "networkidle2", but those seem to wait indefinitely.

---

• node.js 8.10
• puppeteer 1.10.0

Answer 1

My solution is to use chrome's own chrome://downloads/ page to managing download files. This solution can be very easily to auto restart a failed download using chrome's own feature

This example is 'single thread' currently, because it's only monitoring the first item appear in the download manager page. But you can easily adapt it to 'infinite threads' by iterating through all download items (#frb0~#frbn) in that page, well, take care of your network:)

dmPage = await browser.newPage()
await dmPage.goto('chrome://downloads/')

await your_download_button.click() // start download

await dmPage.bringToFront() // this is necessary
await dmPage.waitForFunction(
    () => {
        // monitoring the state of the first download item
        // if finish than return true; if fail click
        const dm = document.querySelector('downloads-manager').shadowRoot
        const firstItem = dm.querySelector('#frb0')
        if (firstItem) {
            const thatArea = firstItem.shadowRoot.querySelector('.controls')
            const atag = thatArea.querySelector('a')
            if (atag && atag.textContent === '在文件夹中显示') { // may be 'show in file explorer...'? you can try some ids, classess and do a better job than me lol
                return true
            }
            const btn = thatArea.querySelector('cr-button')
            if (btn && btn.textContent === '重试') { // may be 'try again'
                btn.click()
            }
        }
    },
    { polling: 'raf', timeout: 0 }, // polling? yes. there is a 'polling: "mutation"' which kind of async
)
console.log('finish')