Puppeteer find array elements in page and then click

Question

Hello I have site that url are rendered by javascript. I want to find all script tags in my site, then math script src and return only valid ones. Next find parent of script and finally click on link. This is what i have:

const scripts = await page.$$('script').then(scripts => {
    return scripts.map(script => {
        if(script.src.indexOf('aaa')>0){
            return script
        }
    });
});
scripts.forEach(script => {
    let link = script.parentElement.querySelector('a');
    link.click();
});

My problem is that i have script.src is undefined. When i remove that condition i move to forEach loop but i get querySelector is undefined. I can write that code in js inside of console of debug mode but i cant move it to Puppeteer API.

from console i get results as expected

let scripts = document.querySelectorAll('script');
scripts.forEach(script=>{
let el = script.parentElement.querySelector('a');
console.log(el)
})

Answer 1

When you use $$ or $, it will return a JSHandle, which is not same as a HTML Node or NodeList that returns when you run querySelector inside evaluate. So script.src will always return undefined.

You can use the following instead, $$eval will evaluate a selector and map over the NodeList/Array of Nodes for you.

page.$$eval('script', script => {
    const valid = script.getAttribute('src').indexOf('aaa') > 0 // do some checks
    const link = valid && script.parentElement.querySelector('a') // return the nearby anchor element if the check passed;
    if (link) link.click(); // click if it exists
})

There are other ways to achieve this, but I merged all into one. Ie, If it works on browser, then you can also use .evaluate and run the exact code and get exact desired result.

page.evaluate(() => {
    let scripts = document.querySelectorAll('script');
    scripts.forEach(script => {
        let el = script.parentElement.querySelector('a');
        console.log(el) // it won't show on your node console, but on your actual browser when it is running;
        el.click();
    })
})