Accessing javascript variable in Puppeteer

Question

I'm trying to render a page using puppeteer, extract the name of a dynamic javascript variable on the page, and return the variable as an object. I'm using puppeteer and node.js to do this.

I'm able to get the name of the javascript variable that is being generated on page but anytime I try to print it out to the console I am getting a "undefined" error. I've spent the better part of the morning trying to figure this out but I'm stumped. Here's the code below.

const puppeteer= require('puppeteer');

var url = 'https://www.website.com';
async function run () {

try {
    const browser = await puppeteer.launch({
        headless: true,
        args: ['--no-sandbox']
    });
    const page = await browser.newPage();
    await page.goto(url);

    const flashVariableName = await page.evaluate( () => {
        var start = document.documentElement.innerHTML.indexOf('var flashvariable_') + 'var '.length;
        var tempDoc = document.documentElement.innerHTML.substr(start);
        var end = tempDoc.indexOf(' =');
        return  tempDoc.substr(0, end);
    });

        console.log(await page.evaluate( (flashVariableName) => flashVariableName ));

        await browser.close();
    } catch(e){
        console.log("Error Occurred", e)
    }
}

run();

flashVariableName is returning the right name for the javascript variable on the page but how can I actually get access to the varialble?

Answer 1

You have two issues.

This is the first one:

console.log(await page.evaluate( (flashVariableName) => flashVariableName ));
//                                ^^^^^^^^^^^^^^^^^
//                                |
//                                +--> this is undefined

If you look at the documentation for Page#evaluate

What you're missing is the ...args bit which supplies parameters to the pageFunction. It should have been this instead:

console.log(await page.evaluate( (flashVariableName) => flashVariableName, flashVariableName ));
//                                                                         ^^^^^^^^^^^^^^^^^

But that is not enough because flashVariableName is just a string at this point. It is the name of a variable but not the variable itself. You still need to "evaluate" it:

console.log(await page.evaluate( (varName) => window[varName], flashVariableName ));

However that only works if the variable you are trying to read is available in the global scope (i.e. window in your case) AND it can be serialised (e.g. if it refers to a function or a DOM element that won't be possible)