Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to fix 'page.goto is not a function' error in a web scraper

Tags:

puppeteer

I'm trying to build a web scraper using puppeteer that scrapes my venmo page to look for payments. When I try to run my script I get an error that says "page.goto is not a function"

I'm honestly not quite sure where to even start with this

const puppeteer = require('puppeteer');

const url = 'generic.com';

(async () =>  {

//running in headless to observe what happens for now 
const browser = await puppeteer.launch({headless: false});
const page = browser.newPage();
await page.goto(url);

let data = await page.evaluate(() => {

    let amount = document.querySelector('span.bold.medium.green').innerText;
    let timePayed = document.querySelector('a.grey_link').innerText;

    return { 
         amount,
         timePayed
    }
});

console.log(data);

debugger;

await browser.close();

})();

This is my error message

UnhandledPromiseRejectionWarning: TypeError: page.goto is not a function
at D:\venmoScraper\scraper.js:12:12
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:13212) UnhandledPromiseRejectionWarning: Unhandled promise 
rejection. This error originated either by throwing inside of an async 
function without a catch block, or by rejecting a promise which was not 
handled with .catch(). (rejection id: 1)
(node:13212) [DEP0018] DeprecationWarning: Unhandled promise rejections 
are deprecated. In the future, promise rejections that are not handled 
will terminate the Node.js process with a non-zero exit code.
like image 650
rupp16 Avatar asked Dec 11 '22 01:12

rupp16


1 Answers

The line,

const page = browser.newPage();

should be written as,

const page = await browser.newPage();

browser.newPage() returns as Promise.

like image 161
Md. Abu Taher Avatar answered Dec 12 '22 16:12

Md. Abu Taher