Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

puppeteer execute a js function on the chosen page

This is the analyzed page https://www.diretta.it/.

In this page the content of the following days is loaded dynamically with the js without changing the URL of the site (you can try it at the top right of the table).

Using puppeteer, with the following code

await page.goto ('https://www.diretta.it/');

it loads the contents of today's page. Is there a way to load the page with tomorrow's content? i have to scrape information from the matches of the following days

the function in js executable from terminal for change day is:

> set_calendar_date ('1')
like image 272
Menne Avatar asked Feb 10 '19 22:02

Menne


People also ask

Does puppeteer execute JavaScript?

Puppeteer pages have a handy evaluate() function that lets you execute JavaScript in the Chrome window. The evaluate() function is the most flexible way to interact with Puppeteer, because it lets you control Chrome using browser APIs like document.

How do you evaluate a page in a puppeteer?

evaluate() method. Evaluates a function in the page's context and returns the result. If the function passed to page. evaluteHandle returns a Promise, the function will wait for the promise to resolve and return its value.

Can puppeteer run in browser?

Puppeteer lets you automate the testing of your web applications. With it, you can run tests in the browser and then see the results in real-time on your terminal. Puppeteer uses the WebDriver protocol to connect with the browser and simulate user interaction with HTML elements or pages.


1 Answers

What you are looking for is the page.evaluate() function. This function lets you run any JS function in the page context.

In simpler terms, running page.evaluate() is akin to opening Dev tools and writing set_calendar_date('1') there directly.

Here is a working snippet, don't hesitate to pass {headless: false} to puppeteer.launch() if you want to see it working with your own eyes.

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.diretta.it/');
  await page.evaluate(() => {
    set_calendar_date ('1');
  });
  await page.waitFor(500); //Wait a bit for the website to refresh contents

  //Updated table is now available
})();
like image 66
MadWard Avatar answered Oct 13 '22 01:10

MadWard