Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Nodejs/Puppeteer - How to use page.evaluate

I know is a noob question, but I want to know when I should use page.evaluate

I also know the documentation exists, but I still do not understand

Can anybody give me an explanation about how and when to use this function when creating a scraper with puppeteer?

like image 973
nanquim Avatar asked Aug 27 '18 19:08

nanquim


People also ask

What is Page evaluate in puppeteer?

Evaluates a function in the browser context. If the function, passed to the frame. evaluate, returns a Promise, then frame. evaluate would wait for the promise to resolve and return its value. If the function passed into frame.

What is Page $$ eval?

page.$$eval(selector, pageFunction[, ...args])This method runs Array. from(document. querySelectorAll(selector)) within the page and passes it as the first argument to pageFunction . If pageFunction returns a Promise, then page. $$eval would wait for the promise to resolve and return its value.

How do you get the title of the page in puppeteer?

Puppeteer has page. title() function to get the title of the current page.


1 Answers

First, it is important to understand that there are two main environments:

  • Node.js (Puppeteer) Environment
  • Page DOM Environment

You should use page.evaluate() when you are seeking to interact with the page directly in the page DOM environment by passing a function and returning a <Promise<Serializable>> which resolves to the return value of the passed function.

Otherwise, if you do not use page.evaluate(), you will be dealing with elements as an ElementHandle object in the Node.js (Puppeteer) environment.

Example Usage:

const example = await page.evaluate(() => {
  const elements = document.getElementsByClassName('example');
  const result = [];

  document.title = 'New Title';

  for (let i = 0; i < elements.length; i++) {
    result.push(elements[i].textContent);
  }

  return JSON.stringify(result);
});

See the simplified diagram below:

Puppeteer page.evaluate() Diagram

like image 156
Grant Miller Avatar answered Oct 10 '22 18:10

Grant Miller