Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do you wait for a selector, but only for a specific time period in puppeteer?

I'm scraping data from youtube and trying to get the number of comments. I'm trying to grab the element that contains the value, but in case the comments are disabled for a video, that element doesn't exist at all and waitForSelector() waits, I think, for about 30 seconds before ending the program. How can I tell puppeteer to wait for that element for, say, 5 seconds and if it doesn't exist, move on with the rest of the code?

Here's the code that I'm using-

await page.waitForSelector("yt-formatted-string.count-text.style-scope.ytd-comments-header-renderer")
let commentCount = await (await (await page.$("yt-formatted-string.count-text.style-scope.ytd-comments-header-renderer")).getProperty("textContent")).jsonValue()
like image 363
roitmi Avatar asked Dec 03 '22 18:12

roitmi


2 Answers

Below code you can try for waiting -

await page.waitForSelector(yourSelector, {timeout: TIMEOUT});

Ex:

await page.waitForSelector(yourSelector, {timeout: 5000});

UPDATED:

To catch timeouterror and do something -

const {TimeoutError} = require('puppeteer/Errors');

try {
  await page.waitForSelector(yourSelector, {timeout: 5000});
} catch (e) {
  if (e instanceof TimeoutError) {
    // Do something if this is a timeout.
  }
}

Reference:

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagewaitforselectorselector-options

like image 135
PySaad Avatar answered May 11 '23 21:05

PySaad


Try code below, just add a timeout option

try {
    await page.waitForSelector("yt-formatted-string.count-text.style-scope.ytd-comments-header-renderer", {timeout: 5000 }); // 5s timeout
} catch (e) {
   // something wrong !!!!!!
}

let commentCount = await (await (await page.$("yt-formatted-string.count-text.style-scope.ytd-comments-header-renderer")).getProperty("textContent")).jsonValue()
like image 20
lx1412 Avatar answered May 11 '23 23:05

lx1412