Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get Puppeteer waitForNavigation Working after click

I am trying to get puppeteer to wait for the navigation to finish before moving on to the next statement. Based on the Docs for waitForNavigation() , the code should work below. but it just skips to the next statement and I have to use a workaround to wait for a specific URL in the response.

I have tried all the waituntil options as well
( load, domcontentloaded, networkidle0 and networkidle2 ) .

Any ideas how I could get that working properly is appreciated.

const browser = await puppeteer.launch({
  headless: false,
})
const page = await browser.newPage()
const home = page.waitForNavigation()
await page.goto(loginUrl)
await home

const login = page.waitForNavigation()
await page.type('#email', config.get('login'))
await page.type('#password', config.get('password'))
await page.click('#submitButton')
await login // << skips over this

// the following line is my workaround and it works , but ideally I don't want 
// to specify the expected "after" page each time I navigate
await page.waitForResponse(request => request.url() === 'http://example.com/expectedurl')
like image 634
Martin Thompson Avatar asked Sep 06 '18 20:09

Martin Thompson


People also ask

Why can't I use waitfornavigation?

The reason the pattern doesn't work for you is because browser doesn't do any "navigation" - instead, current page DOM is modified. Thus, using waitForNavigation won't work.

What is the difference between waituntil and timeout in puppeteer?

The options for page.goto allows us to define an object with timeout and waitUntil. The timeout is how long Puppeteer will wait, in milliseconds, for the page to load before it throws and error and stops running. For waitUntil we can choose from: load, domcontentloaded, networkidle0, networkidle2.

How do I type in the input field in puppeteer?

To type in the input field we have waited for, we can use Puppeteer's page method page.type which takes a CSS selector to find the element you want to type in and a string you wish to type in the field. Lastly we can take a screenshot with page.screenshot or use waitFor to make sure you can see the results of your code.

What are the wait options in a world built on JavaScript?

However, in a world built on javascript, you will likely want to specify some wait options. The options for page.goto allows us to define an object with timeout and waitUntil. The timeout is how long Puppeteer will wait, in milliseconds, for the page to load before it throws and error and stops running.


2 Answers

The function page.waitForNavigation() waits for navigation to begin and end.

The navigation has already been initiated with page.click().

Therefore, you can use Promise.all() to avoid the race condition between the mentioned functions:

const browser = await puppeteer.launch({
  headless: false,
});

const page = await browser.newPage();

await page.goto(loginUrl);

await page.type('#email', config.get('login'));
await page.type('#password', config.get('password'));

await Promise.all([
  page.click('#submitButton'),
  page.waitForNavigation({
    waitUntil: 'networkidle0',
  }),
]);

await browser.close();
like image 85
Grant Miller Avatar answered Nov 15 '22 01:11

Grant Miller


I have been going through the same problem, I used pending-xhr-request

it solved many problems when the requests were expected, but when I have late requests I have faced many problems, it took me a while to solve the problem so I built a package Puppeteer-response-waiter to do that

const puppeteer = require('puppeteer');
const {ResponseWaiter} = require('puppeteer-response-waiter');

let browser = await puppeteer.launch({ headless: false });
let page = await browser.newPage();
let responseWaiter = new ResponseWaiter(page);
await page.goto('http://somesampleurl.com');
// start listening
responseWaiter.listen();
// do something here to trigger requests
await responseWaiter.wait();
// all requests are finished and responses are all returned back

// remove listeners
responseWaiter.stopListening();
await browser.close();

hope this will solve your problem.

like image 29
samyouaret Avatar answered Nov 15 '22 01:11

samyouaret