Proper way to use page.waitForNavigation in Puppeteer [duplicate]

Question

I'm a bit confused about page.waitForNavigation. I know what it is and what it does but I get different results based on the interenet speed (that's what I think is the factor).

Imagine this code:

await page.$eval('#username', (el , setting ) => el.value = setting.username , setting );
await page.$eval('#password', (el , setting ) => el.value = setting.password , setting );
await page.$eval('form', form => form.submit());
await page.waitForNavigation();
await page.goto( 'http://example.com/dashboard'  );

It fills the login form, submits the login form, waits for the form to be submited and then redirects to dashboard.

This works fine on my localhost which has slower internet speed (compared to the server), but when I upload it to the server, I get

Error: Navigation Timeout Exceeded: 30000ms exceeded 

On the server it works fine if I remove await page.waitForNavigation(); from the code and I get redirected to dashboard.

But now on localhost, I get redirected to the dashboard before the form can be submitted. I get you cant see dashboard , your not logged in or something like this.

I think the deciding factor is the speed of internet.

On the server, I have a very high speed so the form gets instantly submitted and it's done before the await page.waitForNavigation() line so I get a nvigation timeout error.

But on the localhost with the slower speed, the form needs more time to be submitted so I need to have await page.waitForNavigation() after submitting the form, otherwise I get redirected to the dashboard before the form has any chance to be submitted.

I'm looking for advice from someone with more experience working with Puppeteer on how to handle this kind of situation. Right now, I keep editing my code when running on server or localhost, which works, but it's very annoying!

After using

async function open_tab(setting) {
  const page = await global_browser.newPage();
  await page.setViewport({
    width: 1000,
    height: 768
  });

  return await new Promise(async(resolve, reject) => {
      await page.$eval('#username', (el, setting) => el.value = setting.username, setting);
      await page.$eval('#password', (el, setting) => el.value = setting.password, setting);
      await Promise.all(
        page.$eval('form', form => form.submit()),
        page.waitForNavigation()
      )
      await page.goto('http://example.com/dashboard');
      resolve();
    }).then(() => {
      console.log(' -> don!  ');
      page.close();
    })
    .catch(() => {
      console.log(' -> somethign went wrong !');
      page.close();
    })
}

I get

(node:14812) UnhandledPromiseRejectionWarning: TypeError: undefined is not a function
    at Function.all (<anonymous>)
    at Promise (D:\wamp\www\gatewayCard\robot\server.js:287:23)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
(node:14812) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
(node:14812) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Answer 1

It happens because the navigation might happen on submit before you wait for it.

Use the submission and waitForNavigation inside one promise using Promise.all, so it will wait for both of them instead of one at a time.

await Promise.all([
 page.waitForNavigation(),
 page.$eval('form', form => form.submit())
])

or,

await Promise.all([
 page.$eval('form', form => form.submit()),
 page.waitForNavigation()
])

either should work.

EDIT 1:

The edit is totally offtopic to your main question. You are not using a proper async...await in your code. Here is a better code.

The beauty of async...await functions are you can do less code and better readability and maintainability. It's a double-edge sword, but worth it if used properly.

async function open_tab(setting) {
  try {
    const page = await global_browser.newPage();
    await page.setViewport({
      width: 1000,
      height: 768
    });
    await page.goto('http://example.com/dashboard');
    await page.$eval('#username', (el, setting) => el.value = setting.username, setting);
    await page.$eval('#password', (el, setting) => el.value = setting.password, setting);
    await Promise.all(page.$eval('form', form => form.submit()), page.waitForNavigation());
    console.log(' -> don!  ');
    await page.close();
  } catch (error) {
    console.log(' -> somethign went wrong !', error);
    await page.close();
  }
}

EDIT 2:

Here in your code,

return await new Promise(async (resolve, reject ) => {

this is wrong on many steps. async functions returns a promise anyway, so you are returning a promise inside a promise without catching it and using await with it. Check out your code early otherwise you will face huge problems soon.

It seems you should learn a bit more about async await first. Here are some helpful links for you, these will explain timeouts and everything you want to learn.

• https://javascript.info/async-await
• https://codeburst.io/javascript-es-2017-learn-async-await-by-example-48acc58bad65