How to get all console messages with puppeteer? including errors, CSP violations, failed resources, etc

Question

I am fetching a page with puppeteer that has some errors in the browser console but the puppeteer's console event is not being triggered by all of the console messages.

The puppeteer chromium browser shows multiple console messages

However, puppeteer only console logs one thing in node

Here is the script I am currently using:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  page.on('console', msg => console.log('PAGE LOG:', msg.text));
  await page.goto('https://pagewithsomeconsoleerrors.com');
  await browser.close();
})();

Edit: As stated in my comment below, I did try the page.waitFor(5000) command that Everettss recommended but that didn't work.

Edit2: removed spread operator from msg.text as it was there by accident.

Edit3: I opened an issue on github regarding this with similar but different example screenshots: https://github.com/GoogleChrome/puppeteer/issues/1512

Answer 1

The GitHub issue about capturing console erorrs includes a great comment about listening to console and network events. For example, you can register for console output and network responses and failures like this:

  page
    .on('console', message =>
      console.log(`${message.type().substr(0, 3).toUpperCase()} ${message.text()}`))
    .on('pageerror', ({ message }) => console.log(message))
    .on('response', response =>
      console.log(`${response.status()} ${response.url()}`))
    .on('requestfailed', request =>
      console.log(`${request.failure().errorText} ${request.url()}`))

And get the following output, for example:

200 'http://do.carlosesilva.com/puppeteer/'
LOG This is a standard console.log message
Error: This is an error we are forcibly throwing
    at http://do.carlosesilva.com/puppeteer/:22:11
net::ERR_CONNECTION_REFUSED https://do.carlosesilva.com/http-only/sample.png
404 'http://do.carlosesilva.com/puppeteer/this-image-does-not-exist.png'
ERR Failed to load resource: the server responded with a status of 404 (Not Found)

See also types of console messages received with the console event and response, request and failure objects received with other events.

If you want to pimp your output with some colours, you can add chalk, kleur, colorette or others:

  const { blue, cyan, green, magenta, red, yellow } = require('colorette')
  page
    .on('console', message => {
      const type = message.type().substr(0, 3).toUpperCase()
      const colors = {
        LOG: text => text,
        ERR: red,
        WAR: yellow,
        INF: cyan
      }
      const color = colors[type] || blue
      console.log(color(`${type} ${message.text()}`))
    })
    .on('pageerror', ({ message }) => console.log(red(message)))
    .on('response', response =>
      console.log(green(`${response.status()} ${response.url()}`)))
    .on('requestfailed', request =>
      console.log(magenta(`${request.failure().errorText} ${request.url()}`)))

The examples above use Puppeteer API v2.0.0.

Answer 2

Easiest way to capture all console messages is passing the dumpio argument to puppeteer.launch().

From Puppeteer API docs:

dumpio: <boolean> Whether to pipe the browser process stdout and stderr into process.stdout and process.stderr. Defaults to false.

Example code:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({
         dumpio: true
    });

    ...

})();

Answer 3

You need to set multiple listeners if you want to catch everything. The console event is emitted when javascript within the page calls a console API message (like console.log).

For a full list of the console API take a look at the docs for console on MDN: https://developer.mozilla.org/en-US/docs/Web/API/Console

The reason you need multiple listeners is because some of what is being logged in the image you posted is not happening within the page.

So for example, to catch the first error in the image, net:: ERR_CONNECTION_REFUSED you would set the listener like so: page.on('requestfailed', err => console.log(err));

Puppeteer's documentation contains a full list of events. You should take a look at the documentation for the version you are using and look at the different events the Page class will emit as well as what those events will return. The example I've written above will return an instance of Puppeteer's request class.

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-page