console.log message is truncated

Question

I'm new to puppeteer. I used to have PhantomJS and CasperJS but while setting a newer server (freebsd 12) found out that support for PhantomJS is gone and CasperJS gives me segmentation faults.

I was able to port my applications to puppeteer just fine but ran into the problem that when I want to capture data from a table, this data seems to be incomplete or truncated.

I need all the info from a table but always end up getting less.

I have tried smaller tables but it also comes out truncated. I don't know if the console.log buffer can be extended or not, or if there is a better way to get the values of all tds in the table.

const data = await page.$$eval('table.dtaTbl tr td', tds => tds.map((td) => {
    return td.innerHTML;
}));

console.log(data); 

I should be able to get all rows but instead I get this

[ 'SF xx/xxxx 3-3999 06-01-16',
'Sample text - POLE',
  '',

 /* tons of other rows (removed by me in this example) <- */

  '',

 /* end of output */ ... 86 more items ]

I need the 86 other items!!! because I'm having PHP pick it up from stdout as the code is executed.

Answer 1

Why console.log does not work

Under the hood, console.log uses util.inspect, which produces output intended for debugging. To create reasonable debugging information, this function will truncate output which would be too long. To quote the docs:

The util.inspect() method returns a string representation of object that is intended for debugging. The output of util.inspect may change at any time and should not be depended upon programmatically.

---

Solution: Use process.stdout

If you want to write output to stdout you can use process.stdout which is a writable stream. It will not modify/truncate what you write on the stream. You can use it like this:

process.stdout.write(JSON.stringify(data) + '\n');

I added a line break at the end, as the function will not produce a line break itself (in contrast to console.log). If your script does not rely on it you can simply remove it.

Answer 2

You can also use

console.log(JSON.stringify(data, null, 4));

instead of

process.stdout.write(JSON.stringify(data) + '\n');