Communicate "out" from Chromium via DevTools protocol

Question

I have a page running in a headless Chromium instance, and I'm manipulating it via the DevTools protocol, using the Puppeteer NPM package in Node.

I'm injecting a script into the page. At some point, I want the script to call me back and send me some information (via some event exposed by the DevTools protocol or some other means).

What is the best way to do this? It'd be great if it can be done using Puppeteer, but I'm not against getting my hands dirty and listening for protocol messages by hand.

I know I can sort-of do this by manipulating the DOM and listening to DOM changes, but that doesn't sound like a good idea.

Answer 1

Okay, I've discovered a built-in way to do this in Puppeteer. Puppeteer defines a method called exposeFunction.

page.exposeFunction(name, puppeteerFunction)

This method defines a function with the given name on the window object of the page. The function is async on the page's side. When it's called, the puppeteerFunction you define is executed as a callback, with the same arguments. The arguments aren't JSON-serialized, but passed as JSHandles so they expose the objects themselves. Personally, I chose to JSON-serialize the values before sending them.

I've looked at the code, and it actually just works by sending console messages, just like in Pasi's answer, which the Puppeteer console hooks ignore. However, if you listen to the console directly (i.e. by piping stdout). You'll still see them, along with the regular messages.

Since the console information is actually sent by WebSocket, it's pretty efficient. I was a bit averse to using it because in most processes, the console transfers data via stdout which has issues.

Example

Node

async function example() {
    const puppeteer = require("puppeteer");
    let browser = await puppeteer.launch({
        //arguments
    });
    let page = await browser.newPage();

    await page.exposeFunction("callPuppeteer", function(data) {
        console.log("Node receives some data!", data);
    });

    await page.goto("http://www.example.com/target");
}

Page

Inside the page's javascript:

window.callPuppeteer(JSON.stringify({
    thisCameFromThePage : "hello!"
}));

Update: DevTools protocol support

There is DevTools protocol support for something like puppeteer.exposeFunction.

https://chromedevtools.github.io/devtools-protocol/tot/Runtime#method-addBinding

If executionContextId is empty, adds binding with the given name on the global objects of all inspected contexts, including those created later, bindings survive reloads. If executionContextId is specified, adds binding only on global object of given execution context. Binding function takes exactly one argument, this argument should be string, in case of any other input, function throws an exception. Each binding function call produces Runtime.bindingCalled notification.

.

Answer 2

If the script sends all its data back in one call, the simplest approach would be to use page.evaluate and return a Promise from it:

const dataBack = page.evaluate(`new Promise((resolve, reject) => {                                                  
  setTimeout(() => resolve('some data'), 1000)                                                                      
})`)
dataBack.then(value => { console.log('got data back', value) })

This could be generalized to sending data back twice, etc. For sending back an arbitrary stream of events, perhaps console.log would be slightly less of a hack than DOM events? At least it's super-easy to do with Puppeteer:

page.on('console', message => {
  if (message.text.startsWith('dataFromMyScript')) {
    message.args[1].jsonValue().then(value => console.log('got data back', value))
  }
})
page.evaluate(`setInterval(() => console.log('dataFromMyScript', {ts: Date.now()}), 1000)`)

(The example uses a magic prefix to distinguish these log messages from all others.)