Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can the browser turned headless mid-execution when it was started normally, or vice-versa?

I want to start a chromium browser instant headless, do some automated operations, and then turn it visible before doing the rest of the stuff.

Is this possible to do using Puppeteer, and if it is, can you tell me how? And if it is not, is there any other framework or library for browser automation that can do this?

So far I've tried the following but it didn't work.

const browser = await puppeteer.launch({'headless': false});
browser.headless = true;
const page = await browser.newPage();
await page.goto('https://news.ycombinator.com', {waitUntil: 'networkidle2'});
await page.pdf({path: 'hn.pdf', format: 'A4'});
like image 408
Sumit Ghosh Avatar asked Jan 26 '23 10:01

Sumit Ghosh


1 Answers

Short answer: It's not possible

Chrome only allows to either start the browser in headless or non-headless mode. You have to specify it when you launch the browser and it is not possible to switch during runtime.

What is possible, is to launch a second browser and reuse cookies (and any other data) from the first browser.

Long answer

You would assume that you could just reuse the data directory when calling puppeteer.launch, but this is currently not possible due to multiple bugs (#1268, #1270 in the puppeteer repo).

So the best approach is to save any cookies or local storage data that you need to share between the browser instances and restore the data when you launch the browser. You then visit the website a second time. Be aware that any state the website has in terms of JavaScript variable, will be lost when you recrawl the page.

Process

Summing up, the whole process should look like this (or vice versa for headless to headfull):

  • Crawl in non-headless mode until you want to switch mode
  • Serialize cookies
  • Launch or reuse second browser (in headless mode)
  • Restore cookies
  • Revisit page
  • Continue crawling
like image 133
Thomas Dondorf Avatar answered Jan 29 '23 12:01

Thomas Dondorf