Python: keep open browser in pyppeteer and create CDPSession

Question

I've got two issues that I can't solve it at them moment.

1. I would like to keep the browser running so I could just re-connect using pyppeteer.launcher.connect() function but it seems to be closed imidiately even if I don't call pyppeteer.browser.Browser.close().

test01.py:

import asyncio

from pyppeteer import launch, connect

async def fetch():
    browser = await launch(
        headless=False,
        args=['--no-sandbox']
    )
    print(f'Endpoint: {browser.wsEndpoint}')
    await browser.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())

$ python test01.py
Endpoint: ws://127.0.0.1:51757/devtools/browser/00e917a9-c031-499a-a8ee-ca4090ebd3fe
$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" http://127.0.0.1:51757
curl: (7) Failed to connect to 127.0.0.1 port 51757: Connection refused

2. How do I create CDP session. This code should open another browser window but it doesn't work as expected:

test02.py

import asyncio
import time

from pyppeteer import launch, connect

async def fetch():
    browser = await launch(
        headless=False,
        args=['--no-sandbox']
    )
    page = await browser.newPage()
    cdp = await page.target.createCDPSession()
    await cdp.send('Target.createBrowserContext')
    time.sleep(5)
    await browser.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())

$ python test02.py
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.',)>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.

Answer 1

How to keep the browser running

You just need to use autoClose flag, here's the docs:

autoClose (bool): Automatically close browser process when script completed. Defaults to True.

In this case you test01.py would look as follows:

import asyncio

from pyppeteer import launch, connect

async def fetch():
    browser = await launch(
        headless=False,
        args=['--no-sandbox'],
        autoClose=False
    )
    print(f'Endpoint: {browser.wsEndpoint}')
    await browser.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())

CDP session

Here it is:

import asyncio
import time

from pprint import pprint

from pyppeteer import launch, connect
from pyppeteer.browser import BrowserContext

async def fetch():
    browser = await launch(
        headless=False,
        args=['--no-sandbox'],
        autoClose=False
    )
    page = await browser.newPage()
    cdp = await page.target.createCDPSession()
    raw_context = await cdp.send('Target.createBrowserContext')
    pprint(raw_context)
    context = BrowserContext(browser, raw_context['browserContextId'])
    new_page = await context.newPage()
    await cdp.detach()
    await browser.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())

Inspired by Browser.createIncognitoBrowserContext from pyppeteer itself. Notice creating additional sessions via CDP doesn't seem to be such a great idea because browser._contexts won't be updated and will become inconsistent. It's also likely that Browser.createIncognitoBrowserContext might fit your needs without resorting to CDP whatsoever