Is there a way to get a webpage's Network activity (which you can see on Chrome Dev Tools) on load via Python?

Question

I want to listen to the Network events (basically all of the activity that you can see when you go to the Network tab on Chrome's Developer Tools / Inspect) and record specific events when a page is loaded via Python.

Is this possible? Thanks!

Specifically:

• go to webpage.com
• open Chrome Dev Tools and go to the Network tab
• add api.webpage.com as a filter
• refresh page [scroll]

I want to be able to capture the names of these events because there are specific IDs that aren't available via the UI.

Answer 1

Update 2021 I had to make few changes to Zach answer to make it work. Comments with ### are my comments

def get_perf_log_on_load(url, headless=True, filter=None):

    # init Chrome driver (Selenium)
    options = Options()
    options.add_experimental_option('w3c', False) ### added this line
    options.headless = headless
    cap = DesiredCapabilities.CHROME
    cap["loggingPrefs"] = {"performance": "ALL"}
    ### installed chromedriver.exe and identify path
    driver = webdriver.Chrome(r"C:\Users\asiddiqui\Downloads\chromedriver_win32\chromedriver.exe", desired_capabilities=cap, options=options) ### installed
    # record and parse performance log
    driver.get(url)
    if filter:
        log = [item for item in driver.get_log("performance") if filter in str(item)]
    else:
        log = driver.get_log("performance")
    driver.close()

    return log

Answer 2

Although it didn't completely answer the question, @mihai-andrei's answer got me the closest.

If anyone is looking for a Python solution than the following code should do the trick:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome.options import Options

def get_perf_log_on_load(self, url, headless = True, filter = None):

    # init Chrome driver (Selenium)
    options = Options()
    options.headless = headless
    cap = DesiredCapabilities.CHROME
    cap['loggingPrefs'] = {'performance': 'ALL'}
    driver = webdriver.Chrome(desired_capabilities = cap, options = options)

    # record and parse performance log
    driver.get(url)
    if filter: log = [item for item in driver.get_log('performance')
                      if filter in str(item)]
    else: log = driver.get_log('performance')
    driver.close()

    return log