Get all XHR Requests of a Site in Python

Question

I want to make a program in python that prints out all the xhr requests of a site.

I can do this with the inspect tool, by going to Network -> XHR, but I would like to run a code that does this.

The code should give out a dictionary with a key of name, and a value of a dictionary with the corresponding Request URL, Request METHOD, content-type, and remote address.

For Example:

aDict = {'randomfile.js?_=1581185731016': {'url': 'https://softwaresat.netlify.com/randomfile.js?_=1581185731016', 'method': 'GET', 'address': '104.248.60.43:443'}}

Answer 1

You can use the selenium-wire python module for getting all the xhr calls of a website which helped me for my projects.

Install module pip install selenium-wire

Sample code to get xhr requests for google.com

from seleniumwire import webdriver  # Import from seleniumwire

# Create a new instance of the Chrome driver
driver = webdriver.Chrome()

# Go to the Google home page
driver.get('https://www.google.com')

# Access requests via the `requests` attribute
for request in driver.requests:
    if request.response:
        print(
            request.url
        )