Menu
I have a webdriver using selenium that opens a browser for me, points it to an ip Address, does a bunch of stuff and closes.
I want to know all of the urls accessed during this time. That is, any ads that are loaded, any css calls that were made out to any url and so on.
Here is the code im using
from selenium import webdriver
browser = webdriver.Firefox(profile) # Get local session of firefox
browser.get(url) # Open a url and wait for it to finish
254
A simple packet sniffer in Python can be created with the help socket module. We can use the raw socket type to get the packets. A raw socket provides access to the underlying protocols, which support socket abstractions.
To capture network traffic, you can use tcpdump. This is a packet sniffer that can observe and record network traffic on an interface. In the following example, we capture 1,000 packets using tcpdump. An easier way to analyze network traffic is to use an actual network traffic analyzer, such as Wireshark.
Practical Data Science using Python Sniffing or network packet sniffing is the process of monitoring and capturing all the packets passing through a given network using sniffing tools. It is a form wherein, we can “tap phone wires” and get to know the conversation.
I did it by loading the firefox plugins Firebug and Netexport. The first is a tool that allows you to see all the exchange of information, the second allows to write all of it in a file (.har extension). So basically selenium has to load the plugins, the website and wait the time you want, and when it closes, you get a file with the result.
86
Its not a python solution.. But you can add fiddler plug in to Firefox. We needed to do exact same thing about a year ago. We used selenium to open browser and all UI stuff and in background Fiddler captured all traffic (http and https) .. This also list all JS CSS src and you can debug later with inspector see what request is sent and what response is received
45
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
