Developer Tools: Follow network requests across popups

Question

We are trying to figure out how something works on the web (for web scraping/automation) and one of the web pages we are working on issues a popup to do some of the work. One of our most commonly used debug tools is the Chrome network tab in Developer Tools, hit "record" do some work, and then examine what was done and then replicate the work done "offline".

However the Developer Tools (in Chrome, Safari and Firefox - all work the same) do not follow requests across a popup, even if you hit "record".

Is there some configuration value I'm missing, or some way to record all network events? We can't use tcpdump/wireshark for this because it's all done over SSL. One option we've considered is a man-in-the-middle https proxy, but I can't find anything pre-written so we'd have to create one ourselves.

Answer 1

I don't know of any way to follow the requests across pop-ups, as each window has its own Web Inspector, however you can use Fiddler to inspect HTTPS requests. It will MITM, and subsequently throw a certificate error, which should allow you to inspect all requests in the order that they happened.

Answer 2

You can use Charles Web Debugging Proxy, which is an app that lets you see all the traffic and even replace some responses with your own. Of course that may break HTTPS so you have to accept the certificate errors, but that's usually a minor problem. It works on Win, Mac and even Linux.