Automatize HAR files generation from Google Chrome

Question

Basically what I need is a way to automatize the result of the following operations:

1. open a new tab;

2. open the Network tab in the developer tools;

3. load an URL;

4. select "Save All as HAR".

Often, proposed solutions involves the use of PhantomJS, browsermob-proxy, or pcap2har; those won't fit my case since I need to work with SPDY traffic.

I tried to dive into the Google Chrome Extensions API and indeed I managed to automatize some tasks, but still no luck for what concerns the HAR files generation. Now this method is particularly promising but I still can't figure out how would I use it.

In other words, I need something like this experiment from the Google guys. Note the following:

We used Chrome's remote debugging interface with a custom client that starts up the browser on the phone, clears its cache and other state, initiates a web page load, and receives the Chrome developer tools messages to determine the page load times and other performance metrics.

Any ideas?

---

Solution

For the curious, I ended up with a Node.js module that automates such kind of tests: chrome-har-capturer. This also gave me the opportunity to dig deeper into the Remote Debugging Protocol and to write a lower-level Node.js interface for general-purpose Chrome automation: chrome-remote-interface.

Answer 1

The short answer is, there is no way to get at the data you are after directly. The getHAR method is only applicable to extensions meant to extend DevTools itself. The good news is, you can construct the HAR file yourself without too much trouble - this is exactly what phantom.js does.

1. Start Chrome with remote debugging
2. Connect to Chrome on the debugging port with a websocket connection
3. Enable "Network" debugging, you can also clear cache, etc - see Network API.
4. Tell the browser to navigate to the page you want to capture, and Chrome will stream all the request meta-data back to you.
5. Massage the network data into HAR format, ala phantom.js
6. ...
7. Profit.

For a head start, I have a post that with sample Ruby code that should you get started with steps 1-4: http://www.igvita.com/2012/04/09/driving-google-chrome-via-websocket-api/

Answer 2

By now there's a browser plugin to do that: https://github.com/devtools-html/har-export-trigger

It uses the WebExtensions DevTools API and I got it to work with both Firefox and Chrome.

See my code for Chrome here: https://github.com/theri/web-measurement-tools/blob/master/load/load_url_using_chrome.py#L175

Automatically installing the plugin in Chrome is a bit more complicated than in Firefox, but feasible - I extracted the plugin archive locally and then link to it in chrome_prefs.json (see same repository).