How to log fetched network resources in JavaScript?

Question

Is there a way to access the list of resources that the browser requested (the ones found in this Chrome inspector's network panel)?

I would like to be able to iterate through these fetched resources to show the domains that have been accessed, something like:

for (var i = 0; i < window.navigator.resources.length; i++) {
  var resource = window.navigator.resources[i];
  console.log(resource); //=> e.g. `{domain: "www.google-analytics.com", name: "ga.js"}`
}

Or, maybe there is some event to write a handler for, such as:

window.navigator.onrequest = function(resource) {
  console.log(resource); //=> e.g. `{domain: "www.google-analytics.com", name: "ga.js"}`
}

It doesn't need to work cross browser, or even be possible using client-side JavaScript. Just being able to access this information in any way would work (maybe there's some way to do this using phantomjs or watching network traffic from a shell/node script). Any ideas?

Answer 1

You can do this, but you will need to use Chrome extensions.

Chrome extensions have a lot of sandbox-style security. Communication between the Chrome extension and the web page is a multi-step process. Here's the most concise explanation I can offer with a full working example at the end:

1. A Chrome extension has full access to the chrome.* APIs, but a Chrome extension cannot communicate directly with the web page JS nor can the web page JS communicate directly with the Chrome extension.

2. To bridge the gap between the Chrome extension and the web page, you need to use a content script . A content script is essentially JavaScript that is injected at the window scope of the targeted web page. The content script cannot invoke functions nor access variables that are created by the web page JS, but they do share access to the same DOM and therefore events as well.

3. Because directly accessing variables and invoking functions is not allowed, the only way the web page and the content script can communicate is through firing custom events.

For example, if I wanted to pass a message from the Chrome extension to the page I could do this:

content_script.js

document.getElementById("theButton").addEventListener("click", function() {
    window.postMessage({ type: "TO_PAGE", text: "Hello from the extension!" }, "*");
}, false);

web_page.js

window.addEventListener("message", function(event) {
    // We only accept messages from ourselves
    if (event.source != window)
      return;

    if (event.data.type && (event.data.type == "TO_PAGE")) {
      alert("Received from the content script: " + event.data.text);
    }
}, false);

`4. Now that you can send a message from the content script to the web page, you now need the Chrome extension gather up all the network info you want. You can accomplish this through a couple different modules, but the most simple option is the webRequest module (see background.js below).

`5. Use message passing to relay the info on the web requests to the content script and then on to the web page JavaScript.

Visually, you can think of it like this:

Full working example:

The first three files comprise your Google Chrome Extension and the last file is the HTML file you should upload to http:// web space somewhere.

icon.png

Use any 16x16 PNG file.

manifest.json

{
  "name": "webRequest Logging",
  "description": "Displays the network log on the web page",
  "version": "0.1",
  "permissions": [
    "tabs",
    "debugger",
    "webRequest",
    "http://*/*"
  ],
  "background": {
    "scripts": ["background.js"]
  },
  "browser_action": {
    "default_icon": "icon.png",
    "default_title": "webRequest Logging"
  },
   "content_scripts": [
    {
      "matches": ["http://*/*"],
      "js": ["content_script.js"]
    }
  ],
  "manifest_version": 2
}

background.js

var aNetworkLog = [];

chrome.webRequest.onCompleted.addListener(function(oCompleted) {
            var sCompleted = JSON.stringify(oCompleted);
            aNetworkLog.push(sCompleted);
        }
        ,{urls: ["http://*/*"]}
 );

chrome.extension.onConnect.addListener(function (port) {
    port.onMessage.addListener(function (message) {
        if (message.action == "getNetworkLog") {
            port.postMessage(aNetworkLog);
        }
    });
});

content_script.js

var port = chrome.extension.connect({name:'test'});

document.getElementById("theButton").addEventListener("click", function() {

    port.postMessage({action:"getNetworkLog"});

}, false);

port.onMessage.addListener(function(msg) {
  document.getElementById('outputDiv').innerHTML = JSON.stringify(msg);
});

And use the following for the web page (named whatever you want):

<!doctype html>
<html>
<head>
    <title>webRequest Log</title>
</head>
<body>
    <input type="button" value="Retrieve webRequest Log" id="theButton">
    <div id="outputDiv"></div>
</head>
</html>