Accessing a Javascript variable inside an HTML file

Question

I'm doing a little bit of reverse engineering on the Rapportive API in Gmail.

I make this request

import requests
url ='https://api.linkedin.com/uas/js/xdrpc.html'
r = requests.get(url)
print r.text

The response is an empty HTML file that has a lot of Javascript in it. On line 3661, it sets the RequestHeader for the subsequent call to Rapportive:

ak.setRequestHeader("oauth_token", ae);

Is there a way I can request that page and then return ae?

Answer 1

I think you can try:

1. Get the page as you already does;
2. Remove all non-javascript elements from the response page;
3. Prepend a javascript (described below) in the page's javascript to override some code;
4. Execute it with eval('<code>');
5. Check if the token has been set correctly;

I'm proposing the following code to override the XMLHttpRequest.setRequestHeader functionality to be able to get the token:

// this will keep the token
var headerToken; 

// create a backup method
XMLHttpRequest.prototype.setRequestHeaderBkp = 
XMLHttpRequest.prototype.setRequestHeader; 

// override the "setRequestHeader" method
XMLHttpRequest.prototype.setRequestHeader = function(key, val)
{
  if ('oauth_token' === key)
    headerToken = val;

  this.setRequestHeaderBkp(key, val);
}