Cookie handling in Google Apps Script - How to send cookies in header?

Question

I'm trying to write a simple script that fetches text from a webpage and processes that string. But, that website requires me to be logged in. I was successful in logging in to that website. This is how I logged in:

var payload = {"name1":"val1","name2":val2"};

var opt ={"payload":payload,"method":"post"};

var respose = UrlFetchApp.fetch("http://website.com/login",opt);

After logging in, the website places me in http://website.com/home. I checked response.getContentText() and I can confirm that I have been logged in successfully as it contains the text from http://website.com/home. Now I need to get the contents of http://website.com/page and process it. I first assumed the script can handle cookies by itself and proceeded with

var pagedata = UrlFetchApp.fetch("http://website.com/page);//Did not work

That obviously didnt work and pagedata.getContentText() says me to login first, which indicates cookies were not successfully passed..

I then tried to extract cookies which the server responded during login and to send it along with this request.

var cookie = response.getAllHeaders()['Set-Cookie'];     

// variable cookie now contains  a legitimate cookie.

// It contains 'JSESSIONID=blabla;Path=/' and 
// it is the ONLY cookie that server responds.

I tried to send that cookie in my page request.

var header = {'Cookie':cookie};

var opt2 = {"header":header};

var pagedata = UrlFetchApp.fetch("http://website.com/page",opt2);

I think even now cookies were not properly sent, as the content again says me to login.

Am I passing cookies correctly? I need help regarding the correct method of sending cookies in a request.

Answer 1

Here you can find cookies specification: http://www.w3.org/Protocols/rfc2109/rfc2109

You have a potential issue in your code: response.getAllHeaders()['Set-Cookie'] can return either a string or a table of string if multiple 'set-cookie' attributes are sent back from the server.

Eric is right, you cannot return the cookie without digesting it.

Second error in your code:

var opt2 = {"header":header};

should be

var opt2 = {"headers":header};

Be aware also that GAS uses Google IPs. It can happen that two consecutive fetch use different IPs. The server your are connecting to may be session-IP dependant.

Are you sure the server only send you back one cookie after an authentification ?

Answer 2

It looks like you are setting the headers correctly in UrlFetchApp.fetch().

I believe that the data in the Set-Cookie header is in a different format than the data that is expected in Cookie header. For example, Set-Cookie contains information about expiration, etc.