How to import firefox cookies to python requests

Question

I'm logged in on some page in Firefox and I want to take the cookie and try to browse webpage with python-requests. Problem is that after importing cookie to the requests session nothing happen (like there is no cookie at all). Structure of the cookie made by requests differ from the one from Firefox as well. Is such it possible to load FF cookie and use it in requests session?

My code so far:

import sys
import sqlite3
import http.cookiejar as cookielib
import requests
from requests.utils import dict_from_cookiejar

def get_cookies(final_cookie, firefox_cookies):
    con = sqlite3.connect(firefox_cookies)
    cur = con.cursor()
    cur.execute("SELECT host, path, isSecure, expiry, name, value FROM moz_cookies")
    for item in cur.fetchall():
        if item[0].find("mydomain.com") == -1:
            continue
        c = cookielib.Cookie(0, item[4], item[5],
            None, False,
            item[0], item[0].startswith('.'), item[0].startswith('.'),
            item[1], False,
            item[2],
            item[3], item[3]=="",
            None, None, {})
        final_cookie.set_cookie(c)



cookie = cookielib.CookieJar()
input_file = ~/.mozilla/firefox/myprofile.default/cookies.sqlite
get_cookies(cookie, input_file)

#print cookie given from firefox
cookies = dict_from_cookiejar(cookie)
for key, value in cookies.items():
    print(key, value)


s = requests.Session()
payload = {
"lang" : "en",
'destination': '/auth',
'credential_0': sys.argv[1],
'credential_1': sys.argv[2],
'credential_2': '86400',
}
r = s.get("mydomain.com/login", data = payload)
#print cookie from requests
cookies = dict_from_cookiejar(s.cookies)
for key, value in cookies.items():
    print(key, value)

Structure of cookies from firefox is:

_gid GA1.3.2145214.241324
_ga GA1.3.125598754.422212
_gat_is4u 1

Structure of cookies from requests is:

UISTestAuth tesskMpA8JJ23V43a%2FoFtdesrtsszpw

After all, when trying to assign cookies from FF to session.cookies, requests works as I import nothing.

Answer 1

It looks like there are two type of cookies in the Firefox - request and response. It could be seen while Page inspector > Network > login (post) > Cookies:

Response cookies:   
    UISAuth 
        httpOnly   true
        path       /
        secure     true
        value      tesskMpA8JJ23V43a%2FoFtdesrtsszpw
Request cookies:    
    _ga            GA1.3.125598754.422212
    _gat_is4u      1
    _gid           GA1.3.2145214.241324

The request cookies are stored in the cookies.sqlite file in the

~/.mozilla/firefox/*.default/cookies.sqlite

and can be load to the python object in more ways, for example:

import sqlite3
import http.cookiejar

def get_cookies(cj, ff_cookies):
    con = sqlite3.connect(ff_cookies)
    cur = con.cursor()
    cur.execute("SELECT host, path, isSecure, expiry, name, value FROM moz_cookies")
    for item in cur.fetchall():
        c = cookielib.Cookie(0, item[4], item[5],
            None, False,
            item[0], item[0].startswith('.'), item[0].startswith('.'),
            item[1], False,
            item[2],
            item[3], item[3]=="",
            None, None, {})
        print c
        cj.set_cookie(c)

where cj is CookieJar object and ff_cookies is path to Firefox cookies.sqlite. Taken from this page.

The whole code to load cookies and import to the python requests using session would looks like:

 import requests
 import sys

 cj = http.cookiejar.CookieJar()
 ff_cookies = sys.argv[1] #pass path to the cookies.sqlite as an argument to the script
 get_cookies(cj, ff_cookies)
 s = requests.Session()
 s.cookies = cj

Response cookie is basically session ID, which usualy expires at the end of the session (or some timeout), so they are not stored.