Passing csrftoken with python Requests

Question

How do you pass a csrftoken with the python module Requests? This is what I have but it's not working, and I'm not sure which parameter to pass it into (data, headers, auth...)

import requests from bs4 import BeautifulSoup  URL = 'https://portal.bitcasa.com/login'  client = requests.session(config={'verbose': sys.stderr})  # Retrieve the CSRF token first soup = BeautifulSoup(client.get('https://portal.bitcasa.com/login').content) csrftoken = soup.find('input', dict(name='csrfmiddlewaretoken'))['value']  login_data = dict(username=EMAIL, password=PASSWORD, csrfmiddlewaretoken=csrftoken) r = client.post(URL, data=login_data, headers={"Referer": "foo"}) 

Same error message every time.

<h1>Forbidden <span>(403)</span></h1> <p>CSRF verification failed. Request aborted.</p> 

Answer 1

If you are going to set the referrer header, then for that specific site you need to set the referrer to the same URL as the login page:

import sys import requests  URL = 'https://portal.bitcasa.com/login'  client = requests.session()  # Retrieve the CSRF token first client.get(URL)  # sets cookie if 'csrftoken' in client.cookies:     # Django 1.6 and up     csrftoken = client.cookies['csrftoken'] else:     # older versions     csrftoken = client.cookies['csrf']  login_data = dict(username=EMAIL, password=PASSWORD, csrfmiddlewaretoken=csrftoken, next='/') r = client.post(URL, data=login_data, headers=dict(Referer=URL)) 

When using unsecured http, the Referer header is often filtered out and otherwise easily spoofable anyway, so most sites no longer require the header to be set. However, when using an SSL connection and if it is set, it does make sense for the site to validate that it at least references something that could logically have initiated the request. Django does this when the connection is encrypted (uses https://), and actively requires it then.