django and python requests - getting a 403 on a post request

Question

I am using requests to log into my Django site for testing (and yes, I know about the Django TestClient, but I need plain http here). I can log in and, as long as I do get requests, everything is OK.

When I try to use post instead, I get a 403 from the csrf middleware. I've worked around that for now by using a @crsf_exempt on my view, but would prefer a longer term solution.

This is my code:

with requests.Session() as ses:

    try:

        data = {
            'username': self.username,
            'password': self.password,
        }

        ses.get(login_url)
        try:
            csrftoken = ses.cookies["csrftoken"]
        except Exception, e:
            raise
        data.update(csrfmiddlewaretoken=csrftoken)

        _login_response = ses.post(login_url, data=data)

        logger.info("ses.cookies:%s" % (ses.cookies))

        assert 200 <= _login_response.status_code < 300, "_login_response.status_code:%s" % (_login_response.status_code)

        response = ses.post(
            full_url,
            data=data,
            )

        return self._process_response(response)

The login works fine, and I can see the csrf token here.

 INFO:tests.helper_fetch:ses.cookies:<RequestsCookieJar[<Cookie csrftoken=TmM97gnNHs4YCgQPzfNztrAWY3KcysAg for localhost.local/>, <Cookie sessionid=kj6wfmta

However, the middleware sees cookies as empty.

INFO:django.middleware.csrf:request.COOKIES:{}

I've added the logging code to it:

def process_view(self, request, callback, callback_args, callback_kwargs):

    if getattr(request, 'csrf_processing_done', False):
        return None

    try:
        csrf_token = _sanitize_token(
            request.COOKIES[settings.CSRF_COOKIE_NAME])
        # Use same token next time
        request.META['CSRF_COOKIE'] = csrf_token
    except KeyError:
        # import pdb
        # pdb.set_trace()
        import logging
        logger = logging.getLogger(__name__)
        logger.info("request.COOKIES:%s" % (request.COOKIES))

Am I missing something with way I call request's session.post? I tried adding cookie to it, made no difference. But I can totally see why crsf middleware is bugging out. I thought the cookies were part of the session, so why are they missing in my second post?

            response = ses.post(
                self.res.full_url,
                data=data,
                cookies=ses.cookies,
                )

This variation, inspired by How to send cookies in a post request with the Python Requests library?, also did not result in anything being passed to csrf middleware:

            response = ses.post(
                self.res.full_url,
                data=data,
                cookies=dict(csrftoken=csrftoken),
                )

Answer 1

For subsequent requests after the login, try supplying it as header X-CSRFToken instead.

The following worked for me:

with requests.Session() as sesssion:
    response = session.get(login_url)
    response.raise_for_status()  # raises HTTPError if: 400 <= status_code < 600

    csrf = session.cookies['csrftoken']
    data = {
        'username': self.username,
        'password': self.password,
        'csrfmiddlewaretoken': csrf
    }


    response = session.post(login_url, data=data)
    response.raise_for_status()

    headers = {'X-CSRFToken': csrf, 'Referer': url}
    response = session.post('another_url', data={}, headers=headers)
    response.raise_for_status()

    return response  # At this point we probably made it

Docs reference: https://docs.djangoproject.com/en/dev/ref/csrf/#csrf-ajax

Answer 2

You could also try to use this decorator on your view, instead of the csrf_exempt. I tried to reproduce your issue, and this worked as well for me.

from django.views.decorators.csrf import ensure_csrf_cookie`

@ensure_csrf_cookie
def your_login_view(request):
    # your view code