python-requests - user-agent is being overriden

Question

I have

    logindata = {
        'username': 'me',
        'password': 'blbla'
    }
    payload = {'from':'me', 'lang':'en', 'url':csv_url}
    headers = {
        'User-Agent': 'Mozilla/5.0'
    }
    api_url = 'http://dev.mypage.com/admin/app/import/'

    with requests.Session() as s:
        s.post(api_url, data=json.dumps(logindata), headers=headers)

        print s.headers

        # An authorised request.
        r = s.get(api_url, params=payload, headers=headers)

I am getting rejected but it is because of 403 forbidden. And I printed the headers, I get:

..'User-Agent': 'python-requests/2.2.1 CPython/2.7.5 Windows/7'..

Why is my 'User-Agent': 'Mozilla/5.0' getting overriden? what am I missing here?

Answer 1

headers are not kept inside session this way.

You need to either explicitly pass them every time you make a request, or set the s.headers once:

with requests.Session() as s:
    s.headers = {'User-Agent': 'Mozilla/5.0'}

You can check that the correct headers were sent via inspecting response.request.headers:

with requests.Session() as s:
    s.headers = {'User-Agent': 'Mozilla/5.0'}

    r = s.post(api_url, data=json.dumps(logindata))
    print(r.request.headers)

Also see how the Session class is implemented - every time you make a request it merges the request.headers with headers you have set on the session object:

headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict),