Python requests can't send multiple headers with same key

Question

I'm trying to send a GET request to a server with two headers which have the same name, but different values:

url = 'whatever'
headers = {'X-Attribute': 'A', 'X-Attribute': 'B'}
requests.get(url, headers = headers)

This obviously doesn't work, since the headers dictionary can't contain two keys X-Attribute.

Is there anything I can do, i.e. can I pass headers as something other than a dictionary? The requirement to send requests in this manner is a feature of the server, and I can't change it.

Answer 1

requests stores the request headers in a dict, which means every header can only appear once. So without making changes to the requests library itself it won't be possible to send multiple headers with the same name.

However, if the server is HTTP1.1 compliant, it must be able to accept the same as one header with a comma separated list of the single values.

---

Late edit:
Since this was brought back to my attention, a way to make this work would be to use a custom instance of str that allows to store multiple of the same value in a dict by implementing the hash contract differently (or actually in a CaseInsensitiveDict, wich uses lower() on the names). Example:

class uniquestr(str):

    _lower = None

    def __hash__(self):
        return id(self)

    def __eq__(self, other):
        return self is other

    def lower(self):
        if self._lower is None:
            lower = str.lower(self)
            if str.__eq__(lower, self): 
                self._lower = self
            else:
                self._lower = uniquestr(lower)
        return self._lower

r = requests.get("https://httpbin.org/get", headers={uniquestr('X'): 'A',
                                                     uniquestr('X'): 'B'})
print(r.text)

Produces something like:

{
  ...
  "headers": {
    ...
    "X": "A,B",
  }, 
  ...
}

Interestingly, in the response the headers are combined, but they are really sent as two separate header lines.

Answer 2

requests is using urllib2.urlencode under the hood (or something similar) in order to encode the headers.

This means that a list of tuples can be sent as the payload argument instead of a dictionary, freeing the headers list from the unique key constraint imposed by the dictionary. Sending a list of tuples is described in the urlib2.urlencode documentation. http://docs.python.org/2/library/urllib.html#urllib.urlencode

The following code will solve the problem without flattening or dirty hacks:

url = 'whatever'
headers = [('X-Attribute', 'A'),
           ('X-Attribute', 'B')]
requests.get(url, headers = headers)