Modify a Python "requests" module response object

Question

I'm requesting an HTTP API with the Python "requests" module.

result = requests.get(api_url)

The response is in JSON format and contains a 'data' key which is a dictionary. For some reasons, I have to make a second request further in the script and I'm trying to update the first dictionary (result.json['data']) with the second (result2.json['data']).

I tried this:

result.json['data'].update(result2.json['data'])

and this:

for key,value in dict(result2.json['data']).iteritems():
    result.json['data'][key] = value

but none of these worked, the final result.json['data'] is not modified and only contains the first items.

So I'm wondering if the Python "Requests" module produces some king of "read-only" objects or if I am just missing something.

Answer 1

result.json() is either a method (requests 1.0 and up) or a property. Store the result of that method or property in a variable, then update that:

json_result = result.json()
# requests < 1.0: json_result = result.json

json_result['data'].update(result2.json()['data'])

(In older requests versions (pre-1.0), result.json is a property, and underneath that is still a method returning your object on the fly).

Otherwise, Response objects are not meant to be mutable. You'd have to alter the response body text to alter what result.json returns, and it would require intimate knowledge of how the Response objects cache the response body to alter that. This would tie you to a specific requests version.

For the current requests series (2.4.x), you can swap out the response._content attribute:

result.encoding, result._content = 'utf8', json.dumps(json_result)

where I've made the assumption that result.content already exists (e.g. a streaming response has already been consumed), and that json_result has already been updated.

If this will work in other requests versions depends; _content is an internal, private attribute. The leading underscore indicates it is not part of the public API, so the developers are free to change the meaning of that attribute or altogether remove it in future releases.