Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to serialize requests Response object as HAR

I want to serialize a requests Response object as json, preferably in HAR format.

import requests
resp = requests.get('http://httpbin.org/get')

har = to_har(resp)  # <--- magic

but couldnt find anything online with my google-fu powers.

it seems that all data exist on the Response object, i hope i dont need to implement the whole HAR spec and there exist some code/utility i can reuse.

a valid answer might give: existing library or refer to a starting point if nothing exists so far for python and/or requests.

currently my simpler 3min solution (not HAR format) serialization to Response object looks like this (might be good start point if nothing exists):

def resp2dict(resp, _root=True):
    d = {
        'text': resp.text,
        'headers': dict(resp.headers),
        'status_code': resp.status_code,
        'request': {
            'url': resp.request.url,
            'method': resp.request.method,
            'headers': dict(resp.request.headers),
        },
    }

    if _root:
        d['history'] = [resp2dict(h, False) for h in resp.history]
    return d

i post this as i think not only me struggle to serialize Response objects to json in general regardless of HAR format.

like image 246
ShmulikA Avatar asked Jun 04 '19 11:06

ShmulikA


1 Answers

currently my simpler 3min solution (not HAR format) serialization to Response object looks like this (might be good start point if nothing exists):

Looks like this is the best solution. I've checked every HAR-related library on PyPI and the only close solution I found (except har2requests) is marshmallow-har. Unfortunately, marshmallow_har.Response.__schema__ match the internal structure neither requests.Response nor urllib3.response.HTTPResponse. So, the solutions I see are:

  1. Use the ad-hoc solution as you already do. To make sure the result has the correct structure, marshmallow-har can be used.
  2. Make your own marshmallow schemas by providing attribute argument to the fields. I'd advise to fork and extend marshmallow-har but it uses factories and other strange magic and can't be easily extended. So, it's better to start from ground zero.

And consider open-sourcing your solution :)

like image 143
Gram Avatar answered Oct 17 '22 06:10

Gram