Passing session from template view to python requests api call

Question

I want to make multiple internal REST API call from my Django TemplateView, using requests library. Now I want to pass the session too from template view to api call. What is the recommended way to do that, keeping performance in mind.

Right now, I'm extracting cookie from the current request object in template view, and passing that to requests.get() or requests.post() call. But problem with that is, I would have to pass request object to my API Client, which I don't want.

This the current wrapper I'm using to route my requests:

def wrap_internal_api_call(request, requests_api, uri, data=None, params=None, cookies=None, is_json=False, files=None):
    headers = {'referer': request.META.get('HTTP_REFERER')}
    logger.debug('Request API: %s calling URL: %s', requests_api, uri)
    logger.debug('Referer header sent with requests: %s', headers['referer'])
    if cookies:
        csrf_token = cookies.get('csrftoken', None)
    else:
        csrf_token = request.COOKIES.get('csrftoken', None)

    if csrf_token:
        headers['X-CSRFToken'] = csrf_token
    if data:
        if is_json:
            return requests_api(uri, json=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
        elif not files:
            return requests_api(uri, data=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
        else:
            return requests_api(uri, data=data, files=files, params=params, cookies=cookies if cookies else request.COOKIES,
                                headers=headers)
    else:
        return requests_api(uri, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)

Basically I want to get rid of that request parameter (1st param), because then to call it I've to keep passing request object from TemplateViews to internal services. Also, how can I keep persistent connection across multiple calls?

Answer 1

REST vs Invoking the view directly

While it's possible for a web app to make a REST API call to itself. That's not what REST is designed for. Consider the following from: https://docs.djangoproject.com/ja/1.9/topics/http/middleware/

As you can see a django request/response cycle has quite a bit of overhead. Add to this the overhead of webserver and wsgi container. At the client side you have the overhead associated with the requests library, but hang on a sec, the client also happens to be the same web app so it become s part of the web app's overhead too. And there is the problem of peristence (which I will come to shortly).

Last but not least, if you have a DNS round robin setup your request may actually go out on the wire before coming back to the same server. There is a better way, to invoke the view directly.

To invoke another view without the rest API call is really easy

 other_app.other_view(request, **kwargs)

This has been discussed a few times here at links such as Django Call Class based view from another class based view and Can I call a view from within another view? so I will not elaborate.

Persistent requests

Persistent http requests (talking about python requests rather than django.http.request.HttpRequest) are managed through session objects (again not to be confused with django sessions). Avoiding confusion is really difficult:

The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use urllib3's connection pooling. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase

Different hits to your django view will probably be from different users so you don't want to same cookie reused for the internal REST call. The other problem is that the python session object cannot be persisted between two different hit to the django view. Sockets cannot generally be serialized, a requirement for chucking them into memcached or redis.

If you still want to persist with internal REST

I think @julian 's answer shows how to avoid passing the django request instance as a parameter.

Answer 2

If you want to avoid passing the request to wrap_internal_api_call, all you need to do is do a bit more work on the end of the TemplateView where you call the api wrapper. Note that your original wrapper is doing a lot of cookies if cookies else request.COOKIES. You can factor that out to the calling site. Rewrite your api wrapper as follows:

def wrap_internal_api_call(referer, requests_api, uri, data=None, params=None, cookies, is_json=False, files=None):
    headers = {'referer': referer}
    logger.debug('Request API: %s calling URL: %s', requests_api, uri)
    logger.debug('Referer header sent with requests: %s', referer)
    csrf_token = cookies.get('csrftoken', None)

    if csrf_token:
        headers['X-CSRFToken'] = csrf_token
    if data:
        if is_json:
            return requests_api(uri, json=data, params=params, cookies=cookies, headers=headers)
        elif not files:
            return requests_api(uri, data=data, params=params, cookies=cookies, headers=headers)
        else:
            return requests_api(uri, data=data, files=files, params=params, cookies=cookies, headers=headers)
    else:
        return requests_api(uri, params=params, cookies=cookies, headers=headers)

Now, at the place of invocation, instead of

wrap_internal_api_call(request, requests_api, uri, data, params, cookies, is_json, files)

do:

cookies_param = cookies or request.COOKIES
referer_param = request.META.get['HTTP_REFERER']
wrap_internal_api_call(referer_param, requests_api, uri, data, params, cookies_param, is_json, files)

Now you are not passing the request object to the wrapper anymore. This saves a little bit of time because you don't test cookies over and over, but otherwise it doesn't make a difference for performance. In fact, you could achieve the same slight performance gain just by doing the cookies or request.COOKIES once inside the api wrapper.

Networking is always the tightest bottleneck in any application. So if these internal APIs are on the same machine as your TemplateView, your best bet for performance is to avoid doing an API call.