Streaming a response with django-import-export

Question

I've been using django-import-export for a while now to provide CSV/XLS export functionality of data for users but as datasets get larger I'm encountering server timeouts.

I understand it's possible to use StreamingHttpResponse but from what I've seen this is done by writing your own CSV writer functions to output your data which, at least initially, doesn't strike me as something I can do with my current approach.

Is it possible to stream a response when you're using a django-import-export model resource to generate your file?

This is the kind of code I've implemented at the moment, with a standard HttpResponse;

class ExportConsolePlacesView(ClientPlacesView, View):
    """
    Export view for all places, either incomplete or complete.
    """
    model = Place
    http_method_names = ['get', ]

    def get(self, request, *args, **kwargs):
        self.object_list = self.get_queryset()
        console_name = self.console.name.replace(' ', '_')

        if kwargs['query'] == u'complete':
            dataset = PlaceResource().export(
                Place.objects.complete_entrants_for_console(self.console)
            )
            filename = '{}_complete_entrants'.format(console_name).lower()

        elif kwargs['query'] == u'incomplete':
            dataset = PlaceResource().export(
                Place.objects.incomplete_entrants_for_console(self.console)
            )
            filename = '{}_incomplete_entrants'.format(console_name).lower()

        export_type = kwargs['format']
        _dataset_methods = {
            'csv': dataset.csv,
            'xls': dataset.xls
        }
        response = HttpResponse(
            _dataset_methods[export_type], content_type=export_type
        )
        response[
            'Content-Disposition'] = 'attachment; filename={fn}.{ext}'.format(
                fn=filename,
                ext=export_type
            )

        return response

Answer 1

I queried this with the developer of django-import-export and he believes that due to the app using tablib it wouldn't be possible to stream the response due to the way tablib behaves.

I think it would not be possible to create stream of export from tablib which django-import-export uses.

https://github.com/django-import-export/django-import-export/issues/206