Django management command doesn't flush stdout

Question

I'm trying to print to console before and after processing that takes a while in a Django management command, like this:

import requests
import xmltodict

from django.core.management.base import BaseCommand


def get_all_routes():
    url = 'http://busopen.jeju.go.kr/OpenAPI/service/bis/Bus'

    r = requests.get(url)
    data = xmltodict.parse(r.content)

    return data['response']['body']['items']['item']


class Command(BaseCommand):

    help = 'Updates the database via Bus Info API'

    def handle(self, *args, **options):
        self.stdout.write('Saving routes ... ', ending='')
        for route in get_all_routes():
            route_obj = Route(
                route_type=route['routeTp'], route_id=route['routeId'], route_number=route['routeNum'])
            route_obj.save()
        self.stdout.write('done.')

In the above code, Saving routes ... is expected to print before the loop begins, and done. right next to it when the loop completes so that it looks like Saving routes ... done. in the end.

However, the former doesn't print until the loop completes, when both strings finally print at the same time, which is not what I expected.

I found this question, where the answer suggests flushing the output i.e. self.stdout.flush(), so I added that to my code:

    def handle(self, *args, **options):
        self.stdout.write('Saving routes ... ', ending='')
        self.stdout.flush()
        for route in get_all_routes():
            route_obj = Route(
                route_type=route['routeTp'], route_id=route['routeId'], route_number=route['routeNum'])
            route_obj.save()
        self.stdout.write('done.')

Still, the result remains unchanged.

What could have I done wrong?

Answer 1

The thing to keep in mind is you're using self.stdout (as suggested in the Django docs), which is BaseCommand's override of Python's standard sys.stdout. There are two main differences between the 2 relevant to your problem:

1. The default "ending" in BaseCommand's version of self.stdout.write() is a new-line, forcing you to use the ending='' parameter, unlike sys.stdout.write() that has an empty ending as the default. This in itself is not causing your problem.
2. The BaseCommand version of flush() does not really do anything (who would have thought?). This is a known bug: https://code.djangoproject.com/ticket/29533

So you really have 2 options:

1. Not use BaseCommand's self.stdout but instead use sys.stdout, in which case the flush does work
2. Force the stdout to be totally unbuffered while running the management command by passing the "-u" parameter to python. So instead of running python manage.py <subcommand>, run python -u manage.py <subcommand>

Hope this helps.