Python Progress Bar THROUGH Logging Module

Question

I have seen different solutions for a progress bar within Python, but the simple stdout solutions are not working for my project. I have multiple classes and use the "logging" module to output information to STDOUT. I have a function of which I want to show a progress bar on one line, flushing the buffer each time.

Example of the simple progress:

for i in range(100):
    time.sleep(1)
    sys.stdout.write("\r%d%%" %i)
    sys.stdout.flush()

When I try to write via STDOUT and then flush the buffer, either the buffer is not flushed or the progress doesn't go anywhere. I am hoping to avoid some sort of threading or complicated process to make this possible. Does someone have a preferred way of making this happen?

Answer 1

I couldn't find a good solution for this, so I wrote enlighten progress bar to handle it. Basically it changes the scroll area of the terminal so logging is done above the progress bar(s) rather than having to redraw the progress bar(s) every time you want to write to STDOUT. This lets you write to the terminal as much as you want without having to modify logging, print, etc.

import logging
import time
import enlighten

# Setup logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

# Setup progress bar
manager = enlighten.get_manager()
pbar = manager.counter(total=100, desc='Ticks', unit='ticks')

for i in range(1, 101):
    logger.info("Processing step %s" % i)
    time.sleep(.2)
    pbar.update()

Answer 2

I solved it like this:

import logging
import time
from tqdm import tqdm
import io

class TqdmToLogger(io.StringIO):
    """
        Output stream for TQDM which will output to logger module instead of
        the StdOut.
    """
    logger = None
    level = None
    buf = ''
    def __init__(self,logger,level=None):
        super(TqdmToLogger, self).__init__()
        self.logger = logger
        self.level = level or logging.INFO
    def write(self,buf):
        self.buf = buf.strip('\r\n\t ')
    def flush(self):
        self.logger.log(self.level, self.buf)

if __name__ == "__main__":
    logging.basicConfig(format='%(asctime)s [%(levelname)-8s] %(message)s')
    logger = logging.getLogger()
    logger.setLevel(logging.DEBUG)

    tqdm_out = TqdmToLogger(logger,level=logging.INFO)
    for x in tqdm(range(100),file=tqdm_out,mininterval=30,):
        time.sleep(.5)

Output

2016-12-19 15:35:06 [INFO    ] 16%|#####9                                | 768/4928 [07:04<40:50,  1.70it/s]
2016-12-19 15:36:07 [INFO    ] 18%|######6                               | 865/4928 [08:04<40:34,  1.67it/s]