how to only show progress bar of the master node of tqdm in distributed training?

Question

tqdm flushes a lot in distributed training setting (torch.distributed.run) Is there a way to only display the bar from master node?

Answer 1

You can switch it off by setting disable=True parameter for non-master processes, for example:

# ...
    
master_process = ddp_rank == 0

# ...
    
for epoch in range(epoch_num):
  with tqdm(dataloader, disable=not master_process) as pbar:    
    # ...