Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Performance using STDOUT (screen) vs regular file

I have been doing some research and I got this situation. If you want to write to the STDOUT (screen), you won't be able to do a multithread script which prints the data faster tan a simple single thread script. But, if you write to a file like this:

myPrinter.perl > myPrint

The result change and you can see that the multithread approach gets better time. My doubt is, since STDOUT (screen) or the output file are both shared resources, wouldn't be the access time similar? why multithread approach only performs better writting to file?

the perl scripts that I used in the experiments are:

Single thread

for my $i  (1..100000000){
    print("things\n");
}

Multithread

use threads;
use Thread::Queue 3.01 qw( );

use constant NUM_WORKERS    => 4;


sub worker {
    for my $i (1 .. 25000000){
        print("things\n");
    }
}

my $q = Thread::Queue->new(); #::any

async { while (defined( my $job = $q->dequeue() )) { worker($job); } }
for 1..NUM_WORKERS;

for my $i (1 .. 4){
    $q->enqueue($i);
}

$q->end();
$_->join for threads->list; 

Credits: the queue implementation was taken from one of the ikegami answers.

like image 747
Iván Rodríguez Torres Avatar asked Feb 23 '17 02:02

Iván Rodríguez Torres


1 Answers

This could be explained if writing to STDOUT requires some form of locking internally.

When STDOUT is connected to a terminal, the output is flushed after every newline. Otherwise, STDOUT is only flushed every 4 KiB or 8 KiB (depending on your version of Perl). The latter scenario presumably required fewer or shorter locks.

You could use |cat instead of >file to get the same effect.

If your actual worker spends a much smaller proportion of time writing to STDOUT, this problem should go away.

like image 90
ikegami Avatar answered Sep 21 '22 10:09

ikegami